The town of Smugglers is built around a natural deep water cove which forms a perfect harbour. The cliff faces around the cove are riddled with caves which are accessable only by row boat at low tide yet which do not fully flood. These were used as bolt holes and storage caves by smugglers long ago, and hence a thriving community of smugglers and pirate types grew here.
The town is now quite a reasonable size and is still home to many a pirate vessel. This is a staging post for raids east into Haven's territory or further east still to the shores of Hishan.
The harbour has been built up and fortified as this is the first settlement which is likely to be hit by Haven navy vessels out for some sport. As such the harbour walls are well reinforced and have balista mounted atop them.
Smugglers is a sea based town and is actually really difficult to reach by land. It is build at the bottom of a cleft in the cliff faces and the only way down to it is via a winding staircase which is obviously well defended. The town cannot really grow any larger due to the geographical restrictions of the cove, unless the pirates choose to built atop the cliff face but this is unlikely as the cove is well hidden behind a small wood and building up there would mean it became much more visible. Besides the cove is extremely well protected and any buildings up on the cliff would not be so.
Visitors are allowed to Smugglers, whether by boat or afoot, as long as they are not military in nature. However the town has a rough nature and outsiders are nearly always tested by the locals to see what they are made of. If they prove weak they will likely be robbed or killed. If they prove they can defend themselves they'll be left alone and respected.