Ride out by day, hire villagers and give them a trade, pay for every wall and tower coin by coin. When dusk falls, portals open on four sides. Hold the line until dawn, rebuild what was lost, and push your settlement outward until the portals themselves fall.