Villagers tend to either walk through the wooden walls or go half way across the map to find a break in the wall instead of using the gate that's right in front of them.