The game never tells you this, but if you demolish the main settlement building, it gets reset to a level 1 of YOUR culture next turn and the garrison begins filling up. Same goes for ports.
Don't click convert and then remain stuck in the same region for 3-4 turns with your stack waiting until the level 2 or 3 or whatever building is converted to an equivalent level of your faction to prevent the enemy from just walking into your settlement that will have 0 garrison until then. Yes, it is cheaper and faster than starting from level 1 and building it up back to the level it was when you conquered it, but sometimes you just need to build up the garrison fast. Also, other times you conquer, say, a level 4, but havent researched the technology to build a level 4 of your own, and therefore cant convert it.