Upgrade your town-level. There should be 10 - Town - Levels. I think its level 2 or 3.

It'd be nice if I could destroy dwarven buildings after conquering their territory.

Originally posted by McWinters:

I Have a similar problem in Durin's Hall, says i can't go to stabilization because i need a mine in Durin's Hall. So i cannot access the Duchy buildings.

You need to find the minerals in the different counties. If you check your activities page it should have a mining expedition, do that and you can find the different minerals inside the mountain... prepare for the snowball effect, you'll start rolling in cash and become so rich quickly you'll not know what to do with it.

edit: once you find out whatever mineral you have in the county you should then be able to build the "Mines" building, so make sure you have a spot open or know what you want to change out. You'll get more building spots as you dig down so it's not a big deal.