You should build the +15 growth pop building(think its called idolz?, and upgrade them no further than t2)in each town/city of a province and then destroy them once you've built everything, pick the +50 growth decree, grab the starting growth research, and then lastly make sure to raze a settlment every once in awhile for the +25 growth. You can hit t5 and max out one military chain FARRR before any other faction can so it most certainly isn't slower. There's also a bunch of other stuff you can do to pad your growth but those are the important things you should be doing right out of the gate.

Besides that consolidating how you apply your growth will make a huge difference; if you build everything up at once you won't get anywhere, instead plop down all your idolz, if its an econ province then build da horde in every town/city but leave them at t1 and then only max your settlements walls. Generally ill be prioritizing one structure chain at a time in order of importance.

Honestly I really like the mechanic, makes their faction feel different and given that orcs arn't exactly known for their civic engineering the tunnel visioned building feels quite orcy.

Was getting my ass kicked by rats in a recent head to head campaign and had to make a go for broke play against the vamp player before I got outscaled. He had just hit t4 when i showed up with grimgor and a 10 stack of blorcs + the immortulz and arty. Punted many a zombie that game.