The Empire's ground units are honestly fine. Have you tried to recruit the riflemen Stormies? They're a straight upgrade.

If I remember as the Empire you can build Scout Troopers and Imperial Comandos which are pretty good if anything I think in space the empire ships need a nerf to where they all don't carry high tier fighters since the Empire focused more on firepower instead of fighters though I'd leave the option to build these high tier fighters as their own units like of like in base game where you could build tie defenders.