I don't think Full Galaxy will be that bloated compared to Known Galaxy, as it only has 10 more planets. Although it really depends wherever you want the Empire of The Hand or not.

I can honestly say that having or not having the hand doesnt much matter unless you are Pentastar, or well into the late game. The real question is if you want to conquer the unknown regions, or potentially have a strong empire sitting in the unknown region at the end game (for me usually pentatstar with dozens of secutors for some reason)