I have observed that the initial manual leveling points seem messed up. For example I was able to give a standard follower over 300 HP in a single row (he was level 6 at that time). Also the points alloted do not seem to match the current level of the follower. Basically, it's easy to make him OP in term of stats but much difficult for his skills (and what about perks?)