One way is to prioritize a lot of ranged units. Let them take down most of the guards and then you the player can jump from the ladder onto the wall and finish off the rest.

Another way is to build high leadership. Get as many mid tier units as you can and have a high relation with your faction. Ask other lords to follow you and then attack using the "send troops with out me".

I do this when their is absolutely no way to get in.