Every castle has a wall and a stairways down to inner castle.

Bring all your man down the wall.

Form 4 lines.

Make infantry and cavalry block the bottom of stairs like 300 thermopolye

Make archers line up on the side of stairs to fire up on enemies coming down.

You can 100% win every sige defense even if you are 1600 men down on numbers.

Here is a reference video:
https://www.youtube.com/watch?v=Tgxfts6q_NA&feature=youtu.be