Starving out the garrison helps, but you can't starve the militia. Try not have enemy parties inside when you set up your siege as well. You do want to have way more troops than they do, the more you bring the less they can deploy at once and the more you can deploy. Unlike vanilla bannerlord you will not get clean sieges with single digit losses.

I try to bring 2x as much, but having 500 or so more makes a big difference.

The typical Shokohu siege will usually involve a meat grinder at the start where their ranged gets to basically hit you for free. Your wall guys will focus on taking out the archers inside, while the battering ram formation generally beelines attacking the gates and blobs of melee units defending gates. There are a bunch of gates to get through before reaching the keep. By the end of it about 50+ nerds have been routed and ran back into the keep. But in the process half your army went down, good surgeons help a lot here.

You can have an outsized effect on sieges with your usual player shenanigans, the units coordinate in a pretty rigid way that opens them up to flanking from a player. I've had ones where I get triple digit kills, when you can stay alive and take out a lot of infantry you will get great results on either offence or defence. On the attack it really pushes your army up, and the more of the castle you clear the less arrows and bullets will hit your guys.