I don't believe it is possible to designate certain groups of prisoners to a canteen, but there are some ways to do this, somewhat.

If you designate a Shared Canteen and designate different sec levels to eat within the canteen at different times, along with segregating the security levels to prevent high risk inmates from interacting with lower sec levels with servos and timers, it can essentially allow for different groups of prisoners to eat together within a given time in the regime. Prisoners will always use the facilities nearest them, so if you separate the canteen regime times and prevent access from other security levels for that time frame, it can achieve this.

You may not be able to double-designate, but you can choose at what time different sec levels eat, which helps in a way. I hope this helps

true, but enter the visit room or maybe the mid ward and you start having problems

what I used to do: make multiple doorways with small "airlock rooms" and designate these airlock rooms as desired security levels.

with a zones mod I can do it without doors and walls