If anyone has the issue where pawn gizmos disappear randomly, and find it's whenever there's a loadout assigned but the pawn has an item in their inventory not part of their loadout, I've managed to track down the culprit thanks to earlier posts in the comment section:

Drop All Inventory

If you have that mod and are experiencing this issue, disable that mod. To be honest, it was only when finding out it caused a problem that I realized the "Drop all inventory" gizmo wasn't there anyway, so if you're using this and that together, you're probably not missing out on it if you disable that anyway

Hope this helps and saves anyone else having to dig through the comments (figured making it a discussion would make it more visible for people)