There is absolutely 0 need for off-game invites for it to work, as we figured out what the potential problem was.
I tested for about a couple hours with a friend and we discovered that some of the mods are corrupting files that cause multiplayer to be missing weapon textures, models, and UIs. What we did was going from 3-4 mods to begin with, and test if it works well, and then increase the number by another 3-4 to see if certain mods are corrupting game files. It turns out to be the case.
For example, a new gun mod like the 24th STS 416 is indeed causing problems in multiplayer. We started with just some of the older mods from Modern Lancer (the author) like the risers and few other guns, they worked fine hosting the game and joining. As soon as we added the STS 416, the notice "unfolding inactive mods" popped up during connection to host (which is a bad indicator that your mods won't be working), and indeed when the host game loads in everything is missing textures and models. Same thing happened when we loaded the QBZ 191 mod and everything got corrupted.
I would simply suggest people to load just a few mods at a time from Modern Lancer, and test it out yourselves by adding one or two more at a time to see if it's also the case for you all.


All the best.