You can put Anya back to the party right after you finish the camp encounter where she temporarily leaves.

This causes a problem when you go to the Ludo cave right after and the game simply doesnt realize Anya is in the party. Based on what you choose to do with Reya Anya is removed from the party(off screen) or stays.