They dont disappear, all cultures will join the same culture weakest faction.

If you are bretonnian, sooner or later they'll start joining you when their faction is destroyed, but you wont be able to convince them since you can't recruit the leading clan of a faction, and all bretonnian dukedoms only have 1 or 2 clans.

Same vanilla rules apply.