It can sometimes happen that there are slight assymetries in a build one doesn't notice visually, but that still affect placement options. I can't be more precise unless you provide an example of what exactly you mean

take all the parts off and rebuild it sometimes the game glitches

Make sure that the pieces are connected to their corresponding matches on both sides. There will be positions where a part can connect to two others, but it's positioning is based off of only 1 of them. You need to make sure the correct piece is highlighted to make sure it is positioned is just the correct manner.