I have custody of my 10 yo son. I want my boyfriend who use to live with us to move back in. Once while my son was not in the home my boyfriend and I had a fight where the police was called. My ex husband says if I let him move back in, he will take me to court and get custody. What is the liklihood of this happening?