Estranged ex-husband wants to see children! Should I let him?

Okay so It's a long story, but I'll try to some it up for you all! I was 5 months pregnant with our 3rd child when my ex-husband, who was always both mentally and physically abusive ran off and left the children and I! I can't say that I wasn't happy to see him go that day because I actually was, I no longer wanted to be around him and more important than that, I didn't want him around my babies for fear that he would turn abusive towards them! I hadn't seen or heard of him until 4 days ago, when out of the blue he calls me and says that he's sorry for ever treating me the way he did and that he never should have left me by myself with the children, he also told me he would like to see them if I would let him! I don't know what to do! He seemed very sincere when he called, but you never can tell! If I let him, I don't think he'll come in here and start punching our children around or anything, more than that I'm just worried what effect it will have on them because their so young! Lillian (10) remembers him a little, Ezra (6) doesn't remember him at all, and Xavier (4) has never met him, to have a strange man come walking into your home, saying that he's your father, I mean what kind of things could that do to a child's mind? Maybe I'm just over reacting! What do you think I should do?