I'm going to guess this woman was your first "true love" so to speak. That has a strong pull. I dated my uni girlfriend for about 2.5 years and thought she would be the one. Of course that was silly thinking.
The other pull is you have kids with her. I couldn't comprehend having them as young as you have and so its difficult for me to work how that would fare in the mix.
Overall, you have to accept the old relationship is over. Going back rarely works from my observations as a 41 year old on his 2nd marriage. You will make better parents to your kids apart then together.
As for this new bird, be very careful o the rebound. After my divorce i was single for a while and then ended up dating a woman who was exotic, fun, hot, and shagged like no tomorrow. But she was also nuts and manipulative and eventually made me miserable. Also, being "hot as fuck" is, quite frankly, not a solid reason for getting into a relationship. I mean this in the nicest possible way - but it does sound like you have some growing up to do. You are not the same person at 27 as you were at 20 or whenever you met your ex and you will change again in another 10 years or so.
Good luck.