Should we have a baby?

To say I'm in a dilemna is an understatement. I've been seeing one of my coworkers now for almost three years and he's married. Now before you tell me what a low life I am know I never planned on falling in love with him and we have both tried to end our relationship several times over the past three years. Two months ago we decided once and for all we were going to stop anything besides platonic friendship. To make matters worse our families are extremely connected. His Dad and my Dad are business partners so we are constantly around each other. It's not like we can stop being around each other because that would send off major red flags.We both realized that we have to end this some time and it's not fair to his family or us. So we stopped. And we both were miserable but we stuck to our guns. Then our families went on vacation to the Bahamas together two weeks ago. While we were down there he sent me a text at 2 am one night that said "I can't take this. I want to be with you!" I was shocked and relieved at the same time. I sent him a text back that we would talk later when we had time alone. When we got back we made plans to meet and we talked about our relationship. I told him that I knew it was wrong and he did too and that I loved him but I didn't see how it was going to work. He said he felt the same way but he couldn't give me up. I asked him what he wanted and he said it didn't matter what he wanted because he can't have it. Then he said I can't stop. I love you.. I told him that I felt that way, but I was scared and that I can't keep going back and forth on this. We were in the middle of talking when he got a phone call. His sister went into labor. He was at the hospital for two days because she had complications and the baby was born a month early. Today as he was leaving the hospital he called me and said he wanted to stop by on his way home. I told him that was fine. When he got there I had no sooner opened the door when he started kissing me. Knowing our previous conversation and how I told him to be sure what he wanted I paused to ask him if he was sure. He answered me with a passionate kiss. Then he asked me if I wanted to go all the way. We've never been all the way before, but close enough. I told him I did but was he sure. He said "You could get pregnant". I told him I understood but is that really what he wants. He never answered me instead he just got this determined look in his eye. I wanted to do it so bad but when it started to happen suddenly things wouldn't work. He was embarrassed and he apologized. I asked him if he wanted to go all the way with me and he said yes. Then I asked him what would happen if I got pregnant and he said he didn't know. He was the one to bring it up and he's been around his nephew for two days now. Am I reading more into this or is he really thinking about us having a baby? Please don't tell me what a piece of crap I am. I'm in a bad situation and I need helpful advice.