Ben Higgins' Bachelor Blog: Why This Week Made Me Felt Like I Had Ruined Everything

After Ben Higgins made viewers swoon as a fan favorite on Kaitlyn Bristowe's season of The Bachelorette, he's now on his own journey to love on The Bachelor. He's been blogging exclusively about each and every rose ceremony for PEOPLE – check back each week and follow him on Twitter at @benhiggi!

Wow. That was intense. Even watching it back I got uncomfortable because that week was just filled with so much angst. It was really the first time since this whole thing started that I started to doubt everything.

And I said from the beginning when all those "Perfect Ben" commercials were on TV that I was far from perfect, and I think over the past two weeks you've seen the evidence of that. I make mistakes, and I think anyone would when on a journey like this, but this was the week where I thought my mistakes might really cost me the love that I came here to find.

Anyway, enough with the generalities, let's start from the beginning and dive in!

Let's start with Olivia. I'll get to the two-on-one date in a bit, but let's start with Mexico City. I expressed how last week really shook me, hearing that women like Emily and Amanda and Jen thought so poorly of Olivia. And while you all have had the benefit of seeing Olivia while she's with the other women and in her interviews, I didn't. All I saw was what she presented to me in person. And that was that of a beautiful, poised woman who may have been controversial by being a little aggressive and talking to me first in group situations, but I didn't see that as any sort of egregious offense. But what I was hearing now was really shaking me. And it was happening all at once. Quickly.

I was struggling to process what was such a game-changing moment, and I likely should've been upfront with my concerns right when Olivia grabbed me after Emily, but I was still reeling from what I heard. But after talking to the other women, I knew I could not go into this rose ceremony without talking to Olivia again.

I try to put myself in the shoes of the women often when I make decisions. And when I put myself in Olivia's shoes, I knew I would want a chance to face these accusations and tell my side of the story before anything drastic were to happen. So that was my intention in pulling Olivia aside. I just wanted to hear what she had to say about all of this. What I heard though honestly just confused me even further. In essence, I heard that Olivia agreed that she and the other women did not get along. But in hearing her side and why it was never brought to me, I respected that. She didn't want to cause a problem if she felt she could handle it. I sincerely wish that she had felt enough comfort with me to come to me with her problems but I also respect that this is an accelerated process and she didn't want to waste our time with problems that she felt weren't pressing.

However, there were also some alarming things in what she said to me. When she was describing her differences, I could see how those statements could be hurtful and condescending to the other women. That she felt herself on a level above them or looked down upon them. And it made me wonder if there was more to Olivia thinking of herself as intimidating than just being a woman I had singled out as a "front-runner." (I use quotations because I hate using that term in what is not a competition. It is a journey of people trying to find out if they can legitimately see a life with one another. I just can't think of a better term at the moment so I'm forced to use it.)

But when all of this was said and done, what I knew is that I didn't yet know enough. I didn't want to make a rash, emotional decision. I had two big fears (amongst many others) coming into this: One was having the wool pulled over my eyes and having a woman make me fall in love with a person that she really wasn't; the second was saying goodbye to someone that I could really see a future with too soon and regretting it. On this night I was being confronted with both.

I decided that I needed more time to make this decision and I would respect the feelings I had had for Olivia and trust my gut to give her that rose in the first place. I mean I would hate to send a woman as amazing as Olivia home simply because she was misunderstood by a group of women also dating the same guy as her. But I won't lie and say that my guard was definitely up at this point and my eyes open.

Bachelor Blog: Why This Week Made Me Felt Like I Had Ruined Everything" data-ad-channel="Brightcove" data-ad-subchannel="" data-auto-play="no">

But I still had to say goodbye to someone that night, and sadly that was Jen. Jen was a woman that frankly wowed me on night one. I mean, how could she not? She's beautiful and smart and intriguing. But I found myself here, five weeks later still just intrigued. I felt like I was only seeing glimpses of her and was still waiting for her to feel comfortable enough to open and let me see who she really was. And I felt like I had really tried – maybe more so than with anyone else outside of Jubilee – but I had gotten to the point where I was simply further along with other woman than with Jen. And I didn't see that changing. This is a difficult environment to open up in, and it didn't seem like Jen was going to be able to do that. She is going to make some man extremely happy one day. That was a hard goodbye.

• PEOPLE's collector's edition The Bachelor, with new interviews and updates on your favorite contestants, is on sale now in stores and online!

As hard as all of that was, I was ready to get to the Bahamas and start a fresh week. The Grand Isle Resort and just Great Exuma Island itself was just so beautiful. I thought his would be the perfect week to put the drama of Mexico City behind us and move forward.

Little did I know that even more drama awaited me here.

I knew that asking Caila on a second one-on-one would cause some controversy. But I felt that it was something I needed to do. Caila had caught my eye and my attention from the very beginning – I mean from the moment she jumped in my arms out of the limo really. But our first date, as fun as it was, really didn't allow us to get to know each other. And for the chemistry that we had together, it still seemed too surface-level to be getting this close to hometown dates.

If last week in Mexico taught me anything it was that I could not afford to waste time and keep women that I just liked. I did not come on The Bachelor to just find a great girl that I'll date for a while. I really came her to find a future wife. So this date needed to happen. I needed answers when it came to Caila.

So as fun as the day was, checking off a bucket list item of deep sea fishing, I went into the night knowing that we needed to get somewhere. What I got in response was really unexpected in a lot of ways. I was so confused, and I wasn't lying when I said there was a point in that conversation where I was pretty positive that Caila was breaking up with me.

But as confused as I was, the more we talked the more I realized that there was so much more to Caila than I had originally thought. This was a woman with real depth of thought. A woman that would challenge me. And what was almost more important than any of that was that – as much as I still didn't understand exactly what Caila was hiding behind that perfect smile – I was 100 percent sure that I wanted to take the time to find out. And that even though she wasn't fully ready to dive in and open up just yet, she wanted to stay here and keep trying with me too. So while I walked away still unsure of whether or not Caila was my future wife, I was absolutely sure that I wanted to take the time to find out.

It was at the group date that this week really started to go downhill. Which is a hard thing to do when you are going to swim with pigs in the Bahamas (I mean c'mon?! swimming pigs!!! How cool is that?!) … but it definitely did. At first everything seemed fine and fun. The boat ride was great, the feeding the pigs was hilarious and fun (they were NOT pork hot dogs guys! Chicken! I promise! I triple-checked!), even if the pigs were WAY more aggressive than we thought they would be. But then I really started to notice some tension amongst the women – but not towards each other. Definitely towards me.

It wasn't until much later that I heard there was some rumor going around that I had supposedly told Lauren B. that she was the only one I cared about (as lovely as Lauren B. is, that was definitely not true) that some of it started to make sense. But initially I was baffled. These women that had just been having such a great time suddenly seemed mad at me or at least didn't care about me anymore. It was like someone flipped a switch.

I started trying to figure out what was going on and my conversations with Becca and JoJo, and I could tell from those that it had something to do with me giving too much attention to Lauren B. I was confused, though, since I really hadn't spent that much time with her that day. And definitely not more than anyone else. And I knew that because I always try to be so conscious of things like that on these group dates with spending too much time with anyone or making anyone feel left out.

So I assumed that was what was upsetting Leah as well, but once I grabbed her to talk it was apparent right from the start that there was more to that. She was really hurt that I had taken Caila on a second date before I had taken her on a first one. I actually think I was so worried about upsetting Emily and Olivia for being on the two-on-one without having had a one-on-one date that I thought Leah would understand. But I was wrong, and I really can see where she is coming from. In fact she told me that she would've actually preferred the two-on-one. Like I said, I'm not perfect and this was definitely one of the mistakes I made this week.

I hated hurting her and felt awful but didn't know how I could fix it in this moment. I really just asked her to try to enjoy the day and that we could talk about it that night when we didn't have a 100-lb. pig butting into our conversation when we were trying to talk seriously (I'm not kidding either … there was one pig that would not leave us alone). Plus at this point I was so on-edge about spending too much time with any one girl after everyone seemed so upset about me and Lauren B. … I don't know. I guess it's safe to say my mind was spinning at this point, and it really felt like I couldn't do anything right.

The ride back to the mainland from the island wasn't shown, but it was really uncomfortable. Girls were completely avoiding me and whispering to each other most of the way. I actually was worried that many of them might leave once we docked, and I half expected some of them not to come to the after party that night. Maybe more than any other time in this entire journey, I felt like I had possibly ruined everything. I had let these women down and maybe there was no repairing it.

But I decided as I was getting ready for the night that I needed to step up and fix what was wrong. I couldn't risk losing these amazing women, and I needed to take this problem head-on. It's why you saw me come to the party with such a sense of purpose. I didn't want to hide from the problem, I wanted to address it and repair it if there was any way to repair it. It wasn't just JoJo and Becca and Leah either. Lauren H. was obviously upset. Amanda seemed extremely upset too. Really the only person that didn't seem upset was Lauren B.

I started talking to each of the women, one by one. I wanted them to know that I cared, that I didn't expect to be chased and that I was willing to put in the work needed to fix things with them. I've said before how much these women had come to mean to me and they needed to know that. And to know that if something was wrong, be it that today or in our potential future as husband and wife, I wasn't going to run from it. And after talking to each one of them things seemed better. I wasn't sure that all the issues were cleared up, but I definitely felt like we were back on track.

That was until my conversation with Leah. Her telling me that Lauren B. was not who I thought she was another game-changer. It felt like Mexico City with Emily and Olivia all over again. Was anyone here being honest?! Was I just being duped by a bunch of great actresses?! I really started to question everything.

RELATED VIDEO: Ben Higgins Makes Some Tough Decisions After the Girls Begin Trash Talking One Another

But if I learned one thing from Mexico City with Olivia, it was that I needed to address this head-on, just as I had done with the other women that were upset with me that day. So when Lauren B. sat down, I brought it up immediately. I am not sure after everything that had happened in the past week that I would have been able to hide it anyway, but either way I needed to figure this out.

Lauren B.'s reaction was what I hoped it would be though. Lauren was bewildered that anyone could think that about her, and I could tell she was truly hurt by it. I feel like if I had someone say something like that to me I would react the same way and that meant something to me.

Despite that, I was still shaken by the accusation. Mostly because it meant that one of those girls was not being honest, and I truly cared for them both. I didn't know what I was going to do yet but like with Olivia, I felt like I needed more time to figure this thing out.

What was important to me, though, was that I gave Amanda the rose that night. It was one of those nights where I truly felt like I wanted to reassure every single one of those women, but I felt with Amanda that perhaps I had taken for granted how wonderful and sweet she always is. And I wanted her to know that I noticed and that I still saw something with her moving forward.

When Leah knocked on my door later that night I was surprised but in a very good way. I actually really appreciate it when women make the effort to do something extra. I feel like every relationship should be filled with that on both ends so I was excited that she was here and that we could make up for some of the time that we hadn't yet had on a date right there.

But the longer it went, the more I realized that we weren't doing that at all. Instead, we were talking more about Lauren B. And while I really do appreciate it when a woman cares enough about me to warn me of a mistake I may be making, the longer this went, the less it felt like that.

And if I am being honest with myself and with Leah, I wasn't sending her home for what she said about Lauren B. It was more about the fact that I saw that OUR relationship wasn't moving forward at all. And it hadn't been for a while now. And the fact that we were sitting here talking about another girl, no matter who that girl was, instead of focusing on us, was just another sign that we had not progressed as a couple since the first week or so. And that was a problem.

I promised these women that if I ever knew that I didn't see myself with them I would let them know the instant I knew that and not lead them on or keep them from their homes and families a second longer than necessary. (I had also asked them to afford me that same courtesy if they felt I wasn't the one for them). At that moment in the conversation I knew that Leah and I were not meant to be together. And that if I didn't tell her then, I would just be telling her at the rose ceremony in a couple days. So despite the awkward timing, it was best for both of us if I said goodbye right then and there.

Ughhhh!

RELATED VIDEO: Keeping Track of Bachelor Ben Higgins's 'Kiss List'

Bachelor Ben Higgins's 'Kiss List'" data-ad-channel="peoplenow" data-ad-subchannel="sharethisnow" data-auto-play="no">

So all this, and I still had a two-on-one date to look forward to the next day. Now I am sure that was an awful thought for Olivia and Emily as well, but I was nearing my emotional breaking point by now.

It was only the excitement I had to see two women that I really liked that kept me in good spirits despite how I knew the day was going to end. I hated the idea of this two-on-one date, but this past week had taught me one thing: When there is conflict, it is best to face it head-on. I knew Olivia and Emily had issues with each other, and I thought the best way to make a decision was to have a true side-by-side comparison of my feelings for each of them and make a final decision. It was important to me that this was not about Olivia vs. Emily though. It was going to be about Ben and Olivia and Ben and Emily. Nothing more. And I knew those two women would respect that.

Of course the weather, which was the result of a nearby hurricane threw a kink into all the plans that day. We were supposed to take a seaplane ride over the islands and then land at a perfect little private island for us to spend the day on. The storm reflected the mood of that day however and even when the weather cleared up enough to fly, the seaplane wouldn't start when the girls got in it. As much as The Bachelor is about cool experiences, safety is obviously the priority, so like with the storm in Las Vegas cutting short the helicopter ride I had planned with JoJo, we decided the plane was not worth the risk.

Unfortunately it also meant we would get less time together. By the time we got the boat and had me ride over to get the women and then off to the island, I knew there was little time to waste.

My conversation with Olivia there reminded me what I liked about her. She was sweet and confident and poised. She put her heart on the table and I can't tell you how much that means to me. But she told me that she loved me in that moment. And those are very serious words to me. Very serious. The second she said them I fell into myself and started examining how I really felt for Olivia. Whether or not I felt like I could say those words back.

Talking with Emily after that I really remembered what I liked about her though as well. I just can't sit with Emily and not smile. She's so endearing and so honest and fun it just is impossible not to. But on top of that, what she was telling me was very meaningful. Sometimes with how fun Emily is, it's easy to dismiss how emotionally intelligent and self-aware that she is. That she knows she has a lot of growth left and that she wants me by her side as she grows as a person really meant a lot to me.

But as I went back and I knew the moment of truth was at hand, I still couldn't get what Olivia said to me out of my mind. She was in love with me. I knew I wasn't in love with her, but could I be?

One thing I feel really badly about was that I picked up the rose when I went to go talk to Olivia. In all honesty, I'm not really sure why I did that. There were so many other things on my mind that it must've just been instinct as I knew I was making my decision and I usually pick up the rose when I do that on other dates. But this wasn't like other dates. Another mistake and one that I regret to this day. Told you, far from perfect.

I think I said it all with what I said to Olivia. Even if I took away all the controversy of last week in Mexico and disregarded the previous red flags that didn't come to my attention until much later, I just couldn't see myself as her husband. This past week had caused me to really step back and take myself out of the bubble that I had been living and dating in to re-examine what I was really here for. It wasn't about having fun with a bunch of great women. It was about finding a wife. I told myself going into this that I would act with intention at every turn and like with Leah the night before, I knew then that Olivia was not the one for me. And I owed it to her to tell her that right then and there.

I want to stress, however, that after watching this back and hearing all the criticism and actually KNOWING Olivia from personal experience rather than watching her from afar, she really is a good person with a huge heart. Far from perfect like myself? Absolutely. And I am sure there are some mistakes she would admit to and things she would do differently if she had the chance to go back. But she did it all in an effort to find true love. She may not always say the perfect thing, but when you really get to know her, she is a great woman that I will always be grateful to have met.

After that, I won't lie, I was spent. I could not handle another cocktail party. Not emotionally. Too much had happened already this week. So while I knew it was hard for the women that may have felt the need to express something more to me, I knew what I needed to do. All of this re-evaluating my priorities that week I knew that I needed to look at this amazing group of women and decide who actually could be my wife and who I just needed to say goodbye to.

It went through my mind at one point that I would send no one home. And just take the remaining group to our next spot. But I knew I had to be honest even if it was hard. And despite what was a great date the week before in Mexico City, deep down, I think I knew that Lauren H. and I were behind where I was with the other women. And that as much as she meant to me and as much as I enjoyed my time with her, she was not my wife. It was really that simple. I could see a future with the other women there. But I knew that if I gave Lauren H. a rose tonight, I would be breaking her heart in the future. Maybe not next week or the week after, but definitely before this whole thing was over. She's too good a woman for that.

So there you have it.

I think this is my longest blog (which is saying something because I tend to be a bit wordy), but it was also the most emotional week I faced the entire time. Until the final one. I know you guys got a little sneak peek of what is to come, and it doesn't even do it justice. The emotion and agony I faced heading towards that final day still haunts me to this day. When I decided to be The Bachelor, I never ever imagined it would end like this. I'll save all that for when we get there because I don't think I have it in me to get into all of that and the two "I love you"s right now. It's just too much.

But if you think this is going to get any easier, think again! Next week we go to a place that is closer to my heart than any other place in the entire world. My hometown of Warsaw, Indiana. Another week that changes everything for me in so many ways. The emotion of being home, seeing what my life would actually be like with these women. Not to mention that it is this week that I make the decision whose families I will be going to meet. And with a single mother of two still someone I am seriously considering, that is not something I take lightly. The Bahamas may have been the most emotionally exhausting week that I faced on this journey, but next week may have been the most important in a lot of ways.

So get some rest, Bachelor Nation: It only gets harder from here!

The Bachelor airs Mondays (8 p.m. ET) on ABC.