You went from this:
My gut says this is a problem. I have to find the courage to tell him I can't be a secret, and I need to do it soon.
To talking with him and *very quickly* coming to this point:
I learned that our relationship will remain as private as possible.
Then this starts happening:
I told him the first weekend how hurtful it was, he was very apologetic, but then a couple of nights ago he did it again, and I sort of laid into him with a long message.
I can't really explain why, but this relationship strikes me as one with a lot of unspoken secrets, and one in which you keep very few boundaries, and like *you* are getting lost in it. In him and his needs driving everything.
It seems like you are trying to read his mind again (something almost everyone does from time to time), but it strikes me as sign of a little enmeshment starting to build up in this relationship. Even more so, the attempts to try to guess what he is thinking is unlikely to be accurate.
Red flags pop up for me when you describe "learning" the relationship will be kept as private as possible. Actually, you learned that he will want to keep the relationship as private as possible. You did not describe him addressing your needs at all. You didn't describe it as two people coming to an agreement that the relationship will be kept private, but one person declaring what will be.
I'm a little concerned his needs are driving this relationship and your needs are being lost, *you* are being lost in it, until you finally can't take it anymore and sort of (respectfully) vent in a possibly ineffective way (rather than sitting down face to face and claiming your space and needs and finding compromise.)
I think it's time to ask your doc for help for finding a non-religious therapist to help guide you through how to set and keep boundaries, and to be able to declare your space to exist when in a romantic relationship. There are also online counseling services too. You have got to learn it's safe and healthy for you to exist, your needs to exist, in small and big ways, on a regular basis, than having another's needs run over your existence to the point that you eventually get burned out or the point of sending a lot of pent up and legit concerns his way, and he pulls away, and your fears of abandonment get triggered and you try to pull him in more and around and around this will go.
I really think it's time to focus leads on what he needs and a little more on you being you.
I could also totally be off as well, and/or not making a lot of sense! Take anything that's helpful and please disregard the rest. I hope you and him do connect soon, in a way where you can both show up fully in the relationship and get your needs met.