I feel like the "odd man out" here being a man. I know with my live-in girlfriend, it was always the understanding that that is the day I get out of the house, rake leaves, go to a friend's place for a beer, go for a drive in the country, whatever. Maybe Anthony should shingle the roof that day, or work in the garage.
Guilt might be your automatic reaction, but the truth is, your man loves you anyway, it's the same for everyone, and it's temporarily not really completely in your control being hormonal in cause. Maybe you need to have a good laugh about it to lighten it up, and lighten up on yourself a little too. You're a sweetheart just the way you are.
Dave