I have a really terrible memory, and it never takes me by surprise any more if it comes out in conversation with someone that I don't remember something we talked about previously. And it's not just when I'm stressed, it's a daily occurrence. My head spends too much time being focused on other things.
There's been a couple of times in the past where this problem has caused major major problems with a person. They accuse me of forgetting something critical, or saying something hurtful. And what's my defence? It makes me really vulnerable to that sort of manipulation, because the thing about bad memory is you don't know what it is that you've forgotten. You don't remember what it is that you don't remember.
These days, if I have an important or emotional conversation with someone, I make a point of jotting down the key points soon afterwards, in my journal, my desk diary, hell even on a napkin. The process of making some notes is usually enough to solidify the memory, but it also means that if the issue comes up again, I can feel confident when I say, "No, that's not how it happened", because I'm the one that made a few notes.
People get used to me having a terrible memory really quickly, and I know that even when it's not malicious, some people try and use that to their advantage from time to time. And if it wasn't an important conversation the first time round then, pfft, whatever, let them have their fun. But if it was important, no no no, I've got notes to back myself up, so don't try those games with me on this one.
One day, when my mental health improves, I know that my memory will improve. In the meantime, I adapt the way I operate so that it doesn't cause me too many problems. Some people will take advantage of any vulnerability they can.