I have the Swiss Cheese brain when everything is "normal", i.e. when there is little (!) work, when there are no (!) problems and life just is. If I can spend that time with things I like, I'm fine. If I'm supposed to still understand what people tell me, well, good luck. :D
On the other hand, when there is a work load that makes all the so-called normal people go wild or scream in anger because it's too much (has happened various times), I thrive (and so does my brain). It's like that high a stress level is my own normal. I really think that's because I am just so effing used to that incredibly high stress level so when it's lower, I can't get my brain to produce what is needed to keep it running smoothly.
My boss (and the former ones) sometimes look at me as if I'm totally stupid (nice, quiet days) and in high-stress jobs or times at work I will just get promoted, offered jobs higher up, etc. Always. Has happened at every single job (but the short-term ones).