I think it must be part of PTSD. I experience the same thing. It takes longer to come on if I'm with my husband - I can usually manage two or three shops, but then I feel light headed, wobbly and out of it.
I occasionally try a shop by myself to see if I can do it, and then I feel more like the panic you experience.
So now I'm focussing on the things I can do rather than reinforcing failure. For some time I've been able to go out for a walk in the country with my husband and the dog, so I'm adding in taking the dog out by myself. Yesterday I managed to laugh at something he did. ( Although right now I'm on here to avoid going out).
Of course, that approach only works for me because H will do the food shopping. And I don't know if it's the right approach, as I'm still waiting for a T, so I'm making up my own management.