Would it help to start with plants, herbs and flowers that have a spiritual meaning? For example, Rosemary has subtle small purple flowers, and has long been considered to have protective qualities. I find that when I plant flowers for their specific spiritual qualities, that meaning overrides pretty much anything, and they're a constant reminder of the spiritual quality that I was trying to invoke. Even flower colours can be important - white flowers typically help cleanse and purify, pink flowers assist with friendship (including making friends with ourself).
Picking flowers that have a spiritual quality that you don't especially link to your trauma or abuser could be a soothing way to get into plants. If you have particular flowers that you do kinda like, do a bit of internet searchkng to see if they have a spiritual quality that you can associate with them to give them an even stronger positive quality when you're around them.
Some are very pungent - lavender brings calm. But some aren't, like a small buxus hedge for self-empowerment.