This is the problem... medication numbs true emotion, which is why at some point you have to go off them, in order to deal with any residual aspects lingering around. Most of it is actually just withdrawal... ie. stop taking meds for a day or two, suddenly your body goes into withdrawal. Maybe you might want to start decreasing the dose and keep it regular vs. dropping it for a day or two at a time, thus sending your body into withdrawal chaos. When I say decrease, I mean slightly... ie. if on 40mg per day, drop it by 5 - 10mg per day only, and keep it that way for a good month or two, so not only you can measure emotions as they arise, but give yourself adequate time to deal with them as the medication begins to slowly evade your system.
I think you are tempting fate though to just drop it for days at a time, as meds just don't work that way or play nice like that. Its not even so much that the emotion you suddenly feel overwhelmed by has been hiding, but more a combination that some of it has been hiding, yet some is being forced by sending your body into withdrawals.
Tapered is the approach... and if quite honestly, you taper over a year, who really cares... just find what works for you, that also gives you time to deal with any residual emotion + give yourself time to sort it out... which will happen quick enough if you have done trauma therapy already... then you should be ok.
The problem though is depending on your lifestyle, ie. whether you can manage your now life without meds, regardless the symptoms from playing russian roulette with withdrawals as you have... that will be another factor to whether you need to remain on them, change for sex drive, or tapering will be sufficient inline with your lifestyle.