I do this a lot. I'm not sure what the cause or solution is, but you're not alone in it. I think it's probably a combination of things - memory issues, anxiety, finding it hard to get things clear in my head/confusion, distraction/concentration problems...
I kind of work on the theory though that, if it's such a problem for the person I'm talking to, then they can go talk to someone else!
I find it frustrating when I catch myself doing it, and I think I probably do it more than I realise, but I have enough problems talking as it, having someone calling me out on it, or picking at it, wouldn't be helpful. It would just make me even more hesitant to talk. (As it does when I pick up on it myself)
How do you feel when your daughter picks you up on it? What age is she?