You are doing what I have envisioned for the future. Kind of stealth. Outed to my family. I want everyone else to continue to think I'm a man. Just be allowed to feel a little more like a woman, first and foremost to my wife, to my children. To open up a bit more and take hormones. A lot of things can then be concealed. I feel very similar. I can't just walk around in women's clothes and feel comfortable in them, because my body will never be that beautiful. That's why I don't want to. But I need allies and I still have to work on that.
So you're not doing anything wrong. Everything is valid and it's nice that it makes you feel better.