My perspective and experience is entirely different. My husband and I would rather have a threesome than any other kind of sex. We like both women and men as our thirds and even though he is straight and not “doing” the other guy, he still really enjoys how hot it makes me and just how fun and sexy it is in general. We’ve been together for a long, long time and although we are still hot for each other and very in love, it’s easy to get in long held lanes and roles. Nothing brings us more fully into the present than another partner in the mix and so even though we often play with the same partners over and over again, there is an element of newness that they bring. The fact that we know them well and also care about them actually enhances the experience.
It sounds like you’ve got some possessiveness going on and although that isn’t inherently wrong (it’s very culturally reinforced) it will tend to diminish your pleasure in this kind of situation, particularly if you are doing it with someone you are romantically involved with already. It need not be said that when it comes to sex, folks ought to do what works for them and feels good for them, but I also think that people’s unexamined stories and hang-ups can often detract from situations that they might actually enjoy if they worked with those a bit.