Perhaps we should simply change the terminology and say that a lot of harassment is not about lust or attraction, even though it is about getting off on the power exerted over someone else and so has a sex component. Hotel maids are not agitating for panic buttons in rooms because as a class of people they tend to be so sexually alluring; it’s because as a class they are so vulnerable that they are frequently targeted.
This is a very different thing than the people (mostly male) who want to excuse harassing behavior because a woman was wearing lipstick or high heels at work. “She was so pretty, I just couldn’t help myself. I’m biologically programmed to find that exciting.” Where in fact, if they knew they would suffer negative consequences, they would have no problem maintaining self-control. It’s only when there is a power imbalance and they think they can get away with it, that this type of behavior takes place. That’s way more about power than sex.
Edit: “Sexual assault is a committed based on control and power. Therefore, give the control and power back to the survivor by letting them choose what they would or would not like to do.” This is excerpted from a survivors advocacy program. https://rvap.uiowa.edu/help/supporting-survivors/
SA victim advocates and support programs are the ones leading the charge to bring the power dynamics associated with rape and harassment into greater awareness. Sometimes women being raped have orgasms. That doesn’t mean that they secretly enjoyed the experience. Just because a sexual body part is involved doesn’t indicate that it is primarily a sexual experience.