I've never asserted that it's about white people as human beings. It's about the dominance hierarchy, and in our culture, those at the apex are white. Ergo, in our culture, it's about them. If you would like to refute what I've quoted from a cognitive scientist who has studied this for decades about how our thought processes work, please feel free. But just saying it could be debated without actually doing that doesn't move the ball.
The color of the skin is only a factor as it relates to a specific experience in our culture. As I noted in another comment, about 50% of blacks also have unconscious bias towards whites, and against those who look like them. So deep is the cultural soup that we swim in, that they cannot escape it either - in part because it's reinforced every day - in the media and elsewhere. It might well be different in other cultures. I'm talking about the US.
I don't know the first thing about Critical Theory. What I'm writing about comes out of sociology and cognitive science. If you find demanding basic human rights to be totalitarian, I don't really know what to say to you on that.
Of course unemployment and poverty are important, but racism is a huge element in those problems, and all the ones you named. Those with ethnic or black-sounding names are less likely to be interviewed or given a chance at a job. Despite laws against it, there is still discrimination based on race in getting loans, which helps keep blacks from building wealth via home ownership. It's all a part of the same larger system, and as a white man, you don't really get to decide what is harmful to someone else. That's not for you to say. Never having experienced what it's like to be a marginalized person, you don't get to dictate how that experience should feel and what's important and not important.
And just for grins, what are the "logical inconsistencies" of asking to be treated with basic respect and to not have artifical barriers put in your way and then justified. That seems like something that has logic to it all.