I didn't say you were racist. I said that you were not neutral, and that there is no way to be neutral. What individuals do has little effect on structural inequality unless what they are doing is dismantling those structures. You personally not behaving in racist ways or saying racist things has almost no impact on say, whether or not black people still are discriminated against in getting home loans.
You might say that if everyone in the loan industry just individually started behaving better it would change that, but the problem is, most of what is happening is taking place in their subconscious. They mostly don't even realize that they find black candidates for loans less qualified than white candidates with similar backgrounds for racist reasons. They just know that they don't feel like as good a risk - because there's implicit bias going on, which is what the vast majority of racism (and other oppression is), not people walking around burning crosses.
Cognitive science says that only about 2% of thought is conscious. The rest is cultural narratives, stereotypes, old experiences of ours, things we were taught as children, etc. That's what runs most people, not their conscious choices to treat people fairly or not.