|Think about Loose Coupling|
Yes, there is a slippery slope. And it isn't a long one, either.
An better example than the ones that you gave is the following, "Blacks are admitted and then get preferential treatment in universities thanks to affirmative action, so we should not consider their degrees to be as good."
I consider this to be better because virtually any educated American whose eyes are open has seen this happen. Far from every black, not even a majority, but there are plenty of painfully bad examples walking around. By contrast I can't say that I've seen any evidence that the women who are interested in programming are significantly worse than the men who are interested in programming. And my personal experience with people who got degrees overseas has been very good.
So I've admitted that this line of thought can lead to horrible conclusions. Obviously nobody reasonable would want to say them. (At least not in public.) But is it wrong?
My answer is that, on a personal level, it is not wrong. Even the reverend Jesse Jackson has admitted that when he is walking down a dark street, he worries more if he is being followed by a black man than a white man. A glance at US crime statistics strongly suggests that this is a fairly reasonable concern. Trying to deny that this is factually the case may make us feel good, but it is intellectually dishonest.
But on a societal level, it is very bad. It creates and perpetuates a permanent underclass. It leads to a nasty cycle where lack of opportunity creates conditions that reinforce the biases that cause the original lack of opportunity.
A major job of government is to make people act in ways that benefit society as a whole when those people individually would not act that way. For instance I know that my personal financial contribution makes an imperceptable difference to how good the roads I commute on, while keeping my money makes a huge difference to my life. Therefore government makes me contribute, by charging me taxes and then spending it on roads. (For more on the intrinsic problems with supplying public goods I heartily recommend The Logic of Collective Action.)
When you combine these facts it makes perfect sense for government to force people not act on certain prejudices that they may have. Which it does - as evidenced by the laws you mention. (The fact that we need such laws is evidence that the issues are not about to trivially resolve themselves.) And I personally support these laws because I think that they are a good thing.
In short, there are reasonable prejudices that I don't want people acting on. But there are also reasonable prejudices that I don't mind people acting on. Where do I draw the line?
I personally mostly draw the line based on how easy it is for someone you're prejudiced against to fix the issue. If you're black, you aren't changing that (OK, Michael Jackson did...) so I don't want people acting on that prejudice because it locks you into a ghetto. OTOH if you don't know how Perl is supposed to be capitalized, it is trivial for you to change that. Therefore I feel that my stigmatizing you for that fact is not a significant barrier to you.
I say mostly because I also find it very reasonable to be prejudiced against people because of characteristics that directly affect performance. The canonical example is that you don't pick short people to be basketball players. Now perhaps the short person actually is better. But physical height is such a direct factor in your ability to play basketball that it is going to be very hard for you to overcome that handicap.