The trouble with Larry
Liberals blasted that it was absurd and insulting to even consider innate differences. Conservatives piously wondered what had happened to free speech in academia.
To my mind, there were two key problems with his speech, and neither was the consideration of innate differences.
The first is simply that he ordered his list of potential reasons in precisely the order that left him least culpable for the problem. Quite apart from the merits of the arguments, one ought to be immediately suspicious of an argument that runs (stripped to its fundamentals) "There are many possible reasons for this problem. The most likely by far is that it's not our fault and we can't do anything about it. It's also possible (but much less probable) that it's entirely our fault, for at least seven reasons."
The bigger problem, though, is that his argument assumes as a matter of course that what gets you onto a science faculty is raw, native ability, and that top-notch professors were simply born with vastly more ability than everyone else. Some math and science types like this idea, because it makes them sound a bit like Michael Jordan (despite having no other obvious similarities). Since Mr. Summers in an economist (a field that tends to throw around a lot of math), I suspect he doesn't mind thinking of himself as six standard deviations out from the mean in ability.
The problem is that you already see vastly fewer women in math and science as graduate students or even undergraduates. Take it from me--you don't have to be a math god to be a graduate student in math. You just have to be reasonably good and put a lot of work in.
That's where the "native ability" idea gets pernicious. Math is honestly hard. So's physics. You have to do a lot of work to get good at it. If you think it's all native ability, then there's a temptation not to do the work---after all, if you find it tough, you must not have been born with the "math gene," so why bother?
Did Mr. Summers have any right to make a speech like that? Of course he did! We all have an inalienable right to make fools of ourselves--or to point it out when others have done so.