The 99th percentile

You might have heard of the recent study in Science that compared the performance of boys and girls on school math tests and concluded that there were no noticeable differences. The study generated plenty of headlines along the lines of “girls are as good at math as boys”. Inevitably, the ghost of Lawrence Summers’s notorious remarks on the cause of underrepresentation of women in science was summoned and exorcised. Heather Mac Donald at the City Journal takes issue with this, noticing that the study found twice as many boys as girls above the 99th percentile in 11th grade (hat tip to 3 Quarks Daily):

On the contrary, Science’s analysis of math test scores only confirms the hypothesis that cost Summers his Harvard post: that boys are found more often than girls at the outer reaches of the bell curve of abstract reasoning ability. If you’re hoping to land a job in Harvard’s math department, you’d better not show up with average math scores; in fact, you’d better present scores at the absolute top of the range.

Actually, hiring decisions at Harvard and other math departments are not based on “math scores”, but rather on proven research ability (and other factors that we won’t go into right now). Math professors don’t spend their days solving test problems. We engage in research, a complicated, messy creative activity where you can’t check the answer at the back of the textbook and your work is not graded on a scale from 0 to 100.

A successful career in research depends on many things, not just the raw problem-solving ability. There’s drive, motivation and independence. Creativity. Communication skills. Short-term and long-term strategic planning of a research program where, just like on the stock market, possible benefit is always associated with risk. A good taste – we’re often guided by the beauty and elegance of our theorems as much as by their utility. Entrepreneurial skills: soliciting funds, organizing research activities, managing a group of graduate students and junior personnel. Then there are less obvious but nonetheless important factors, such as Dr. James Austin’s “Chances I- IV” (the quote and further explanation can be found on Marc Andreessen’s blog).

[You] have to look carefully to find Chance IV for three reasons.

The first is that when it operates directly, it unfolds in an elliptical, unorthodox manner.

The second is that it often works indirectly.

The third is that some problems it may help solve are uncommonly difficult to understand because they have gone through a process of selection.

We must bear in mind that, by the time Chance IV finally occurs, the easy, more accessible problems will already have been solved earlier by conventional actions, conventional logic, or by the operations of the other forms of chance. What remains late in the game, then, is a tough core of complex, resistant problems. Such problems yield to none but an unusual approach…

[Chance IV involves] a kind of discrete behavioral performance focused in a highly specific manner. […]

Chance IV favors those with distinctive, if not eccentric hobbies, personal lifestyles, and motor behaviors.

We were talking about, what, exactly? High school standardized tests vs. aptitude for research? Yeah. Right.

Sure, a low SAT score in math suggests that you might not be a professional mathematician in the making. But I don’t see much difference, in terms of research potential, between a student who scored 99% on a high school test and one who scored “only” 95%. The former may have spent more time specifically preparing for the exam, the latter may have had a headache. So what? Both have met high enough standards to merit further consideration. I couldn’t find the full text of the Science article online, but this article reports that girls and boys were almost equally represented in the top 5%.

Which still does not either prove or disprove that girls and boys have the same potential for research in math or science. Or that we should be instituting hiring quota based on the gender breakdown among the top 5% of SAT scores. My point is, the Science study does disprove the popular widesweeping assertions about how girls are generally not as good at math as boys – but the only reliable way to determine your aptitude for research is by doing research (and I suppose that the same is true of other high-end careers in science).

Nor does it say anything about whether women in science departments face gender discrimination. The question there is: if we only consider the small (for whatever reasons) number of women with a proven research capacity who are pursuing an academic career, are those women treated, respected and promoted the same way as their male counterparts? You could bring that up with those undergraduates who insist on calling me “Miss”, as opposed to my male colleagues who are addressed as “Professor” or “Doctor”. You could read my earlier post on the subject. You could read this blog, especially entries like this one here.

It’s not a comfortable subject to talk about. One would have to mention specific people and incidents – people who are often adamant that they do not discriminate against anybody and may well be offended by the suggestion that, in fact, they do. It’s easier to discuss anonymous statistical data, to get into a pointless argument about what SAT scores imply about academic hiring. That, unfortunately, is not going to solve the problem.

Author: Izabella Laba

Mathematics professor at UBC. My opinions are, obviously, my own.

17 thoughts on “The 99th percentile”

  1. The big point is that the difference is decreasing. If there is an inherent difference, shouldn’t it be fairly constant?
    The burden of proof should be on those who believe in an inherent difference. Especially considering that there are studies that show there is obvious sexism out there (women musicians are 50% more likely to be chosen if they audition behind a curtain; when people are given the same resume with just the first name changed, the male resume does better; …).

  2. This is quite a complicated problem. I hope Professor Laba could write more articles on this issue.

  3. There is a point that is not made often enough, in my opinion. Why does it matter if members of a given large group have more potential for mathematics, whatever that means, than members of another large group? After all, we should accept a person into a graduate program, or hire them into a position, based on their individual qualifications, not based on group membership.

    Suppose that a study comes out tomorrow showing conclusively that people with brown eyes are typically much better in mathematics than people with blue eyes. So what?! If a blue-eyed person applies for a job afterwards, one would hope that they will not be dismissed without a thorough examination of their record.

  4. It does matter for political reasons.

    Whilst it is absolutely correct that when making individual decisions which large group you belong to (ideally) is irrelevant, when — as in the UK — governments look at what proportion of an academic department belongs to each large group when making funding decisions it matters what proportion you would expect if each of those decisions is made fairly.

    If it were true that people with brown eyes were typically better at (research) mathematics than people with blue eyes then it would be outrageous for funding for a maths department to be dependent on the proportion of those in the department with brown/blue eyes to be consistent with that in the general population.

    It seems to me that this is a category-mistake made too often. If someone says tall people are generally cleverer than short people then this doesn’t mean they are saying that all tall people are clever and all short people are stupid and there is no reason for people in either group to be offended by the suggestion. It doesn’t even mean that the cleverest person in the world is tall. It just means that if you select on intelligence you would expect to find more tall people than short people in the group.

  5. Some of my bad experience in academia I could easily explain with sexism – except that I am a man. Do you really think that male professors are more respected by their undergraduates? In my experience there is no correlation between respect and the gender. (Of course your undergraduates will notice your female body, but this is totally caused by the human nature. If you want to change this, you must literally kill them.)

    There are lots of socialist professors, although every socialist experiment let to poverty. There are lots of professors whom believe in man made global warming, ever though reality proofs each and every climate model wrong. And not surprisingly there are a few sexist professors, notwithstanding that political correctness has not yet established a quota.

    The main fact for the gender gap is that man have a higher variability than women.

    fredtopeka: “The big point is that the difference is decreasing.”

    That is wrong. There were just no tough math in these test. One could even create a math test in which the girls are better than the boys. But at research level the variability will always cause its effect.

  6. You could bring that up with those undergraduates who insist on calling me “Miss”, as opposed to my male colleagues who are addressed as “Professor” or “Doctor”.

    When I call someone by the formal title, sometimes it is just a hidden insult. Everyone knows the title anyway. To emphasize the title makes sense when that person does not perform to the expected standard.

    I know a female professor, whom the students refer to by her forename. And this professor is highly respected.

    On the other hand a lot of male professors are not at all respected by their undergraduates. But this disrespect is normally not stated publicly.

  7. Max:

    I really hope that students aren’t just being ironic whenever they act polite. That would be just too depressing to contemplate. 🙂

    Truth be told, I’m not that big on formal titles. Reasonable and polite conversations can be had without using any titles at all. I’m usually on a first-name basis with people I work with, including graduate students and some undergraduates.

    On the other hand, if you are going to use a formal title, it’s a good idea to get the right one. When you’re addressing a faculty member at a university, the generally accepted norm is “Professor” or “Dr.” Anything else, such as “Miss” or “Mister”, stands out as not normally used in this context. As far as I can tell, women professors are called “Miss” much more often than male professors are called “Mister”. We’re not happy about that.

    Yes, in my experience students do treat female and male professors differently. Not all students, of course, and not to the same extent, but nonetheless. I won’t give specific examples here, because I’m writing this blog under my own name and any instructors or students I might write about could be easily identified, but there’s a whole discussion about this over here. See in particular this comment – I could not agree more. You might also want to read this article on gender bias in teaching evaluations, and here are some additional references.

  8. Here are a couple bits from a Slate article. The first shows that outright sexism can have a big impact:

    “M.A. Paludi and W.D. Bauer conducted a study in which 180 men and 180 women were asked to grade a paper on a five-point scale. When the author was “John T. McKay” rather than “Joan T. McKay,” the men on average graded the paper a point higher—and the women scoring the test weren’t much more egalitarian.”

    The second notes that math variability is not greater for men in all countries and this might imply it’s not inherent:

    “Strikingly, a new analysis of math data from 22 countries (not yet published but presented at several conferences) finds men with the expected spread in scores in many countries—but not in Lithuania, Germany, the Netherlands, Slovenia, or Denmark. In these places, female variability is either greater, or there’s little difference between the sexes.”


  9. Call me weird, but I would like to see more women in math because I enjoy the company of women at least as much as I enjoy the company of men. And I certainly do not like being in some place (i.e. our department) where four out of five people are men — that’s what I call weird. Incidentally, I don’t have the same feelings about eye color (actually I go for green eyes but that doesn’t seem to be an option for Professor Iosevich).

    P.S. Why does anyone read “absolute truths” into these studies that reflect where people’s attitudes/ confidence etc are today. What does it really mean (for the long term) if young women don’t do as well on these tests at the moment? Evidently more and more young women are doing better on these tests as society changes (and young women believe they can and should excel in this direction) — it is our job to encourage and to hire them, to make math a more balanced profession!

  10. I have thought about this fact a lot, and I can never seem to reach a “real” explanation that actually makes sense to me.

    Why are there so many more men then women in mathematics and physics?

    It bothers me a lot as it almost pushes forward some modern stereotypes. But I can never find a nice solid reasoning for why; Is it social reasons? Is it actually harder for women to succeed in this fields because of how it’s set up? Are men more attracted to math? (I really have no idea!?)
    Please tell me what you think??

  11. Anonymous grasshopper, you have still lot a learn. You are experiencing what we call a cognitive bias. One answer to your problem is to claim that there are no objective truths.

    Cheers, Panonymous

  12. “But I don’t see much difference, in terms of research potential, between a student who scored 99% on a high school test and one who scored “only” 95%.”


    I disagree with this paragraph.

    Terence Tao was not exactly “only” top 95% at HS. Last 3 of last 6 Fields medalists combined 4 gold, 3 silver, and 1 bronze at the IMO, including 3 perfect scores. The other 3 did not participate; I’m doubtful, that they were at only 95% of their HS peers.

    In my opinion, 95th percentile is too low for a math researcher.

    The headache argument seems fallacious, I think — if a person were really sick, perhaps they scored top 80%. Does that mean top 80% has the same potential as top 99%? Furthermore, you are discussing statistics for girls and boys later in the paragraph; the headaches would presumably average out…

    Interesting article, though.

  13. davidek: In that case, it’s not clear that I should be a math researcher.

    Terry Tao was not only talented, but also had quite favourable conditions in which these talents could develop. He told me for example that his parents would drive him to different schools around town for different “gifted” classes when he was growing up. My parents didn’t have a car until I graduated from university, and if they’d had one, there weren’t any places in particular where they could have taken me, nor would they have had the time to do it. I was “accelerated” (by 4 years), but that was all that anyone could have done for me. I did not always have perfect grades. I came a couple of points short of a bronze medal at the IMO. I wasn’t ranked in the top 1% in college. I do know that I have since done better than many of my HS and college peers who might have been ranked ahead of me back then.

    And, you know, at least I did have a decent school that I could attend without undue hardship, and nobody in my family has ever told me that math isn’t for girls. So, I was luckier than many.

  14. Izabella,

    [Apologies for misspelling of your name earlier.]

    [Also, this is a bit tangential to the main scope of the article.]

    I am confused why you say: “In that case, it’s not clear that I should be a math researcher.”

    1) You went to IMO, I think that is much, much better than 95th percentile (in math). In most countries, that is better than top 0.01%.

    2) You were “accelerated by 4 years” — does that mean you skipped 4 years of school? If so, again, way above 95th percentile.

    3) You might not have been ranked top 1% in college (though, I suspect that you were top 1% in math), however, presumably the students at your college were not a representative sample of the general population — top 1% in college might translate to, say, top 0.1% in general population (depending on the college).

    I agree, there is a caveat that I did not think of — the “95th percentile is too low” does not apply to people who came back from underprivileged background (or were somehow strongly discouraged from doing math).

    Sadly, giving equal opportunity in college might be already too late; for example, if one believes in Malcolm Gladwell thesis on 10,000 hours* described in his book Outliers.

    *Basically, in order to be a “world expert” in something, you need to spend 10,000 hours doing it (regardless of your exceptional your talent might be). And, in most cases, that means you need to spend (most of) that time in HS…

  15. I agree, there is a caveat that I did not think of — the “95th percentile is too low” does not apply to people who came back from underprivileged background (or were somehow strongly discouraged from doing math).

    You’re saying this as if the “underprivileged” people were rare special cases in which an exception could be made. In reality, it’s a substantial percentage of the population, and the numbers increase further if you look beyond the most developed countries. Between the underprivileged, the discouraged (as you say), those who had other interests back in high school, and so on, I think that there are good reasons to look beyond the top 1% according to SAT. And yes, girls are certainly more likely to be discouraged.

    In some hypothetical ranking of “overall performance”, I’m pretty sure that I would have placed well within the top 1%, at least in math. That’s different from raw test scores. You shouldn’t think that I’m the only person out there with a story to tell.

    Edited to add: no, I don’t particularly believe in Gladwell’s theories.

  16. “. But I don’t see much difference, in terms of research potential, between a student who scored 99% on a high school test and one who scored “only” 95%. ”

    Maybe all the meaning in that sentence is driven by the term “research potential,” but let’s not minimize the difference between 95th percentile and 99th percentile — that’s a full standard deviation. It’s the same as the difference between the 50th percentile and the 84th percentile. Do you really think that a full standard deviation in ability isn’t meaningful?

  17. Yes, I was indeed talking about research potential. If you would like to make the case that students who regularly get 99% on school tests perform better on school tests than students who regularly get 95% on school tests, then I have no argument with that.

Comments are closed.

%d bloggers like this: