On a plane flight back from a recent trip, I watched the movie 21. The plot, advertised as “based on a true story”, is roughly as follows. (In case you have not seen the movie and would like to, I will try to avoid major spoilers.)

Ben Campbell, an idealistic and somewhat naive MIT student, impresses a math professor (Kevin Spacey) by answering correctly a couple of tricky questions in class. Soon afterwards, the professor asks Ben to join his card-counting blackjack team in return for a share of the profits. The team travels to Las Vegas on weekends, plays blackjack at major casinos, and wins millions of dollars by placing themselves strategically at the right tables and employing the card-counting techniques taught by the professor. Ben refuses at first (”and if you tell anybody, I’ll make sure that you won’t graduate”), but there’s no other way that he can pay for his dream med school, and he’s attracted to a female student on the team, and besides, if he didn’t join the team, there would be no movie, so guess what happens.

That’s about the first quarter of the movie, and I’ll leave it there, because this is already enough to raise serious questions about just how close to a “true story” we are here.

My first question was, has there really been an MIT math professor who made a fortune off a team of student card players? Wouldn’t that be serious professional misconduct, and would an MIT professor (not a bad job) really take this sort of risk? As it turns out, the movie is somewhat loosely based on the adventures of a real-life MIT card counting team (one of several that MIT has had over the years). However, the teams were all entirely composed of, and run by, the students. There were no professors involved, and the Spacey character is completely fictional.

Which also preempted my follow-up question. There’s a classroom scene where the Spacey character asks his students what applications of Newton’s method they know. A student suggests, “Nonlinear equations?…” Spacey responds along the lines of “Yeah, that’s very clever, because this course is called Nonlinear Equations. Why don’t you tell me something I don’t already know.” My impression was that Spacey’s demeanor, and this exchange in particular, was a little bit too snarky. A professor is not supposed to do that if he (or, especially, she) wants to get good student evaluations.

The user comments on IMDB include several reviews by authors who appear to be well familiar with casinos, blackjack and card counting, and are not entirely happy with the treatment of the subject. I’ve never played blackjack, or been to a casino, but their criticism makes sense to me.

But here’s the reason why I’m writing this post. How exactly does Ben manage to impress Spacey’s character? Spacey asks him the following question:

You’re on a game show. The host asks you to choose one of three doors. Behind one of them there’s a car, behind each of the other two there’s a goat. You pick one door, say Door 1. The host, who knows where the car is and is not allowed to reveal that information, opens Door 3, behind which there is a goat. He then asks you if you want to choose Door 2 instead of 1. Is it to your advantage to switch?

In case you’d like to think about it, the rest of post is behind the cut.

Yes, you should switch to Door 2. Why? The short version is as follows. Initially, each door is the winning one with probability 1/3. If you stay with Door 1, you win if and only if your initial choice was the winning door, an event with probability 1/3. If you switch to Door 2, you win if and only if your initial choice was not the winning door, an event with probability 2/3. Therefore you are more likely to win if you switch. That’s the answer Ben gave (though I didn’t think that either he or Spacey explained it very clearly).

More details here, including a discussion of just how the details of the set-up can make all the difference. Enjoy!

More at I Can Has Cheezburger?

You might have heard of the recent study in Science that compared the performance of boys and girls on school math tests and concluded that there were no noticeable differences. The study generated plenty of headlines along the lines of “girls are as good at math as boys”. Inevitably, the ghost of Lawrence Summers’s notorious remarks on the cause of underrepresentation of women in science was summoned and exorcised. Heather Mac Donald at the City Journal takes issue with this, noticing that the study found twice as many boys as girls above the 99th percentile in 11th grade (hat tip to 3 Quarks Daily):

On the contrary, Science’s analysis of math test scores only confirms the hypothesis that cost Summers his Harvard post: that boys are found more often than girls at the outer reaches of the bell curve of abstract reasoning ability. If you’re hoping to land a job in Harvard’s math department, you’d better not show up with average math scores; in fact, you’d better present scores at the absolute top of the range.

Actually, hiring decisions at Harvard and other math departments are not based on “math scores”, but rather on proven research ability (and other factors that we won’t go into right now). Math professors don’t spend their days solving test problems. We engage in research, a complicated, messy creative activity where you can’t check the answer at the back of the textbook and your work is not graded on a scale from 0 to 100.

A successful career in research depends on many things, not just the raw problem-solving ability. There’s drive, motivation and independence. Creativity. Communication skills. Short-term and long-term strategic planning of a research program where, just like on the stock market, possible benefit is always associated with risk. A good taste - we’re often guided by the beauty and elegance of our theorems as much as by their utility. Entrepreneurial skills: soliciting funds, organizing research activities, managing a group of graduate students and junior personnel. Then there are less obvious but nonetheless important factors, such as Dr. James Austin’s “Chances I- IV” (the quote and further explanation can be found on Marc Andreessen’s blog).

[You] have to look carefully to find Chance IV for three reasons.

The first is that when it operates directly, it unfolds in an elliptical, unorthodox manner.

The second is that it often works indirectly.

The third is that some problems it may help solve are uncommonly difficult to understand because they have gone through a process of selection.

We must bear in mind that, by the time Chance IV finally occurs, the easy, more accessible problems will already have been solved earlier by conventional actions, conventional logic, or by the operations of the other forms of chance. What remains late in the game, then, is a tough core of complex, resistant problems. Such problems yield to none but an unusual approach…

[Chance IV involves] a kind of discrete behavioral performance focused in a highly specific manner. [...]

Chance IV favors those with distinctive, if not eccentric hobbies, personal lifestyles, and motor behaviors.

We were talking about, what, exactly? High school standardized tests vs. aptitude for research? Yeah. Right.

Sure, a low SAT score in math suggests that you might not be a professional mathematician in the making. But I don’t see much difference, in terms of research potential, between a student who scored 99% on a high school test and one who scored “only” 95%. The former may have spent more time specifically preparing for the exam, the latter may have had a headache. So what? Both have met high enough standards to merit further consideration. I couldn’t find the full text of the Science article online, but this article reports that girls and boys were almost equally represented in the top 5%.

Which still does not either prove or disprove that girls and boys have the same potential for research in math or science. Or that we should be instituting hiring quota based on the gender breakdown among the top 5% of SAT scores. My point is, the Science study does disprove the popular widesweeping assertions about how girls are generally not as good at math as boys - but the only reliable way to determine your aptitude for research is by doing research (and I suppose that the same is true of other high-end careers in science).

Nor does it say anything about whether women in science departments face gender discrimination. The question there is: if we only consider the small (for whatever reasons) number of women with a proven research capacity who are pursuing an academic career, are those women treated, respected and promoted the same way as their male counterparts? You could bring that up with those undergraduates who insist on calling me “Miss”, as opposed to my male colleagues who are addressed as “Professor” or “Doctor”. You could read my earlier post on the subject. You could read this blog, especially entries like this one here.

It’s not a comfortable subject to talk about. One would have to mention specific people and incidents - people who are often adamant that they do not discriminate against anybody and may well be offended by the suggestion that, in fact, they do. It’s easier to discuss anonymous statistical data, to get into a pointless argument about what SAT scores imply about academic hiring. That, unfortunately, is not going to solve the problem.

The Dynamical Systems school in Bedlewo is coming to an end. Today we took a trip to the Kórnik castle.

The Kórnik castle.

According to our tour guide, the town of Kórnik used to be called Kurnik, the Polish word for “henhouse”. In the 19th century, this part of Poland was occupied by Prussia. There was concern that the town might be renamed Hühnerhaus, the German word for “henhouse”. To prevent that, the residents decided to change the spelling to Kórnik. This is pronounced the same way as Kurnik (well, there is a slight difference if you pay very close attention), but it no longer derives from anything hen-related, nor does it mean anything else in Polish, so it couldn’t be translated into German. The town kept its name throughout the Prussian, then German, occupation.

The bridge over the castle moat.

A little mystery castle on Lake Góreckie

More photos under the cut.

Kórnik castle: the sitting room

Fine woodwork in Kórnik

Once were warriors...

An archeological site on Ostrów Lednicki

The Gniezno cathedral tower, from up close.

How the tower clock works.

The wild boar feast.

A few weeks ago, the Brookings Institution published a report on a study investigating why there are so few women in public offices. The study was conducted by Jennifer L. Lawless and Richard L. Fox in early 2008; I found out about it via articles in The Washington Post and Los Angeles Times. (That was several weeks ago. I would have blogged it then if I’d had the time.)

In this report, we argue that the fundamental reason for women’s under-representation is that they do not run for office. There is a substantial gender gap in political ambition; men tend to have it, and women don’t.

That’s a little bit like saying that I don’t play the lottery because I would not want to win a million dollars.

I’ve never run for public office or thought seriously of doing so, but I’m familiar enough with the glass ceiling in science, including the actual research and the administration of science. Based on that experience, I can say confidently enough that a woman who doesn’t run for something or other is not necessarily lacking in ambition. She may very well have other reasons, such as a realistic and sober assessment of the level of support that her candidacy will attract, the expected resistance she will face if elected, or the cost to her professional and personal life.

Politics is the art of the possible. It’s also the art of knowing what is possible, what is not, and what is not worth its price.

On to the details. The study surveyed two roughly equal-sized groups, male and female, of professionals in occupations that often yield political candidates (law, business, politics, education). The factual findings are valuable, if not too surprising. Women are less likely to consider running and to take concrete steps in that direction. They are more easily deterred by the prospect of campaining. They are less likely to be recruited to run and to believe that they are qualified for the position.

But the conclusions are sometimes a little bit off.

In terms of fundraising and vote totals, the consensus among researchers is the complete absence of overt gender bias. [...] In other words, when women run for office - regardless of the position they seek - they are just as likely as their male counterparts to win their races.

Given that fewer women run for office to begin with, it might be possible that those who do run are preselected more carefully than their male counterparts in terms of qualifications, political talent, ambition and determination. Urban legend has it that women must be more skilled and work harder than men in order to achieve equal success.

Is the urban legend correct? The study reports that women are statistically less likely to “feel qualified”, but that their credentials are comparable to their male conterparts. The second claim is based on responses to several “yes or no” questions concerning relevant experience (has conducted policy research, has organized events, etc). I would point out here that these activities are all pretty common among this group of professionals; what’s more relevant is the actual level of excellence and accomplishment. We learn, for instance, that 65% of women and 69% of men in the study engage in regular public speaking. Fine, but does a woman have to be more skilled and accomplished (in terms of types of engagements, generating enthusiasm in the audience, etc) in order to feel qualified to run? Does she have to be more skilled to be elected? We don’t get an answer to that.

Despite the general disclination toward the mechanics and personal trade-offs involved in running for office, women are significantly more likely than men to allow their negative feelings toward the various aspects of a campaign to prevent them from entering the electoral arena.

It’s also possible that women have more vitriol thrown at them during the campaign. We all know how Hillary Clinton was the media’s darling. Of course, most women aren’t Hillary Clinton. The study says that women “are more likely [...] to determine that they lack the thick skin required to succeed in politics”. How much actual hostility - if any - does a woman face when she runs for the city council? District attorney? State governor? How does it compare to what male candidates face? I’d like to know.

Then, unfortunately, there is this:

As we mentioned at the outset of the report, when women run for office, they are just as likely as men to win their races. The lack of gender bias in fundraising receipts and election outcomes, however, is only as good as the dissemination of the message. That is, if women think the system is biased against them, then the empirical reality of a playing field on which women can succeed is almost meaningless.

In other words: if someone could just explain to women that there is no glass ceiling, the glass ceiling would cease to exist. It’s all in your head, darling.

I would suggest that the data collected by Lawless and Fox points to a different conclusion. Specifically, the study shows that women are less likely to be encouraged to run for public office, both by political gatekeepers and by their co-workers, family and friends. I see it as evidence of a statistically significant bias: the community is less likely to see women as suitable candidates. We are not just making it up.

But if a woman is qualified and wants to take a shot at it, she can just put forward her own candidacy, right? Well, the study emphasizes the importance of encouragement and finds that those who have received it are twice as likely to run. There is a very simple explanation for this. If you’ve been recruited or encouraged to run for something, it’s a pretty reliable indicator that your candidacy will be taken seriously. Conversely, the same people who wouldn’t have suggested a woman as a viable candidate might also be less than eager to support her campaign, vote for her, or accept her as a leader if she gets elected.

In my own experience, women are rather more likely to be discouraged from being too ambitious. Just keep doing your own work, honey, don’t get out of line, and we will all be happier. You think that something or other should be done differently, or perhaps there’s an agenda you want to advance? Doesn’t mean that you should be in charge of doing that. Go talk to the man who’s currently holding the office. Perhaps he’ll let you do the work for him. You insist that you want to run yourself? It’s unbecoming to admit your ambition so openly. Politicians always run for office out of the selfless desire to work for the greater common good. You think that Mr. X is a little bit less qualified? That’s so not collegial. You should learn to work with people before you try running for something.

The bottom line is: it would be nice if ambitious women really faced a level playing field, but the study has not convinced me of that and has in fact provided some evidence to the contrary.

The results of the 2008 Discovery Grants competition are now available: GSC 336, GSC 337. This is clearly an improvement over last year’s situation, when the budget for the mathematics GSCs was cut quite severely. I don’t have the time to do detailed statistics as I did in an earlier post on the subject, but for a quick comparison, let’s look at the top tier of GSC 336 grants. In 2007, the three highest awarded amounts were 57K, 39K and 35K; all other grants were valued at 32K or less. In 2008, the top three grants are 52K, 48K, 44K, and five more are valued at 42K each. No, it does not mean that all the best researchers applied this year. It means that last year there was much less money available, as I’m hearing from sources close to the bean counter.

I would hope that the 2007 GSC budget was a one-time screw-up and that the GSC budget will stay around the 2008 level in the future. There are, however, a couple of additional considerations that unfortunately have to be brought up.

First, in this less than perfect world, the quality of our research is often judged by the size of our grants. It takes time and expertise to read the actual papers or to make merit-based comparisons between people working in different fields. You only need a few seconds, though, to look up someone’s most recent Discovery Grant. If Professor A gets 30K per year and Professor B gets 40K, then B must be better than A, right? Well, that’s one possibility, but another is that A got his grant in 2007 and B in 2008. But, honestly, how many people beyond those affected directly will even be aware of the cuts? Or remember in which year that happened? The bias is there and can affect A’s ability (at least compared to his counterpart B) to attract further funding, recruit graduate students and postdocs, win awards, and so on.

Second, and related: upon renewal, most researchers receive grants within 10% of their previous amount. The report of the Discovery Grants International Review Committee has detailed statistics on that and suggests that the researcher’s funding history influences the judgement of the GSC members. To continue with the above example - this is how A’s temporary problem can become permanent.

To address the inertia problem (which seems to be quite widespread and by no means limited to situations such as the one just described), the committee suggests a change to the NSERC procedures. Instead of ranking proposals and allocating the funds at the same time, as is currently done, GSCs should first rank the proposals based on merit alone, without any reference to current or new level of funding, and only allocate the funds after the ranking process is complete.

As it happens, a similar recommendation was made independently by the Grants Selection Committee structure review committee. Their report is now posted on the NSERC web page - you’ll find it if you follow the link. NSERC is currently working on implementing the suggestions of both committees. At the NSERC information session at the Second Canada-France Congress in Montreal about a month ago, we were given a rough sketch of how NSERC proposes to do this.

I have not yet had the time to read the report in detail, or to think much about the subject, so I’ll come back to it later when I have something intelligent to say. As for the inertia problem, I’m not convinced that the above suggestion will resolve it completely. The GSC members will still be able to look up people’s previous grants and make judgements based on that. People are human, you know. Then again, I’ll be interested to read the committee’s arguments when I have the time.

… it’s time for some summer music.

This band is called Nouvelle Vague. What they do is cover versions of 1980s new wave songs, bossanova-style. The first time I heard them was on a trip to Europe two years ago. I arrived at my destination late in the afternoon, cranky and tired, after 3 plane flights and over 20 hours of travel. The time difference between there and Vancouver was 9 hours. I hardly got any sleep that night and was wide awake again by 6:30 am. That’s when I heard this song on the radio. Quite unbelievably, it put me in a good mood for the rest of the day.

It’s my favourite Nouvelle Vague song, by far. The lead singer here is Gerald Toto, who also records solo and with various other groups. For an encore, here’s Gerald Toto again, this time in French:

Let be a set of real numbers. We will write and . What can we say about the minimum size of or ?

It’s easy to prove that , and the equality holds if and only if is an arithmetic progression. Similarly, , and the equality holds if and only if is a geometric progression. On the one hand, both of the lower bounds above are sharp; but on the other hand, the sets minimizing and look quite different. In fact, if is an arithmetic progression, then the product set is rather large, with . Conversely, if is a geometric progression, then must be large.

Erdös and Szemerédi conjectured that at least one of and must always be large. Specifically, it is expected that for any we must have

The reason for this post is that, just very recently, Jozsef Solymosi proved the estimate

improving the previously known bounds, the most recent of which was also due to himself. (Earlier results were due to Erdös, Szemerédi, Nathanson, Ford, and Elekes.)

You might or might not call Solymosi’s proof “simple”, depending on how you define that word. The argument, as it stands, is quite elementary and does not use anything more sophisticated than the Cauchy-Schwartz inequality. That doesn’t mean, though, that you or I could have easily found it. Many of the best combinatorialists around have thought about this particular problem and were not able to find an improvement.

Jozsef explains the main idea as follows. Let , , . Assume also that , and that the set also has cardinality . This assumption is, just like Jozsef says, “unjustified and usually false”; however, if we accept the notion that is small if and only if has “multiplicative structure” (e.g. a geometric progression), it shouldn’t come as a surprise that the sizes of and might not be completely unrelated. There are in fact rigorous results of this type, but they are not used in Jozsef’s paper.

We will further assume that every element of occurs the same number of times: for every , there are exactly pairs such that . This again is usually not true, but, at the cost of losing a log factor, we can reduce to this case by pigeonholing.

Now, let , where the are arranged in increasing order. For each , consider the line in . By the above assumption, each such line contains points of . For every two consecutive lines , the points are all distinct, and there are of them. Moreover, all these points lie in the segment of enclosed by the two lines; hence different pairs of consecutive lines generate disjoint sets of points. That’s at least points total.

On the other hand, all these points lie in the set , the cardinality of which is exactly . It follows that , hence and at least one of must be greater than or equal to , as claimed.

Wasn’t that nice?

In addition to a rigorour version of the above argument, Jozsef also has a variant covering the case of , where are different sets, and a result on iterated sumsets of sets whose product sets are very small. If you’re interested, you should check his paper if you have not done so already.

Or, if you’re Dennis Lehane, you might call it “the hamster wheel.”

In an age when reading for pleasure is declining, book publishers increasingly are counting on their biggest moneymaking writers to crank out books at a rate of at least one a year, right on schedule, and sometimes faster than that.

Many top-selling writers, such as John Grisham and Mary Higgins Clark, have turned out at least one book annually for years. Now some writers are beginning to grumble about the pressure, and some are refusing to comply.

Me, I’ve given up on several authors because their work felt, well, forced. Patricia Cornwell’s Scarpetta series, mentioned in the article, started out with several tight and highly enjoyable forensic thrillers, but then evolved into a combination of soap opera and introspection. Introspection, in this context, means using about 100 pages to say the equivalent of “it’s scary when a serial killer is after you.” But in the end it was the soap opera part - a familiar character confesses that she’s actually a millionaire, another character gets resurrected all of a sudden - that turned me off. Now, Cornwell is certainly not the worst of the gang. I’ve singled her out because she used to be one of my favourite writers, and perhaps she would still be if she had taken the time to get off the hamster wheel and think about what she really wanted to write.

A writer can “keep her face out there” by writing a book every year, or she can do it by writing books that the readers won’t forget. If she can do both, that’s great, but if not, I’m more likely to buy her books if she does the latter.