Lessons from an Overlooked Mathematical Proof

This story about a retired mathematician proving a notoriously elusive statistical conjecture hits some really interesting themes.

Thomas Royen, a retired statistician who had worked in pharmaceuticals and academia, was brushing his teeth one morning when his brain burped out a method to prove something called Gaussian Correlation Inequality (the article has a very good general audience explanation for what exactly that is). Royen wrote up his proof, sent it to some colleagues, published it in an obscure journal no one reads, and then sat back and watched a whole lot of nothing happen for several years. Finally someone realized that Royen’s proof was in fact valid and constituted a significant achievement in the field. And next thing you know, here he is in Wired Magazine.

Apart from the fact that this happened in the first place, there are a few interesting aspects to this story that are worth dwelling on. The first is how easy it is for important developments in knowledge to get overlooked. Here is how the article describes the publication of Royen’s proof:

Proofs of obscure provenance are sometimes overlooked at first, but usually not for long: A major paper like Royen’s would normally get submitted and published somewhere like the Annals of Statistics, experts said, and then everybody would hear about it. But Royen, not having a career to advance, chose to skip the slow and often demanding peer-review process typical of top journals. He opted instead for quick publication in the Far East Journal of Theoretical Statistics, a periodical based in Allahabad, India, that was largely unknown to experts and which, on its website, rather suspiciously listed Royen as an editor. (He had agreed to join the editorial board the year before.)

With this red flag emblazoned on it, the proof continued to be ignored.

For a guy who just proved an important and intractable math problem, Royen sure seems to have gone out of his way to hide it in plain sight. There’s a wide berth between not wanting to go through the slog of peer review and publishing in an Indian journal with what sounds like single-digit readership. If Royen were twenty or thirty years younger I would wonder why he just didn’t publish his proof on Medium, or Stack Exchange, or even on Quora, which has a pretty strong math community.

But the paper was published. There it is in Volume 48 of the Far East Journal of Theoretical Statistics, right alongside papers on Bayesian Modelling of Growth Retardation among Children, Penalty Spline Estimators, and Simulated Hellinger Disparity Estimation. It’s not exactly click-bait, but it is clearly a serious academic venue.

And the Far East Journal is not entirely unread: its articles have been cited over 700 times since 2012, according to Google Scholar. (Interestingly, Royan’s is only the tenth most cited article in the Far East Journal’s history. The article with the most citations is titled “Parallel computing and Monte Carlo algorithms” by Jeffrey Rosenthal. It was published in 2000, giving it a 14-year head start.)

One way to view this is as a markets problem. The demand for theoretical statistical knowledge by savvy consumers is vastly smaller than the supply of knowledge on offer. The supply of both scholarly articles and the journals that publish them is large and the quality is uneven due to the need among academics to publish constantly. There are various curatorial functions, such as the formal curation that exists at arxiv.org and more informal method such as people simply sharing quality articles with each other over email. But collectively the curation is not robust enough to comprehensively cover the entire output across every venue and identify quality.

The market for specialized academic knowledge outside of the prestigious peer-reviewed ecosystem looks less like the New York Stock Exchange and more like the over the counter pink sheets market. You could hit it big and 100x your money, but it is far more likely that you loose your shirt.

Another part of this story that I found very interesting is the apparent simplicity of the proof. Statistician Donald Richards had been trying to find a proof for the Gaussian Correlation Conjecture for 30 years. But when he first saw Royen’s paper he said “I knew instantly that it was solved.” It is apparently an almost viscerally simple proof. Here is how the article describes it:

Any graduate student in statistics could follow the arguments, experts say. Royen said he hopes the “surprisingly simple proof … might encourage young students to use their own creativity to find new mathematical theorems,” since “a very high theoretical level is not always required.”

Just as Royen’s article was hiding in plain sight for three years before getting noticed, could it be that the proof was also hiding in plain sight for decades while statisticians dedicated to finding it looked elsewhere? If this proof was so simple why did it take so long to solve? And why did the proof come from someone so seemingly removed from the front lines of theoretical statistics research?

One possible answer is path dependency. Forty years ago a special, limited case of the conjecture was proved only for two-dimensional shapes. It was not a comprehensive proof, but it led researchers to believe that it contained the seeds for a comprehensive proof. Much work went into extending the two dimensional proof to a full proof. This turns out to have been the wrong approach, and it blinded researches to potential solutions elsewhere. Here is Loren Pitt, the mathematician responsible for the limited proof, describing this blindness:

Despite hundreds of pages of calculations leading nowhere, Pitt and other mathematicians felt certain—and took his 2-D proof as evidence—that the convex geometry framing of the GCI would lead to the general proof. “I had developed a conceptual way of thinking about this that perhaps I was overly wedded to,” Pitt said. “And what Royen did was kind of diametrically opposed to what I had in mind.”

I also wonder if there is currently a bias in favor of complex solutions, at least among academic researchers. Looking at the field it’s easy to assume that all the simple proofs have been found, and if a problem remains unsolved for as long the Gaussian Inequality Conjecture then the solution must be complex. For instance, it took mathematician Andrew Wiles over 150 pages (and his adult entire life) to prove that an+bn = cn, which is about as simple a mathematical statement as you could come up with.

The lessons I take away from this story are that simplicity is underrated, the default mode of complexity ought to be more regularly questioned, and maybe, just maybe, the truth is indeed out there, published in an obscure academic journal you’ve never heard of halfway around the world.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s