Information Processing

Just another WordPress.com weblog

Archive for the ‘malcolm gladwell’ Category

Annals of psychometry: IQs of eminent scientists

with 31 comments

I recently came across a 1950s study of eminent scientists by Harvard psychologist Anne Roe (The Making of a Scientist, published in 1952). Her study is by far the most systematic and sophisticated that I am aware of. She selected 64 eminent scientists — well known, but not quite at the Nobel level — in a more or less random fashion, using, e.g., membership lists of scholarly organizations and expert evaluators in the particular subfields. Roughly speaking, there were three groups: physicists (divided into experimental and theoretical subgroups), biologists (including biochemists and geneticists) and social scientists (psychologists, anthropologists).

Roe devised her own high-end intelligence tests as follows: she obtained difficult problems in verbal, spatial and mathematical reasoning from the Educational Testing Service, which administers the SAT, but also performs bespoke testing research for, e.g., the US military. Using these problems, she created three tests (V, S and M), which were administered to the 64 scientists, and also to a cohort of PhD students at Columbia Teacher’s College. The PhD students also took standard IQ tests and the results were used to norm the high-end VSM tests using an SD = 15. Most IQ tests are not good indicators of true high level ability (e.g., beyond +3 SD or so).

Average ages of subjects: mid-40s for physicists, somewhat older for other scientists

Overall normed scores:

Test (Low / Median / High)

V 121 / 166 / 177

S 123 / 137 / 164

M 128 / 154 / 194

Roe comments: (1) V test was too easy for some takers, so top score no ceiling. (2) S scores tend to decrease with age (correlation .4). Peak (younger) performance would have been higher. (3) M test was found to be too easy for the physicists; only administered to other groups.

It is unlikely that any single individual obtained all of the low scores, so each of the 64 would have been strongly superior in at least one or more areas.

Median scores (raw) by group:

group (V / S / M)

Biologists 56.6 / 9.4 / 16.8
Exp. Physics 46.6 / 11.7 / *
Theo. Physics 64.2 / 13.8 / *
Psychologists 57.7 / 11.3 / 15.6
Anthropologists 61.1 / 8.2 / 9.2

The lowest score in each category among the 12 theoretical physicists would have been roughly V 160 (!) S 130 M >> 150. (Ranges for all groups are given, but I’m too lazy to reproduce them all here.) It is hard to estimate the M scores of the physicists since when Roe tried the test on a few of them they more or less solved every problem modulo some careless mistakes. Note the top raw scores (27 out of 30 problems solved) among the non-physicists (obtained by 2 geneticists and a psychologist), are quite high but short of a full score. The corresponding normed score is 194!

The lowest V scores in the 120-range were only obtained by 2 experimental physicists, all other scientists scored well above this level — note the mean is 166.

My comments:

The data strongly suggests that high IQ provides a significant advantage in science. Some have claimed that IQ is irrelevant beyond some threshold: more precisely, that the advantage conferred by IQ above some threshold (e.g., 120) decreases significantly as other factors like drive or creativity take precedence. But, if that were the case it would be unlikely to have found such high scores in this group. The average IQ of a science PhD is probably in the 130 range, and individuals with IQs in the range described above constitute a tiny fraction of the overall population of scientists. If IQ were irrelevant above 130 we would expect the most eminent group to have a similar average.

Conversely, I think one should be impressed that a simple test which can be administered in a short period of time (e.g., 30 minutes for Roe’s high-end exams) offers significant predictive power. While it is not true that anyone with a high IQ can or will become a great scientist (certainly other factors like drive, luck, creativity play a role), one can nevertheless easily identify the 99 percent (even 99.9 percent) of the population for which success in science is highly improbable. Psychometrics works!

The scores for theoretical physicists confirm an estimate made to me by a famous colleague many years ago, that only 1 in 100,000 people could do high level theoretical physics.

Feynman’s 124: in this context one often hears of Feynman’s modest grade school IQ score of 124. To understand this score we have to remember that typical IQ tests (e.g., administered to public school children) tend to have low ceilings. They are not of the kind that Roe used in her study. One can imagine that the ceiling on Feynman’s exam was roughly 135 (say, 99th percentile). If Feynman received the highest score on the mathematical portion, and a modest score of 115 on the verbal, we can easily understand the resulting average of 124. However, it is well known that Feynman was extremely strong mathematically. He was asked on short notice to take the Putnam exam for MIT as a senior, and received the top score in the country that year! On Roe’s test Feynman’s math score would presumably have been > 190, with a correspondingly higher composite IQ.

I thought I should put this post up now, as the new book by Malcolm Gladwell, Outliers: Why Some People Succeed and Some Don’t is out soon and will surely handicap the discourse on this subject for years to come 🙂

Advertisements

Written by infoproc

July 7, 2008 at 4:02 pm

Gladwell amongst the patent trolls

with 11 comments

Malcolm Gladwell writes about Nathan Myhrvold‘s company Intellectual Ventures in the recent New Yorker. (Myhrvold is the former cosmologist who left physics and eventually became consigliere to Bill Gates, founding Microsoft Research and charting Microsoft’s blue sky research direction. He famously missed the importance of the Internet until the mid 90’s.) If you read this blog often, you know my opinion about Gladwell: he has a good nose for interesting topics, but not enough brainpower or common sense for reliable analysis. The same is true here: he produces an interesting profile of Myhrvold (although see here for a much better one from 1997 by Ken Auletta) and friends, but seems to entirely miss a number of important points. Intellectual Ventures is not about real inventions, but about patenting around ideas so that they have a future claim on the ones that turn out the be useful. In other words, they are patent trolls. Gladwell does not seem to realize the difference between rampant speculation and true invention: the hours of painstaking work in the lab required to convert an idea into reality.

Here’s an excerpt about how the “invention” process works — get some smart guys in a room and let them talk (every theory group lounge is a fount of commercializable ideas ;-). Yes! if your inventors are smart enough, they can produce 36 new inventions at dinner! Is this a statement about real innovation, or about what a patent attorney might manage to get the understaffed, overburdened USPTO to approve? It makes a mockery of what real inventors and innovators do. Why start a company and hire engineers to build a prototype? Just get a few lawyers and patent everything in sight…

How useful is it to have a group of really smart people brainstorm for a day? When Myhrvold started out, his expectations were modest. Although he wanted insights like Alexander Graham Bell’s, Bell was clearly one in a million, a genius who went on to have ideas in an extraordinary number of areas—sound recording, flight, lasers, tetrahedral construction, and hydrofoil boats, to name a few. …

But then, in August of 2003, I.V. held its first invention session, and it was a revelation. “Afterward, Nathan kept saying, ‘There are so many inventions,’ ” Wood recalled. “He thought if we came up with a half-dozen good ideas it would be great, and we came up with somewhere between fifty and a hundred. I said to him, ‘But you had eight people in that room who are seasoned inventors. Weren’t you expecting a multiplier effect?’ And he said, ‘Yeah, but it was more than multiplicity.’ Not even Nathan had any idea of what it was going to be like.”

The original expectation was that I.V. would file a hundred patents a year. Currently, it’s filing five hundred a year. It has a backlog of three thousand ideas. Wood said that he once attended a two-day invention session presided over by Jung, and after the first day the group went out to dinner. “So Edward took his people out, plus me,” Wood said. “And the eight of us sat down at a table and the attorney said, ‘Do you mind if I record the evening?’ And we all said no, of course not. We sat there. It was a long dinner. I thought we were lightly chewing the rag. But the next day the attorney comes up with eight single-spaced pages flagging thirty-six different inventions from dinner. Dinner.”

For the cognoscenti out there, yes, the Wood mentioned in the article is none other than Star Warrior Lowell Wood, former head of the zany (useless?) O Group (NYTimes 1984) at Livermore. Wood is perfect for Myhrvold’s purposes — for decades his group bamboozled the US defense establishment with wild ideas that cost taxpayers billions of dollars. Follow the link to the Times article and tell me how many of the ideas mentioned turned into something useful, now almost a quarter century later.

Rather than leave you with a completely negative impression of the article, I include the following excerpt, which has Wood noticing something about cancer cells in the bloodstream that seems to have eluded biologists and medical researchers for some time. It is true that there are great ideas out there just waiting to be discovered, but lots of people can have the same idea. The hard part is making the idea into a practical, commercially viable reality.

…Last March, Myhrvold decided to do an invention session with Eric Leuthardt and several other physicians in St. Louis. Rod Hyde came, along with a scientist from M.I.T. named Ed Boyden. Wood was there as well.

“Lowell came in looking like the Cheshire Cat,” Myhrvold recalled. “He said, ‘I have a question for everyone. You have a tumor, and the tumor becomes metastatic, and it sheds metastatic cancer cells. How long do those circulate in the bloodstream before they land?’ And we all said, ‘We don’t know. Ten times?’ ‘No,’ he said. ‘As many as a million times.’ Isn’t that amazing? If you had no time, you’d be screwed. But it turns out that these cells are in your blood for as long as a year before they land somewhere. What that says is that you’ve got a chance to intercept them.”

How did Wood come to this conclusion? He had run across a stray fact in a recent issue of The New England Journal of Medicine. “It was an article that talked about, at one point, the number of cancer cells per millilitre of blood,” he said. “And I looked at that figure and said, ‘Something’s wrong here. That can’t possibly be true.’ The number was incredibly high. Too high. It has to be one cell in a hundred litres, not what they were saying—one cell in a millilitre. Yet they spoke of it so confidently. I clicked through to the references. It was a commonplace. There really were that many cancer cells.”

Wood did some arithmetic. He knew that human beings have only about five litres of blood. He knew that the heart pumps close to a hundred millilitres of blood per beat, which means that all of our blood circulates through our bloodstream in a matter of minutes. The New England Journal article was about metastatic breast cancer, and it seemed to Wood that when women die of metastatic breast cancer they don’t die with thousands of tumors. The vast majority of circulating cancer cells don’t do anything.

“It turns out that some small per cent of tumor cells are actually the deadly ones,” he went on. “Tumor stem cells are what really initiate metastases. And isn’t it astonishing that they have to turn over at least ten thousand times before they can find a happy home? You naïvely think it’s once or twice or three times. Maybe five times at most. It isn’t. In other words, metastatic cancer—the brand of cancer that kills us—is an amazingly hard thing to initiate. Which strongly suggests that if you tip things just a little bit you essentially turn off the process.”

That was the idea that Wood presented to the room in St. Louis. From there, the discussion raced ahead. Myhrvold and his inventors had already done a lot of thinking about using tiny optical filters capable of identifying and zapping microscopic particles. They also knew that finding cancer cells in blood is not hard. They’re often the wrong size or the wrong shape. So what if you slid a tiny filter into a blood vessel of a cancer patient? “You don’t have to intercept very much of the blood for it to work,” Wood went on. “Maybe one ten-thousandth of it. The filter could be put in a little tiny vein in the back of the hand, because that’s all you need. Or maybe I intercept all of the blood, but then it doesn’t have to be a particularly efficient filter.”

Wood was a physicist, not a doctor, but that wasn’t necessarily a liability, at this stage. “People in biology and medicine don’t do arithmetic,” he said. He wasn’t being critical of biologists and physicians: this was, after all, a man who read medical journals for fun. He meant that the traditions of medicine encouraged qualitative observation and interpretation. But what physicists do—out of sheer force of habit and training—is measure things and compare measurements, and do the math to put measurements in context. At that moment, while reading The New England Journal, Wood had the advantages of someone looking at a familiar fact with a fresh perspective.

That was also why Myhrvold had wanted to take his crew to St. Louis to meet with the surgeons. He likes to say that the only time a physicist and a brain surgeon meet is when the physicist is about to be cut open—and to his mind that made no sense. Surgeons had all kinds of problems that they didn’t realize had solutions, and physicists had all kinds of solutions to things that they didn’t realize were problems. At one point, Myhrvold asked the surgeons what, in a perfect world, would make their lives easier, and they said that they wanted an X-ray that went only skin deep. They wanted to know, before they made their first incision, what was just below the surface. When the Intellectual Ventures crew heard that, their response was amazement. “That’s your dream? A subcutaneous X-ray? We can do that.”

Let me close with my usual observation (specifically aimed at venture capitalists, research lab directors and university administrators) concerning an asymmetry in cognitive depth: yes, physicists can casually read the New England Journal of Medicine and come up with interesting insights, but, no, biologists and medical doctors cannot read Physical Review.

Gladwell and genius

leave a comment »

Malcolm Gladwell exhibits exquisite taste in the topics he writes and talks about. I just wish his logical and analytical capabilities were better (see also here). This talk at the New Yorker’s recent Genius 2012 conference is entertaining, but I disagree completely with his conclusion. Ribet, Wiles, Taniyama and Shimura are probably the real geniuses, not Michael Ventris, the guy who decoded Linear B. (Gladwell also can’t seem to remember that it’s the Taniyama-Shimura conjecture, not Tanimara. He says it incorrectly about 10 times.) My feeling is that Gladwell’s work appeals most to people who can’t quite understand what he is talking about.

Gladwell is confused about the exact topic discussed in James Gleick’s book Genius. In a field where sampling of talents is sparse (e.g., decoding ancient codexes) you might find one giant (even an amateur like Michael Ventris) towering above the others, able to do things others cannot. In a well-developed, highly competitive field like modern mathematics, all the top players are “geniuses” in some sense (rare talents, one in a million), even though they don’t stand out very much from each other. In Gleick’s book, Feynman, discussing how long it might have taken to develop general relativity had Einstein not done it, says “We are not that much smarter than each other”!

To put it really simply, if I sample sparsely from a Gaussian distribution, I might find a super-outlier in the resulting set. If I sample densely and have a high minimum cutoff for acceptable points, I will end up with a set entirely composed of outliers, but who do not stand out much from each other. Every guard in the NBA is an athletic freak of nature, even though they are evenly matched when playing against each other.

To counteract the intelligence-damping effect of Gladwell’s talk, I suggest this podcast interview with Nassim Taleb, about his new book The Black Swan. Warning: may be psychologically damaging to people who fool themselves and others about their ability to predict the behavior of nonlinear systems.

Written by infoproc

May 12, 2007 at 2:43 pm