Fraud In The Social Sciences

Megan McArdle has an excellent piece on the nature of the discipline and its perverse incentives:

The system was rewarding a very, very specific thing: novel but intuitively plausible results that told neat stories about human behavior. Stars in that field are people who consistently identify, and then prove, interesting but believable results.

The problem is that reality is usually pretty messy, especially in social psychology, where you tend to be looking for fairly subtle effects. Even a genius will be wrong a lot of the time: he will invest in hypotheses that sound convincing but aren’t actually true, or come up with data that is too messy to tell you much one way or another. Sadly, the prestige journals aren’t looking to publish “We tested this interesting hypothesis, and boy, the data are just a mess!” They want a story, the clearer, the better.

Academics these days operate under enormous pressure to churn out high volumes of these publications. Hitting those targets again and again is the key to tenure, the full professorship, hopefully the lucrative lectures. Competition is fierce for all of those things, and it’s easy to get knocked out at every step. If getting good results is somewhat random, then all those professors are very vulnerable to a string of bad luck. The temptation to make your own luck is thus very high.

Again, I do not excuse those who resort to cheating. But as the consumer of these publications, we should be worried, because this system essentially selects for bad data handling. The more you manipulate your data (and there are lots of ways to massage your data so that it shows what you’d like, even without knowing you’re doing it), the more likely you are to come up with a publishable result. Peer review acts as something of a check on this, of course. But your peers don’t know if, for example, you decided to report only the one time your experiment worked, instead of the seven times it didn’t.

It would be much better if we rewarded replication: if journals were filled not only with papers describing novel effects, but also with papers by researchers who replicated someone else’s novel effects. But replicating an effect that someone else has found has nowhere near the prestige–or the publication value–of something entirely new. Which means, of course, that it’s relatively easy to make up numbers and be sure that no one else will try to check.

Most cases are not as extreme as Stapel. But if we reward only those who generate interesting results, rather than interesting hypotheses, we are asking for trouble. It is hard to fake good questions, but if the good questions must also have good answers . . . well, good answers are easy. And it seems that this is what the social psychology profession is rewarding.

Emphasis mine.

What I found fascinating about this is that you can substitute the phrase “climate science” for “social psychology” and (say) “Mann” for “Stapel,” and it makes just as much sense.

This is probably worth a PJMedia piece.

[Update a few minutes later]

One other phrase that would have to change: “that told neatpolitically appealing stories about human behaviorhumanity’s impact on the environment.”

15 thoughts on “Fraud In The Social Sciences”

  1. I just read a summary about the Stapel investigation.

    A choice quote: “It is to the credit of the three universities involved that they gave the Committees a free hand to analyse all of Mr Stapel’s publications.”

    Somebody needs to hit UVa with a cluebat.

  2. Sociology is not a science, never has been a science and should never be considered a science. Psychology is the handmaiden of sociology and “climate science” is an oxymoron.

    1. “Sociology is not a science, never has been a science and should never be considered a science. Psychology is the handmaiden of sociology and “climate science” is an oxymoron.”

      This should be carved in stone in front of every sociology and psychology dept in every College and University in the country.

      1. Paraphrasing Robert Heinlein; “Anything that includes the word ‘science’ in the name of the field isn’t one, such as ‘social science’. “

  3. ‘Struth. Affecting human behavior has always been an art, not a science. Convincing people to regard it as science was a most artful deceit.

  4. “Social Sciences” ? The portmanteau itself is an admission of fraud. There is no “there” there.

    1. Indeed, if it was a science there wouldn’t be the need to prepend the adjective “social”. Same goes for Social Justice – if it was Justice, it wouldn’t need the adjective.

  5. While I am adamant in condemning fraud in science, and am just as adamantly a Constitutional conservative, I am surprised by the attitudes expressed here.

    If you condemn “social science,” then how can you make valid conclusions about groups of people? Just pronounce your position and believe it to be true? That is not only prejudice, but it is a fallacy of assertion, and a tactic common among the political left.

    Granted, most social “scientists” are leftists who use their positions to propagandize, but that does not mean that there can be no science in “social sciences.” In fact, conservatives in social science could do a tremendous good to help understand what appears to be the mental disorders and mob mentality that underlie the yearning for statist solutions to personal problems.

    1. There’s a big problem with scientific analysis of societies: the sample size. There simply haven’t been enough societies to provide a decent sample. Maybe we’ll have a large sample after we settle the Galaxy…

    2. Science uses math; if it can’t be expressed mathematically, it isn’t (yet?) a science. “Social science” does not allow you to make predictions about how your subjects will react.

      Joseph Hertzlinger; Please note that even in the Foundation books, it’s revealed that Hari Seldon isn’t so much a scientist as a puppet. A Galaxy-wide sample probably won’t help much.

  6. Progressives learned this on the knee of “The Silent Spring” and Rachael Carlson. One set of eggs proved her ideas about DDT endangering birds, the other set did not. She just threw out the data she didn’t like, and lo and behold, the newly formed EPA banned DDT not without evidence, but in spite of more accurate evidence by true scientists that did not follow her conclusion.

    Considering her success and fame, why is anyone surprised that our system has been warped the way it is?

  7. Climate Science: The Navier Stokes equations describe fluid flow with changes in temperature and density. They are non-linear, chaotic, with sensitive dependence on initial conditions. That means no finite set of past recorded states are sufficient to predict distant future states. This has been known since Edward Lorenz’s paper “Deterministic Aperiodic Flow” in 1963. That means trying to predict future weather or cilmate states from records of past states MUST fail. Global Warming predictions are are a hoax.

    1. That means trying to predict future weather or cilmate states from records of past states MUST fail

      DonM, no. It just means you are limited in what you can say about future states of a chaotic system. The predictions degrade rather quickly in quality over time until you effectively can do as well just from studying the statistical distribution of such state measurements, which incidentally is quite feasible to do with a chaotic system. Just let it run out over a long enough time.

      And climatology is more about statistical long term properties. You wouldn’t use it to predict a particular future state at a particular time, but rather to determine things such as an approximation of the rules that govern climate dynamics or particular long time scale, large spatial scale physical properties and trends (such as global mean temperature).

  8. It’s interesting that Judah Folkman, the discoverer of the angiogenesis concept in biology and cancer therapy, was rejected for a long time because he was a pediatric surgeon, not a biology researcher, and because the early efforts to duplicate his work were unsuccessful. For years, he was considered a charlatan in basic research although a renowned pediatric surgeon. It isn’t easy sometimes to pick the good guys from the chaff.

Comments are closed.