Cooking stove use, housing associations, white males, and the 97%

The Cook et al. (2013) 97% paper included a bunch of psychology studies, marketing papers, and surveys of the general public as scientific endorsement of anthropogenic climate change. Let's go ahead and walk through that sentence again. The Cook et al 97% paper included a bunch of psychology studies, marketing papers, and surveys of the general public as scientific endorsement of anthropogenic climate change. I only spent ten minutes with their database -- there will be more such papers for those who search. I'm not willing to spend a lot of time with their data, for reasons I detail further down. This paper is vacated, as a scientific product, given that it included psychology papers, and also given that it twice lied about its method (claiming not to count social science papers, and claiming to use independent raters), and the professed cheating by the raters. It was essentially voided by its invalid method of using partisan and unqualified political activists to subjectively rate climate science abstracts on the issue on which their activism centers -- a stunning and unprecedented method. I'm awaiting word on retraction from the journal, but I think we already know that this paper is vacated. It doesn't represent knowledge of the consensus. I discovered that the following papers were included as endorsement, as "climate papers", again in just ten minutes of looking. They are classified as either implicit or explicit endorsement, and were evidently included in the 97% figure: Chowdhury, M. S. H., Koike, M., Akther, S., & Miah, D. (2011). Biomass fuel use, burning technique and reasons for the denial of improved cooking stoves by Forest User Groups of Rema-Kalenga Wildlife Sanctuary, Bangladesh. International Journal of Sustainable Development & World Ecology, 18(1), 88–97. Ding, D., Maibach, E. W., Zhao, X., Roser-Renouf, C., & Leiserowitz, A. (2011). Support for climate policy and societal action are linked to perceptions about scientific agreement. Nature Climate Change, 1(9), 462–466. Egmond, C., Jonkers, R., & Kok, G. (2006). A strategy and protocol to increase diffusion of energy related innovations into the mainstream of housing associations. Energy Policy, 34(18), 4042–4049. Gruber, E., & Brand, M. (1991). Promoting energy conservation in small and medium-sized companies. Energy Policy, 19(3), 279–287. Ha-Duong, M. (2008). Hierarchical fusion of expert opinions in the Transferable Belief Model, application to climate sensitivity. International Journal of Approximate Reasoning, 49(3), 555–574. Palmgren, C. R., Morgan, M. G., Bruine de Bruin, W., & Keith, D. W. (2004). Initial public perceptions of deep geological and oceanic disposal of carbon dioxide. Environmental Science & Technology, 38(24), 6441–6450. Reynolds, T. W., Bostrom, A., Read, D., & Morgan, M. G. (2010). Now what do people know about global climate change? Survey studies of educated laypeople. Risk Analysis, 30(10), 1520–1538. Semenza, J. C., Ploubidis, G. B., & George, L. A. (2011). Climate change and climate variability: personal motivation for adaptation and mitigation. Environmental Health, 10(1), 46. In Table 1, page 2, the authors claimed that social science papers were classified as "Not climate related" and not included as endorsement cases. This is a false claim, and the authors should be investigated for fraud. (There were some papers that were classified as "Not climate related" in my quick search, but the above papers were not -- they were classified is implicit or explicit endorsement.) I assume this is obvious, but I'll spell out that a survey of what the general public knows or thinks about climate science is not scientific evidence of anthropogenic warming. It doesn't matter what the results of such a survey are -- it has nothing to do with the scientific evidence for AGW. It's not a climate paper. A survey of people's cooking stove use and why they don't like the new, improved cooking stoves, is not scientific evidence of anthropogenic warming. It's not a climate paper. An investigation of the psychology of personal motivation for adaptation or mitigation is not evidence of anthropogenic warming. It's not a climate paper. This also makes us throw out the claim that 97% of the authors of the papers that were counted as taking a position said that their papers endorsed the consensus. That claim is vacated until we find out how many of those authors were authors of social science, psychology, marketing, and public survey papers like the above. It would be amazing that the authors of such papers responded and said that their papers counted as endorsement, but at this point all bets are off. (Only 14% of the surveyed authors responded at all, making the 97% figure difficult to take at face value anyway. But the inclusion of non-climate science papers throws it out.) I want to note here that the authors are still misrepresenting their 97% figure as consisting of "climate papers". For an upcoming event, Cook claims "They found that among relevant climate papers, 97% endorsed the consensus that humans were causing global warming." Clearly, this is false. There is no way we'll be able to call the above papers "relevant climate papers". Don't let these people get away with such behavior -- call them out on it. Ask them how psychology papers can be "relevant climate papers", raise your hand at events, notify journalists, etc. Make them defend, explicitly, what they did. Hopefully, it will be retracted soon. But until then, make them defend what they did. For one thing, Cook should now have to disclose how many psychology and other irrelevant papers were included. In a scenario where retraction wasn't justified, they would have to rewrite the paper. In this case, the false statements, fraud, and absurd method mandate retraction, and some sort of penance. The paper lied about its method in another way. It claimed to use independent raters (bottom of page 2), when in fact they set up an online forum to discuss their ratings. This was not limited to cases where raters disagreed, which the paper said were resolved by a third person -- this was when they were actually making their ratings. Here's a good example of a forum discussion, released by whistleblower Brandon Shollenberger, where they discuss how to rate a psychology paper about white males and "denial". Yes, they're seriously discussing how to rate a psychology paper about white males*. These people are deeply, deeply confused. The world thought they were talking about climate science. (Unlike the psychology papers listed above, this white males study didn't make the cut, for unknown reasons.) In that discussion, several raters wanted to count the paper as evidence of endorsement. Some say it should count as a "mitigation" paper... One rater said "I have classified this kind of papers as mitigation as I think that's mostly what they are related to (climate denial and public opinion are preventing mitigation)." I want to pause here and make clear that this alone could vacate the study, and would normally require swift investigation and clarification to avoid a retraction. We have explicit evidence that a rater counted a bunch of psychology or social science papers as mitigation, which was an endorsement category for them. (Mitigation in general should not be counted, and it's a vast number of papers in this study -- that's a major flaw, maybe the seventh major, invalidating flaw in this paper. We've been asleep at the wheel, haven't carefully thought about the epistemology of consensus, and what should count as evidence. I don't think mitigation papers can count in most cases -- they usually aren't primary evidence, they don't report something new about our understanding of climate change or processes. A number of consensus paper authors have gotten away with counting them without discussing or validating why they should count.) Another rater said "I have classified many social science abstracts as mitigation; often they are studying how to motivate people to act to slow AGW." There's a soundtrack that goes with that quote, perhaps a big thud, maybe the sound of a record player needle going off track, or the sound of someone opening a bottle of Tylenol. This is a disaster. These disclosures alone, of what raters were doing, should lead to the retraction of the paper, ideally by the authors, but if necessary by the journal or IOP. This was not a valid or credible study. We knew that already, and we had plenty of reason to retract before this, but now we have explicit evidence here that these people had no idea what they were doing, were largely motivated by ideology, and should probably submit to drug testing. These people have no business being raters, and whoever was running this rodeo should've corrected this immediately, but they didn't. Other raters, like Dana Nuccitelli, say it should count as "methods" (I think methods papers were excluded from endorsement), but that "It's borderline implicit endorsement though, with all the 'climate change denial' phrases.  If you read the paper I'd bet it would be an explicit endorsement." Nuccitelli thinks that if a psychology paper uses the phrase "climate change denial", it might count as scientific endorsement of anthropogenic climate change. We should linger on that. This is a staggering level of stupidity with respect to what would count as scientific evidence of AGW. The implied epistemology there is, well, I don't know that it has a name. Maybe it's some kind of postmodernist view of reality being based on belief, anyone's belief (except for the beliefs of skeptics) -- perhaps a grotesque misreading of Kuhn. Even if we thought reality was best understood via consensus, it's not going to be created by consensus, and the only consensus we would care about would be that of climate scientists. That Marxist or neo-Marxist sociologists pepper their paper with "climate change denial" does not add to our confidence level about AGW -- it is not evidence of anything but the ideology of two American sociologists [McCright & Dunlap (2011) -- the theoretical framework of that paper, laid out in its introduction, was Marxist.] It doesn't test the energy balance model, or revise or validate or estimates of transient climate sensitivity. It has no input into our knowledge of AGW. In any case, I'm stunned by Nuccitelli's behavior in these rater forum pages, and his behavior as a climate science writer – he and Jenny McCarthy should jointly surrender to some sort of authority. No one who would want to count a paper about white males as evidence has any business being a rater in subjective rating-based study of the climate science consensus. There are lots more of these alarming online discussions among raters, including disclosures that they violated the protocol and cheated by looking at the entire paper after suspecting that it was written by a skeptic (veteran climate scientist Richard Lindzen in this case.) That post vividly captures the absurd level of ideological bias and motivation of the raters. (It also captures fraud, because the authors state that raters were blind to authors, and that everything but the title and abstract was hidden.) I honestly think at least part of the issue here is intelligence and knowledge. It think this is a pervasive issue in the climate debate, but is rarely called out, and it's easy for it to be lazy ad hominem. Intelligence can be a real, functional constraint. For example, I think some climate science skeptics simply aren't smart enough -- they're not smart enough to understand climate science or its methods. They'll never understand what these "computer models" are doing, or why calling something a computer model doesn't invalidate it. I think if the Higgs boson had political implications of the sort that AGW is presumed to have, some of those same people would express similar arguments against the existence of the Higgs or the validity of its detection, saying that it's all "computer models", or that we can't really "see" it. In such a case, I think it would come down to them not being smart enough to understand the methods, or the nature of that particular reality. Reality isn't structured such that any scientific field will be understandable to any outsider with an IQ of 100 or better -- it would be arbitrary to assume that it was. The people who conducted the Cook study don't understand rudimentary epistemology, or what counts as evidence of anthropogenic climate change. Cook's e-mailed response to my call for retraction also struck me as that of someone who just isn't equipped to deal with these sorts of issues. Nuccitelli's comments in the forum about the white males study is more evidence that these people aren't equipped for this. In any case, this study is vacated regardless of how smart or not smart they are. We just need to keep in mind for the future that if people don't understand basic epistemology, or what counts as evidence, they won't be able to do a valid consensus study -- not even a survey of scientists, because the construction of the questions and response scales will require such knowledge, basic epistemological competence. Anyone who does a scientific consensus study really needs to think deeply about epistemology, needs to be comfortable with it, able to understand concepts of confidence, proof, the nature of different kinds of claims, and the terms scientists are most comfortable using to describe their claims (and why.) That's all I have for that issue. I step back and start at a more fundamental place below. Let's retrace our steps.. The above papers have nothing to do, epistemologically, with the scientific consensus on global warming. The consensus only pertains to climate science, to those scientists who actually study and investigate climate. To include those papers was either a ridiculous error or fraud. I didn't expect this -- I expected general bias in rating climate papers. I never imagined they'd include surveys of the public, psychology papers, and marketing studies. In retrospect, this was entirely predictable given that the researchers are a bunch of militant anti-science political activists. As I said, I found those papers in ten minutes with their database. I'm not willing to invest a lot of time with their data. The reason is what I've argued before -- a method like theirs is invalid and perverts the burden of proof. The world is never going to be interested in a study based on militant political activists reading scientific abstracts and deciding what they mean with respect to the issue that is the focus of their activism. That method is absurd on its face. We can't do anything with such studies, and no one should be burdened with going through all their ratings and identifying the bias. We can't trust a study where the researchers are political partisans who have placed themselves in the position to generate the data that would serve the political goals -- the position of being subjective raters of something as complex and malleable as a scientific abstract. That's not how science is normally done. I've never heard of researchers placing themselves in the position of subjectively rating complex text, articles and the like, on an issue on which they happen to be activists. I don't think I spelled this out before because I thought it was obvious: There is enormous potential for bias in such a method, far more potential than in a normal scientific study where the researchers are collecting data, not creating it. Having human beings read a complicated passage, a short essay, and decide what it means, is already a very subjective and potentially messy method. It's special, and requires special training and guidelines, along with special analyses and statistics. The Cook paper is the only subjective rating study I've ever seen that did not report any of the statistics required of such studies. It was amazing -- they never reported interrater reliability. I can't imagine a rater study that doesn't report the reliability of the raters... This study is a teachable moment, a future textbook example of scientific scams. But having humans read a scientific abstract and decide what it means is even more challenging than a normal rater study. For one thing, it's very complicated, and requires expert knowledge. And in this case, the researchers/raters were unqualified. Most people aren't going to be able to read the abstracts from any given scientific field and understand them. Climate science is no different from any other field in this respect. The raters here included luggage entrepreneurs, random bloggers, and an anonymous logician known only by his nom de guerre, e.g. "logicman", among others. Normally, we would immediately stop and ask how in the hell these people are qualified to read and rate climate science abstracts, or in logicman's case, who these people are. To illustrate my point,  here's a sample climate abstract, from LeGrande and Schmidt (2006): We present a new 3-dimensional 1° × 1° gridded data set for the annual mean seawater oxygen isotope ratio (δ18O) to use in oceanographic and paleoceanographic applications. It is constructed from a large set of observations made over the last 50 years combined with estimates from regional δ18O to salinity relationships in areas of sparse data. We use ocean fronts and water mass tracer concentrations to help define distinct water masses over which consistent local relationships are valid. The resulting data set compares well to the GEOSECS data (where available); however, in certain regions, particularly where sea ice is present, significant seasonality may bias the results. As an example application of this data set, we use the resulting surface δ18O as a boundary condition for isotope-enabled GISS ModelE to yield a more realistic comparison to the isotopic composition of precipitation data, thus quantifying the ‘source effect’ of δ18O on the isotopic composition of precipitation. You think a layperson, even a smart, educated layperson would understand what this means? How would they? Would they know what the GISS ModelE is? What GEOSECS data is? What 3-dimensional 1° × 1° gridded data is? Even scientists in other fields wouldn't know what this means, unless they did a lot of reading. The Cook study was ridiculous on these grounds alone. The burden lies with the authors trotting out such a questionable method to first establish that their raters had the requisite knowledge and qualifications to rate climate abstracts. No one should ever publish a study based on laypeople rating scientific abstracts without a big pile of evidence that they're qualified. This is technically ad hominem, but I don't think ad hominem is a fallacy in such cases (which I detail in my book on valid reasoning), any more than wanting a doctor to have gone to medical school is a fallacy. I'm fine with laypeople criticizing academics and scientists, and I don't think such criticism should be dismissed out of hand (which I call the Insider Fallacy), but if you give me a study where laypeople rated thousands of abstracts, I'm not going to accept the burden of proving that their ratings were bad, one by one. Thousands of ratings is a much different situation than one critical essay by a layperson, which we can just read and decide if it's valid or right. With thousands of ratings, I think the burden has to be on the researchers to establish that they were qualified. When we add the fact that the raters were militant political activists motivated to achieve a particular result from the study, we go home. The normal response might be several minutes of cognitive paralysis over the absurdity of such a method, of such a "study". ERL should be ashamed of what they did here. This is a disgrace. Political activists rating abstracts pertaining to their political aims? Have we decided to cancel science? Are we being serious? It's 2014. We have decades and mountains of research on bias and motivated reasoning, scientific research. The idea of humans reading and rating abstracts on an issue central to their ideology sparks multiple, loud, shrieking alarms. A decently bright teenager would be able to identify this method as absurd. This really isn't complicated. It shouldn't be happening in the modern world, in modern scientific journals. I assume this article will be retracted. At some point, we need to be able set politics aside in favor of transcendent commitments to integrity, scientific rigor, valid methods, and a basic posture against fraud and in favor of using our brains. Speaking of using our brains, I think we might also want to think about why we would ever count papers, and take a percentage, as a measure of the consensus on some issue in scientific field. There are several obvious issues there that we'd need to address first. And on this particular topic, it doesn't address the arguments skeptics make, e.g. publication bias. The publication bias argument is unruffled by counting publications. If we care about engaging with or refuting skeptics, this method won't do it. But again, there are several obvious issues with counting papers as a method (which is layered on top of the issues with having humans read abstracts and decide what they mean with respect to a contentious environmental issue.) I won't go into them now, but we need careful work by epistemologists here (I know there is some work -- I apologize if I've missed a paper where an epistemologist or philosopher of science has directly addressed the paper counting method.) Those of you who have shaped yourself into pretzels defending this study should be ashamed. And you should also be prepared to enthusiastically assimilate and circulate the findings of Heartland or Heritage if they stoop to using a bunch of their political activist staffers to subjectively rate scientific abstracts. And if ERL doesn't retract, for some unimaginable reason, they should enthusiastically publish subjective rater studies conducted by conservative political activists on climate science, Mormons on the science of gay marriage, and Scientologists on the harms of psychiatry (well, if it weren't just an environmental journal.) I get the impression that Cook and company don't think they're militant political activists, as though being a staunch leftist is the default rational position, not partisan (even though they talk about politics more than science on some of their pages, savage skeptical scientists, Republicans, and oil companies, ignore scientific evidence and papers that conflict with their views, ignore a large swath of economics, and are just off the charts in their "denier! denier!" hostility) -- if you seriously think that the only "partisans" are people who disagree with you, then you've not yet achieved mature adulthood. (It's often the mark of a hardcore partisan ideologue that they don't see themselves as ideological -- they've converted their ideology into descriptive reality. This is the nature of Haidt's "tribal moral communities". In the case of Cook's gang, they know very little about the intellectual landscape, the different views and frameworks people can have about the environment, economics, values, etc. -- their world is just them and "deniers". When a researcher uses the word denier to tag their adversaries, you should probably ignore them. Credible people don't use unscientific, unvalidated smears.) I think some of you who've defended this "study" got on the wrong train. I don't think you meant to end up here. I think it was an accident. You thought you were getting on the Science Train. You thought these people -- Cook, Nuccitelli, Lewandowsky -- were the science crowd, and that the opposition was anti-science, "deniers" and so forth. I hope it's clear at this point that this was not the Science Train. This is a different train. These people care much less about science than they do about politics. They're willing to do absolutely stunning, unbelievable things to score political points. What they did still stuns me, that they did this on purpose, that it was published, that we live in a world where people can publish these sorts of obvious scams in normally scientific journals. If you got on this train, you're now at a place where you have to defend political activists rating scientific abstracts regarding the issue on which their activism is focused, able to generate the results they want. You have to defend people counting psychology studies and surveys of the general public as scientific evidence of endorsement of AGW. You have to defend false statements about the methods used in the study. Their falsity won't be a matter of opinion -- they were clear and simple claims, and they were false. You have to defend the use of raters who wanted to count a bad psychology study of white males as evidence of scientific endorsement of AGW. You have to defend vile behavior, dishonesty, and stunning hatred and malice as a standard way to deal with dissent. I think many of you have too few categories. You might have science and anti-science categories, for example, or pro-science and denier. The world isn't going to be that simple. It's never been that simple. Reality is a complicated place, including the reality of human psychology and knowledge. Science is enormously complicated. We can't even understand the proper role of science, or how to evaluate what scientists say, without a good epistemological framework. No serious epistemological framework is going to lump the future projections of a young and dynamic scientific field with the truth of evolution, or the age of the earth. Those claims are very different in terms of their bodies of evidence, the levels of confidence a rational person should have in them, and how accessible the evidence is to inquiring laypeople. Cognition is in large part categorization, and we need more categories to understand and sort people's views and frameworks when it comes to fresh scientific issues like AGW. If our science category or camp includes people like Cook and Nuccitelli, it's no longer a science category. We won't have credibility as pro-science people if those people are the standard bearers. Those people are in a different category, a different camp, and it won't be called "science". Those climate scientists who have touted, endorsed, and defended the Cook et al. study – I suggest you reconsider. I also suggest that you run some basic correction for the known bias, and cognitive dissonance, humans have against changing their position, admitting they were wrong, etc. Do you really want to be on the historical record as a defender of this absurd malpractice? It's not going to age well, and as a scientist, certain values and principles should matter more to you than politics. If you're always on the side of people who share your political views or aims, if you're always on the side of people who report a high AGW consensus figure, no matter what they do, something is wrong. It's unlikely that all the people who share your political perspective, or all studies conducted by them, are right or valid -- and we know that in advance. We need more honesty on this issue, less political malice, better epistemology. * The white males paper is an invalid paper that arbitrarily and unscientifically labels people as deniers for not being worried enough about global warming, and for rating the media coverage as overhyped. It was written by sociologists, not psychologists, and they did nothing to validate this "denial" construct as real. They presented no evidence that a rational person has to worry a certain amount about distant future changes to earth's climate, that not worrying the specified amount is evidence of a process of "denial", that the appropriateness of media coverage of an issue is a descriptive fact that can be denied, rather than a subjective and complex, value-driven judgment, and that it is an established and accessible fact that the media does not overhype AGW. We have a serious problem with what journals are doing.   Comments Barry Woods 08/29/2014 5:53am ref the white males paper - the one with the Cool Dudes in the title..? http://judithcurry.com/2011/07/29/cool-dudes/ the only response to that sort of 'psychological research' is humour http://www.cartoonsbyjosh.com/cool_dude_scr.jpg http://www.cartoonsbyjosh.com/index.old.html Reply Graham Strouts link 08/29/2014 7:04am Awesome and amazing. Incredible post. As someone who has this "study" thrown at me by scientists on a weekly- nearly daily- basis in debates with scientists on FB and elsewhere, you have said everything I argue and far more, only far more eloquently and authoritatively. This could mark a turning point in the politics of AGW. Thankyou. Reply Vieras 08/29/2014 8:06am If you want to be even more amazed (or depressed), post this story to the Google+ science forum. You'll get attacked by the moderators there. I kid you not. Reply Steve Ta 08/29/2014 8:12am I assume if if they were to rerun the study, they could include the previous study results as yet another endorsement of AGW. Reply Randizzle 08/29/2014 9:52am Brilliant Jose! This needs airing,,,,, widely. will send it along to others. Reply HK 08/29/2014 10:21am "which I detail in my book on valid reasoning" What is the book's title? Is it published yet? Reply Jim Pettit 08/29/2014 1:42pm Irony: a psychology student whining that a published paper is invalid because it was based in part on psychology studies. Awesome, dude. Simple awesome... Reply MikeR 08/29/2014 1:46pm "The world is never going to be interested in a study based on militant political activists reading scientific abstracts and deciding what they mean with respect to the issue that is the focus of their activism." Ah, but they are. This paper is very widely quoted by people who like the results, including the President of the United States. Note that even aside from your complaints, there are _real_ climate papers which will surely be counted as supporting the consensus which are irrelevant, if we use your definition of "test the energy balance model, or revise or validate or estimates of transient climate sensitivity." Say, for instance, that there's a study done on how melting the Himalayan glaciers will affect Indian flowers. Presumably the paper will take for granted AGW as being true; that's probably the reason for the paper. But the author may know nothing about energy balance or transient climate sensitivity or the attribution problem. Which is fine - but if those issues are the ones you are focused on it makes no sense to ask him, or even figure out what he assumes. It would really be much more helpful to identify a number of critical links in the AGW hypothesis, and test them one at a time. What percentage of _scientists who study it_ think that CO2 is increasing due to human activity? (Probably 100%?) What percentage of _scientists who study it_ think that temperature has increased in the last century by about __ degrees? What do _those who study it_ think is the percentage of that increase caused by AGW? Equilibrium climate sensitivity? Etc. Reply Geoff Chambers link 08/29/2014 1:50pm This post is getting some publicity at http://bishophill.squarespace.com/blog/2014/8/29/more-on-cooks-97.html Here’s the comment I left there: “Duarte’s article links to Cook’s upcoming event at Bristol University’s Festival of Ideas. Bristol lists him as “of the University of Queensland” without noting that he’s a student there - a student who got into the University circuit via his collaboration with Professor Lewandowsky, and who then lied to his collaborator Lewandowsky and to me about his collaboration on Lewandowskys’ paper, and then lied again in a paper he wrote with Lewandowsky which tried to hide the lies in the first paper. I’ve been calling out Lewandowsky and Cook as liars at Chris Mooney’s, the New Yorker, Huffington Post, the Conversation, and anywhere else I can. Mooney and his colleagues at the Guardian, Telegraph, New York Times, Los Angeles Times etc. are only journalists. If they wants to repeat lies and conduct fawning interviews with known liars that’s their business. Bristol University is a bit different, and so is the Conversation, since it’s financed by a number of state-financed universities and institutions. If they lie and continue to publicise the work of known liars people just might start to ask questions. Or they might like to sue me. Or they could start telling the truth.” source