Monday, June 27, 2005

Expert Opinion

After a speech at UC Berkeley, I asked former Colorado Senator and erstwhile Presidential candidate Gary Hart a question about experts and opinions. I wanted to know how to translate the amazing policy relevant research I had been exposed to in my economics courses at Berkeley into action. It seemed to me that all the wonderful knowledge residing in the university was not being used much in policy making. Do supply side tax cuts stimulate the economy? Will a stricter bankruptcy law hurt entrepreneurship? Does class size impact student performance? We know a lot about the answers to these questions, but it does not always seem to matter. It was government bureaucrats, not academic experts, making important decisions on economic policy, health policy, education policy and environmental policy. And though academics are hardly perfect or unbiased, they actually have norms and procedures to evaluate new research, which should lead to better policies.

Hart answered quite honestly that during his time in the Senate, he had seen a parade of experts during every relevant committee session. The problem was not too few experts, but too many! During important debates on tax policy, climate change, public schools, and other issues, both parties employed their own band of experts, who predictably confirmed the party line, with their own “research.” For the Senators without solid training in social science, it was difficult to discern between the experts, and moreover, it was always politically expedient to place higher value on the research that supported the party agenda. As a result, the research on every controversial issue will always be “mixed,” even when the rigorous, peer reviewed publications all point the same way. This morass of research not only frustrates serious academics and policy wonks. It also leads to bad policy.

Still, it is difficult to combat bad research in the political arena, because discussions of t-statistics, sample size, and selection bias float above the median voter’s head. It is simply too difficult to expose bad science in 30 seconds, especially when the other guy’s story sounds so intuitively appealing. Think about how easy it is to argue that smaller classes always improve student performance. The sound bite culture of our political discourse makes careful, fact-based persuasion even more difficult.

The debate over global warming suffers from “this proliferation of experts” problem. The WSJ published a problematic editorial on the subject recently, which was met by a technical and reasoned response in cyberspace at a site called But will anyone on the other side ever read such analysis? Or will they simply pull out another study (probably poorly designed) that supports their position that global warming in not caused by human activity? The average voter will find it hard to decipher the academic jargon in both bodies of research, despite one being of much inferior quality. This has to be expected, because not everyone can invest in becoming an expert, although improvements in education, particularly economics and math would help here.

Today’s NYT had an article on the possible links between autism and the small amounts of mercury contained in vaccines for children. The large sample studies (there are at least three discussed) find no link, but other experts (predictably using “alternative” methods) are finding different results. Sadly, “alternative methods” usually means using bad statistics and ignoring academic convention. But it won’t stop there from being government hearings on the issue. (I would prefer more research on this very important issue)

The effect of abstinence programs, the impact of school choice, the wisdom of supply side tax cuts and other pertinent policy questions are all being studied by serious scholars, but unfortunately by non-serious scholars as well. With too many “experts,” all expert opinion is devalued and decisions get made on ideology not on empirical evidence. The onus is certainly on academics to make their work clear and accessible, but it is useless if there are no standards to judge research on. Perhaps academics would be better off to monitor and rate the research being used in government policy and find ways to differentiate good advice from bad. It is a Herculean task, but until it is carried out, every “expert” can be dismissed by ideologues as just one opinion.



Return to Top