Freshwater Feedback Part 1: "Everybody does it"

July 31, 2015

You can boil my claim about mathiness down to two assertions:

  1. Economist ||N|| did ||X||.
  2. ||X|| is wrong because it undermines the scientific method.

Claim 1 is a positive assertion, a statement about “what is …” Claim 2 is a normative assertion, a statement about “what ought …” As you would expect from an economist, the normative assertion in claim 2 is based on what I thought would be a shared premise: that the scientific method is a better way to determine what is true about economic activity than any alternative method, and that knowing what is true is valuable.

In conversations with economists who are sympathetic to the freshwater economists I singled out for criticism in my AEA paper on mathiness, it has become clear that freshwater economists do not share this premise. What I did not anticipate was their assertion that economists do not follow the scientific method, so it is not realistic or relevant to make normative statements of the form “we ought to behave like scientists.”

In a series of three posts that summarize what I have learned since publishing that paper, I will try to stick to positive assertions – assertions about the facts – concerning this difference between the premises that freshwater economists take for granted and the premises that I and other economists take for granted.

In my conversations, the freshwater sympathizers generally have not disagreed with my characterization of the facts in claim 1–that specific freshwater economists did ||X||. In their response, two themes recur:

In the rest of this post, I’ll address “everybody does it.” In a subsequent post, I’ll address “helping those guys.” Then in a third post, I’ll observe that in my AEA paper, I also criticized a paper by Piketty and Zucman, who are not freshwater economists. The response I heard back from them was very different from the response from the freshwater economists. In short, Piketty and Zucman disagreed with my statement that they did ||X,|| but they did not dispute my assertion that ||X|| would be wrong because it would be a violation of the scientific method.

Together, the evidence I summarize in these three posts suggests that freshwater economists differ sharply from other economists. This evidence strengthens my belief that the fundamental divide here is between the norms of political discourse and the norms of scientific discourse. Lawyers and politicians both engage in a version of the adversarial method, but they differ in another crucial way. In the suggestive terminology introduced by Jon Haidt in his book The Righteous Mind, lawyers are selfish, but politicians are groupish. What is distinctive about the freshwater economists is that their groupishness depends on a narrow definition of group that sharply separates them from all other economists. One unfortunate result of this narrow groupishness may be that the freshwater economists do not know the facts about how most economists actually behave.

In my informal conversations, no one from the freshwater camp has articulated exactly what they mean by the adversarial method, so I’ll try to fill in the blanks here. (It would be helpful if someone who supports this view indicated whether I have characterized it accurately.)

In an equilibrium where everyone follows the adversarial method, each side tries to make the best possible case for its position. What we might call the rules of evidence are that an advocate cannot make statements that are false, but it is to be expected that an advocate will withhold information that does not support the advocate’s position.

As always, it helps to be specific. One of the issues that I raised in a conversation concerned the support (in the mathematical sense) of the distribution of productivity in the Lucas-Moll model. Their assumption 1 states that at time 0, the support for this distribution is an unbounded interval of the form ||[x,\infty)||. In response to objections of the general form “this assumption is unacceptable because it means that everything that people will ever know is already known by some person at time 0,” Lucas and Moll present a “bounded” model in which the support for the distribution of productivity at time zero is ||[x,B]||. Using words, they claim that this bounded model leads to essentially the same conclusions as the model based on assumption 1. (I disagree, but let’s stipulate for now that this verbal claim is correct.) I observed that Lucas and Moll do not give the reader any verbal warning that because of other assumptions that they make, the support for the distribution of productivity jumps discontinuously back to ||[x,\infty)|| at all dates ||t>0|| so it is a bounded model in only the most tenuous sense.

When I suggested that this omission concerning the behavior of the support of the distribution was a sign of something wrong, the response I got went like this:

In presentations of the paper, members of the audience and/or discussants pointed out that this is how the support behaves. This implied that it was evident to someone who digs into the math that the support for the distribution is bounded only at date 0 and not at any other date. Because this result could be inferred from a careful examination of the math, the authors met their obligations to the reader. Because the support is bounded at date 0, they did not, technically, make a false statement when they use the word “bounded” to describe the model’s behavior. Beyond this, they should not be expected to say in words anything that would weaken their argument.

I defined mathiness as a combination of words and symbols in which the meaning of the words diverges from the meaning implied by the symbols. One of the ways they can diverge is if the words selectively disclose only some of the formal results. Calling their model “bounded” is an example of this type of divergence.

The freshwater sympathizers agreed, for example, that Lucas and Moll strategically refrained from verbal disclosures about some of the properties of the underlying mathematical formalism. Where we disagreed was whether this was a sign of behavior by the authors that is wrong. In effect, their response was caveat emptor; this is what all economists do.

For me, the most interesting thing to emerge out of these conversations was the realization that the difference between us, which seems like an intractable difference in values or beliefs about right and wrong, may simply reflect different inferences the facts. We agreed about what about what Lucas and Moll did but we disagreed about what other economists do.

One way to characterize the underlying disagreement about what is wrong is to say that we are both commenting on strategic interaction between economists that takes the form of a repeated, multi-player prisoner’s dilemma. In this game, following the scientific method corresponds to cooperation; following the adversarial method corresponds to defection. My claim is that economics is characterized by a trigger strategy/reputational equilibrium that sustains the scientific method. In calling attention to defection by the freshwater economists, I am following a strategy that is equilibrium play in this reputational equilibrium.

In contrast, the freshwater economists believe that we are already in the noncooperative adversarial equilibrium, so it is wrong to express disapproval of economists who are simply engaging in the type of behavior that is rational in that equilibrium. The freshwater economists might agree that in some first-best sense, an equilibrium based on the scientific method would be preferable, but they apparently believe that we are not in such an equilibrium; that it is not possible to get back to such an equilibrium; and that even if we did, it would not be possible to sustain it.

If so, what looks superficially like a deep and intractable disagreement about values or morality may simply reflect disagreement over the facts about what most economists do. When the freshwater types say “everybody is following the adversarial method,” what they may honestly be saying is that “everybody I know is following the adversarial method and they all believe that everyone else is doing this too.” But because freshwater economists have so sharply isolated themselves from the rest of the profession, they may be generalizing from an unrepresentative set of observations.

To me, the facts seem to be that freshwater economists are following a coordinated strategy based on the adversarial method whilst most other economists remain committed to the clarity of the scientific method.

If I could persuade the thoughtful freshwater types of just one point, it would be that they owe it to themselves to assess the factual basis for their belief that all economists follow the adversarial method instead of the scientific method. Everyone they talk to may honestly believe this, but a bit of empirical inquiry may reveal that this is a fact about whom they talk to, not a fact about the behavior of most economists.