Skip to content

The devil’s in the details…and also in the broad strokes. Is this study ridiculous, or am I badly misjudging it?

This post is by Phil Price, not Andrew.

Something caught my eye in a recent MIT Technology Review: an article in Nature Communications entitled ‘The greenhouse gas impacts of converting food production in England and Wales to organic methods.’ This is a subject that interests me, although I have no expertise in it whatsoever, so I clicked through and read it and became increasingly baffled as I worked my way through. On the one hand, as I said I have no expertise so who am I to say that they’re missing something huge? On the other hand, they are obviously missing something huge.

Here’s what they say (the second half of the abstract): “Here we assess the consequences for net GHG emissions of a 100% shift to organic food production in England and Wales using life-cycle assessment. We predict major shortfalls in production of most agricultural products against a conventional baseline. Direct GHG emissions are reduced with organic farming, but when increased overseas land use to compensate for shortfalls in domestic supply are factored in, net emissions are greater. Enhanced soil carbon sequestration could offset only a small part of the higher overseas emissions.”

Certainly believable from what I know. I would also find it believable that organic farming is about the same, or is somewhat better than conventional by this measure. No idea. Like I said: not an expert. But sure, yields per acre are probably going to be lower — you’ll lose more to pests and fungus and such — so you’ll need more acres if you want to grow the same amount of the same foods. Of course you will shift your production from some foods to others, but it’s not hard to understand the mechanism by which you’d need to clear more forests to have farmland, or something other change that would be net negative from a carbon standpoint.

Anyway, I started reading through the article and, as I said, grew increasingly baffled. My bafflement was focused initially on one thing: the lack of discussion of the price. The words ‘price’ and ‘cost’ came up only when discussing how to allocate a given amount of carbon emissions among the different components that go into producing food, and not (as far as I can tell) into any model of what foods will be produced. This seems crazy. With organic farming it costs more to produce the same amount of food, which is why most farming is non-organic: if organic were cheaper why would anyone use conventional methods, especially since people are willing to pay more for organics? I’m not an economist and it’s only been a week since I posted something that chided economists for tending to believe too deeply in my own theories, but there it’s not like they’re wrong about everything and one thing they’re right about is that if something gets more expensive people will buy less of it.  If, under an all-organic regime, meat triples in price and all other food doubles in price, people are going to (a) eat less food, (b) waste less food, and (c) eat a lot less meat. How can you ignore this?

But I thought that maybe I just didn’t understand how their model works, maybe there’s some way they implicitly take price into account and therefore don’t need to model price directly. At any rate I kept reading and got to their Methods section, where they explain what they did. Here are a few points:
1. “The Objective Function of the model, which is maximised subject to constraints on resource availabilities, is the sum of total crop and livestock production, expressed as ME [Metabolizable Energy, i.e. Calories].” This is a weighted sum over agricultural products and ‘rain classes’ of the “fresh weight per unit crop area or livestock number per year.” The assumption is that this quantity will be maximized in an all-organic regime.
3. The doozy: “In each farm type, the set of crop and livestock production activities available are fixed, as evidence suggests that the dominant agricultural activity (e.g., dairy farming) will usually stay in place post conversion to organic management, due to existing farm infrastructure, farming knowledge and local conditions.”
4. “The land areas under each farm type are fixed, reflecting the areal coverage of their conventional equivalents recorded in the June Survey of Agriculture in 2010”.

So, combining 3 and 4: If raising livestock becomes half as profitable per acre (or even becomes completely unprofitable), doesn’t matter, we’ll still have the same number of acres in livestock. If raising livestock becomes twice as profitable per acre, same thing, no change in acreage. According to #3 (as I understand it) you’re able to switch between types of livestock — raise fewer sheep and more cows — but if an acre is in livestock now, it’s going to be in livestock in an all-organic world too, no matter what.

I can imagine something like that in the face of small changes in agricultural practices, like the relatively small amount of acreage that has changed to organic production over the past twenty years. But they are talking about an agricultural regime that they estimate to generate “a drop in total food production expressed as metabolisable energy (ME) by of the order of 40% compared to the conventional farming baseline.” How could this possibly be close enough to the truth to be a useful model? Perhaps there’s an implicit assumption that the price of food won’t change much because imported food won’t change much in price, so people won’t switch their dietary habits? But if that’s true, won’t people simply switch almost completely to imported food? If English (and Welsh!) farms switch to organic production, but foreign farms do not, then English/Welsh farms will go out of business as people spend their food dollars on the cheaper (foreign non-organic) competition. The model does not allow that: assumptions 3 and 4 guarantee that the same acreage will be farmed for the same purposes, no matter how unprofitable.

Or perhaps non-organic foreign food will not be allowed for import. Then all food will get more expensive, some foods more than others. The study doesn’t appear to look at that, but we can imagine: food would get a lot scarcer and a lot more expensive.  People would eat less of it, and there would be some switching from relatively expensive foods to cheaper ones. Surely this would render assumptions 3 and 4 ridiculous? It takes a whole lot of ‘metabolisable energy” to raise a cow or even a sheep. If people eat half as much meat — hey, it’ll be a whole lot more expensive, they’ll certainly reduce their consumption — and that land is switched to producing plants for human conception then conceivably the amount of ‘metabolisable energy’ available for human consumption wouldn’t go down at all. But not only does the model not predict that (I’m not saying it should!) it doesn’t even allow that shift to take place.

To me this whole exercise seems like an example of a fallacy Andrew has discussed before, the ‘all else equal’ fallacy (which he has probably assigned a cute name). You’ll see an article that says something like (for example) walking to work costs about the same as driving, because the average commute is 10 miles and to walk 10 miles takes 1100 Calories and that costs about $3 or whatever, and of course that’s ridiculous because if everybody actually walked to work there’s no way the average round-trip commute would be 20 miles (I just made up the numbers and the example). It’s not that these comparisons are uninformative, indeed they can be quite informative and thought-provoking, it’s just that you can’t take them seriously as predictions of what would happen in the counterfactual world that they envision.

Similarly, I think there may be interesting stuff to be learned from this study about switching to organic farming, but one thing I don’t think you can learn is how much ‘metabolisable energy’ would be produced in England and Wales if they switched entirely to organic food production. The assumptions seem completely unreasonable to me.

And yet here it is in Nature Communications. This not only seemed reasonable to the authors, it seemed reasonable to the editor and the reviewers. So probably it should seem reasonable to me, too. But it doesn’t.  Can someone enlighten me? How can it make sense to envision this huge change in food production without even a nod to how it would change prices and therefore people’s consumption choices, which would in turn change farmers’ decisions about what foods to produce?

This post is by Phil, not Andrew.





  1. Alex B says:

    I love watching Steve Jobs on YouTube, but did you put in the right link to the article?

  2. Phil, Phil, Phil… if people wanted to have reasonable and accurate models of a process, they wouldn’t be allowed to publish them in Nature !!! :-)

  3. Tom says:

    Just playing devil’s advocate, if you actually modeled the switch to organics, wouldn’t the groundswell opposition to tripling food prices and bankrupting livestock farms prevent you from ever getting to the new equilibrium of “all organics”? (It’s not like a dictator hasn’t actually run such a supply shock experiment before, perhaps Zimbabwe? Cambodia?) So any answer doesn’t really make sense.

    I agree that adjusting across multiple margins makes more sense for small changes (perhaps for a non-ruinous carbon tax), but the “all else equal” kind of makes sense here. If the policy question is JUST switching to organics without ripple effects (i.e., skipping over the transition and assuming enough supply to get back to roughly the same prices), then this is the answer. If your policy is actually switch to organics, collapse supply, drive up prices, and drive those livestock ranchers out of business to get carbon savings, then say that.

    • Phil says:

      Tom, your point seems similar to Mathijs’ (below): if you’re going to posit a switch to organic agriculture, well, how are you going to achieve that, and doesn’t it matter?

      This does come down to what it does and doesn’t make sense to hold constant in an ‘all else equal’ scenario. To me it seems nutty to hold the mix of agricultural land use constant in the face of what would be an enormous reduction in in-country food production, because I think such a reduction would be accompanied by a large change in land use and in the mix of foods that are grown. But it seems less crazy to pretend that yeah, OK, we can switch to organic agriculture, and yes, prices and land uses will change, but let’s pretend we can get the electorate to approve that would make this happen even though we probably can’t. Why does one seem nutty to me and the other not? I’ll have to think about that. One thing does spring to mind immediately: in the latter scenario we don’t have to pretend anything about the current political climate remains fixed, we just have to envision a plausible political path that leads to organic agriculture. I can do that. Whereas I can’t envision a plausible path to a switch to organics that doesn’t involve changes in how the land is used. Maybe that’s just poor imagination on my part, I dunno.

      • Tom says:

        1) you may disagree on the organic farming inefficiency
        2) they answered the question “is this policy easy and effective?”
        2) they never specified the question of interest or ended up answering a different question than they stated.

        Your post acknowledges that organic farming is a less efficient food production technology (hence the price and land use changes). Therefore the GHG savings must outweigh the costs to using an inefficient technology.

        Let’s assume organic farming is actually fairly efficient relative to non-organic farming but at dramatically reduced GHG emissions, and the overseas production easily compensates for the shortfall without appreciably changing total GHG emissions. That results would suggest that organic farming is an “easy” fix and we should sharpen our pencils to see how it all plays out.

        In the paper (based on your description), the shortfalls are dramatic. Organic farming is so inefficient that you would either need to enlist a large amount of foreign cropland and import food (otheir solution) or vary multiple margins (your solution) with a dramatic change in prices, choices, and lifestyle. Therefore, it is not an “easy” fix (no savings with no behavioral adjustments or massive adjustments with potential savings). You could do the hard work to figure out all the pain and tradeoffs in such a multiple margin adjustment regime but is it worth it for a policy that is not viable?

        You want the paper to answer the question of “what would happen?” but it seems more like a first pass at “is this policy easy?” Lakeland mentioned you were a physicist. This is more like a back of the envelope calculation on whether a solution is viable rather than the final analysis. Perhaps it was not billed as such (like others, I’m going by your description), which is a shame that they overpromised but that does not seem to invalidate this type of analysis.

  4. I don’t understand what you’re so upset about. The paper seems pretty clear in its aim of assessing, if all else were equal, what the difference in greenhouse gas emissions would be if the same land, etc., were used for organic versus conventional agriculture. Obviously if all else isn’t equal this won’t be a *prediction* of what all the effects will be, but it’s neither misleading nor inept to just consider the effect of change along one axis. It’s like taking a partial derivative, rather than a full derivative — often very useful. As an added bonus, you can consider partial derivatives along multiple variables, such cost if you like, and combine them make a fuller picture.

    • In order for a partial derivative to be meaningful you should have a small change in the associated variable. So if you want to switch say 5% of ag to organic, you can figure out what might happen using some kind of partial derivative…

      But if you take a partial derivative of say demand of icecream vs temperature when the outdoor temperature is say -20C and then argue that it’s essentially zero, so that when the temperature is 20C you’ll sell the same amount of icecream… meh…

      I think Phils point is mostly like that… who cares what you get when you assume you can adjust all production to organic and nothing else changes, it doesn’t describe anything close to real, might as well assume you can convert organic to be 1/2 as costly to produce too, and that there’s nuclear fusion reactors making all our electricity, and also that every good girl gets a pony.

      • Phil says:

        Yes to what Daniel said.

        I agree, Raghuveer, that the paper is clear in its assumptions. I just think those assumptions are totally unrealistic, so what’s the point?

        • I haven’t looked at the paper carefully enough to know this, nor do intend to, but I wonder if the model is (roughly) linear, in which case Daniel’s objection that partial derivatives are relevant only for small changes doesn’t matter. In other words, one could interpret the results as saying “here’s what happens if all production shifts to organic, all other things being equal; if 5% shifts to organic, you’ll get 5% of this change in greenhouse gas emission.” I wouldn’t structure the study this way — directly considering small deviations along multiple axes makes more sense — but I still think there’s value in what the authors have shown.

          Or maybe I’m biased to accentuate positive features where I can find them, since most of the papers I read these days are depressingly awful…

          • Phil says:

            I would be a lot less critical of a paper that took the sort of approach you propose. You could look at the very modest changes to organic farming that have taken place thus far, and extrapolate a bit: what if the rate of changeover increases by such-and-such, what would we expect to see.

            And I don’t want to discourage people from trying to do the “whole agricultural economy” calculation, I don’t think that’s inherently useless, not at all. I just don’t see how you can come anywhere near the right answer if you don’t allow for land use change. But here it is in Nature!

  5. Jackson Monroe says:

    They are assuming it seems (from a brief look) that a market with all organic products would clear, and so everything produced could be sold somehow.

    “But if that’s true, won’t people simply switch almost completely to imported food? If English (and Welsh!) farms switch to organic production, but foreign farms do not, then English/Welsh farms will go out of business as people spend their food dollars on the cheaper (foreign non-organic) competition.”

    This assumption is not allowed by the paper, and while possibly true misses the point as I understand it. The point is when you maximize the amount of food you make under an organic regime you still raise greenhouse emissions, it is taken for granted that the food is sold and eaten, because the paper is trying to be generous. “Even if we all switched to organic and ate that better food, the shortfall would induce greater greenhouse emissions elsewhere, so you still made us worse off.”

    I didn’t read too closely, but I think when you accept that they are being generous to the “other side” and making assumptions in their favor you can see the point they’re after.

    • Phil says:

      It’s not that I think there’s no point to a study like this. In fact, I said so in my penultimate paragraph: “Similarly, I think there may be interesting stuff to be learned from this study about switching to organic farming…” But I also said “…one thing I don’t think you can learn is how much ‘metabolisable energy’ would be produced in England and Wales if they switched entirely to organic food production”, which is supposedly what the study is about.

  6. Mathijs Janssen says:

    Let me make the nitpick observation: there is nothing in economic theory that suggests that, if the price of a good goes up, you will buy less if it. In principle, you could buy more, the same or less. Google “Giffen good” for a discussion. Obviously, economic experience suggests it’s typically less…

    That said, some model of production response to the intervention seems very useful. Not easy though. For one, you would have to specify how the change from non-organic to organic production would be implemented. Are there subsidies for organic production, so that farmers switch voluntarily? Is there a ban on non-organic production, but are farmers still allowed to choose what and whether to produce? Or are farmers simply forced to produce the same goods as before, but organically. The price implications will be very different under the three scenarios.

    Having specified the intervention, the hard work starts. It will involve estimating a lot of demand elasticities and then extrapolating them well outside of the range in which they have been observed. Very messy stuff.

    A much simpler thing to do, and more in the style of the original article (which I have not read), is to do an optimal production exercise: keeping the total output (measured in Metabolizable Energy) fixed at the current level, so that everyone can eat as much (energy) as before, minimize the GHG emission by choosing the production methods (i.e. the best mix between organic/non-organic and between crop/livestock). If the optimal production does not reduce GHG by much, or it does not involve a significant switch towards organic production, that would be an argument against organic production.

    • Phil says:

      Mathijs, I know about Giffen Goods. I also know that an economist once told me “there has been only one example of a Giffen Good in all of human history, and that example is faulty.” He was talking about potatoes in the Great Potato Famine. And he was exaggerating. But it is clear that examples, if they are to be found, are extremely rare. It’s hard to see how they’d be relevant to this topic.

      Your last paragraph would, I agree, be way better than what the authors actually did. Go ahead and write it up! Publish in Nature!

  7. D Kane says:

    > Is this study ridiculous, or am I badly misjudging it?

    Depends. I suspect you are judging it on the basis of your ideology.

    This study produces a result you don’t like, so you criticize it. The vast majority of studies on climate change which conclude that global warming will be hugely dangerous/costly use exactly the same methodology. (Assume that X changes a lot and assume that lots of other variables — which are highly likely to respond to big changes in X — don’t change at all.) You don’t criticize those studies — or at least you haven’t here or perhaps I missed your criticisms — because you agree with their conclusions/ideology.

    I think all these studies (both this one and the world-will-end global warming ones) are ridiculous, mostly for the reasons you provide in this post.

    • I actually think if you asked Phil to read a study like that on GW he’d probably criticize it in the same way… But Phil’s a physicist and so to the extent he’s read GW stuff it’s probably mostly been about physical questions like how much heat will be trapped in the ocean etc, not what the ecological / economic / land use consequences will be.

      Phil is famous for stirring up a big pot of hornets about housing prices, my impression is Phil knows enough economics to ridicule economic analysis from people who don’t even take into account basic ideas in economics, as he did here. So I don’t think your characterization of Phil is in any way accurate.

      To the extent that Phil doesn’t attack that kind of GW consequences study I suspect it’s because he doesn’t waste his time reading them in the first place.

      • bop says:

        It’s pretty clear at this point that Phil does not understand basic econ 101.

        • Phil says:

          Dude, I’m the one saying they completely ignored economics in this article!

          • bop says:

            Okay, I actually do think you understand economics 101, but you seem to think you have graduate-level knowledge. You just wrote a fairly long blog post that basically boiled down to two words: general equilibrium. I’m not sure a lengthy blog was necessary. The authors of the article were not accounting for any equilibrium effects, that simple. To me it seems like you write these blogs as though you have discovered a new concept but in actuality it is contained in most introductory economics classes. Plus the SF housing price debacle was just unforgivable, the way you shifted goal posts in the comment sections and refused to admit the basic supply/demand mishap you made in the original post.

            • Andrew says:


              I’d like to make a comment, not on Phil’s post or this particular discussion, but on the more general point of “economics 101” or “economics 201” or whatever. Yes, equilibrium is a natural idea, but it’s an idea that real-world econ Ph.D.’s often seem to miss. See my posts from a few years ago on the “all else equal” fallacy (which Phil refers to in his post but did not link to).

              To put it another way: we can’t trust professional economists to get this issue right. We’re on our own. Phil did not discover a new concept (nor did he claim that he did!). If this is indeed covered in most introductory economics classes, then maybe someone should send Steven Levitt to spend a sabbatical taking Economics 101 and taking careful notes.

            • Phil says:

              Bop, it’s a pity if you have a knee-jerk anti-Phil reaction to everything I write, but I understand that sort of thing sometimes happens and I have it a lot better than some. (But her emails! Benghazi Benghazi Benghazi!).

              Not only did I refuse to admit the basic supply/demand mishap I made in the original post, I still deny it. I said in the original post that if you build more housing the price of housing will, on average, go down. My numerous critics say that I got it wrong because I don’t necessarily agree that the price of housing will go down near where you built the housing, I think that in certain market conditions the price goes up there and down elsewhere (specifically, in the places the rich people vacate when they move to the new place). I still don’t understand how people (such as you) can be so sure I’m wrong, absent a model that captures something of the spatial variability in housing costs and the reasons for that variability. I’d even take a 2-box model, ‘city’ and ‘suburbs’ if that’s what you’re offering. But just to say ‘supply and demand!’, that doesn’t explain anything. Housing prices differ by location, and buyers and renters do not consider different locations perfectly interchangeable: you know what ‘they’ say about housing prices and location! The conceptual model I was plugging, in which (under certain conditions) wealthy people move to an area and bring their disposable income with them, thus increasing economic activity and making property values rise there, is one that I know is believed by some house-flippers, business owners, and renters who are afraid of being priced out when the rich people move in. I believe it too. I didn’t realize when I posted that economists don’t think it’s right!

              I did make a mistake in that post, and it’s one I regret: I wrote with a lot more certainty than I should have. It sounds like you think I did that (and other bad things) in the comments, not just the post, and I’d have to go back and look to see if that’s true, maybe it is. So I’ll accept the criticism of acting like I’m an expert when I’m not. And for that, I apologize. But I’m not gonna grovel.

    • Anoneuoid says:

      The vast majority of studies on climate change which conclude that global warming will be hugely dangerous/costly

      I haven’t seen this. The “threat” they are concerned about is less than average. The real problem (from what I’ve seen) is the models vastly underestimate the degree of “natural” climate change.

      For example:

      Since at least the start of the 20th century, the average global sea level has been rising. Between 1900 and 2016, the sea level rose by 16–21 cm (6.3–8.3 in).[2] More precise data gathered from satellite radar measurements reveal an accelerating rise of 7.5 cm (3.0 in) from 1993 to 2017,[3]:1554 which is a trend of roughly 30 cm (12 in) per century.


      For example, in 2007 the Intergovernmental Panel on Climate Change (IPCC) projected a high end estimate of 60 cm (2 ft) through 2099,[6] but their 2014 report raised the high-end estimate to about 90 cm (3 ft).[7] A number of later studies have concluded that a global sea level rise of 200 to 270 cm (6.6 to 8.9 ft) this century is “physically plausible”


      Since the last glacial maximum about 20,000 years ago, the sea level has risen by more than 125 metres (410 ft), with rates varying from less than a mm/year to 40+ mm/year, as a result of melting ice sheets over Canada and Eurasia.

      Basically the average rate of sea level rise since the last glacial max has been ~6 mm/year, and going as high as 40 mm/year for 500 out of 20k years (2.5%). So the average is 2x higher than current, and about 75% of the max predicted by the IPCC. If we take the max in the literature, it is about 75% of the historical max.

      And we should also remember that last century levees were raised ~5-6 meters all along the Mississippi river during a time when the US became the most powerful and wealthy country in known human history.

      This stuff isn’t even a greater than usual threat.

      • Anoneuoid says:

        Found my earlier post a source on the levees:

        That link is down now, but here is another source on the history. On page 68 you see 1882-1972 the levees were raised from 9-30.5 feet (2.7-9.2 m) or 6.5 meters:

        Backup screenshot:

        So such engineering projects over the course of a century are hardly unprecedented.

        • Phil says:

          About fifteen years ago I did a very small consulting project for an organization that is mostly funded by a consortium of insurance companies. Not relevant but just so you don’t wonder about it: Most of the companies had experienced recent weather-related losses that were historically unusual, and they wanted to know if they were seeing a signal of climate change. They gave me some data to look at. My answer was that most of what had happened was a very large increase in property that was insured against weather-related damage and was in places where such damage was likely. I said if they wanted to know whether extreme storm events were getting more likely or bigger they should look at the weather data, rather than the extremely noisy filter of insurance payouts, which are subject to all sorts of additional variation.

          Anyway, as part of that little project I read some reports and data sheets from insurance companies. One of them was a ‘letter from the President’ (or the CEO, or the Chairman of the Board, I don’t remember)…I think it was from Munich Re, a large reinsurance company. The gist of the letter was: as individuals, both employees and staff of Munich Re might have different preferences for what climate they want, and might or might not be worried about climate change. But the world’s infrastructure is highly optimized towards the climate we have had for the past few hundred years: our ports and all of our waterfront houses and industry is built for the current sea level, our dams are built where the rivers flow, our farm towns are built where the climate is good for agriculture. Any significant change is going to cause substantial economic damage, not because the new climate will necessarily worse but simply because it will be different: if you are optimized for current conditions and the conditions change, you are no longer optimized. I have the words wrong but that was the point.

          Yes, sure, there will be plenty of places where we can just build a few thousand miles of seawalls or levees or something else that is not extremely difficult or expensive. But there will be plenty of places where we can’t. For example, Miami is built on limestone; you can build all the seawalls and levees you want, the water will come right up out of the ground. Florida has a big problem. Of course, it’s nothing compared to Bangladesh.

          Basically I think you’re too sanguine about this. We are (collectively) in for a world of hurt.

          • Anoneuoid says:

            How am I too sanguine? I expect more extreme climate change than the IPCC just based on the historical rates. As my post said, the sea level rose at 1/3 to 1/2 the average rate over the last century. We should at least expect a reversion to the mean.

            Luckily we have access to better technology and cheap energy so it shouldnt be too hard to do the equivalent work of 2k km of 6 m tall levees over the course of a century. But what if the opposite happens first and we get 50 years of cold dry climate. That would be even worse.

            I think if anyone ever did a real cost benefit on this climate change + other risks like solar flares, asteroids, financial collapse, nuclear war, etc the conclusion would be to first do things that will help in general. Ie, even in the bible they knew to stockpile food, etc to prepare for hard times.

            • Anoneuoid says:

              A major phase of deglaciation from ∼16.5–7 ka BP. The total esl change in this interval is ∼120 m
              A high rate of sea-level rise starting at ∼14.5 ka BP of ∼500 y duration… the globally averaged rise in sea level of ∼20 m occurs at a rate of ∼40 mm⋅y−1 or greater.


              So half the time since the LGM has been spent in a state where sea levels were rising at 12.5 mm/yr, and 2.5% of the time at 40 mm/yr. That is the type of stuff to prepare for when it comes to sea level, not a measly 6-9 mm/yr (which are the *high* IPCC estimates).

              So I don’t know what wrong with their models but it seems they exclude these possibilities. Really… something that happens 50% of the time since the LGM is now outside the upper range according to their models? Something that happened 2.5% of the time is physically impossible?

              Let’s act on the assumption there is a 1/40 chance of 500 years of 40 mm/yr rise. That is 4 m/century and 20 m total, starting this century. What is the plan then?

              • jim says:

                Sea level is an extremely complex phenomenon. There is no justification for anticipating a reversion to the mean of any particular period.

                The strongest mid-term (~10Kyear – 1000k year scale) controls on sea level are the Milankovitch orbital cycles (eccentricity, axial tilt and procession), which operate independently of one another and have periodicities up to ~400K years. The most rapid sea level rise will occur when these cycles are aligned at maxima or minima, but the pattern is complex because of the multiple cycles of different periodicity.

                Milankovitch cycles are amplified / dampened by CO2, albedo and other feedbacks, so as the orbital cycles change sea level change may lag – first change more slowly than while feedback dampens change, then accelerate as the feedback signal shifts, then decelerate as the impacts of the feedbacks taper off.

                We haven’t even gotten to tectonics yet which, at the moment, is the dominant factor in local sea level outside of tides. In the Gulf of Alaska, local sea level is falling steeply, while further south in Techie land it’s roughly stable, so we won’t be building any sea walls.

              • Anoneuoid says:

                Sea level is an extremely complex phenomenon. There is no justification for anticipating a reversion to the mean of any particular period.

                I don’t follow your argument. If something is complex, we cannot use simple arguments like “things that happened before can happen again, let us prepare for it to the degree feasible”?

                The strongest mid-term (~10Kyear – 1000k year scale) controls on sea level are the Milankovitch orbital cycles

                The LGM was only ~20k years ago. Do you know what caused the 500 years of 40 mm/yr rise 14.5k years ago? Basically can you explain what we see in figure 1 here:

                I doubt you can with Milankovitch cycles…

              • jim says:

                At first glance of your paper I notice the glaring lack of error bars on all the results in the abstract. I probably wouldn’t want them in there either. The period of the highest rate of sea level rise not surprisingly has relatively few observations which all seem to have very large error bars.

                It’s also interesting that there is no error at all given for the ages. I’d be interested to know what’s up with that. New dating techniques coming out all the time but no error at 20ka? Seems a bit hopeful.

              • Anoneuoid says:

                At first glance of your paper I notice the glaring lack of error bars on all the results in the abstract. I probably wouldn’t want them in there either. The period of the highest rate of sea level rise not surprisingly has relatively few observations which all seem to have very large error bars.

                It’s also interesting that there is no error at all given for the ages. I’d be interested to know what’s up with that. New dating techniques coming out all the time but no error at 20ka? Seems a bit hopeful.

                Well, do you have a better paper? Do you believe the sea level was 120 m lower 20k years ago? Do you believe in the LGM?

                I am open to all of those things being some sort of artifact, but if you want to start questioning stuff I need to know how far you are willing to go. That paper was just the one linked to by the wikipedia page on sea level rise.

            • Anoneuoid says:

              Then there is this:

              If you sum the sunspot number (which is roughly the number of sunspots) for each solar cycle, you see cycles 17-23 (~1933 – 2009) were more active than average. The last cycle (# 24) was the least active since cycle 6 (1810 – 1822), and the next is projected to be half of that:

              The forecast for the next solar cycle says it will be the weakest of the last 200 years. The maximum of this next cycle – measured in terms of sunspot number, a standard measure of solar activity level – could be 30 to 50% lower than the most recent one. The results show that the next cycle will start in 2020 and reach its maximum in 2025.


              So basically the next solar cycle is supposed to be the least active since decent records began (which was right after the Maunder Minimum). What does it mean? No one knows.

              Also we seem to be at crucial point in a ~5300 year cycle seen in relative carbon-14 levels, which are also the lowest since the LGM:

              Others have noticed the same (in older + other data) and called it a 7k + 3.5k yr cycle (which averages to ~5.3k):

              Periodicities of several kyr are intriguingly close to the pacing of climate oscillations recorded in Greenland ice and North Atlantic sediment records [Johnsen et al., 1992; Bond and Lotti, 1995]. However, in view of the good fit to the geomagnetic data (particularly in times of low field/high production) the most likely explanation for the 36-Cl fluctuations is probably that they too represent production cycles driven by 3500- year geomagnetic oscillations. A generally weaker 7000- year geomagnetic component may also be present, though the evidence for this is slight.


              Once again, no one knows what this means… but as you can see I am very concerned about climate change. But not really about CO2 emissions.

      • JimV says:

        As I understand it the last several interglacial periods have lasted about 20,000 years and the Earth is at about 10,000 years since the the start of this one. So if I understand correctly, absent AGW, sea level rise should have leveled off as we approach the next ice age. In other words, we are in the process of melting all the residual ice which normally lasts between ice ages. Dr. Edward Teller (of hydrogen bomb fame or infamy) told the U.S. Petroleum the following in 1959:

        “At present the carbon dioxide in the atmosphere has risen by 2 per cent over normal. By 1970, it will be perhaps 4 per cent, by 1980, 8 per cent, by 1990, 16 per cent [about 360 parts per million, by Teller’s accounting], if we keep on with our exponential rise in the use of purely conventional fuels. By that time, there will be a serious additional impediment for the radiation leaving the earth. Our planet will get a little warmer. It is hard to say whether it will be 2 degrees Fahrenheit or only one or 5.

        But when the temperature does rise by a few degrees over the whole globe, there is a possibility that the icecaps will start melting and the level of the oceans will begin to rise. Well, I don’t know whether they will cover the Empire State Building or not, but anyone can calculate it by looking at the map and noting that the icecaps over Greenland and over Antarctica are perhaps five thousand feet thick.”

        There are many other problems which AGW may cause, such as the famine in Syria which fostered instability in the region. There are many things which could be done to solve or prevent such problems, but too many people seem to have a laissez-faire attitude for the work to begin in seriousness.

        • Anoneuoid says:

          Yes, there are many reasons to think it is about to get cold in the next few centuries, possibly much sooner. You won’t find a single public study on a climate model used to study this possibility. I’ve asked before and no one could find one for me.

          I’ve met so many people who claim to be concerned about climate change but have not done a single thing to prepare for it. Some even ridicule those who do prepare… Those are the people with a laissez-faire attitude.

          • Phil says:

            Back in the seventies some scientists talked seriously about the coming ice age. The reason you don’t see that stuff now isn’t that it’s suppressed in a nefarious way, it’s just that nobody credible believes it anymore: greenhouse gas forcing is high enough that it is overwhelming any small amount of cooling that might otherwise be happening.

            As for preparing for climate change, I think most people don’t know what they can do now that will prepare, or at least not consciously. But I know lots of people who are already “adapting”/reacting.

            • Anoneuoid says:

              greenhouse gas forcing is high enough that it is overwhelming any small amount of cooling that might otherwise be happening.

              This is false, there is no opposition between two forcings.

              The 1970s ice age concern was due to CO2 induced warming as well. The difference was the feedbacks in the models and that the current temperature trend was down. Back then they figured warmer air over poles -> more precipitation -> more ice -> more albedo.

              There is a documentary from the time where they explain this quite clearly on YouTube. Pretty sure it is this one:

              There is nothing fundamentally wrong with this mechanism, it is just the trend switched from lower to higher temperatures. You will see that mechanism return when temperatures start trending down for some reason.

    • Phil says:

      D Kane,
      You say “This study produces a result you don’t like, so you criticize it.” I do not in fact criticize publications just because I don’t like the results they produce. It’s notoriously hard to resist confirmation bias but I think I do a lot better than average on this score.

      In this case, I am very willing to believe that a switch to organic farming would have negative climate change implications. I do think that’s unfortunate — give me spots on my apples, but leave me the birds and the bees — but if I were forced to bet right now I’d say that most ways we could switch to organics would have negative climate change implications. It’s just hard to overcome the land use penalty that comes with lower yields per acre. Most mechanisms I can imagine for getting the whole world to switch to organic agriculture would necessarily lead to converting even more land to agriculture than we already have. If I had to bet, I’d bet that switching to organics would somewhat increase global warming.

      That said, a little basic math shows that we could make up the agricultural land deficit by eating a lot less meat — worldwide, about 60% of agricultural land is used to grow food for livestock, and humans and our livestock account for more than 95% of all mammal biomass. But unless there’s a causal mechanism by which switching to organics leads to lower meat consumption, this isn’t relevant. We could continue using conventional agriculture, eat a lot less meat, and free up even more land from agriculture than in the organics-with-low-meat consumption case.

      In any case I think you wrong me by saying I’m judging on the basis of ideology. I’m judging on the basis that I think two of the key assumptions are stupid.

      • D Kane says:

        Perhaps I am being uncharitable. But then you give us nonsense like this:

        > For example, Miami is built on limestone; you can build all the seawalls and levees you want, the water will come right up out of the ground. Florida has a big problem.

        So, if this reality, you would predict that real estate prices in Miami specifically (but also in any coastal, low-lying region) would be trending down, both over the last few decades, but especially in the last 10 years, as the (unavoidable) dangers of climate change become universally accepted. Right?

        And, yet, the exact opposite has happened. Miami (and Florida coastal real estate in general) has been an amazing investment for the last decade.

        If it is so obvious that “Florida has a big problem” then why don’t the thousands of (rich, educated, sophisticated) buyers of Florida coastal property not see it? Are all of them stupid?

        • Phil says:

          D, maybe that depends on your definition of ‘stupid.’ There are people who are generally pretty smart but who are in denial about sea level rise. What I said about Florida isn’t remotely controversial. I wouldn’t call you ‘stupid’ if you don’t accept it, but your position doesn’t seem tenable.

          This article says some coastal properties are being hurt more than others. This one says properties in Miami-Dade County that are subject to flooding have been appreciating in price, but at a substantially slower rate than those that aren’t.

          Property website Zillow says“…And in some states, the fraction of properties at risk of being underwater is alarmingly high. More than 1 in 8 properties in Florida are in an area expected to be underwater if sea levels rise by six feet, representing more than $400 billion dollars in current housing value. ” (Six feet is going to take a while, though).

          The Miami Herald just ran an article saying even the state GOP accepts that the sea level is rising and that this will have major negative effects. It notes that “King tides and sunny-day flooding are disrupting postal delivery in many communities, eroding utility boxes, requiring law enforcement to manage traffic corridors where flooding has closed roads, [Jennifer Jurado, chief resilience officer for Broward County], said.”

          But if you’re only going to read one thing, read this article on flooding in Miami.

          • D Kane says:

            From the article you recommend:

            And just about everyone who can afford to buys near the water. Not long ago, Kenneth Griffin, a hedge-fund billionaire, bought a penthouse in Miami Beach for sixty million dollars, the highest amount ever paid for a single-family residence in Miami-Dade County (and ten million dollars more than the original asking price). The penthouse, in a new building called Faena House, offers eight bedrooms and a seventy-foot rooftop pool. When I read about the sale, I plugged the building’s address into a handy program called the Sea Level Rise Toolbox, created by students and professors at Florida International University. According to the program, with a little more than one foot of rise the roads around the building will frequently flood. With two feet, most of the streets will be underwater, and with three it seems that, if Faena House is still habitable, it will be accessible only by boat.

            Since the time Griffin bought, the market value of that penthouse, and residences like it, is probably up around 30%.

            For how many years will housing prices in Florida need to go up, especially housing prices on the beach, before you would admit that Florida does not “have a big problem?”

            If there is no amount of house appreciation that will cause you to change your views, then you are an ideologue, not a scientist.

            I, on the other hand, view prices changes — especially aggregated over thousands of buyers, over a decade or more — as an excellent sign of whether or not geographic region X has a “problem.” If prices of coastal real estate fell a lot, then that would cause me to change my views.

            • Phil says:

              D Kane, boy, I hardly know what to say. I thought believing in rational markets went out of fashion in 1929. Or maybe it was 2008.

              There are evidently plenty of people who are willing to ignore the science on sea level rise. That’s all it takes for prices to keep going up. Buying Miami real estate is like buying Bear Stearns in 2007: it could work out, just make sure you get out in time.

              • D Kane says:

                > Buying Miami real estate is like buying Bear Stearns in 2007: it could work out, just make sure you get out in time.

                If Miami real estate prices are higher in five years (or 10 or 20 or 50 years), will you admit you are wrong?

              • Phil says:

                I’m sure you know the saying “the market can stay irrational longer than you can stay solvent.” If I thought I knew when market corrections would happen, I’d either be a lot richer or a lot poorer than I am (depending on whether I was right or wrong.) I would not be surprised if the Miami waterfront market crashes within five years, but also wouldn’t bet on it. Ten years feels better than fifty fifty, and twenty feels like a lock. Yeah, no way does this take twenty years.

                It feels pretty unsatisfying to have to wait fifteen years or so to settle this. On the other hand I think it’s unlikely it will take that long. Maybe we can find something shorter term that doesn’t rely on an assumption of rational markets? A bet based on frequency of flooding, or dollar cost of (flooding plus flood mitigation), something like that? In all of these things I am open to a wager if we find the right one. I think there are websites that facilitate, if they haven’t been shut down. And when I say a wager I mean more than “I was wrong about Miami bring in trouble due to flooding”, which I’m very willing to say if I am in fact wrong, but we could put some money on it. Of course, if you’re so sure you should invest in Miami real estate, that would be a larger wager than I’m talking about.

  8. jim says:

    Hi Phil,

    I don’t have time to read the thing, but I might help if we understood the larger research context of the model. My guess is that this is a single step in a much longer and far more complex process.

    As a geology student I got interested in the origin and composition of granitic magmas. We know how and why granitic magmas, lavas, and explosive products form now, but it took thousands of experiments, most of which were done on binary compositional elements of the granitic system.

    In effect, these authors are exploring a few dimensions of a multidimensional system. Later, they can explore other dimensions and build a bigger view of the entire system.

    • Phil says:

      I’m all for exploring this kind of stuff! Build a toy model, exercise it, see what it tells you. But if you’ve got some extremely influential assumption that seems divorced from reality, like this one does, don’t try to convince yourself (or me!) that your toy model behaves like the real world. You can learn stuff from it, but you can’t take it literally. These authors are taking it literally. I don’t trust their quantitative results in the slightest.

  9. Dale Lehman says:

    I think the criticism is somewhat misplaced. This study does what many studies have done. For example, The Limits to Growth, was essentially based upon false assumptions – resource use would deplete exhaustible resources but there were no price rises to inhibit consumption, nor spur exploration or technological advances. Yes, that study has profound influences on people’s thinking. I think we should probably be discussing whether that is a good or a bad thing. We (at least some of us) knew the Limits projections were nonsense. At the same time they highlighted that physical limits were real and would require real adaptations and/or pain. I never much liked that study (and those of its ilk) but many people saw it as a valuable eye-opening type of exercise. In that way, it is not unlike Wasnink’s work – flawed, but pointing to nutrition as important and often neglected. It’s easy to draw a distinction – Wasnink made up data and/or had sloppy research, the Limits made blatantly false assumptions (much like the study being discussed here). But I think those distinctions are of secondary importance.

    Is there value in work that rests on false (and important) assumptions? Linearity will produce plenty of ridiculous results, as will failure to distinguish between incremental and large changes (in a nonlinear world). Such studies can be valuable in pointing to things that are not possible, so people start thinking carefully about the adjustments and intervening factors will actually work. For me, it comes down to how such studies are presented. When they are hyped, they are propaganda, and regardless of the worthiness of their cause, I don’t like them. But if they are presented as a “toy model” to provoke thinking in ways that have been under-studied, then I see value in them. I have not read this particular study nor have I even looked at how Nature Communications has presented it. But I don’t think the flaws in the model should be the point – it is the way the model’s results are presented that matters.

  10. Steve says:

    In defense of Phil, I think the study is not helpful. It says, “Direct GHG emissions are reduced with organic farming, but when increased overseas land use to compensate for shortfalls in domestic supply are factored in, net emissions are greater. Enhanced soil carbon sequestration could offset only a small part of the higher overseas emissions.” Sure, unrealistic models can be useful. But, this looks like the Laffer curve. The question what effects tax increases have on economic growth, so let’s see what happens when the marginal rate goes to 100%. Oh gosh tax increases are bad. The question here is will organic farming ameliorate green house gas emissions. Let’s see what happens when all farming is organic and everyone consumes the same amount. Oh its not a net positive. But, the assumptions are so unrealistic that they tell us nothing about what will happen in the real world. Organic farming uses different methods, will work better and have different rates of return for different crops. So, switching to organic farming will change land use, consumer habits, etc. It is clear that the authors mean to provide some guidance on the question of whether switching to organic farming is beneficial. It is not just about how the study is hype. There is no way such a study cannot be interpreted as evidence that organic farming is not environmentally beneficial when it really provides no evidence at all.

    • Phil says:

      Thanks Steve! But actually I’m quite willing to give them their assumptions about agricultural yield per acre and stuff like that. Those could be right or could be wrong, but they don’t seem wacky. Assuming you’ll produce 40% fewer calories but you’re not going to make significant changes to the mix of agricultural products you produce, that does seem wacky.

  11. Phil says:

    It’s funny how many people who are defending the paper say they haven’t read it.

    A theme of many respondents is that we can learn from simplified models, even extremely simplified models. Indeed we can! I used to be a physicist. Simplified models were, and still are, my bread and butter. Spherical cows, frictionless planes, ‘in absence of air resistance’…bring ’em on, they’re great…but only for certain things. If you’re designing an airplane, don’t use a model with no air resistance and a frictionless plane. (Haha).

    The study I’ve referred to here doesn’t claim to be a first-order look, or to provide an order-of-magnitude estimate, or words of that type. Here, this is from the abstract: “Organic farming might contribute to this through decreased use of farm inputs and increased soil carbon sequestration, but it might also exacerbate emissions through greater food production elsewhere to make up for lower organic yields. To date there has been no rigorous assessment of this potential at national scales.”

    Rigorous assessment.

    I do think you can learn from studies like theirs. I learned from reading about it! But I don’t think you can learn what they claim to have demonstrated.

    Some commenters (who haven’t read the paper) have said I’m being too picky, and maybe that’s true. My expectations were set by the MIT Technology Review article and by the paper’s abstract, both of which claimed that this study tells us what would happen if England and Wales switch fully to organic agriculture. If they had emphasized up-front that they were doing so under some extreme assumptions that are unlikely to hold in real life, but that we could still learn from such a study — and if they then couched the results in terms of what actually _can_ be learned from such a study — I wouldn’t have found it so jarring. It’s not that they used this model or that they came up with these results that seems wrong to me, at all. What bothers me is that they (seem to) think the results of the model are close to what would happen in the real world. It’s another example of mistaking the map for the territory.

    • Dale Lehman says:

      Fair enough – and this is exactly my point. What you have described here is a study that is being misrepresented by the authors and being picked up by other media outlets. You focus on the commenters who focus on the belief that simple models can be valuable. I would focus on those attacking this study because of its unrealistic assumptions. That is not the problem with this study – it is the claims that are being made about what it shows. The quote from the abstract is to the point: they did not conduct a rigorous assessment. They designed an oversimplified model that, at best, demonstrates that a complete view of organic farming should include effects on where food is produced. It would then be easy for them to point out the myriad other secondary impacts (perhaps of primary importance) that they did not model: price effects on consumers are farmers, for example. If they were honest about their study, it would be to draw attention to one of these factors that is often ignored – and use that as a platform to shed light on other factors that they continue to ignore.

      • I can’t help but think that here is a *perfect* example of a question containing both economic / decision making content as well as fundamental physical laws and constraints, and that it’s absolutely *perfect* for an agent based modeling approach.

        How would that model look?

        1) Create some land areas that correspond to farm and city
        2) Create a large number of agents that correspond to statistical aggregates of farmers and city dwellers
        3) Specify a relatively constant external import/export food market from which agents have the option of buying or selling, but which is essentially unaffected by the local production changes.
        4) Define rules describing fairly basic physics of food growth both under conventional growing and organic methods.
        5) Define rules describing fairly basic decision making about food consumption and food production.
        6) Run the model forward in time to equilibrium using “conventional” growth assumptions throughout.
        7) Flip the switch to convert all local food production to organic, as if a policy were put in place banning fertilizer and pesticide sales or something.
        8) Run the model dynamically forward and observe the behavior as food production changes, pricing changes, importation changes, land use changes…
        9) Run long enough to observe a new equilibrium.
        10) Describe the complexity and the dependence of the results on various aspects of the model, be realistic about the limitation and the range of assumptions that are realistic. If possible use Bayesian methods to calibrate the results to some real world observations…

        I can already hear the hordes of certain social scientists telling me about how as a physics person I can never appreciate the complexity of social science phenomena and how this is a fruitless and pointless task and how what we really need is a good accidental policy discontinuity and a difference-in-differences analysis with a lot of p values…

        But seriously, this kind of agent based approach offers a lot of value if your goal is to understand mechanism.

        • matt says:

          Maybe it does, but this isn’t even empirical. As a theoretical quantitative exercise, sure it’s fine. This stuff exists in economics, it just has been woefully unsuccessful at producing anything useful to the real world. I can’t see specific context would be any different. Why would you create a model this complex when you have no data to even identify anything in it?

          More generally Daniel, you need to stop touting this modelling approach for domains in which you have no evidence that it can be useful. Show me a paper that does something like this in the social sciences and produces a useful insight, and you’ll gain some credibility. Otherwise, you are beating a dead horse by outlining how you’d model some new problem each week, but never actually following through with it. It’s cheap talk: you can spout off at no cost because you have no skin in the game, and you have no intention of actually following through on any of these suggested approaches. And no, don’t ask me to chip in to a crowdsourcing project that allows you to work on this; that again is cheap talk, because you know it won’t happen.

          • You’re right, I’m not going to do this, because I’m not getting paid to be an economist, and doing this kind of thing is a full time job. But the fact that people don’t make progress doing this isn’t evidence that you can’t make progress doing this. If you’d like to show me an example of this kind of model being created in econ and then going nowhere, I’m happy to read the paper. So far, I have yet to see anyone even try. I don’t read the econ literature all day long though… so maybe I just haven’t seen it.

            If you’d like to see evidence that this kind of modeling technique can be useful in areas people haven’t been particularly trying it, you can read my E-Life paper on bone growth: sure it helps we have some empirical data we can inform the model with. But acting like we don’t have any empirical data to inform the proposed economic models with doesn’t make it true. We have literally *decades* of rather detailed data on farm production, prices, and farms that convert from conventional to organic… it’s out there if you want to look for it.

            But, even if you really do have no data to identify parameters with, why would you create a model this complex?

            several reasons:

            1) It can identify questions to ask so that you know what data to even collect
            2) It can identify parameters that make a difference to predictions vs those that don’t
            3) It can tell you whether there are several mechanisms that can explain the same kinds of observations
            4) Maybe you actually *do* have data that can identify parameters in the model, in fact, often you might but don’t know it until afterwards.
            5) Strong models are generally much easier to identify than weak ones. Mechanisms themselves constrain behavior.
            6) Because this is the purpose of science, to describe the regularities in the world. The purpose of science isn’t just to figure out what would happen in the UK if in 2020 someone passed some specific policy, but rather questions like “how does the efficiency of plant based energy production vs meat based energy production affect how rapidly humans could be forced to adapt to certain kinds of shocks”. You simply *can’t* study that “empirically” through difference in difference type regressions because you can’t run several hundred experiments the way you can answer questions about how say roughness of pipes affects water flow by trying different roughnesses.

            If you can point me to 10 or so papers in econ with agent based type approaches that “go nowhere” and “offer nothing of use” or whatever. I’d be grateful. I certainly have seen multiple econ papers highlighted here offering nothing of any generalizable use from other more “standard” approaches.

            • That E-Life paper corresponds to something like 10 years of biologists collecting *experimental* data on a particular process, expecting certain kinds of results, getting confusingly contradictory results based on their intuitive descriptions of the process, trying to describe to me what their intuitions were and why they were confused by the data… and finally I said they should just encode some of those descriptions into an actual model and run it. After a couple of weeks we had a running model that reproduced the semi-quantitative results they were seeing, but only under certain regimes of the various parameters… it identified how parameters must necessarily be changing together for example (cell replication rates change together with cell death rates for example, and cell replication and death rates are intimately connected through exposure to diffusive chemicals with the cell type distribution)… So, in the end a paper they couldn’t get accepted anywhere because no-one could understand what any of the data meant… was all tied nicely into a package in which everything made some kind of sense.

              So, I interpret you as saying basically “put up or shut up” and now I’ve “put up” so now if you’d put up 10 similar types of papers in economics doing nothing of any use we’ll see.

            • matt says:

              Daniel, look at modern macro. This is all it is.

              • If you’d like to show me 5 or 10 papers on modern macroeconomic agent based models I’d be happy to read them. It’s gotta be a lot easier to link to 5-10 papers than it is to do what you suggest where apparently I should complete a masters thesis on each weekly topic or just not have any opinions…

            • matt says:

              I really don’t know that much is being learned by adding that model in. I only skimmed the paper, but I certainly wouldn’t be taking those parameter estimates seriously. You’ve built a predictive model; great, but what use is that?

              • Well, for example, the model predicted quantitative changes in the rate of cell replication which was related to the Apaf knockout genotype, a fact not previously considered by the biologists involved and then, when an experiment was run and the number of dividing cells quantified, sure enough the quantified rates of replication followed the predicted patterns.


                So that constitutes all three stages of Feynman’s science cartoon: Guess, compute the consequences, and compare to data.

                It’s sufficient to say that the biologists found that the model unified all the observed data in a way that made qualitative sense to them, where they previously couldn’t figure out why the 4 genotypes had the phenotypes they observed, and they could only offer halting verbal descriptions of why it was confusing. In particular it explained why knocking out genes whose absence should eliminate cell death successfully eliminated cell death, and yet led to smaller, and more distally specified and more stochastically variable-sized rib segments, with no proximal type bone segments.

                In a situation like this conventional vs organic farming analysis, an agent based model might show things like “across all reasonable values for the parameters involved, in a scenario where 100% of farming is forced to be organic, land use would rapidly convert to a monoculture of 3 particular high value crops whose demand at higher prices and lack of overseas supply due to inability to be transported effectively would compensate for the fact that production costs had risen substantially. Imports of other crops would increase, but due to the distribution of the bulk of English population concentrated at or near port cities, and the transportation efficiency of sea transport relative to ground transport within the UK, total energy costs would rise by only between approximately 2 and 15%. In most plausible scenarios, total meat consumed would drop by 5 to 15%. Due to the cost of land use conversion we expect substantial consolidation of farms, and essentially 100% of small family farms would sell to conglomorates”

                and that would be considered a terrible outcome of course, because while it provides all sorts of quantitative testable predictions, and a mechanism to see how the outcomes are related specifically to the assumptions in the model that specify decision making actions of the individual actors, it doesn’t look anything like any example in Angrist and Pishcke and we all know that they say “fancier econometric techniques are typically unnecessary and even dangerous” QED.

              • I’m sure for example that the USDA modeling forest management dynamics in the presence of wild-fires are all just wasting their time, and they really just need wait for a wild fire to happen, and then do a difference in difference analysis of something or other using linear regression and some kind of clustered standard errors. Economists have got this all figured out, if only ecologists knew?


                After all, the forestry model is probably just a theoretical exercise… except that from the article: “the simulated relationship of ERC and fire size and frequency captured the distribution of the historical data from 1992–2009.” and later:

                “The size distributions of remote sensing and simulated management activity patches (thinning from below, heavy partial harvest, clear cutting) were similar for federal lands, which occupy approximately 40% of the forest lands (Appendix 12). For other owners, the sizes of the simulated patches were smaller than those recorded by remote sensing, suggesting that the rules governing management unit size for nonfederal owners need further development. The challenges of model validation with independent data mean that the findings of our study should be viewed with caution and that the focus should be on the relative differences between scenarios, which are less subject to errors in model assumptions and parameterization.”

                Wow, humility and a willingness to admit the limitations of a study, while still making relatively strong structural assumptions, and identifying which components of the model are sensitive vs not sensitive to errors in the assumptions. Who would have thought?

          • For example, perhaps you’d like to discuss why this paper is a total disaster that “isn’t even empirical”


            perhaps the real truth is that these techniques are effective enough that they’re kept proprietary rather than publicized… I know AnyLogic is certainly selling enough copies of their software to financial services companies that evidently people are willing to bet money on the general techniques’ usefulness.

      • Phil says:

        Dale, if you look at my comments I believe I’ve agreed with every commenter who said we can learn from simplified models. I don’t think I’m focused on commenters who focus on the belief that simple models can be valuable…certainly I am not getting on them in a bad way. I agree with them!

        You’re right that I should have made it more clear from the outset that I am not opposed to someone doing a study that makes the assumptions that were used in this study. Hey, study whatever you want, it’s fine with me. What I object to isn’t the assumptions, it’s the strong implication that the assumptions are a good match to the real world. I think I said this OK in the closing paragraphs of the post, but perhaps by then it too late because I had already focused everyone’s attention on the assumptions themselves.

        • Phil, I think your complaints are similar to a lot of my complaints when I was in my PhD… it’s not so much that people were doing a lot of wrong things… it’s more that people were doing a lot of pointless things and acting like they were very insightful and interesting and informative.

          There are kind of two main reasons why people do this kind of thing:

          1) They don’t know any better… they were trained by people who did this kind of stuff, or they’re trying the best that they know how to make some contribution, but often maybe the field is dominated by pretty weak studies.

          2) They know better, but after all they have to justify their salary and grants somehow so they just publish some stuff that’s easy to crank out.

          there are also kind of combinations of these things… people don’t really know better, but they also feel like they should crank out some stuff to prove they’re not just wasting everyone’s time… or to be able to cite things when applying for grants or whatever.

          So, try this for example. Search based on similar key words for all the papers you can find on this topic… And then skim them, and rate them on a scale of 0 to 10 with 0 being “hasn’t got a hope in hell of improving our understanding of the real world” to 10 being “even if it’s in some simplistic way, this actually tries hard to move some important aspect of our understanding of the world forward”

          But before you do that, what is your prior on the mean value you would rate all the studies you find from say the last 10 years?

          • To further anchor the scale, let’s say

            2 = “at least considers questions that are relevant to actually understanding the real world”

            5 = “makes some effort to understand the real world, but honestly makes some kind of obvious mistakes about reasonable assumptions”

            8 = “makes a good effort to understand the real world, makes decent core assumptions, but maybe makes too many simplifying assumptions or doesn’t examine a wide enough range of possibilities”

          • Phil says:

            By your scale (below) I’d give this study a 5. I guess I’d expect 7 to be more typical. I hope there aren’t many below 5.

            Seems like a lot of trouble to do this with a lot of papers…and I’m not expert enough to judge, in a lot of cases. But maybe I’ll do a dozen of ’em. In any case I take your point. Most stuff out there isn’t very good, why should I expect this to be different? In this case I did expect it to be different: it got written up in MIT Technology Review, and it was published in Nature Communications, which, yeah, is not the same as ‘Nature’ but is still pretty prestigious.

            Now you’re going to know I shouldn’t be so naive as to expect those two facts to indicate anything. And yet, hope springs eternal in the human heart.

            • Phil, if I had found a paper on a similar topic and it was published in Nature instead of Nature Communications, and that’s all you know, would your prior expectation for its score increase or decrease? Empirically if we do it, do you think your posterior would predict an increase in quality score or decrease?

            • As for hope springs eternal, that’s why we stick around here right? I mean, if it were just bashing papers rather than offering some kind of constructive criticisms and discussion of what should or shouldn’t matter and a hope that maybe a bunch of silent observers are reading in the wings and thinking about these things and discussing them with their colleagues… would we still be here commenting on the blog? Isn’t that other stuff what Twitter is for anyway ;-)

        • Anoneuoid says:

          The way they are supposed to check the validity of the assumptions is to derive a prediction from them and compare it to the real world. If the paper is missing that, then why are you paying any attention to it to begin with?

          • Evidently it derives predictions… unfortunately there is no real world in which we can run the experiment and then see if the predictions played out. Does this mean we can’t do science here? Not necessarily. It can be useful to derive various sub-predictions and compare those, or to derive predictions in less drastic changes maybe those that occurred in the past, etc.

            Not every group has to do the entire science thing, have a theory, derive some predictions, carry out some experiments, collect the data, do the comparison, etc. It’s fine for people to specialize. You might call this a theoretical exercise. Should we ignore all theoretical exercises until the whole cycle is complete? I don’t think so, but we should judge theory without empirical followup on the basis of various measures of good theory. Like for example, there are plenty of theories that don’t even really count as theories, in that they don’t really generate even vague quantitative predictions… some things called theory in social sciences just predict directions, like fatter arms = more whatever it was in that study, or more power poses = higher social dominance or whatever… This does substantially better than that at least.

            • Anoneuoid says:

              They should put that they made predictions in the first sentence of the paper since that is the main point of it then. Once someone checks the predictions, it would be worthwhile for others to look into it further.

Leave a Reply