Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages

Go Home

Modeling for Impact in the Era of Big Data

Modeling for Impact in the Era of Big Data

On the occasion of the field’s 50th anniversary, in 2007, Jay Forrester spoke to us (SDR 23:359-370) about prospects for the field of System Dynamics—his view of the situation summarized by the phrase “aimless plateau”.  He lamented our “slight impact” in government and placed the blame primarily on our failure to ask the big questions, to write books to influence the public (in the tradition of Urban Dynamics, World Dynamics, and The Limits to Growth), and to maintain rigor rather than “dumbing down” our models with “unwise simplification”.

In calling for courage and rigor in SD modeling, Jay was echoing the words of Dana Meadows and Jenny Robinson more than 20 years earlier, in “The Electronic Oracle” (Wiley 1985).

  Meadows and Robinson added a third component that they felt was critical for having influence in the public sphere, namely working collaboratively with other researchers and influencers.

Jay, Dana, and Jenny all hoped and believed that, if only we acted with quality and high purpose, the unique attributes of SD modeling would emerge and ultimately be recognized as more powerful than the intrinsically limited modeling paradigms of statistics, econometrics, and input-output analysis.  

I have to admit I’m not so sure.  Looking at the world of modeling today and who has influence, I’m struck by how statistical approaches still seem to have a strong hold on much of public policy—whether the subject is climate change economics, sustainable development goals, or even the (obviously dynamic and nonlinear) COVID-19 pandemic.  There are some exceptions, such as the Forrester award-winning work by Thompson and Tebbens on polio eradication, but such examples of strong SD public policy impact in recent years are few and far between. 

Consider, for example, climate change and its likely damages for the global economy.  The prominent Yale economist William Nordhaus (a vocal opponent of SD since the 1970s) first published work in this area in 1991, and in 2018 was awarded the Nobel Prize for it.  Yet, this econometric work is surprisingly narrow in its outlook, with its damage estimates biased downward, as described by the Australian economist Steve Keen in his recent article, “The appallingly bad neoclassical economics of climate change”. 

It doesn’t necessarily require an SD model to overcome some of the shortcomings of the Nordhaus work.   An influential study from 2015 is also statistical, but looks at data not only cross-sectionally across many geographical locations (as Nordhaus does) but also longitudinally across 50 years (1960-2010).  This study’s estimates of economic damages over the next several decades are at least ten times greater than those of Nordhaus.  This longitudinal analysis does not shy away from data but dives further, and more productively, into it.

Conclusions?  First, it seems to me that if one is hoping to impact public policy these days, it is important to draw from a large number of cases across diverse locations—the more the better.  This is the era of Big Data, after all, and your model needs to be calibrated and applied to many separate cases to prove its worth if you want to influence the public conversation. 

A good recent example is a study of the effect of weather and air pollution on COVID-19 transmission.  This collaboration between SD modelers and quantitative health scientists drew from data on more than 3,700 different locations around the world, and it garnered significant public attention.

Second, I think we should emphasize our ability to work with longitudinal data.  We know (more than most statisticians and economists do) how to move from longitudinal data to properly estimated parameters, and how to situate these parameters within robust dynamic models.

Of course, we should always remember the importance of courage, rigor, and collaboration in modeling.  But, these days, to be heard widely in the public forum we must also employ both cross-sectional and longitudinal data and calibrate our models to many individual cases when possible.

Recent Posts

Society Governance Updates

Society Governance Updates Welcome, Allyson! New President Allyson Beall King joined the Policy Council as our 2024 President. Her primary role is as director of the Washington State University School of the Environment, which focuses on regional ecologies and our...

Call for Presenters: Seminar Series

Call for Presenters: Seminar Series We at the System Dynamics Society are continually seeking vibrant and knowledgeable presenters for our ongoing Seminar Series. As we unfold the calendar, there’s always a place for more insights, experiences, and expertise to enrich...

Upcoming Events

Recent Business cases

Fast-Track Cities Uses System Dynamics to Enhance HIV Care

Fast-Track Cities Uses System Dynamics to Enhance HIV Care EXECUTIVE Summary Low levels of viral suppression at 69% for people with HIV make it hard to believe the 95% target level will be achieved by 2030 in St. Louis, USA. As a solution, Fast-Track Cities-STL opted...

Join us

Models Don’t Make Predictions, People Do!

Models Don’t Make Predictions, People Do!

Models and I go back a long way. I created relatively simple spreadsheet models by hand for 6 years at Arthur Andersen & Co. These were done on 14-column paper with a box of Ticonderoga #3 pencils and a Pink Pearl eraser. I understood the source of every input and the consequence of every output. When Lotus 1-2-3 came along, it merely automated the calculations. These computer-based models were no more influential in changing my thinking than my handmade pencil and paper models. Skip forward to System Dynamics (“SD”).

Circa 1975, when I first met Jay Forrester, Peter Senge, John Sterman and all the people at MIT, I was astonished by SD models. They could do things my models could not. But I realized these SD models had authors…filled with brilliance and know-how and biased perspectives. I also noted that Jay never (ever) said: “The model is predicting….” Instead, he stood behind the logic and reasoning in any model he authored. I don’t recall Peter ever saying that either. I’m not so sure about others, but I suspect they were willing to defend their thinking as well.

In my early SD days, I worked with Nathan Forrester, John Morecroft, Alan Graham, Jack Homer, David Peterson, Bill Arthur, Bob Eberlein, Jim Hines and many, many others. All understood clearly that the structure of the model generated its behavior, and that the structure was their representation. My task back then was to understand what was in their minds through what they included in the models. I learned SD that way.

Jim Hines coined the term, computer-aided thinking, for SD. However, we should not depend on computers to analyze/check/test our model for us.  Tools and features are useful aids, but, at the end of the day, I believe one should understand every facet, every equation, in one’s SD models and be able to explain observed model behavior in real-world terms. The SD model should be, as George Richardson noted in his paper titled Problems for the Future of System Dynamics, a slice of one’s perception of reality.  As such, the model is silent until provoked to simulate that slice, and its output does not speak for the modeler. The model merely tells the modeler, “Here’s what your view of the world says. If you don’t agree, your debate is with yourself and how you represented your thinking in these equations.”

I’ve worked with scores of clients over the years and we’ve built models together. Most did not understand the formal logic (the “maths”), but they grew to understand how a system’s structure generates its behavior. In most cases, the change in their thinking was gradual. Ultimately, the System Dynamics model replaced their mental model. This phenomenon is called reification. In short, the clients reified a formal model and it came to be their knowledge.

So that is why I say that models do not make predictions. People do.

Thompson places an interesting emphasis on Richardson’s point about models being one’s ‘slice of reality.’ In the SD community, we have a strong tradition of making our assumptions explicit in striving for model transparency. In one arena, COVID-19, the Society has joined a special call for model transparency in Science Magazine. On our site, members have heeded this call and openly their COVID-19 work for people to question parameters and equations–question our perception of reality–and to allow people to modify the model to represent their ‘slice.  Please visit these resources and use them to help build your own predictions.

Recent Posts

Society Governance Updates

Society Governance Updates Welcome, Allyson! New President Allyson Beall King joined the Policy Council as our 2024 President. Her primary role is as director of the Washington State University School of the Environment, which focuses on regional ecologies and our...

Call for Presenters: Seminar Series

Call for Presenters: Seminar Series We at the System Dynamics Society are continually seeking vibrant and knowledgeable presenters for our ongoing Seminar Series. As we unfold the calendar, there’s always a place for more insights, experiences, and expertise to enrich...

Upcoming Events

Recent Business cases

Fast-Track Cities Uses System Dynamics to Enhance HIV Care

Fast-Track Cities Uses System Dynamics to Enhance HIV Care EXECUTIVE Summary Low levels of viral suppression at 69% for people with HIV make it hard to believe the 95% target level will be achieved by 2030 in St. Louis, USA. As a solution, Fast-Track Cities-STL opted...

Join us