REPLY Experimental Economics (SD6457)

[SDMAIL John Sterman]

Posted by  John Sterman <jsterman at MIT.EDU>


Martin Schaffernicht asks about the relationship between experimental  
economics and system dynamics.

Experimental studies of dynamic systems are growing.   Studies in SD  
and related fields cover both the "theory building and testing" and  
"concrete problems" purposes Martin describes.

The MIT SD group has been doing experimental studies of dynamic  
decision making since the mid 1980s.  Some studies include:

Sterman, J. and L. Booth Sweeney (2007). "Understanding Public  
Complacency About Climate Change:  Adults' Mental Models of Climate  
Change Violate Conservation of Matter." Climatic Change 80(3-4): 213-238.
    
Sterman, J. D. and L. Booth Sweeney (2002). "Cloudy Skies:
  Assessing Public Understanding of Global Warming." System Dynamics  
  Review 18(2): 207-240.

Booth Sweeney, L. and J. D. Sterman (2000). "Bathtub Dynamics:   
Initial Results of a Systems Thinking Inventory." System Dynamics  
Review 16(4): 249-294.
    
Croson, R., K. Donohue, E. Katon, J. Sterman (working paper). Order  
Stability in Supply Chains: The Impact of Coordination Stock. MIT  
Sloan School of Management Working Paper No. 4513-04.
    
Diehl, E. and J.  Sterman (1995). "Effects of Feedback Complexity on  
Dynamic Decision Making." Organizational Behavior and Human Decision  
Processes 62(2): 198-215.
    
Kampmann, C. and J. Sterman (1998). Do Markets Mitigate  
Misperceptions of Feedback in Dynamic Tasks? Cambridge, MA 02139,  
Sloan School of Management, MIT.

Paich, M. and J. Sterman (1993). "Boom, Bust, and Failures to Learn  
in Experimental Markets." Management Science 39(12): 1439-1458.

Sterman, J. (1987). "Testing Behavioral Simulation Models by Direct  
Experiment." Management Science 33(12): 1572-1592.
    
Sterman, J. (1989). "Misperceptions of Feedback in Dynamic Decision  
Making." Organizational Behavior and Human Decision Processes 43(3):  
301-335.
    
Sterman, J. (1989). Misperceptions of Feedback in Dynamic Decision  
Making. Computer Based Management of Complex Systems. P. Milling and  
E. Zahn. Berlin, Springer Verlag: 21-31.
    
Sterman, J. (1989). "Modeling Managerial Behavior: Misperceptions of  
Feedback in a Dynamic Decision Making Experiment." Management Science  
35(3): 321-339.

See also the PhD theses of Bent Bakken, Christian Kampmann, Ernst  
Diehl, Mark Paich, Linda Booth Sweeney (all MIT except Linda's, which  
is Harvard).

See also the experimental studies of Erling Moxnes (Univ. of Bergen)  
on the dynamics of renewable resource management (for which Erling  
won the Forrester Award, for Erling Moxnes (1998) Not Only the  
Tragedy of the Commons:  Misperceptions of Bioeconomics: Management  
Science.  44: (9) 1234-1248.).

Tarek Abdel-Hamid (Naval Postgraduate School) and colleagues carried  
out a number of experiments in which people manage simulated software  
development projects under different information and feedback  
conditions.

Jim Ritchie-Dunham's PhD thesis reports experiments investigating the  
impact of different modeling and information display tools such as  
balanced scorecards on simulated organizational performance.

Shayne Gary (AGSM) is active in experiments studying misperceptions  
of feedback in various simple dynamic systems.

There is also a robust literature of experimental studies of the beer  
game (see references in the Croson, Donohue, Katok and Sterman  
working paper, available on http://web.mit.edu/jsterman/www), and a  
growing literature in what is now called "behavioral operations  
management".  Many BOM studies are experiments with dynamic systems  
such as supply chains, newsvendor settings, sequential choice, and  
resource allocation.  There have been several conferences, with  
another coming up this July at the U. of Minnesota (hosted by Karen  
Donohue).  See also http://www.ombehavior.com/ and a forthcoming  
special issue of MSOM on behavioral operations.

In the broader field of judgment and decision making, experimental  
studies of dynamic decision making have been an important thread of  
work for decades.  See work of A. Rapaport, W. Edwards, J. Busemeyer,  
D. Kleinmuntz, R. Hogarth, B. Brehmer, D Dörner, J. Funke, C. Plott,  
V. Smith, C Camerer, A. Wearing and many others.  A google search for  
"dynamic decision making" will yield many useful sites and  
references; the papers above also include lit review.  And  
experimental studies in behavioral economics, behavioral finance, and  
game theory increasingly utilize dynamic tasks.  I am sure this  
abbreviated list omits many interesting and important works -- my  
apologies in advance.

It is important to distinguish between tasks that are"static",   
"repeated" and "dynamic":  Static tasks are "one-shot":  you are  
presented with information and asked to make a judgment or decision,  
receiving no feedback or opportunities for subsequent decisions.  An  
example is the one-shot prisoner's dilemma:  you choose once between  
cooperation and defection.  The iterated prisoner's dilemma is a  
"repeated" task.  You receive outcome feedback as you play multiple  
rounds, and have the chance to learn about the behavior of others and  
update your strategies.  But there is no "action feedback" -- that  
is, there is no feedback (in the classic IPD paradigm) between your  
decisions and future payoffs.  A true dynamic decision task includes  
such action feedbacks:  as in most real-world tasks, your decisions  
alter the state of the system (potentially including payoffs,  
probabilities, and available choices), which then condition your  
future decisions.  The beer game is an example:  your ordering  
decisions alter the inventories and backlogs in the system -- both  
yours and those of other players -- which then condition future  
ordering decisions.  You receive outcome feedback each period but the  
situation you face is also different each period as the decisions you  
and others take alter the state of the system.  The research shows  
that as the dynamic complexity of the system grows (as there are more  
time delays, feedbacks (especially positive feedbacks), accumulations  
(stock and flow structures) and nonlinearities, the worse human  
performance typically is, and the slower the rate of learning.   
Similarly, Dennis Meadows' Fishbanks game is a dynamic task:  you not  
only receive outcome feedback on the investment and fishing effort  
choices of others, but your actions and those of others alter the  
stock of fish available in the future and hence the payoffs to future  
investment and fishing effort.


John Sterman
Posted by  John Sterman <jsterman at MIT.EDU>
posting date  Thu, 31 May 2007 09:35:42 -0400