WED 2:30 PM Parallel Session: Model Analysis and Experimentation
Sensitivity Analysis of a Real Estate Price Oscillations Model, by Birnur Ozbas, Onur Ozgun, and Yaman Barlas
The model upon which the formal output analysis was applied was Modeling of Real Estate Price Oscillations in Istanbul, which the authors presented at the ISDC in Boston in 2007. The current paper applied two methods of parameter sensitivity analysis, fractional factorial design and Latin hypercube sampling, to first show the strengths and weaknesses of each design and to use the results of each analysis to identify key sensitive parameters which might indicate leverage points for potential policy recommendations.
Fractional factorial design is a subset analysis of factorial design, which would consider all parameter combinations using all possible parameter values, and would be less time consuming (hence more useful) for most purposes. In the fractional factorial design, seven parameters were selected, each with two potential values. There were three response variables: amplitude, period, and mean of the price oscillations that were measured. One-hundred-forty-four data sets were analyzed, using different combinations of parameters and levels. The results of the analysis on each response variable were given in a table in the paper.
Latin hypercube sampling required each input parameter to have its values divided into N strata of equal probability. From the N possibilities for each input parameter, one value is randomly selected. This constitutes the set of values for a trial. Trials are repeated without replacement for N iterations. An assumption is made that the input parameters are independent of each other. The same number of input parameters, two values each, and three response variables, were designated as with the fractional factorial design analysis. Only one-hundred-twenty-three runs were required using this method. Again the results of the analysis are summarized in a table in the paper.
The authors conclude that Latin hypercube sampling is more complete, easier to understand, and requires fewer tests, but requires the parameters be independent. If parameter independence is not possible, fractional factorial design is better, even though it is more difficult to interpret. It is also important to note that the two designs identified some difference in sensitivity of parameters as they affected each response variable.
Testing Dynamic Decision Making Under Real-Time Pressure: A Scuba Diving Simulator, by Yaman Barlas, and Evrim Dalkiran
A scuba diving simulator (system dynamics model) was developed to determine whether real-time pressure, material and/or information delays, and experience made a difference in successfully completing a task. The task was to have a simulated scuba diver attain and maintain a depth of 10 meters using a scuba diving apparatus (a diving vest that can be inflated or deflated to change the diver's vertical position in the water). Note that as the diver goes deeper more pressure compresses the jacket, so less air can be put into the jacket. This introduced an interesting non-linear feedback. There were sixteen subjects who participated in the experiment.
The Latin Square analysis found that game speed and delays influenced performance significantly. Each participant worked with the simulation six or eight successive times. In the repeated measures analysis, it appeared that improvement in performance each time the simulator was used was statistically significant. Blocking was used to analyze whether those participants with scuba diving experience were better at the simulation. There appeared to be no statistically significant difference between those with and without actual scuba diving experience in the performance with this simulator.
An Attempt to Automate the Analysis of Complex System Dynamics Models: An Example of WORLD 3, by Pedro Retortillo, Margarita Mediavilla, Luis Miguel, and Carlos de Castro
The difficulty with the uncertainty of parameter value selection in system dynamics models requires significant analysis on the part of the modeler to both validate and understand the behavior of a model of even medium-level sophistication. The authors suggest that it should be possible to use the computer to automate the parameter testing process and connect the output to the structure segment of the model. If this can be done, then the modeler will be able to explain to his/her audience how the model behaves using a more clear narrative.
The authors tested this hypothesis by translating the World 3 model to a Simulink-MATLAB platform. The paper describes the first trials in trying to analyze the World 3 model.
A small stock/flow diagram is shown in the equivalent Simulink circuit type diagram. The Simulink interface to the World 3 model is also shown. The authors identified the following parameters for analysis: initial value of non-renewable resources, inherent land fertility, index of absorption of pollution, average life of industrial capital, and the year that industrial capital is stabilized. A range of values is specified for the simulation runs, but the actual value for each parameter is selected at random from within the specified range by the program. Also shown is a fifty run sensitivity graph where it is possible to plot final output values using multiple parameter sets.
Fuzzy logic is also introduced as a potential method of automatically judging whether the output of a simulation run is classified into two or more categories (i.e., poor, good, excellent). There is an explanation about the steps needed to set up fuzzy logic analysis and how it could be applied to inputs and outputs of a model (with some simple examples).
The authors have presented a first trial of their idea in the submitted paper. The examples are simple, which is helpful to the reader.
The ideas in this paper suggest interesting potential for building more functionality into the current system dynamics software packages in the future. It would be worth watching how successful these authors are in their continued efforts to automate some of the analysis process for validation/exploration of system dynamics model behavior.