REPLY Meaning of Stock/Level (SD6907)

[SDMAIL Tom Fiddaman]

Posted by  Tom Fiddaman <tom at ventanasystems.com>


At 10:40 PM 4/9/2008, Alan McLucas wrote:
> I am yet to be convinced that representations of soft variables that
> often appear in SD models are routinely and comprehensively validated,
> rather than just appearing to be plausible.  I would dearly love to be
> proven wrong.  If I am wrong, then this creates an opportunity to
> compile a compendium of models that have been validated and, hence, we
> can take as being proper representations of soft variables.

Ahh ... that is a different beast. I'd hesitate to assert that much of 
anything gets "routinely and comprehensively" validated (not that that's 
OK). But that doesn't mean that soft variables are unvalidatable. All 
are at least subject to tests of robustness in extreme conditions and 
dimensional consistency. Some, like expectations that get written down 
(official forecasts) can be directly checked against data. In most other 
cases, soft variables can be at least be indirectly verified. That is, 
one can say something about the plausibility of a structure including 
soft variables by observing it in context; if the aggregate model yields 
implausible behavior then the soft variables may be at fault. One can 
use the same strategy to indirectly measure soft variables (e.g., 
estimating the parameters of a perception process), which is not 
fundamentally different than drawing conclusions about a physical 
process by observing its inputs and outputs.

The idea of creating a compendium of validated soft variable structures 
is attractive, as long as the compendium includes information about the 
circumstances in which each was derived and its possible limitations. I 
suspect that this would be rather straightforward for some of the 
typical expectation-formation models, and perhaps also for more 
existential concepts like hunger. It would also be quite useful. I think 
at least one common formulation, the TREND function typical in SD 
software, is flawed and ought to be replaced in most usage. I think the 
idea of a compendium gets much stickier when we start talking about 
love, fear and loathing, conservatism, etc. There we have not only 
ambiguity about structure but also about definition, which may not be 
widely agreed upon. Modeling love might actually be a good way to refine 
and communicate that definition, though that may just be my nerd 
perspective, certainly less satisfying than the corresponding 
experience. Some of the ambiguity around soft variables may eventually 
be resolved as technology for directly measuring the state of the brain 
improves.

I think the caution in Alan's note is worth taking to heart. It's easy 
to fill a model with SMOOTHs and other soft variable structures without 
paying much attention to whether they are meaningful. At the very least, 
one ought to explore extremes and try alternative and parameters to see 
whether the soft variables are dominating the behavior. For example, in 
some economic models, shortening perception delays and eliminating 
biases leads to an approximation of general equilibrium, and certainly 
the difference between behavioral and equilibrium perspectives is 
important. 

Regarding measurement, quantification, representation, operationalization:

I agree with Bob that it's not necessary to directly measure every 
variable in order to have a useful and valid model. What I had in mind 
when I said representation was really "operationalization" and 
"quantification." Those deserve careful attention when constructing soft 
variables, particularly for things other than expectations of measurable 
quantities. Operationalization and quantification mean more than just 
assigning a scale. One must also consider the underlying process and 
information sources that would cause the variable to change, and 
possible limits to the scale in extremes. Jay Forrest provides a nice 
example of the thought process:

> ... I think there may be a better way of modeling
> trust. My thought arrives from a perception that trust has a limit - a 
> cap -
> where additional acts of trust building don't create further trust. So
> perhaps there is a stock of trust building acts that has some form of 
> time
> depletion and a stock of trust destroying acts with a different time
> depletion factor ... Trust is I think directionally a (graphically S 
> shaped) fxn of the balance
> of the two where acts of either kind CAN affect the outflow rate of the
> other.

This has at least four nice features:
- it provides a description of the mechanics by which trust goes up or 
down on the basis of events that could potentially be measured, for 
example by asking individuals to recall them
- by distinguishing trust building and destroying acts, it recognizes 
that there may be asymmetry in time constants for response to each
- it generates saturation naturally as a consequence of the behavior of 
truths/(truths+lies) rather than via an arbitrary function
- it captures an important internal dynamic, that the stocks of building 
and destroying acts might influence one anothers' outflows (I'd add that 
they might also influence inflows, i.e. when one has a large stock of 
trust building acts, one might at first neglect to notice trust 
destroying acts, just as scientists with strong faith in a model might 
at first reject apparent anomalies in its performance)
These features could easily be absent in a more simplistic and less 
operational representation, like a single stock of trust adjusting to an 
indicated level, with saturation captured by a lookup table.

While measurement is not necessary, I think the Six Sigma attitude is 
useful. I'm reminded of a company I worked with a few years ago. People 
there actively talked about the sense of loyalty instilled by the 
company's stable employment and internal promotion policies. Then, out 
of the blue and within a few days of Christmas, they suddenly hacked off 
a huge chunk of staff, destroying morale and any semblance of a social 
contract. We all "know" that morale is important, but quarterly results 
are important and measurable. If someone had built a model of the 
company with morale in it, and searched for a way to operationalize and 
measure morale, it would be possible to get a read on (a) whether morale 
actually matters to performance and (b)  whether that act actually 
destroyed morale. The mere prospect of measurement might have influenced 
the decision.

Tom
Posted by  Tom Fiddaman <tom at ventanasystems.com>
posting date  Thu, 10 Apr 2008 20:41:16 -0600