REPLY Getting a Good Problem Statement (SD6531)

[SDMAIL Schuette, Wade]

Posted by  "Schuette, Wade" <wschuett at jhsph.edu>

I agree with Bill (below) and expand one comment.
 
I used to run an executive intelligence group at Cornell University that 
various VP's would come to for answers to questions they had, where the 
data or proxies for the data were somewhere in some combination of 
databases we could get to. I was also taking a computer science course 
in the development of "natual language query systems."
 
I didn't make myself popular when I told the CS people that the best way 
to "optimize" most queries from execs was to throw them away and start 
fresh.   I can't recall a single instance where the question someone asked 
me ("the presenting complaint") was even close to the answer they actually 
needed.  
 
One example:  I was asked "How many employees do we have in the 
Agriculture School?"  I replied with a polite version of "Help me 
understand why you want to know that.  What issue are you trying to 
decide?"   In this case, he was trying to decide whether there would 
be sufficient business at lunchtime to build a new cafeteria in that 
vicinity.    So, yes, he was interested to know that directly across 
the street was a building with 200 federal employees in it that wasn't 
shown on our payroll.  And, yes, come to think of it, I guess students 
do eat lunch too, don't they!   Etc.  
 
The question recommended by my late statistics professor,  Jack Kieffer,  
( a member of the National Academy of Sciences) was "What does it cost 
you to make this decision incorrectly?" Jack asserted that there was no 
way to create an appropriate statistic without knowing the client's risk 
curve - how much did they win if they were right? How much did they lose 
if they were wrong?   That had to be factored into the analysis. There was, 
in his opinion,  so such thing as a "generic" answer.   Statisticians 
battle over that, but, as an MBA, I agree with Jack. 
 
In fact, I think that's a big factor in why most academic research is 
completely ignored by policy makers -- they don't care about "p-values", 
they care about re-election, or power.  Of course, the actual agenda may 
be hard to extract or a secret.   
 
Regardless,  real people will have to take real action and face real 
consequences if they are right or wrong, based on what you are telling 
them, so it seems wise to include that in the analysis.   Most adults 
can deal with major losses, if they entered into the deal knowing what 
they actually were risking. 
 
Others,  academic purists, may strongly disagree with that, of course - 
and argue that they are not responsible for the "misuse" of their 
conclusions.   Hmm.  US Product Liability law would hold that one is 
responsible for "expectable misuse" of equipment.   So far that hasn't 
been applied much to software or analyses, but maybe it should . Again, 
opinions will differ.   Still, I'd like to know who's staking what on the 
answer being "right", and what they would define as "right." 
 
As to the questions to ask, Bill (below) suggests "What are the most important 
questions we should be asking?" I'd add to that,  "Who actually has first-hand 
knowledge of the situation that we could talk to?"  weighted by finding the 
disempowered people low in the power structure and asking them "What do you 
think is going on here?"  Their answers will differ markedly from the "view 
from the top", and will bring in unexpected factors that are completely 
invisible up the chain of command -- some of which will be true show-stoppers.
 
Earl Brooks, a B-school faculty member at Cornell, used to consult for GM. 
They'd bring him in for big bucks to deal with issues they were having.  
Earl would arrive a day early, and spend the day buying drinks for 
employees at the bar nearest the exit, and just ask them to tell him what 
was going on at the plant.   Then he'd meet with the top guns,  stroke 
his chin wisely,  and come back with suggestions at $50,000 apiece (30 
years ago) that management could have gotten themselves for free, if they'd 
asked.   Usually his suggestions worked and management was delighted and 
astounded at his perceptive insight.   
 
Rule of thumb -- in multilevel organizational hierarchies,  the critical 
details at any level are completely lost in the oversimplified graphics 
and mental model two-levels up or higher.  Toyota's "Lean" approach suggests 
"Genchi Genbutsu" - or,  go down and look for yourself before making 
pronouncements. 
 
Similarly, I think the critical variables two layers above one and higher 
in such organizations are also completely unexpected and invisible, or 
meaningless at lower levels -- it's like our body's cells grasping the
 concept "Accepted by Harvard."  It's literally another world.    
 Loosely coupled, but different dimensions.
 
So, by that reasoning, either a "problem" is generally multidimensional 
and multi-level,  or it will be perceived as "different" problems at 
different levels of stakeholders.    
 
So, that suggests again that opinions on the relevant factors should be 
solicited from as many different levels as possible, because they may be 
wildly different.   Similarly, there's no reason built into physics that 
the benefit of solutions will be some sort of constant at any radius of 
consideration in space, time, or scale.  There are many ways to win the 
battle and lose the war.    It may be good to check the sensitivity of the 
answer to your planning horizon - if the sign changes as you expand your 
horizon (from good to bad, or vice versa),  be very cautious. 
 
Other opinions may vary of course. 
 
Wade Schuette
Ann Arbor, MI
Posted by  "Schuette, Wade" <wschuett at jhsph.edu>
posting date  Tue, 28 Aug 2007 12:23:57 -0400