Two separate groups of people came to see me about measuring progress in our area of operation and gave me an opportunity to pontificate in my very best style. I am doing my best to deploy all my skill and experience on how to assess and measure. I am delving way back to my MBA days when I studied marketing research, but Iraq presents a researcher with almost the perfect storm of confusion. I am not sure how to measure progress in Iraq and I am not sure that information is knowable even in theory.
One of the guys who came to visit was a practicing anthropologist. I didn’t know they had that kind of career path, but it makes sense. Anthropologists study relationships between people, institutions, traditions and society. The skills of an anthropologist are more appropriate in Iraq than those of a public pollster. I don’t believe the usual polling methods can produce valid results in a place like Iraq. Figuring out the situation here is more an art than a science, more anecdotal than analytical. My study of marketing research methods gave me a good feeling for the strengths and weaknesses of statistical studies.
Graphically Misleading
The most misleading sort of study is the pseudo-scientific one, with lots of numbers and graphs w/o valid grounding in reality. Such things are usually based on a kind of snowballing of the power of a few guesses. A few people make estimates that are locally valid for decision making but not scientific. For example, “How much traffic is there on the road?” “Lots.” You could make a decision based on that, but it is a soft estimate. Somebody aggregates these guesses and gives them numerical weight. As the aggregations get farther from the original sources, they get less and less related to reality BUT more and more impressive in terms of certainty of numbers and presentation.
In my traffic example, if you aggregate traffic information from downtown Manhattan and rural Wyoming, you might conclude that traffic is a moderate concern in both places and you could produce graphs and charts to support your position. I learned a long time ago that if you want to enhance the power of your own gut estimate, you should put it into writing and if possible draw a chart or a graph. I know this works, but I also know that it is primarily a presentation ploy. Even in the best cases, it is used to simplify information and make it easier to understand. In the process, we trade some degree of accurate detail for presentation. Anyway, I think we are demanding more of the information we have than it has to teach us and much of our precision is unjustified.
Spock Trap
I remember in the old Star Trek when Spock would say something like “impact in 10.5 seconds.” How stupid is that? That is why I prefer Picard. By the time he says 10.5, the number has changed. It is unjustified precision, but it is easy to fall into the Spock trap. It is attractive and makes you seem intelligent. BTW – my own experience in using deceptive numbers is that you are much better off using precise odd numbers. For instance, 97 is a more credible number than 100 or 90. (Remember that Ivory Soap was 99 and 44/100ths percent pure, not 100 %.)My feeling about the part of Iraq that I know best, the places I have actually set foot and looked at with my own eyes, is that things are much better now than they were when I arrived six months ago. I use the word “feeling” because that is what I have. I have observed that people seem friendlier. Markets are fuller. There seems to be less fear. Local people were once afraid to talk to us or work with us. Not any more. It just feels better.
Dreadful Conditions
I am convinced that conditions here are better than our measurements will be ever able to detect. Iraqis have a long history with oppression. Smart people learned to hide their prosperity from predatory authorities. If Saddam’s henchmen found out you had something good, you might not be able to keep it. We also saw the age-old desire to hide assets from the tax collectors. As a result of all this, people have become accustomed to lying to anybody asking questions and trying to make conditions seem as dreadful as possible.
Sing the Body Electric
A good example of a statistic we cannot use – but we do – is electricity. Iraqis get some hours of electricity from the grid. This power is essentially free, since the authorities have generally lost the capacity to meter and charge for it. Naturally, everybody wants as much of this free power as they can get and when the power comes on they plug in everything they own. It makes demand appear much higher and shortfalls more acute. If asked, people complain bitterly about the lack of power. BUT if you fly over Anbar or drive thorough a city at night, you see plenty of lights even when there is ostensibly no power. The fact is that many communities and even individuals have generators. They prefer not to use these generators because it means that electricity is no longer free. However, when they say that they do not have electricity, they really mean that they do not have FREE electricity.
Demand for electricity in Iraq is growing at around 12% a year, as people buy more things like refrigerators, microwaves and DVD players. Supply can never catch up with demand as long as electricity is de-facto free. I am convinced that if/when the authorities figure out how to meter and charge for it, the “problem” of electricity will be mostly solved, or more correctly it will stop being a problem and become an expense.
Fear v Greed
There are some sorts of statistics that I think we might be able to use IF we could assess them. One is the risk premium that contractors and others demand. Six months ago we had to pay relatively more for services because people thought it was risky to deal with us (i.e. they were afraid the insurgents would target them in retaliation). They charged us more to compensate. Now the prices we are paying for our projects are dropping. Of course that could be because we are getting better at knowing local conditions and negotiating better deals. I think that if I could figure out a reliable way to estimate the risk premium, I would have a very good measure of improvement. It is a kind of greed v fear measurement.
Banana Index
One of my own assessment methods is a “banana index”. I observe fruits in the market especially bananas. No bananas are grown locally. They all have to be imported from somewhere else. It is very hard to get a banana to market exactly at the right time. They will usually be either green or brown. A banana stays yellow for only a short time and if it is mishandled it gets easily bruised. If you see lots of good quality bananas in the market, you know that the distribution system is working reasonably well and that good are moving expeditiously through the marketplace. Anyway, I shared my methods with the researchers. They are just rules of thumb, but if you call them heuristics they sound almost scientific.