Measuring success in public affairs is hard because we don’t control all, or even most of the key factors. Beyond that, we are essentially trying to measure a cascading set of conditional probabilities, each more fuzzy than the one before. First we are trying to measure attitudes that nobody really understands. Then we are asking where those attitudes come from. After that we want to know the strength of the conviction and how attitude make practical differences. Do they change behaviors or outcomes? Complicating analysis is that effects may be significantly separated from the causes in both time and space and you have to account for the effects of temporary circumstances and random chance.
You begin to see the problem? All we really need to care about is what people do, but to explain that adequately, we have to consider all the things mentioned above.
Does the Rooster Make the Sun Rise?
It only gets worse. Public affairs can be a little like peeing in the Pacific Ocean saying it caused the rising tide and practitioners, me included, can sometimes strut like roosters taking credit for the sunrise. In other words, we are not sure how the attitudes affected behavior, nor are we sure where those attitudes came from or the strength of conviction. On top of that we are trying to figure out how our small input created a big output.
Not that we are always merely mendacious when taking credit, BTW. Public affairs is indeed all about leverage. Very small input can often create monumental outputs using leverage of the public affairs environment as it pulls in outside resources. Even this good thing, however, is just another problem for measurement. The equation would look like this.
Our input + lots of other resources we don’t control + luck + time = output, which MAY grow into a useful outgrowth. We cannot control most of the factors in this equation and often cannot even know what they are, so instead we measure the reach (not the effectiveness) of OUR own inputs. Let me illustrate with one of my usual examples, not surprisingly an oak tree
Mighty Oaks From Tiny Acorns Grow – But a Bushel of Acorns is Not an Oak Forest
If I plant an acorn, it may grow into a mighty oak. How much credit do I deserve? Maybe a squirrel would have planted an acorn if I didn’t. Maybe one would just grow by itself. Besides that, I didn’t make the acorn. I didn’t create the soil. I cannot control the rain nor can I anticipate every destructive storm nor control all the bugs. The oak tree will grow according to its form and DNA. I cannot demand that it become a pine tree. In fact there is little I can do expect remove obstacles to it becoming the best it can be. But if you come back 100 years later, maybe some kid will say, “My grandfather planted that tree.”
In public affairs we are not dealing with acorns. Our analogous measure is reach. We can get a reasonably good measure of the number of people who COULD have received our message. It doesn’t mean they DID receive our message or that they paid any attention. So reach is a problematic measure.
Don’t Count the Same Guys too Many Times
A look at Facebook shows examples of opportunity, challenge & problems associated with this kind of measurement. You might have a thousand friends or a big rock star might have a million fans. But how much are they getting the messages? We also habitually overestimate the connections. If you have 100 Facebook friends and each of them has 100 friends, you do not have 100 x 100 or 10,000 friends because the sets overlap. If your friends are also each other’s friends you may have only 100 in total. Overlap is usually not 100% and the real number is probably more than just 100, but it is far less than 10,000.
Reach is not a very useful measure, but we like it because it is a relatively easy number to find or estimate AND it tends to be the largest number we are can get, especially if we engage in some willful ignorance about human attention spans and math 101 concepts of overlapping sets, as above.
Reach Exceeds Grasp
And reach is relatively easy to astro-turf, especially in the new media. There is an interesting article talking about how you can BUY Facebook friends and fans for as little as $.076 and $.085 respectively. What reach! If you have big bucks you can reach the all the world in theory. Who can you blame if your reach exceeds your grasp, if you have a million fans who cannot remember your name or hear your message?
Hey, the numbers are good, even if they probably overlap and may represent meaningless relationships. We might become a little suspicious if our extrapolated fan bases (i.e. our estimate of our own fans to the exponent of their fans & friends) exceeded the total population of the earth, but achieving that might take a couple of months anyway.
I am not saying we should not rejoice at successful numbers, but let’s not try to fool others and let’s not fool ourselves. Reach provides ONLY the opportunity to engage and engagement provides only the opportunity to communicate and communication provides only to opportunity to make a difference. You need to start with the acorns, but that doesn’t mean you automatically have a grove of big oak trees.