Meta-Purposes & Why Measurements of Public Diplomacy are Usually Flawed

Something of Lasting Value – A Community

I knew an interesting woman called Eva Sopher who ran the Theatro São Pedro in Porto Alegre.  She helped me understand the meaning of cultural treasures.  The Theatro was being refurbished and put under the direction of a foundation to conserve the building and protect its traditions.  They weren’t doing many plays, so the “output” was low.  If you wanted to put on plays, you could have much more efficiently done so in many other locations or built a new theater.  Donna Eva explained why we should in a different direction.

The plays actually performed, she explained, were just a small part to the output of a cultural institution.   From the cultural point of view, the preparation, rehearsals, production and venue were probably more important.   The Theatro created a cultural community that included not only the theater goers and actors, but also the myriad of others who supported the enterprise.  This was part of a tradition that stretched back centuries and with any luck would continue for centuries into the future. It was a task that was never done and could never be done.  There was no finish line.  She didn’t use the tired cliché, but I will.  The journey was more important than the destination.  In fact the real purpose of the “product” – a successful play – was to support the other parts of the community that made it happen. Eva Sopher was impressive and it was her force of personality that drove this lesson home to me.  It takes year to develop this kind of personal integrity.  That too is a cultural output.  Her personal story was compelling.  She was born in Germany to a well off Jewish family.  They wisely left Germany and took refuge in Brazil when it became clear that the Nazis were literally out for their blood.   She embraced the country of her choice and enriched its culture.

“Objective” Measures Don’t Capture Unique Value

Imagine trying to measure Eva Sopher’s effectiveness with “objective measures.”  What did she really do that you could capture in numbers?   Twenty-five years ago she spoke to a first-tour American diplomat and convinced him to give her a very small grant and sponsor a musical program that drew less than thirty attendees.  Yet she gave me something I could keep and remember.  Her influence on me was never manifest in any way a bean-counter could capture. My subsequent influence on others is completely out of the picture.  I don’t remember what kind of grant we gave her, but it as within my discretionary money so it could not have been more than a couple hundred dollars and that is all the research would count.

How about from my side? Did I waste my time having tea with this old lady?  I would be hard-pressed to show a concrete public diplomacy outcome from having her as a friend and having the Consulate reach out to her and ensconcing us as an honorary part of her community.


What Good is a Speaker?

I was talking to a researcher about our speaker program.  I was a consumer of speakers when I worked in posts overseas and used to run the speaker program in Washington, so I know something about it.    The idea is the measure the effectiveness of the speakers we send overseas.  It costs a fair amount of money to send someone overseas, so it is good to measure, but the measures are inadequate.

They talked about measuring the number of people who listened to the speaker.  Moving a step up the sophistication pyramid, they also talked about estimating the number of people who may know the direct listeners, a secondary audience.  Of course, they would measure any media that came out.   What is wrong with this? Time.  It doesn’t measure the effects over time.  Refer again to my Eva Sopher example.  But there is a bigger flaw in this sort of measurement.  It doesn’t account for the meta-purposes.   When I gave the grant to the Theatro São Pedro, I really didn’t care if they did a performance or did anything “useful” with the money at all.  My grant was a kind of an ante-up or earnest money.   I was buying my way – the Consulate’s way – into the game, making us a part of that community. 

This is how good public diplomacy folks use the speaker program.   Bringing a speaker to an event is a way of opening a door to a community.  We cab piggy-back on the speaker’s expertise.  Bringing an expert on architecture, for example, makes us honorary experts too.  It puts the Embassy’s public affairs into the game.  Frankly, the message delivered on any particular occasion is usually the secondary effect.  The primary goal is relationship and community building.   So if you measure effectiveness by number of people who received a message, you have problems.

“Who?” May be More Important than “How Many?”

And that doesn’t even account for the “who” question.  If I shout out my window I may reach 100 people with my message. But if they don’t care or cannot do anything about what I am saying, it is a complete waste.  We often fall into the numbers trap.  It is seductive but pernicious.

Is There a Better Way?

You might be expecting me to say something about what we should do.  After all, I made such a big run up to it.   But I can’t.   I think that we should indeed measure numbers, reach, output etc.  But we have to recognize that there is a very big area of unknown and objectively unknowable stuff out there.   It is like the dark matter of public affairs.   It is the place where we have to apply judgment.  So I have no universal fix.  You have to use judgment in particular cases.

Indispensable Judgment

Judgment – now that is a slippery term and not a very popular one.  We like to get every process down to close detail so that we can be perfectly fair and consistent.   But the world doesn’t work like that.   We can program something only to the extent that we completely understand it and with the expectation that things will happen in the future as they has in the past.  Human affairs don’t work that neatly most of the time. So let’s indeed gather information and analyze it.   But then trust the judgment of the people we have trained and educated to make the right choices.   

Otherwise we will go with today’s tabloids and ignore the Eva Sophers.