On What we Want, What we Do, and What we Really Measure

This year I've twice visited Varanasi - an ancient city in India. Varanasi is the oldest living city in the world -- it is the seat of Hinduism and the place where Buddha gave his first sermon to establish Buddhism. On both trips I had the opportunity to meet an extraordinary gentleman -- Ajeet Singh -- someone who has dedicated his life to fighting child sexual trafficking (See http://www.guriaindia.org/about-us.php ).  What he does is fascinating, noble, and humbling and you can read more about it in the link above. However, he shared a couple of thoughts that got me thinking about something that is the focus of this blog.

Ajeet Singh Talking to Students of the Walton

College of Business' India Study Abroad

Program. 

Ajeet Singh and I discussed working with funding organizations. He was, quite obviously, very grateful for the support he received. However, he had an interesting challenge– Working with non-profits has increased pressure on him to professionalize and there is a need to assure funding agencies that their money was being used well. A key metric often related to whether children stayed in school for two years after they were rescued. This seemed appropriate, but as Ajeet Singh went on to explain further, the challenges with this measure started to become obvious. He described the girls he had rescued from the sex trade. How, these young girls, were fed growth hormone at the ages of 8 or 9 so that they could enter the sex trade. How they had free and easy access to alcohol and drugs. While a variety of means could be used to keep the girls in school for two years, such girls could not easily be mainstreamed into society. That he could easily maximize his score on that metric (two years in school) and that right after that, the girls would be right back in the red light area. Indeed, sometimes these children have to be taught how to laugh, how to smile, just how to be children. That both he and the funding agencies need an outcome quite different from the kids receiving two years of schooling – but that was often times what they could measure. My students and I were privileged to visit a school he runs for children of sex workers. As we saw the children engaging in uproarious laughter, or in just making noise, some of the issues he described became obvious.

What we saw while visiting Project Guriya was fascinating in itself and probably a rich subject for another post. However, the above conversation stayed in my mind. If we look

Our students working with students of sex workers in a sp-

ecial school in Varanasi, India. 

around us we will see
 that we oftentimes choose to measure our performance in a way that is convenient, but not necessarily correct. And I was reminded of a classical article I had read when I was working on my PhD. The article: “On the folly of rewarding A, while hoping for B” is a true management classic. The author, Steve Kerr, makes the case that organizations, society and individuals often times incentivize behaviors that are not really the desired ones. For instance, in the context of societal incentives Kerr states that in a democracy, the people want candidates to provide details about their objectives – how a proposed program would work, how it would be funded, and so on. However, since these details can be criticized and picked apart, during the election, the populace actually punishes the politicians who provide this information. This leads to most officials going beyond platitudes when campaigning (e.g. healthcare for all). In the context of organizations, Kerr uses the example of universities with a teaching mission. These universities are hoping for excellence in teaching, while rewarding and promoting individuals almost primarily on their research quality and productivity.

Kerr’s article highlights another common problem that plagues organizations. Units that are supposed to implement activities, are also charged with measuring how effective those activities were. This often leads units to look for evidence that would support their actions (as opposed to a balanced approach). As Kerr states: Units and individuals who have … “convinced top management to spend money, say, on outside consultants, usually are quite animated afterwards in collecting rigorous vignettes and anecdotes about how successful the program was.” In my experience even worse, is when the means with which activities designed to meet certain objectives, become the objectives themselves. So an organizational HR unit charged with improving the quality of applicants applying for jobs may identify attending university career fairs as one of the ways through which the objective can be achieved. But I have observed units, who at the end of the year, measure their performance based on the number of career fairs attended – a much more easily measured objective.

I always tell my students that constantly evaluating your own performance with the right metrics is key to being successful. But oftentimes we fall victim to measuring the wrong thing. As a PhD student in the 90s, I knew that a key objective was producing high quality research, relatively quickly. I often found myself falling victim to measuring my performance on the basis of number of hours worked. The greater the number of hours I worked the more I thought I was doing well. And yet, at some point, a few words of advice from an experienced Professor made me pause. And remember that my objective was high quality research that had to meet the standards of top notch journals. The journal editors would not ask me the number of hours I put in. They would look at the work in terms of creativity, uniqueness and contribution. And that perhaps to achieve all of the above, I needed to focus on taking time off, and do other things that would get the creative juices flowing. It’s a lesson I try to pass off to my current students – measuring ourselves with convenient but inappropriate measures breeds complacency as we venture down a path that takes us far away from our real goals.