The next day the team’s Scrum Master was catching up on long-ignored e-mail when Dave, the development manager, called. “Say Sue,” he said, “Great job your team did! I’ve been waiting for the product launch before I bothered you with this, but the appraisal deadline is next week. I need your evaluation of each team member. And if you could, I’d like you to rank the team, you know, who contributed the most down to who contributed the least.”
Sue almost heard the air escaping as her world deflated. “I can’t do that,” she said. “Everyone pitched in 100%. We could not have done it otherwise.” “But Sue,” Dave said. “certainly there must have been a MVP. And a runner-up. And so on.” “No, not really,” Sue replied. “But what I can do is evaluate everyone’s contribution to the effort.”
Sue filled out an appraisal input form for each team member. She rated everyone’s performance, but found that she had to check the ‘far exceeded expectations’ box for each team member. After all, getting out the product on time was a spectacular feat, one that far exceeded everyone’s expectations.
Two days later Sue got a call from Janice in human resources. “Sue,” she said, “Great job your team did! And thanks for filling out all of those appraisal input forms. But really, you can’t give everyone a top rating. Your average rating should be ‘meets expectations’. You can only have one or two people who ‘far exceeded expectations’. Oh and by the way, since you didn’t rank the team members, would you please plan on coming to our ranking meeting next week? We are going to need your input on that. After all, at this company we pay for performance, and we need to evaluate everyone carefully so that our fairness cannot be questioned.”
Sue felt like a flat tire. In the past, when she had a particularly difficult problem she always consulted the team, and she decided to consult them once again. At 10:00 the next morning, the entire team listened as Sue explained her problem. The had always come up with a creative solutions to problems she presented before, and she could only hope they would be able to do it again. She thought she might convince them to elect an MVP or two, to help her put some variation into the evaluations.
Sue suspected that Dave and Janice might not approve of her approach, but she didn’t realize that when the team members heard her dilemma, they would deflate just as quickly as she had. The best they could do was insist that everyone had given 200% effort, they had all helped each other, and they had thought that every single person had done a truly outstanding job. They were not interested in electing a MVP, but they were willing to choose a LVP. It would the unnamed manager who was asking Sue to decide amongst them.
Now Sue really had a problem. She had no idea how to respond to Dave and Janice, and the New Venture team had turned angry and suspicious. Tomorrow they would have to start working together on the next release. How could something that was supposed to boost performance do such a through job of crushing the team’s spirit?
Deming’s View
Sue is not the only one who had trouble with merit pay evaluation and ranking systems. One of the greatest thought leaders of the 20th century, W Edwards Deming, wrote that un-measurable damage is created by ranking people, merit systems, and incentive pay. Deming believed that every business is a system and the performance of individuals is largely the result of the way the system operates. In his view, the system causes 80% of the problems in a business, and the system is management’s responsibility. He wrote that using exhortations and incentives to get individuals to solve management problems simply doesn’t work. Deming opposed ranking because it destroys pride in workmanship, and merit raises because they address the symptoms, rather than the causes, of problems.
It’s a bit difficult to take Deming at face value on this; after all, companies have been using merit pay systems for decades, and their use is increasing. Moreover Deming was mainly involved in manufacturing, so possibly his thinking does not apply directly to knowledge work like software development. Still, someone as wise as Deming is not to be ignored, so let’s take a deeper look into employee evaluation and reward systems, and explore what causes them to become dysfunctional.
Dysfunction #1: Competition
As the New Venture team instinctively realized, evaluation systems which rank people for purposes of merit raises pit individual employees against each other and strongly discourage collaboration. Even when the rankings are not made public, the fact that they happen does not remain a secret. Sometimes ranking systems are used as a basis for dismissing the lowest performers, making them even more threatening. When team members are in competition with each other for their livelihood, teamwork quickly evaporates.
Competition between teams, rather than individuals, may seem like a good idea, but it can be equally damaging. Once I worked in a division in which there were two separate teams developing software products that were targeting similar markets. The members of the team which attracted the largest market share were likely have more secure jobs and enhanced career opportunities. So each team expanded the capability of its product to attract a broader market. The teams ended up competing fiercely with each other for the same customer base as well as for division resources. In the end, both products failed. A single product would have had a much better chance at success.
Dysfunction #2: The Perception of Unfairness
There is no greater de-motivator than a reward system which is perceived to be unfair. It doesn’t matter if the system is fair or not, if there is a perception of unfairness, then those who think that they have been treated unfairly will rapidly loose their motivation.
People perceive unfairness when they miss out on rewards they think they should have shared. What if the vice president had given Sue a big reward? Even if Sue had had acknowledged the hard work of the team, they would probably have felt that she was profiting at their expense. You can be sure that Sue would have had a difficult time generating enthusiasm for work on the next release, even if the evaluation issues had not surfaced.
Here’s another scenario: What would have happened if the New Venture team was asked out to dinner with the VP and each member given a good sized bonus? The next day the operations people who worked late nights and weekends to help get the product out on time would have found out and felt cheated. The developers who took over maintenance tasks so their colleagues could work full time on the product would have also felt slighted. Other teams might have felt that they could have been equally successful, except that they got assigned to the wrong product.
Dysfunction #3: The Perception of Impossibility
The New Venture Team met their deadline by following the Scrum practice of releasing a high quality product containing only the highest priority functionality. But let’s try a different scenario: Let’s assume that the team was given a non-negotiable list of features that had to be done by a non-negotiable deadline, and let’s further speculate that the team was 100% positive that the deadline was impossible. (Remember this is hypothetical; surely this would never happen in real life.) Finally, let’s pretend that the team was promised a big bonus if they met the deadline.
There are two things that could happen in this scenario. Financial incentives are powerful motivators, so there is a chance that the team might have found a way to do the impossible. However, the more likely case is that the promise of a bonus that was impossible to achieve would make the team cynical, and the team would be even less motivated to meet the deadline than before the incentive was offered. When people find management exhorting them to do what is clearly impossible rather than helping to make the task possible, they are likely to be insulted by the offer of a reward and give up without half trying.
Dysfunction #4: Sub-Optimization
I recently heard of a business owner who offered testers $5 for every defect they could find in a product about to go into beta release. She thought this would encourage the testers to work harder, but the result was quite different. The good working relationship between developers and testers deteriorated as testers lost their incentive to help developers quickly find and fix defects before they propagated into multiple problems. After all, the more problems the testers found, the more money they made.
When we optimize a part of a chain, we invariably sub-optimize overall performance. One of the most obvious examples of sub-optimization is the separation of software development from support and maintenance. If developers are rewarded for meeting a schedule even if they deliver brittle code without automated test suites or an installation process, then support and maintenance of the system will cost far more than was saved during development.
Dysfunction #5: Destroying Intrinsic Motivation
There are two approaches to giving children allowances. Theory A says that children should earn their allowances, so money is exchanged for work. Theory B says that children should contribute to the household without being paid, so allowances are not considered exchange for work. I know one father who was raised with Theory B, but switched to Theory A for his children. He put a price on each job and paid the children weekly for the jobs they had done. This worked for a while, but then the kids discovered that they could choose amongst the jobs and avoid doing the ones they disliked. When the children were old enough to earn a paycheck, they stopped doing household chores altogether, and the father found himself mowing the lawn along side the teenage children of his neighbors. Were he to do it again, this father says he would not tie allowance to work.
In the same way, once employees get used to receiving financial rewards for meeting goals, they begin to work for the rewards, not the intrinsic motivation that comes from doing a good job and helping their company be successful. Many studies have shown that extrinsic rewards like grades and pay will, over time, destroy the intrinsic reward that comes from the work itself.
One Week Later
Sue was nervous as she entered the room for the ranking meeting. She had talked over her problem with Wayne, her boss, and although he didn’t have any easy solutions, he suggested that she present her problem to the management team. Shortly after the meeting started, Janice asked Sue how she would rank her team members. Sue took a deep breath, got a smile of encouragement from Wayne, and explained how the whole idea of ranking made no sense for a team effort. She explained how she had asked for advice from the team and ended up with an angry and suspicious team.
“You should never have talked to the team about this,” said Janice. “Hold on a minute,” Wayne jumped in. “I thought our goal in this company is to be fair. How can we keep our evaluation policies secret and expect people to consider them fair? It doesn’t matter if we think they are fair, it matters if employees think they are fair. If we think we can keep what we are doing a secret, we’re kidding ourselves. We need to be transparent about how we operate; we can’t make decisions behind closed doors and then try to tell people ‘don’t worry, we’re being fair’”
Sue was amazed at how fast the nature of the discussion changed after Wayne jumped to her defense. Apparently she wasn’t the only one who thought this ranking business was a bad idea. Everyone agreed that the New Venture team had done an excellent job, and the new product was key to their business. No one had thought that it could be done, and indeed the team as a whole had far exceeded everyone’s expectations. It became apparent that there wasn’t a person in the room who was willing to sort out who had contributed more or less to the effort, so Sue’s top evaluation for every team member was accepted. More importantly, the group was concerned that a de-motivated New Venture team was a serious problem. Eventually the vice president agreed to go to the next meeting of the New Venture team and discuss the company’s evaluation policies. Sue was sure that this would go a long way to revitalize the team spirit.
Now the management team had a problem of its own. The knew that they had to live within a merit pay system, but they suspected they needed to rethink the way it was implemented. Since changes like that don’t happen overnight, they formed a committee to look into various evaluation and pay systems.
The committee started by agreeing that evaluation systems should not be used to surprise employees with unexpected feedback about their performance. Performance feedback loops must be far shorter than an annual, or even a quarterly, evaluation cycles. Appraisals are good times to review and update development plans for an employee, but if this is the only time an employee finds out how they are doing, a lot more needs fixing than the appraisal system.
With this disclaimer in mind, the committee developed some guidelines for dealing with various forms of differential pay systems.
Guideline #1: Make Sure The Promotion System Is Unassailable
In most organizations, significant salary gains come from promotions which move people to a higher salary grade, not merit increases. Where promotions are not available, as is the case for many teachers, merit pay systems have a tendency to become contentious, because merit increases are the only way to make more money. When promotions are available, employees tend to ignore the merit pay system and focus on the promotion system. Of course this system of promotions tends to encourage people to move into management as they run out of promotional opportunities in technical areas. Companies address this problem with ‘dual ladders’ that offer management-level pay scales to technical gurus.
The foundation of any promotion system is a series of job grades, each with a salary range in line with industry standards and regional averages. People must be placed correctly in a grade so that their skills and responsibilities match the job requirements of their level. Initial placements and promotion decisions should be carefully made and reviewed by a management team.
Usually job grades are embedded in titles, and promotions make the new job grade public through a new title. Thus a person’s job grade is generally considered public information. If employees are fairly placed in their job grade, and promoted only when they are clearly performing at the new job grade, then salary differences based on job grade are generally perceived to be fair. Thus a team can have both senior and junior people, generalists and highly skilled specialists, all making different amounts of money. As long the system of determining job grades and promotions is transparent and perceived to be fair, this kind of differential pay is rarely a problem.
The management team at Sue’s company decided to focus on a promotion process that did not use either a ranking or a quota system. Instead, clear promotion criteria would be established for each level, and when someone had met the criteria, they would be eligible for promotion. A management committee would review each promotion proposal and gain a consensus that the promotion criteria were met. This would be similar to existing committees that reviewed promotions to fill open supervisor or management positions.
Guideline #2: De-emphasize The Merit Pay System
When the primary tool for significant salary increases is promotion, then it’s important to focus as much attention as possible on making sure the promotion system is fair. When it comes to the evaluation system that drives merit pay, it’s best not try too hard to sort people out. Studies show that when information sharing and coordination are necessary, organizations that reduce pay differences between the highest and the lowest paid employees tend to perform better over time.
Use evaluations mainly to keep everyone at an appropriate level in their salary grade. Evaluations might flag those who are ready for promotion and those who need attention, but that should trigger a separate promotion or corrective action process. About four evaluation grades are sufficient, and a competent supervisor with good evaluation criteria and input from appropriate sources can make fair evaluations that accomplish these purposes.
Even when annual raises are loosely coupled to merit, evaluations will always be a big deal for employees, so attention should be paid to making them fair and balanced. Over the last decade, balanced scorecards have become popular for management evaluations; at least in theory, balanced scorecards ensure that the multiple aspects of a manager’s job all receive attention. A simple version of a balanced scorecard might also be used for a merit pay evaluations, to emphasize the fact that people must perform well on many dimensions to be effective. A supervisor might develop a scorecard with each employee that takes into account team results, new competencies, leadership, and so on. It is important that employees perceive that the input to a scorecard is valid and fairly covers the multiple aspects of their job. It is important to keep things simple, because too much complexity will unduly inflate the attention paid to a pay system which works better when it is understated. Finally, scorecards should not be used to feed a ranking system.
Guideline #3: Tie Profit Sharing To Economic Drivers
Nucor Steel decided to get into the steel business in 1968, and thirty years later it was the biggest steel company in the US. When Nucor started up, Bethlehem Steel considered it a mere gnat, but 35 years later Bethlehem Steel was not only bankrupt, but sold off for assets. So Nucor Steel is one very successful company that has done a lot of things right in a tough industry. Quite surprisingly, Nucor has a decades-old tradition of paying for performance. How does the company avoid the dysfunctions of rewards?
Nucor Steel started with the realization that profit per ton of finished steel as its key economic driver, and based its profit sharing plan on the contribution a team makes to improving this number. So for example, a team that successfully develops a new steel-making process or starts up a new plant on schedule will not get see an increase in pay until the process or plant has improved the company’s profit per ton of steel. Thus Nucor avoids sub-optimization by tying its differential pay system as close to the economic driver of its business as possible.
Guideline # 4: Reward Based on Span of Influence, Not Span of Control
Conventional wisdom says that people should be evaluated based on results that are under their control. However, this kind of evaluation creates competition rather than collaboration. Nucor makes sure that its profit sharing formula rewards relatively large teams, not just the individuals or small groups who have direct responsibility for an area. Following this principle, if a software program creates a significant profit increase, everyone from those who brought the idea into the company to developers and testers to operations and support people to the end users should share in any reward.
Nucor Steel works hard to create a learning environment, where experts move from one plant to another, machine operators play a significant role in selecting and deploying new technology, and tacit knowledge spreads rapidly throughout the company. It’s reward system encourages knowledge sharing by rewarding people for influencing the success of areas they do not control.
How, exactly, can rewards be based on span of influence rather than span of control? I recommend a technique called ‘Measure UP’. No matter how hard you try to evaluate knowledge work or how good a scorecard you create, something will go unmeasured. Over time, the unmeasured area will be de-emphasized and problems will arise. We have a tendency to add more measurements to the scorecard to draw attention to the neglected areas.
However, it is a lot easier to catch everything that falls between the cracks by reducing the number of measurements and raising them to a higher level. For instance, instead of measuring software development with cost and schedule and earned value, try creating a P&L or ROI for the project, and help the team use these tools to drive tradeoff decisions.
Guideline #5: Find Better Motivators Than Money
While monetary rewards can be a powerful driver of behavior, the motivation they provide is not sustainable. Once people have an adequate income, motivation comes from things such as achievement, growth, control over one’s work, recognition, advancement, and a friendly working environment. No matter how good your evaluation and reward system may be, don’t expect it to do much to drive stellar performance over the long term.
In the book “Hidden Value” , Charles O’Reilly and Jeffrey Pfeffer present several case studies of companies that obtain superb performance from ordinary people. These companies have people-centered values which are aligned with actions at all levels. They invest in people, share information broadly, and rely on teams, and emphasize leadership rather than management. Finally, they do not use money as a primary motivator; they emphasize the intrinsic rewards of fun, growth, teamwork, challenge and accomplishment.
Treat monetary rewards like explosives because they have will a powerful impact whether you intend it or not. So use them lightly and with caution. They can get you in trouble much faster than they can solve your problems. Once you go down the path of monetary rewards, you may never be able to go back, even when they cease to be effective, as they inevitably will. Make sure that people are fairly and adequately compensated, and then move on to more effective ways to improve performance.
Six Months Later
The New Venture Team is having another celebration. They had all been surprised when the VP came to their team meeting six months earlier. But they quickly recovered and told her that they each wanted to be the best, they wanted to work with the best, and they did not appreciate the implication that some of them were better than others. When the VP left, the team cheered Sue for sticking up for them, and then got down to work with renewed enthusiasm. Now, two releases later, the customers were showing their appreciation with their pocketbooks.
There haven’t been any dramatic pay increases and only the occasional, well-deserved promotion. But the company has expanded its training budget and New Venture team members have found themselves mentoring other teams. Sue is rather proud of them all as she fills out the newly revised appraisal input forms that have more team-friendly evaluation criteria. This time Sue is confident that her judgment will not be questioned.
