Stir the pot

This is for all you PM's out there who are constantly trying to make the spot between the rock and the hard place better for all.

I'm moved to write based on the following stimuli:

• The umpteenth request from a client that wants more for less - only this time do it faster
• The team, that despite their best intentions, did too much of this and not enough of that
• Management who wants to know, "What do PM's do?"

Pretty much a typical day at the agency.

Please share your trials, tribulations and triumphs herding the cats.

Train-Wreck Management

“On October 5, 1841, two Western Railroad passenger trains collided somewhere between Worchester, Massachusetts and Albany, New York, killing a conductor and a passenger and injuring seventeen passengers. That disaster marked the beginning of a new management era." [1] These words open Peter Scholtes classic book on leadership. He goes on to explain how the term "management" was unknown in the days of cottage industries. As business grew and became geographically disperse in the 1800's, a way to run these businesses had to be found. But there were no models outside the church and the military, so investigators into the train-wreck disaster looked to the Prussian army for a model. And there they found the classic organization chart - the one we know so well today. Scholtes calls it the "train-wreck" chart. It was revolutionary at the time.

The purpose of what became today's organization chart was clear: The assignment of responsibility would enable "prompt detection of derelictions of duty... and point out the delinquent." Scholtes says: "A fundamental premise of the 'train-wreck' approach to management is that the primary cause of problems is 'dereliction of duty'. The purpose of the organizational chart is to sufficiently specify those duties so that management can quickly assign blame, should another accident occur."[1].

Blame
Note the thinking here: Problems are caused by people who don't do their job well, so finding someone to blame is the first step to correcting problems. Scholtes notes: "The era of management that began in the mid-1800's can be characterized as "management by results.".... Since managers could no longer do the work themselves or direct others in the doing of the work, managers exercised their authority by holding people accountable for results.... In the 1950's, management by results reached its epitome in MBO (Management By Objectives) and performance appraisals, the Harvardization of train-wreck management."[1] He goes on to say that at the time, this theory of management was the best available, and it succeeded in creating order out of chaos. "People like Whistler, McCallum, Frederick Taylor or Henry Ford in the United States or Darby, the Stephensons, or Brunel in England were pioneers.... they did their best and, by and large, what they did was very good."

"Meanwhile, in Japan...." is the title of the next section Scholtes' book. He chronicles how a better approach to management emerged in Japan in the 1950's assisted by W. Edwards Deming. Deming taught that most of the problems we encounter (perhaps 90%) are the result of multiple influences, they generally cannot be attributed to a single cause. Assigning blame for a problem to the last person involved is worse than counterproductive, it will probably make the bad situation worse. Exhorting people to "be careful," "try harder," and "work smarter" is not useful if individuals have little effect on results. Rewarding or punishing people for outcomes that are not under their control can only result in discouragement - or in gaming the system. Instead, chronic problems must be fixed by finding their underlying causes and addressing these effectively. As Deming points out, this usually involves changing the system - the way things are done. And according to Deming, it is management's job to change the system.

Process or People?
Agile software development places a strong emphasis on putting change into the hands of front-line people on self-directed teams - isn't this contrary to Deming's philosophy? Writing in 1995, Scholtes lists what he calls "fads" for addressing systemic problems: "empower people, put them into self-directed teams, motivate them, offer incentives, reengineer and reinvent them." And then he says: "All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system.... A well-run organization with well-functioning systems allows people from top to bottom do work of which they can be proud."[1] So where does this leave us? Which is more important - process or people?

It helps if we trade in the overloaded word "process" and use "system."

In the article "Managing a Living System, not a Ledger"[2] H. Thomas Johnson says "Managers at Toyota believe that improving the system is the surest way to improve long term financial results." He points out that Toyota takes lots and lots of measurements, but they do not use these as performance measurements. Johnson writes: "...Toyota makes virtually no use of management accounting targets (or 'levers') to control or motivate operations... Toyota focuses its operations on continuous system improvement through endless rapid problem solving. And they emphasize genchi genbutsu, or 'going to the place,' to see where a problem occurs, firsthand. They don't rely on second-hand reports or tables and charts of data to achieve a true understanding of root cause. Instead they go to the place (gemba) where you can watch, observe, and 'ask why five times.' This attitude shows a deep appreciation that results (and problems) ultimately emanate from, and are explained by, complex processes and concrete relationships, not by abstract, quantitative relationships that describe results in simple, linear, additive terms." Winding up the article, Johnson says: "Financial quantities cannot reveal if a system is improving or not... No company that talks about improving performance can know what it is doing if its primary window on results is financial information and not system principles.... Companies that intend to perform like Toyota should recognize that... they will never get there by trying to motivate and direct 'lean' initiatives with 'lean accounting' and management accounting 'levers of control.'"

Taiichi Ohno on Standard Work
Let's go back to the source of the Toyota Production System, Taiichi Ohno, and see what he had to say about process - how it is established and how it is changed.[3]

There is something called standard work, but standards should be changed constantly. Instead, if you think of the standard as the best you can do, it's all over. The standard work is only a baseline for doing further kaizen. It is kai-aku [change for the worse] if things get worse than now, and it is kaizen [change for the better] if things get better than now. Standards are set arbitrarily by humans, so how can they not change?

When creating Standard Work, it will be difficult to establish a standard if you are trying to achieve 'the best way.' This is a big mistake. Document exactly what you are doing now. If you make it better than it is now, it is kaizen. If not, and you establish the best possible way, the motivation for kaizen will be gone. That is why one way of motivating people to do kaizen is to create a poor standard. But don't make it too bad. Without some standard, you can't say 'We made it better' because there is nothing to compare it to, so you must create a standard for comparison.

Take that standard, and if the work is not easy to perform, give many suggestions and do kaizen.

We need to use the words 'you made' as in 'follow the decisions you made.' When we say 'they were made' people feel like it was forced upon them. When a decision is made, we need to ask who made the decision. Since you also have the authority to decide, if you decide, you must at least follow your decision, and then this will not be forced upon you at all.

But in the beginning, you must perform the Standard Work, and as you do, you should find things you don't like, and you will think of one kaizen idea after another. Then you should implement these ideas right away, and make this the new standard.

Years ago, I made them hang the standard work documents on the shop floor. After a year I said to a team leader, 'The color of the paper has changed, which means you have been doing it the same way, so you have been a salary thief for the last year.' I said 'What do you come to work to do each day? If you are observing every day you ought to be finding things you don't like, and rewriting the standard immediately. Even if the document hanging there is from last month, this is wrong.' At Toyota in the beginning we had the team leaders write down the dates on the standard work sheets when they hung them. This gave me a good reason to scold the team leaders, saying 'Have you been goofing off all month?'

If it takes one or two months to create these documents, this is nonsense. You should not create these away from the job. See what is happening on the gemba and write it down.

Process AND People
Ohno believed that the primary job of team leaders (first line supervisors) is the constant improvement of the way work gets done. Work standards should be written and posted, but this had better not take very long because the standards should change all the time - at least once a month. Standards are not about how work should be done, but how work is being done. You don't want the standard to be too perfect, because that leaves no incentive for workers to improve their standards. If workers are annoyed by a standard, they are expected to change it. They do not drop a suggestion in a suggestion box, they do kaizen. That is, workers - led by their team leader - do many rapid experiments, find a better way, agree on the improvement, quickly document the new way, and use it. When a standard is improved, the decision for the change must be made by the people doing the work, so they won't feel it is being forced upon them.

People like to use effective processes, and they also like to have control over their own environment. The Toyota Production System provides for both. Ohno made it clear that people must be at the center of improving their own processes. Process improvement may be done only "at the gemba" and it is up to the workers to decide whether or not a proposed improvement should be implemented. Workers are expected to keep changing the way they do their job; in fact, it is bad leadership to have a process so perfect that workers have little incentive to improve it!

Assessment and Certification
Scholtes takes process improvement assessment programs such as ISO 9000 to task because even though they seem good on the surface, they have some problems:[1]

1. The pursuit of quality must be guided by a larger context than certification - it requires a holistic, integrated, long term commitment.
2. Certification is not equal to satisfied customers - you can do the wrong thing as long as you do it consistently.
3. Assessment has a tone of paternalism and mistrust - it replaces internal motivation with external motivation.
4. Assessment assumes that inspectors are all the same - but inspections are not standardized.
5. A certified process is difficult to change - Ohno would be appalled.

Conclusion
When Deming said "change the system", he was talking about changing the complex, interrelated processes used to get work done. Deming believed that changing the system is management's primary job, and in order to do this, managers need competency in four areas:

1. Appreciation for the overall system in which work is done
2. An understanding of variation - and the true relationship between cause and effect
3. Constant pursuit of learning (improvement) through designed experiments
4. An understanding of the psychology of people

When all of these areas are balanced and working together, great things can happen.

References
[1] The Leader's Handbook, by Peter R. Scholtes, McGraw-Hill, 1998.

[2] "Managing a Living System, Not a Ledger,", by H. Thomas Johnson, Lean Manufacturing 2007, Supplement to Manufacturing Engineering, SME, August 2007. Johnson also coauthored Profit Beyond Measure: Extraordinary Results through Attention to Work and People, Free Press, November, 2000.

[3] Workplace Management, by Taiichi Ohno, originally published in 1982, from translation by Jon Miller, Gemba Press, 2007.

Screen Beans Art, © A Bit Better Corporation

Bringing Lean to Software Development

Software is not about making something, it’s about making something work. Take software out of a cell phone, a car, a factory, a business, and it won’t work anymore. Software is the brains behind just about everything complicated that we try to do.

In 1960, J.C.R. Licklider, the visionary behind the Internet, wrote: “The main aims [of software] are 1) to let computers facilitate formulative thinking as they now facilitate the solution of formulated problems, and 2) to enable men and computers to cooperate in making decisions and controlling complex situations without inflexible dependence on predetermined programs.”[1] In 1962, Licklider articulated his vision of an “intergalactic network” connecting everyone in the world.[2] Throughout his life, Licklider could be found in a leadership role in one of the organizations that worked to convert his vision into reality: Bolt Beranek and Newman(BBN), ARPA, IBM Research, MIT. He died in 1990, just as the ideas he championed finally grew into the Internet.

If we look at the Internet through Licklider’s eyes, we see the real purpose of software – it does the routine thinking for us so that we can turn our minds to more complex problems. If we look at the evolution of the Internet, we see what developing software is really all about. At its core, software development is the process of gradually finding ways to turn over more and more of what we know to computers so that we have more space left in our minds to discover ever more interesting things.

The Truck-Driving Problem
Imagine that you are responsible for driving a truck across America, along highways, through cities and around detours, dealing with whatever idiosyncrasies that weather and traffic might throw at you. Now imagine that your job is not to drive the truck, but program a computer to drive the truck for you. How would you go about this task?

Many people – especially those who have never built software – would break this problem into a series of steps: first design the logic of the system; second, translate that logic into code; and finally, add the code to the truck and test that it works. But you can imagine that a problem as complex as the truck-driving problem is far too complex to solve with this approach. Instead, the problem is being tackled over time with a lot of experimentation and testing, one module at a time. Today we have laser following cruise control and tomorrow we will have lane following capability. Soon we might expect congestion detectors and GPS guidance. Someday the truck-driving problem might be solved, but by the time it is, the system will no doubt be a lot different than what we imagine today.

Conventional Wisdom
Unfortunately, it has been conventional wisdom in software development to separate design from coding, coding from testing, and software from the system it supports. This gives us development processes that ignore the learning loops necessary to discover the logic of a complex system, create concrete representations of the system, make sure that these work together, and over time, grow a system that becomes capable of accomplishing the complete job. In software development, we have been asked to solve too many truck-driving problems. And when it turns out that we have been handed an impossible problem, it’s usually the developers – not the process or the scale of the problem – that are held responsible for the failure.

When I was an IT manager in a manufacturing plant in the early 1980’s, we discovered that the conventional wisdom about the way that manufacturing should be done had become obsolete. We figured this out because our competitors in Japan started doing things that produced far better results than we were able to achieve. In response to this serious competitive threat, we threw out conventional wisdom and decided to try the counterintuitive approach being used in Japan, called “Just-in-Time.” As a management team, we came to agree on a few basic principles and enlisted the wisdom of everyone in the plant to figure out how to make these principles work in practice. Indeed, it took some time and a lot of hard work to implement Just-in-Time in our plant. As part of the effort, we completely changed the way we thought about quality and used statistics to gain a deeper insight into how our equipment worked. Slowly, it seemed to us, we got better. In retrospect, the improvement was dramatic, but it certainly didn’t feel like it at the time.

Today, the name “Just-in-Time” has been replaced with “Lean,” but to me it remains largely about the same thing: abandoning conventional wisdom that has become obsolete and moving on to a new way of thinking. The transformation will be multifaceted, principle-based, and entail a lot of very hard work by everyone, workers and managers alike. And – probably – it will only happen if there is something that shakes people up enough to cause them to abandon conventional wisdom and look for a new way of thinking about how to do their job.

Analogies
For those who look upon Lean as a set of operational practices, it doesn’t make sense to apply Lean to software development. Inventory, motion, variation, overproduction – the wastes of an operational process – don’t translate easily to the job of creating the brains of a system and making it smarter it over time. In fact, software development has long been plagued by inappropriate comparisons with other fields. Too often, analogies are created by someone who has only a passing acquaintance with a field. For example, someone who knows little about construction might say: “Buildings are completely designed before construction starts – if it works for buildings, why can’t it work for software?” I once discussed this analogy with a group of construction superintendants, who just about laughed me out of the room. They live in a world of incomplete and conflicting drawings, master schedules that bear no resemblance to reality, owners who regularly change their minds, and the constant threat of lawsuits.

Manufacturing analogies have been foisted upon software development by those who think that “cutting code” is pretty much the same thing as cutting metal. It’s painful to imagine the damage that has been done to software development by the manufacturing slogan “Do it Right the First Time.” I often wonder how this slogan came to mean that code should never have to be changed once it is committed to the code base. In our plant, product specs and manufacturing processes changed all the time; we just tested a product after each tiny step to be sure that no defects had appeared. Declaring that we should not have to change code once it is written is like decreeing that once we start our truck out across the country we should never have to change the planned route, no matter what detours or bad weather or flat tires we might encounter along the way.

I was only vaguely aware of the dangers of analogies when I set out to compare Lean software development to Lean manufacturing. I like using analogies – consider the truck-driving problem for example. I think of analogies as tools to help us understand and agree upon principles; but they should never be used as implementation guidelines. Copying specific practices from another field – or even from another company in our field – is a lazy way to implement change. First class execution is possible only when teams carefully think through the principles behind new ideas, understand how these principles apply to their world, and carefully adapt specific practices so that they make sense in the new context.

Paradigm Shift
I didn’t run into the term “waterfall” until 1999, and when I learned what it meant, I did not understand how it could possibly work. Therefore I was not surprised to discover that “waterfall” processes really don’t work – what surprised me was the fact that conventional wisdom held that they should work. I decided that the time had come for a paradigm shift in software development, similar to the paradigm shift that manufacturing experienced during the 1980’s and early 90’s.

Paradigm shifts require that someone with a clear understanding of a core problem in a particular field demonstrate a new and significantly better way to address this core problem. There’s no question in my mind that in manufacturing, the core problem was push scheduling – at least it certainly was in our plant. Simply implementing an effective pull system drove all sorts of other good behavior, and led eventually to significant cost reductions. We decommissioned a lot of software when we implemented our pull system, because the logic of our existing scheduling system was fundamentally flawed.

In that 1999 waterfall project, the core problem I experienced was the idea that development should be a series of handoffs: Analysis, Design, Coding, Testing, Deployment. The problem wasn’t that these steps were unimportant, but that each step was supposed to be completed before the next step began – and since the next step would be done by a different group of people, the handoff had to be done through documentation. The counterintuitive concept in Lean software development is to turn this process on its side and deploy completely done software in small increments. Let’s not even try to develop a system that will drive a truck, let’s see if it is possible to add some intelligence to cruise control and get that working while there is still a driver behind the wheel. Of course this means we are not sure how a fully automated truck will work, when it might be ready or how much it might cost. But if we keep the ultimate goal of an automated truck in mind and design our laser following cruise control wisely, we will be a step closer to a self-driving truck.

It might seem obvious that a completely automated truck is far too ambitious an undertaking, so splitting it up into incremental steps is the best way to make progress. What is counterintuitive is that this approach should scale downward. But it does. And if you look at any really successful complex system, this is the way it was developed. Lean development requires a visionary leader and a sponsor willing to fund the vision rather than a promise that is unlikely to be kept. However, such a sponsor can be hard to find; sponsors always seem to want a “plan.”
 
The “Plan”
The core problem in software development can be boiled down to the intense pressure for a “plan” that lays out what the software will be able to do, how long it will take, and what it will cost – and the subsequent expectation that development will follow this “plan”. The problem is magnified when the software is expected to be developed independently from the system for which the software will supply the brains. Why is planning a problem? Isn’t a plan essential? As Dwight Eisenhower once said: planning is indispensable, but plans are useless.

The problem with software is that the logic behind any system is inherently complex, and developing that logic independently from the system increases the complexity by at least an order of magnitude. As Fred Brooks once noted, “The complexity of software is an essential property… Many of the classical problems of developing software products derive from this essential complexity and its nonlinear increase with size.”[3] Trying to plan a large software development effort is not much different than trying to plan the development of a software package to drive a truck across America – without access to the truck.

You might notice that the core problem of software development is beginning to bear some resemblance to the core problem of manufacturing – we suffer from push scheduling in manufacturing and push planning in software development. But the antidote is somewhat different. In manufacturing, switching from push to pull scheduling gives dramatic results. In software development, switching from forecast-driven to feedback-driven development gives equally dramatic results.

The Antidote
Embedded in every development “plan” is a forecast – a guess really – of what will happen in the future. Plans that separate the logic of a system from the system itself are based on an additional layer of guessing – ahem – forecasting. If we base the development of a complex system on forecasts, we are doing forecast-driven development. If we develop and deploy small useful feature sets and get them working incrementally, then we are doing feedback-driven development. For simple systems, either approach will work. For large, complex or evolving systems, the only feasible approach is feedback-driven development.

There is a great appetite for predictability in any business that has to answer to shareholders or any organization that is under public scrutiny. This is entirely appropriate. However, the idea that predictability comes from creating a plan and then following the plan is fundamentally flawed when complex systems are involved. Following a plan will create predictability only to the extent that the plan is based on an accurate forecast of the future and a thorough understanding of the complexity of the system to be developed. Since I am a process control engineer, I would like to call to your attention something we learned a long time ago: When you really need predictability, you create a feedback system that can respond to change as it occurs, rather than a predictive system that is incapable of dealing with variation.

Many people are concerned about feedback-driven development – it seems to imply a lack of control. But when we are trying to create the brains behind a system, control is hardly what it is all about. The vision behind the Internet had nothing to do with control – and everything to do with finding ways for people to communicate and think more effectively. It took decades of detailed work engaging great minds from around the world for the Internet to become a reality, and it will continue to emerge for decades to come. This great network of software started with a vision and grew with careful guidance, detailed technical work, and extensive collaboration. If you want to understand how Lean software development works, you need look no further.

Originally Published in Lean Manufacturing 2007, a Supplement to Manufacturing Engineering

References and Notes
[1]Licklider, L.C.R. (1960). "Man-Computer Symbiosis" IRE Transactions on Human Factors in Electronics, v.HFE-1. p.4-11.

[2] This was described in a series of memos by Licklider.

[3] The Mythical Man Month, 20th Anniversary Edition, by Fred P. Brooks, Jr., Addison Wesley,1995.

How Does Toyota Do it? (Book Review)

When Matthew May was asked to help translate the Toyota Production System for the knowledge worker, he thought: “Huh? It doesn’t make any sense. Everyone knows factory work isn’t creative, right?”

How wrong he was. He found that factory workers were more engaged and creative than their corporate counterparts. Their jobs weren’t creative, their job were to be creative. He found that the Toyota organization implements a million new ideas a year – three thousand ideas a day. May notes that these new ideas are the real reason Toyota makes over twice as much money as any other carmaker, with under 15% of the market. These thousands upon thousands of implemented ideas are the engine of Toyota’s innovation.

Who needs another book on innovation? May asks in the introduction to his book, The Elegant Solution: Toyota’s Formula for Mastering Innovation. You do. This book is a great read – short lessons, lots and lots of case studies and examples both in the text and in the sidebars liberally sprinkled throughout the book. This is a book unlike any other on innovation; it is the result of May’s deep thinking and experience in trying to identify for Toyota University what innovation in Toyota actually means.

The book outlines three principles and ten practices for driving innovation in knowledge work, and concludes with a couple of brief chapters on how to get started down this path.

Principles:

1. The Art of Ingenuity. The pressure to innovate falls on the individual – every single individual in the company. First, ingenuity means connecting with your work, whatever it is, understanding why it is important, and making sure that it is a good fit with your skills and interests. If it isn’t create a job that is. Second, ingenuity means constantly experimenting to figure out how to do that job better. Third and most important, everybody is expected to use their ingenuity – everybody – all of the time.
2. The Pursuit of Perfection.  There is no such thing as perfection – but aspiring to achieve perfection is nevertheless the goal. A good example of this principle can be found in the Fast Company article No Satisfaction at Toyota, an article about constant innovation at the Georgetown Kentucky factory. Read the article, it does a fantastic job at explaining the pursuit of perfection.
3. The Rhythm of Fit. Innovation is always obvious – after the fact. But discovering the obvious is not such an easy task. It requires that you are grounded in today, yet have a clear vision of tomorrow. It requires systems thinking, rather than program thinking. And it requires the right social context – one that inspires rather than suppresses creativity.

Practices:

1. Let Learning Lead. Real learning is not about books or lectures or workshops. It is about constantly trying things and finding out what works. Learning involves asking the right questions much more than finding the right answers. Learning means moving the Scientific Method from PhD programs in Universities to the shop floor and the knowledge worker. Learning comes first.
2. Learn to See. Walk in the shoes of the front line worker. Live the life of the customer. Data is important, but it’s interpretation that converts data into information. Learn to walk around and observe what is really happening. Watch your customer, become your customer, involve your customer.
3. Design for Today. As good as Toyota is at anticipating the future, their innovations are always grounded in clear and present needs – demographic shifts, energy shortages, safety concerns. The idea is, as hockey great Wayne Gretzky once said: “Skate to where the puck is going to be, not where it has been.” Market leaders don’t so much create markets as they understand where the market is going to go, and skate there.
4. Think in Pictures.

5. Capture the Intangible. The most compelling solutions are often perceptual and emotional.” May says. For example, when the Lexus team was designing the car, the entire team spend three months living in luxury in southern California, just to feel what their customers felt. They learned that luxury cars were not transportation, they were a safe sanctuary and quiet escape. Provide an ‘experience’ instead of a ‘product’.
6. Leverage the Limits. This chapter centers on a wonderful story about Toyota’s North American Parts Operations (NAPO) stretch goals. In the year 2000, Jane Beseda, newly appointed vice president and general manager, challenged NAPO with ten audacious and mutually competing goals, including inventory reduction, response time reduction, and waste reduction. The goals were such that they could only be achieved by departments working together across what had been organizational boundaries, and the results were nothing short of amazing.
7. Master the Tension. A McKinsey study on global productivity in the late 1990’s found that the companies which improved productivity the most were invariably companies in highly competitive industries. These companies had to find a better way of doing things – it was a matter of survival – but could not afford to throw money at the problem. We agree with May – constraints coupled with challenge provide the breeding ground for innovation.
8. Run the Numbers.There is a place for instinct, but temper it with facts. Think hard about what the important numbers are – and they are often not the obvious ones. With insight into what numbers are important, capturing and analyzing data will confirm instinct (or not) and uncover patterns that are otherwise invisible. 
9. Make Kaizen Mandatory. Standards exist to be challenged and changed. They are the current best-known way of doing things, and they are documented and followed by everyone. But they objective is to change the standard, to keep on improving the way things are done. Taiichi Ohno once said “Something is wrong if workers do not look around each day, find things that are tedious or boring, and then rewrite the procedures. Even last month’s manual should be out of date.”
10. Keep it Lean. Scale it back, keep it simple, make it flawless, let it flow. May notes “We are hardwired to hunt, gather, and horde. To think more. So lean runs counter to human nature. Getting lean requires fighting the basic instinct to add, accumulate, store. Lean requires a precise understanding of value: the who, what, when, where, how, and why of the customer’s need. It means getting that value to them without complexity creeping in.

Get this book. Read it. It will change the way you think about innovation.

Cause and Effect

Is low cost achieved by focusing on cutting costs?
Is high utilization achieved by trying to utilize resources full time?
Does standardized work mean that work processes are followed without challenge?
Does everyone in your organization agree on the answers to these questions?

Cost
When Jane Beseda took over Toyota’s North American Parts Operation (NAPO), she knew that really dramatic results required breaking down the barriers between departments [1].  So she set three year Stretch Goals that were all but impossible – a 50% reduction in inventory, 25% increase in throughput, 25% reduction in freight costs, 50% reduction in packaging expense, 25% increase in space utilization, 50% decrease in backorders.  After a year of effort, department managers began to realize that that they were not going to achieve the Stretch Goal targets unless they changed their focus to cross-organizational projects.  Project teams had been struggling to coordinate their work through the traditional functional department planning approach. This was changed; the departments started to consider potential impact of their plans on other functions and areas of the broader organization. Only then were truly significant gains in eliminating waste and reducing cost realized. After three years, the results at NAPO were truly amazing; almost all of the Stretch Goals were achieved. But this would never have happened if everyone in the organization had not focused on overall system waste rather than individual department costs.

Low overall costs rarely come from lowering costs in individual departments; they come from lowering system costs. Lean companies have learned that this requires a keen understanding of underlying cost drivers and a system-wide effort to eliminate waste.  Consider Zara, a huge women’s fashion clothing chain headquartered in western Spain. It fills retail store orders twice a week, shipping garments around the world in two days – not folded compactly in boxes – but ironed and hanging on hangers ready to display. Zara realizes that it lowers overall costs by getting the clothing that women are asking for on shelves very rapidly.  Zara has fewer markdowns and unsold clothes, and it drives more business to stores with less advertising than its competitors. These tremendous savings would disappear if Zara focused on reducing shipping costs. This is systems thinking at work.

Over time, individual departments will find ways to drive out costs, but rarely do organizations attack the larger costs that occur between departments, divisions, functions, and companies.  A quick and easy cost reduction approach for a department might be to outsource work to obtain lower labor costs.  However, in most cases, outsourcing simply adds more boundaries to cross, and boundaries typically add 20 to 30% to overall costs [2]. One company I know outsourced the process of engaging contractors.  Since training is included in this outsourcing arrangement, all trainers must work through the outsourced contractor management company.  The amount of time and paperwork necessary to set up a training engagement at this company is about an order of magnitude greater than at any other company I have dealt with. However, the arrangement appears to be a cost reduction for the company, because contractors pay the contractor management firm through a fee imposed on their payment. This financial arrangement hides the tremendous waste in the engagement process, which includes a great deal of problem resolution by employees in the company receiving the training. But because of organizational barriers, these same employees have no way to change or improve a very burdensome process.

Standardized Work
In studying Toyota, many companies notice that standardized work processes are a key element of the company’s success.  What they generally miss is that at Toyota, standards exist to be challenged and changed by the front line employees doing the work.  Toyota actively encourages all employees to question any part of their job that is annoying or gets in the way of doing a good job. Employees are expected to look for a better way, prove with experiments that the new way is better, and then implement a new standard.  Taiichi Ohno, father of the Toyota Production System, wrote: “Something is wrong if workers do not look around each day, find things that are tedious or boring, and then rewrite the procedures. Even last month’s manual should be out of date.”

There are those who believe that standardized processes should be imposed from the outside and followed without question. However, companies with a Lean perspective believe that the essence of eliminating waste is to encourage everyone to attack and change the things that annoy them about their jobs. Only by engaging people in doing their job better on a daily basis can you get the sustainable gains that companies like Toyota enjoy. Work standards in a lean company spell out the current best known way of doing things, and they are a baseline for change. New standards are developed through constant experimentation by work teams, using the current work standards as the baseline against which improvement is measured.

At Toyota’s Georgetown, Kentucky plant the paint department has always been the biggest bottleneck preventing one-piece-flow of vehicles through the plant.  Over the past few years, however, things began to change [3].  First the people in the department devised a way to paint any color in any order – instead of piping paint to a robot through a tube that needs cleaning between colors, they now pipe each paint color to a canister just the right size to paint a car.  The robot picks up the right canister and Viola! the cost and time needed to clean out paint lines is eliminated.  The bottom line:  30% less paint used, a huge drop in the use of cleaning solvents, and cars can now be painted any color in any order. The throughput of the shop has increased from 33 to 50 an hour while the space required for painting was reduced by a third and employees were freed up for other work.

At most auto companies, this achievement would be cause for celebration – but at Toyota, constant improvement is the day-in-day-out job of the paint manager. There are no black belts, no outside process experts – just the people doing the work led by their first line managers. Everyone’s job is to keep on improving their work processes, every day, every week, every month. All workers are engaged in the relentless pursuit of perfection – the elusive goal which keeps everyone looking forward rather than patting themselves on the back.

In software development, we have been led to believe that “maturity” involves documenting best practices and making sure that they are followed. On the contrary, a truly mature organization expects development teams to constantly challenge and improve their processes.  The day an organization settles into believing that it is perfect is the day it invites its people to stop thinking. Real, sustainable improvement on all fronts comes only when all workers are expected to challenge and fix anything about their job that annoys them or keeps them from taking pride in their work.

Utilization
Managers in manufacturing plants, supervisors of computer operations, and highway engineers agree on one cause and effect relationship:  if your resources approach full utilization, response time slows to a crawl. Queuing theory is widely applied to servers and highways and manufacturing equipment.  However, in a development environment, managers seem to be unaware that attempting to achieve full utilization slows response time to a crawl and decreases actual utilization. Development organizations are no more immune from the laws of queuing theory than airlines. We see how constantly full airlines with no slack cause cascading problems whenever a slight perturbation is introduced into the airline system. And weather being what it is, perturbations happen all the time.

Similarly, when a slight perturbation is introduced into a fully scheduled development organization, cascading problems are inevitable. Knowledge work being what it is, there will always be perturbations to the most carefully laid out schedules. The only way to deal effectively with these perturbations is to follow the advice of queuing theory:  work in small batches, minimize the length of queues of work to be done, and never, ever schedule an organization beyond its capacity to deliver.  Do this and utilization will increase.  Focus directly on utilization and it is guaranteed to be sub-optimal.

Many organizations queue up requests for software development into large batches of work to be done, presumably because better decisions can be made if the whole picture is visible before work begins.  Challenge this assumption.  Why is it better to respond very slowly to a pile of accumulated requests rather than respond very quickly to the current most important outstanding request? What good, really, does it do to have a long list of work to do?  Looked at closely, it's hard to make a case for piling up lists of work-to-do that are far longer than you have a hope of accomplishing. You're pretty much wasting your time keeping track of stuff you can never get around to, and you're probably setting incorrect expectations on the part of the requestors. Just say no.

Unfortunately, once software has been around for a while, we often see that it takes longer and longer to run the regression tests as each new feature is added, since every feature increases the regression test load.  We call this increasing regression load the regression deficit, and unless it is systematically tackled and reduced, the code base will become increasingly difficult to change.  You will be tempted to release software in larger and larger batches, because regression testing takes so long. Similar to the paint shop manager, a software development manager’s job should be to help development teams chip away at the regression deficit every day, every week, every month, until throughput is increased and one-piece-flow of small feature sets becomes practical.

Cause & Effect
In order for organizations to perform brilliantly, there are two prerequisites:  First, everyone has to agree on what they want, and second everyone has to agree on cause and effect [4].  Let’s assume that everyone in your organization agrees on the results they want, their values and priorities, and the trade-offs they are willing to make in order to achieve those results.  The question to ask is – does everyone agree on cause and effect?

Is there general agreement on what actions will result in system-wide cost reduction?  Are individual departments expected to reduce costs independently, or is it clear that departments must work together to reduce overall system costs, even at the expense of costs in individual departments? Are the measurements in place to reduce the costs of crossing boundaries?

Is there general agreement on the best way to achieve consistent results through standardization?  Are standards always followed?  If not, do you really understand the root cause – are the standards irrelevant or inaccurate, too complex, or too far from the actual work being done? Are standards maintained by a central process group, or are they continually improved by the people doing the work?  Are front line people expected and encouraged to continually challenge and change standards?

Do you have a project scheduling system aimed at optimizing utilization?  Does the management team believe that the system is in fact optimizing utilization? Do people believe that full utilization is the right measurement to emphasize? Is time-to-market an important factor in your business? Does everyone agree on the relationship between time-to-market and utilization?

Lean thinking is counterintuitive; it flies in the face of conventional wisdom concerning cause and effect. No matter how well proven the results, no matter how intellectually sound the arguments of systems thinking are, it is almost impossible for those who have been successful with conventional wisdom to change their habits. It is very difficult for leaders to address the system costs that flourish between department boundaries if they believe that lowering costs in each department will add up to lower overall costs. When leaders believe that standardized processes are an end rather than a beginning, they have a difficult time leveraging the wisdom of front line employees. For those leaders who believe that focusing on full utilization will increase productivity, the outstanding productivity gains that come from focusing instead on throughput will not be available.  It all comes down to measures – small measures cause small results; global measures promote globally optimized results.

Those who have seen the dramatic results of Lean Thinking have come to believe that concentrating on throughput, making front line employees the center of focus of the company, and eliminating waste across the system are the tools of choice in a competitive environment.  Most management teams that have been threatened by fierce competition and survived have changed their habits and adopted the counterintuitive tenets of Lean Thinking. Virtually all companies facing fierce competition from Lean companies that have failed to adopt similar thinking have failed to thrive in the long run.
_________________
Footnotes:

[1] This story is from The Elegant Solution: Toyota’s Formula for Mastering Innovation, by Matthew E. May, Free Press, 2007, p. 139.

[2] Management Challenges for the 21st Century, by Peter Drucker, HarperBusiness, 2001, p. 33.

[3] From “No Satisfaction” by Charles Fishman, Fast Company, Dec 2006 / Jan 2007

[4] See “The Tools of Cooperation and Change,” by Clayton Christensen and co authors, Harvard Business Review, October 2006.


Screen Beans Art, © A Bit Better Corporation

Managing the Pipeline

“I used to have everyone I needed in my business unit,” she told me. “But then we reorganized in the interest of efficiency, and all of my software people were moved into a central group. Before the reorganization, I had a great software architect who helped design all of our new products. Today we’re supposed to figure out exactly what we want before we get a project approved. Then the IT people assigned to my projects don’t know anything about my business, so the team spins its wheels for a long time before it gets traction. Efficiency? What a joke.”

A couple years earlier, this rapidly growing young company (call it XYZ) saw its revenue flattening and had regulators questioning its processes. In an attempt to impose discipline and cut costs, XYZ centralized software development. Executives felt that this would give them more visibility into the development portfolio and ensure that a standard process was used throughout the organization. They also hoped to even out the software development workload, increase utilization of resources, and eventually be able to outsource a good deal of development.

The problem was, XYZ’s products were largely based on software. When the software developers left the business units, the company's track record for fielding successful new products plummeted. Time-to-market stretched out for even the most important initiatives, and market share dropped. What went wrong?

People or Resources?
Company XYZ did very well when new products were designed and implemented inside of a business unit. When a key part of the product development team, those developing the software, were removed from the business unit, many important features of product development were lost. For example, funding was no longer incremental, based on stages and gates. Complete funding for product development had to be justified and allocated in order to get software development resources assigned. In turn, a complete specification and estimate of spending was required before work could begin. Software architects were no longer involved at the fuzzy front end of new product development, and the discovery loops that used to be part of product development were no longer acceptable.

Even as feedback loops were removed from product development, they became more important than ever, because the software development people on the product team were usually not familiar with the business, and worse, they were not even familiar with each other. In places where tacit knowledge and team cohesiveness are important, treating people as interchangeable resources just doesn’t work.

Scheduling
Prior to the reorganization, when software people were embedded in divisions, some people were regularly assigned to several projects at a time, while others appeared to be less than fully utilized. The company decided that it would be more effective to assign individuals to only one project at a time, and if possible, the projects should be small. Unfortunately, this created a scheduling nightmare, so XYZ invested in a computerized resource scheduling system to help sort out the complex resource assignments.

Computerized scheduling systems have a well known problem: they do not accommodate variation. XYZ discovered that if projects didn’t end when they were scheduled to end, the system’s assumptions were invalid, so the system’s resource assignments were often out of touch with reality. The company tried to fix such problems by keeping some teams intact and by holding weekly management meetings to arbitrate the conflicts between the computer’s schedule and reality. In practice, the overhead of management intervention and idle workers waiting for teams to assemble outweighed any efficiencies the system generated.

Company XYZ also tried to reduce the variability of project completion by urging teams to make reliable estimates and rewarding project managers who delivered on schedule. Unfortunately, such attempts to reduce variability generally don’t work. The reason for this becomes clear with a quick look at the theory of variation.

Variation
W. Edwards Deming[1] first popularized the theory of variation, which is now a cornerstone of Six Sigma programs. Deming taught that there are two kinds of variation: common variation and special variation. Common variation is inherent in the system, and special variation is something that can be discovered and corrected. Common variation can be measured and control charts can be used to keep the system within the predicted tolerances. But it is not possible for even the most dedicated workers to reduce common variation; the only way to reduce common variation is to change the system. And here’s the important point: Deming felt that most variation, (95%+)[2] is common variation, especially in systems where people are involved.

The other kind of variation is special variation, which is variation that can be attributed to a cause. Once the cause is determined, action can be taken to remove it. But there is danger here: “tampering” is taking action to remove common variation based on the mistaken belief that it is special variation. Deming insisted that tampering creates more problems that it fixes.

In summary: The overwhelming majority of variation is inherent in a system, and trying to remove that variation without changing the system only makes things worse. We can assume that most of the variation in project completion dates is common variation, but since computerized scheduling systems are deterministic, they can’t really deal with any variation. The bottom line: a computerized scheduling system will almost never work at the level of detail that XYZ was trying to use it. Exhorting workers to estimate more carefully and project mangers to be more diligent in meeting deadlines is not going to remove variation from projects. We need to change the rules of the game.

We know that estimates for large systems and for distant timeframes have a wide margin of uncertainty, made wider if the development team is an unknown. We should stop trying to change that; it is inherent in the system. If we want reliable estimates, we need to reduce the size of the work package being estimated and limit the estimate to the near future. Furthermore, estimates will be more accurate if the team implementing the system already exists, is familiar with the domain and technology, makes its own estimates covering a short period of time, and updates these estimates based on feedback. The good news is, once such a team establishes a track record, its variability can be measured and predicted.

Utilization
Unfortunately, Company XYZ believed that efficiency would be improved by increasing resource utilization. Trying to maximize utilization can have serious unexpected side effects, not the least of which is decreased efficiency and reduced utilization. If this seems odd, think about how efficient our highways are during rush hour. Most systems behave like traffic systems; as utilization of resources passes a critical point, non-linear effects take over, and everything slows to a crawl. Even the most brilliant scheduling system cannot prevent delays if you insist on 100% utilization.

When a computer operations manager looks at the utilization history of her equipment, she would never say: “Look at that – we’re only using 80% of our server capacity and 85% of our SAN’s. Let’s use them more efficiently!” She knows that such high utilization is a warning that the systems are operating on the edge of their capacity, and even now response times are probably slowing down.

But when a development manager takes a look at the utilization history of his department, he will often say: “Look at that – we are only using 95% of our available hours. We have enough free time to add another project!” At the same time, he is probably asking himself, “I wonder why I’m getting all these complaints about our response time?” And all too often his solution is, “We’ll just have to set more aggressive deadlines.”

Response Time
Consider the release manager of a software product. Assume she has service level agreement which calls for critical defects to be found and patched in four hours, serious defects to be found and patched in 48 hours, and normal defects to be fixed in the next monthly release. You can be sure that her primary measurement is response time and she adjusts staffing until the service level is achieved. Because of this, there will always be people available to attack defects, and occasionally people may have a bit of spare time.

In one of my classes, two teams did value stream maps for almost the same problem – deliver on a feature request which would take about 2 hours of coding. One team documented an average response time of 9 hours to deployment, the other team documented an average response time of 32 days. In the first case, the policy was: “When a request is approved, there will always be someone available to work on it.” In the second case, the request got stuck twice in two-week-long queues waiting for resources. The interesting thing is that the first organization actually did more work with fewer people, because they did not have to manage queues, customer queries, change requests and the like. They were more efficient despite, or perhaps because of, a focus on response time rather than resource utilization.

The bottom line is that managing response time, or time-to-market, is more efficient and more profitable than managing utilization. You need some slack to keep development and innovation flowing. As any good operations manager already knows, when work flows rapidly and reliably through an organization, its efficiency and utilization will be higher than in a organization jammed up with too much work.

Rules of the Game
Queuing theory gives us six rules for reducing software development cycle time:

1. Limit work to capacity
2. Even out the arrival of work
3. Minimize the number of Things-in-Process
4. Minimize the size of the Things-in-Process
5. Establish a regular cadence
6. Use pull scheduling

1. Limit work to capacity
The biggest favor you can do for your organization is not to accept any more work than it can handle. Of course, to do this, you have to know the capacity of your organization. One way to estimate the capacity is to look at output. If you currently complete one large system a year, deliver about three services a quarter, and respond to about seven change requests per week, this is a rough approximation of your capacity, and a good limit on the amount of new work you should accept.

Next you might calculate how much work you have already accepted. In one of my classes, an executive did the math and discovered that he had seven years worth of work in a queue that was reviewed every week. He decided that he could toss out all but a few months of work; the rest would never get done, but it was consuming a lot of time.

2. Even out the arrival of work
At Company XYZ, one of the scheduling headaches was caused by a huge workload during the first six months of the year, and a relatively low demand for the second half of the year. At first this puzzled me, because the company’s business was not seasonal, so there seemed to be no reason for the uneven demand. I suspected that there was a sub-optimizing measurement somewhere that might be the cause.; When I asked if the annual budgeting cycle or executive performance measurement system might be driving uneven demand, my suspicions were confirmed. I recommended that the organization work to change the measurement system, rather than accommodate it.

3. Minimize the number of Things-in-Process
One of the basic laws of queuing theory is Little’s Law[3]:

According to Little’s Law there are two ways to improve response time: you can spend money to improve the Average Completion Rate, or you can apply intellectual fortitude to reduce the number of Things-in-Process. For example, assume you can respond to about six feature requests per month. If twelve requests are released for work, they will take an average of two months to complete. If, however, only three requests are released at a time, it will take an average of two weeks to respond to a feature request.

4. Minimize the size of the Things-in-Process
We’ve already noted the effect of high utilization on cycle time; we should also note that as batch size increases the effect is much more pronounced.[4] This is shown in the graph:

So if you want high utilization, you should develop in very small batches. For example, you will get much faster throughput and higher utilization if you develop ten services one at a time, rather than developing all ten at the same time.

5. Establish a regular cadence
In a lean factory, every process is runs at a regular cadence called ‘tact time.’ If you want to produce 80 cars in 8 hours, you produce 10 cars per hour, so one car rolls of the line every 6 minutes. In software development the recommended practice is to establish an iteration cadence of perhaps two weeks or a month, and deliver small batches of deployment-ready software every iteration.

A regular cadence, or ‘heartbeat,’ establishes the capability of a team to reliably deliver working software at a dependable velocity. An organization that delivers at a regular cadence has established its process capability and can easily measure its capacity.

A regular cadence also gives inter-dependent teams synchronization points that they can depend on. Synchronization points are good places to get customer feedback, they are useful for coordinating the work across multiple feature teams, and they can help decouple hardware development from software development.

6. Use pull scheduling
Once both batch and queue size have been reduced and a cadence has been established, pull scheduling is the best method to compensate for variation and limit work to capacity. At the beginning of an iteration, the team ‘pulls’ work from a prioritized queue. They pull only the amount of work that they have demonstrated they can complete in an iteration. When a team is first formed or the project is new, it may take a couple of iterations for the team to establish its ‘velocity’ (the amount of work it can complete in an iteration). But once the team hits its stride, it can reliably estimate how much work can be done in an iteration and that is the amount of work it pulls from the queue.

There are other points where queues might be established: there could be a queue of proposed work that needs a ROM (Rough Order of Magnitude estimate). There may be a queue of work for a preliminary architecture assessment. (See figure below.) Note that these queues should be short, and a team should not pull work from a queue until it has available time to do the work.

A pull system assures that everyone always has something to do (unless a queue is empty), but no one is overloaded. The development process is managed by managing the queues. Management intervention is accomplished by changing the priority or contents of the queues. The cadence should be fast enough that changes can wait until the next iteration; in which case, changes are accommodated at the cadence of the process.

Conclusion
Development teams can do a lot to control their own destiny. They can make sure they have the right information, the necessary skills, and the appropriate processes to do a good job. But some things that impact the performance of the development team are outside of their control. Managing the pipeline is one of those things. If a development organization is swamped with work, no amount of good intentions or good process can overcome the laws of physics. If deterministic rules are applied to an inherently variable system, no amount of exhortation, reward, or punishment can make the system work. When the rules of the game have to change, the six rules for reducing cycle time are a good place to start.

References
[1]W. Edwards Deming – 1900-1993. Thought leader of the Quality Movement in Japan and the US.

[2] W. Edwards Deming, The New Economics, Second Edition, MIT Press, 2000, p 38.

[3] Usually the numerator is WIP (Work-in-Process). The term Things-in-Process comes from Michael L. George and Stephen A. Wilson, Conquering Complexity in Your Business, McGraw-Hill, 2004, p.37.

[4] See Factory Physics by Wallace Hopp and Mark Spearman, McGraw-Hill, 2000

Disclaimer: XYZ Company is not a real company, it is an amalgamation of companies I have worked with.

Breaking The Quality–Speed Compromise

In the hotly contested commodity business of assembling computers, Dell enjoys a 50% cost advantage over its competitors.[1] This commanding advantage comes from Dell’s exceptional responsiveness to customers, flawless operations, and remarkable speed of execution. Conventional wisdom once held that the low cost producer could not provide customized high quality products. But Dell decided that its customers could have it all – low cost computers with the latest technology, built to order and delivered in a week.

In the book Hardball, George Stalk notes that when an industry imposes a compromise on its customers, the company that breaks the compromise stands to gain a significant competitive advantage.[2] For example, the airline industry imposes a big compromise on travelers: if you want low cost tickets, you have to make your plans early and pay a stiff penalty to change them. Southwest Airlines breaks this compromise: its customers can apply the cost of unused tickets to a future flight without a change fee.

In the software development industry, we impose many compromises on our customers. We tell them that high quality software takes a lot of time; we ask them to decide exactly what they want when they don’t really know; we make it clear that changes late in the development process will be very expensive. There’s a significant competitive advantage waiting for companies that can break these compromises. In particular, I’d like to focus on breaking the compromise between quality and speed, because many companies have achieved great leverage by competing on the basis of time.

When I teach classes on Lean Software Development, the first thing we do is draw value stream maps of existing software development processes. Starting with a customer request, the class draws each step that the request goes through as it is turned into deployed software which solves the customer’s problem. The average time for each step is noted, as well as the time between steps, giving a picture of the total time it takes to respond to a customer.

Next the class determines how much of the time between request and deployment is spent actually working on the problem. Typically, less than 20% of the total time is spent doing work on the request; for 80+% of the time the request is waiting in some queue. For starters, driving down this queue-time will let us deliver software much faster without compromising quality.

But reducing wait time is not the only opportunity for faster software development. Typically the value stream maps in my classes show a big delay just before deployment, at a step which is usually called ‘verification’. Now, I don’t have any problem with verification just before deployment, but when I ask, “Do you find any defects during verification?” the answer is always “Yes.” Therein lies the problem. When a computer hits the end of Dell’s assembly line, it is powered on and it is expected to work. The verification step is not the time to find defects; by the time software hits verification, it should work.

The way to get rid of the big delay at verification is to move testing closer to coding – much closer. In fact, testing should happen immediately upon coding; if possible the test should have been written before the code. New code should be integrated into the overall system several times a day, with a suite of automated unit tests run each time. Acceptance tests for a feature should pass as soon as the feature is complete, and regression testing should be run on the integrated code daily or perhaps weekly.

Of course, this testing regime is not feasible with manual testing, automated unit and acceptance tests are required. While this may have been impractical a few years ago, the tools exist today to make automated testing practical. Obviously not all tests can be automated and not all automated test suites are fast enough to run frequently. But there are many ways to make automated testing more effective; for example, each layer is usually tested separately – ie. the business rules are tested below the GUI with most database calls mocked out.

In most of the value stream maps I see in my classes, there is a huge opportunity to move tests far forward in the process and catch defects at their source. Many companies spend a great deal of time tracking, prioritizing, and fixing a long queue of defects. Far better to never let a defect into the queue in the first place.

There is another area of my classes’ value stream maps that raises a flag. Toward the beginning of the map there is usually a step called ‘requirements’ which often interacts with a queue of change requests. Dealing with change requests takes a lot of time and approved changes create significant churn. There has been a feeling that if only we could get the requirements right, this ‘change churn’ would go away. But I generally find that the real problem is that the requirements were specified too early, when it was not really clear what was needed. The way to reduce requirements churn is to delay the detailed clarification of requirements, moving this step much closer to coding. This greatly reduces the change request queue, because you don’t need to change a decision that has not yet been made!

Toward the end of my classes, we draw a future value stream map, and invariably the new value stream maps show a dramatically shortened cycle time, the result of eliminating wait time, moving tests forward, and delaying detailed specification of requirements. We usually end up with a process in which cross-functional teams produce small, integrated, tested, deployment-ready packages of software at a regular cadence.

This kind of software development process exposes another compromise: conventional wisdom says that changes late in the development cycle are costly. If we are developing small bits of code without full knowledge of everything that the system will require, then we are going to have to be able to add new features late in the development process at about the same cost as incorporating them earlier.

The cost of adding or changing features depends on three things: the size of the change, the number of dependencies in the code, and whether or not the change is structural. Since we just agreed to keep development chunks small, let’s also agree to keep changes small. Then let’s agree that we are going to get the structural stuff right – including proper layering, modularization that fits the domain, appropriate scalability, etc.

We are left to conclude that the cost of non-structural change depends on the complexity of the code. There are several measurements of complexity, including the number of repetitions (the target is zero), the use of patterns (which reduce complexity), and McCabe scores (the number of decisions in a module). It has been shown that code with low complexity scores has the fewest defects, so a good measure of complexity is the number of defects.

Which brings us back to our testing regime. The most important thing we can do to break the compromises we impose on customers is to move testing forward and put it in-line with (or prior to) coding. Build suites of automated unit and acceptance tests, integrate code frequently, run the tests as often as possible. In other words, find and fix the defects before they even count as defects.

Companies that respond to customers a lot faster than their industry average can expect to grow three times faster and enjoy twice the profits of their competitors.[3] So there is a lot of competitive advantage available for the software development organization that can break the speed–quality compromise, and compete on the basis of time.

References

[1] Michael L. George and Stephen A. Wilson, Conquering Complexity in Your Business, McGraw-Hill, 2004, p.48.

[2] George Stalk and Rob Lachenauer, Hardball: Are you Playing to Play or Playing to Win, Harvard Business School Press, 2004

[3] George Stalk, Competing against Time: How time-based Competition is Reshaping Global Markets, Free Press, 2003, originally published in 1990. p.4.

Screen Beans Art, © A Bit Better Corporation