Lean Programming

About the time of the 1980 NBC documentary ‘If Japan Can, Why Can’t We?’, I was the System Manager in a video cassette manufacturing plant, and our management team was asking this question every day.  Our Japanese competition was selling superior products at much lower prices, and we couldn’t figure out how they did it.  We knew we needed to make dramatic changes or close up shop, but we didn’t know what to change.

As far as we could tell, we were doing everything right.  We relied on optimized forecasting methods to determine economic lot sizes, and we used the latest MRP (Manufacturing Requirements Planning) software to launch daily schedules into the plant.  We had a sophisticated computer system that analyzed QC results and process parameters, to pinpoint the causes of defects.

We had some quality problems, and it took a month to fill most orders.  In any given week, we were able to pack out about 60% of the planned line items for the week.  But this was okay, because the other 40% of the week’s packout went into finished goods inventory.  Usually we had plenty of on-hand inventory for shipping standard orders.  Special orders were another matter, however.  The division vice president would often call to expedite special orders for important customers.

We moved in-process video cassettes from one workstation to another on carts, and we had a lot of carts.  There was never enough room to store all the carts at the next workstation, so carts full of inventory would get misplaced.   Sometimes cassettes were stacked on top of carts, and occasionally they would spill on the floor.  Video cassettes piled up in front of testing stations, so whenever a process drifted out of control, it took a while to discover that we were producing marginal product.  We had plenty of rework stations, to be absolutely sure that everything we shipped was good product.

All-in-all, we had about a month’s worth of work-in-process inventory. At the time, we blamed our inability to rapidly fill orders on bad forecasts from marketing. Later we were surprised to learn that the real culprit was our in-process inventory.  Today it is well known that the average shipping time of most supply chains is about the same as the average level of inventory in the supply chain.

Lean Manufacturing
At the end of World War II, Sakichi Toyoda, founder of Toyoda Spinning and Weaving company, dreamed of providing cars for the general public, much like Henry Ford’s dream thirty years earlier.  He chartered Taiichi Ohno to put in place an efficient production system to produce high quality automobiles.  Over the next three decades, Ohno developed the Toyota Production System, now known world-wide as Lean Manufacturing[1].  The foundation Ohno’s system was the absolute elimination of waste.

Ohno studied US manufacturing techniques, and learned a lot from Henry Ford’s pioneer work in assembly line flow.  However, the assembly line produced large lots of identical cars.  Ohno didn’t have the customer base to imitate the US practice of manufacturing in ‘economic’ (ie. large) lot sizes.  He was captivated by US supermarkets, however, where a small quantity of every product was placed on shelves, and as shoppers removed  products, the shelves were rapidly replenished.  He decided to place inventory ‘supermarkets’ throughout his plant, and found that this technique dramatically lowered the ‘waste’ of in-process inventory.  He named these inventory supermarkets ‘kanban’.

Because Ohno was converting a spinning and weaving company to an automobile manufacturer, he already knew how to avoid making bad product.  Founder Toyoda Sakichi had invented an automatic shut-off mechanism that stopped a weaving machine the minute a flaw such as a broken thread was detected.  Ohno moved this concept to car manufacturing, where he insisted that each part be examined immediately after it was processed, and the line stopped immediately if a defect was found.

To maximize product flow, standard work sheets were created, but these were not developed at a desk by engineers.  They were developed on the shop floor by the workers who know the process.  Standard cycle times and kanban shelf space for each item was determined and workflow was leveled.  Production workers were like a relay team, handing off the baton (product) to the next person.  The handoff required 100% quality and tight timing.  If things got delayed, teammates were expected to help each other set up a machine or recover from a malfunction.

Ohno’s aggressive elimination of waste led him to the twin values of rapid product flow and built-in quality.  Over time, Ohno discovered that these two values led to the highest quality, lowest cost, shortest lead time products possible.

Total Quality Management
About the same time, Dr. W. Edwards Deming was teaching Quality Management in Japan.  In fact, the Total Quality Management (TQM) movement cannot be separated from Lean Manufacturing.  Demming’s photo is in the lobby of  Toyota’s headquarters, bigger than the photo of founder Toyoda Sakichi. Demming didn’t find an audience in the US after WW II, because managers at the time thought that poor quality was caused by people who just didn’t want to do a good job.  They didn’t think there was much managers could do to improve quality except exhort employees to do a better job.

Demming’s basic message was that quality is a management responsibility, and poor quality was almost always the result of systems imposed on workers which thwarted people’s desire to do high quality work.  He taught the Japanese managers how to empower production workers to investigate problems and systematically improve processes.  He taught that teamwork and long term, trust-based relationships with suppliers were far better than adversarial relationships.  He emphasized a culture of continuous improvement of both processes and products. 

In the 1980’s, Demming’s fourteen points (See Appendix 1) were studied by virtually every manufacturing manager.  Among these fourteen points are the well known mantra’s:

• Don’t Inspect Quality In.
• Constantly Improve the System.
• Break Down Barriers Between Departments.

But a few of Demming’s fourteen points might seem revolutionary even today, such as:

• Drive Out Fear.
• Eliminate Quotas, Numerical Goals and Merit Ratings.
• Don’t Award Business Based on Price; Minimize Total Cost.

Paradigm Shift
When we first heard about Lean Manufacturing, we thought it was a hoax.  Get rid of safety stock, don’t run machines at full capacity, have suppliers deliver small lots on a daily basis?  This was so counter-intuitive, so against the paradigm of the day, that the Japanese manufacturing techniques were widely discounted.  TQM concepts were more intuitive, but alone they were not enough to lift us out of our dire situation.  Desperate for a change, we decided to give Lean Manufacturing a try, and in the end, it saved our plant.

The critical step in implementing Lean Manufacturing in our plant was a carefully planned changeover from push scheduling to pull scheduling.  We decided that we could not do it part way, we had to switch plant-wide, cold turkey, over a weekend.  We devised a simple simulation which we taught to every one in the plant – managers, shift supervisors, and operators.  Using the simulation, teams of workers designed the layout and flow in their areas, including the kanban cards and rapid changeover methods.  The entire plant held its collective breath as the pull system went into effect, but the workers knew what to do – they had developed the methods themselves.  The first week packout accuracy was 92%, and it got better from there.  We were able to fill special orders in two weeks, so the vice president could stop expediting orders.  In a short time we were down to one week of inventory and could fill any order in the same amount of time.  We had lots of extra space, and quality had never been better.

The most difficult part of implementing Lean Manufacturing was the paradigm shift it required.  Everyone ‘knew’ that large lot sizes were necessary to keep expensive machines running at full capacity.   They also ‘knew’ that machine changeovers took a long time, and every minute a machine was idle its burden rate went up.  In addition, large warehouse inventories were necessary to make sure that when a customer ordered a product it could be shipped immediately.  After all, customers didn’t want to wait the month it took us to produce the product.

One of the reasons why Lean Manufacturing has been so difficult to implement is because people must question established, known truths, and this is not easy.  Another reason is that practices which create local optimization at the expense of the overall system are difficult to recognize, let alone change.  Local optimization points provide attractive points of measurement, and inevitably, what is measured is optimized.

Simple Rules
In a January 2001 article in Harvard Business Review titled ‘Strategy as Simple Rules’, Kathleen Eisenhardt describes how smart companies thrive in a complex business environment by establishing a set of simple rules which define direction without confining it.[2]  She suggests that instead of following complex processes, using simple rules to communicate strategy is the best way to empower people to seize fleeting opportunities in rapidly changing markets.

The 1980’s were a time of profound change in US manufacturing, and the change was guided by a set of simple rules.  Simple rules gave guidance to every level of the organization, and got everyone on the same sheet of music.  They empowered people at all levels of the organization, because the provided guidance for making day-to-day decisions.  With simple rules, work teams were able to continuously improve the processes and products without detailed guidance or complex processes.  

The basic practices of Lean Manufacturing and TQM in the 1980’s might be summed up in these ten simple rules:

   1. Eliminate Waste
   2. Minimize Inventory
   3. Maximize Flow
   4. Pull From Demand
   5. Empower Workers
   6. Meet Customer Requirements
   7. Do it Right the First Time
   8. Abolish Local Optimization
   9. Partner With Suppliers
  10. Create a Culture of Continuous Improvement

These Lean Manufacturing rules have been tested and proven over the last two decades.  They have been adapted to logistics, customer service, health care, finance, and even construction.  The application of the rules may change slightly from one industry to the next, but the underlying principles have stood the test of time in many sectors of the economy.

Lean Programming
Recent work in Agile Methodologies, Adaptive Software Development, and Extreme Programming have in effect applied the simple rules of Lean Manufacturing to software development.  The results, which we call Lean Programming, are as dramatic as the improvements in manufacturing brought on by the Just-in-Time and Total Quality Management movements of the 1980’s.

Lean Rule #1:  Eliminate Waste
The first rule of Lean Programming is:  Eliminate waste.  That is, eliminate anything which does not add value to the final product.  In Lean Manufacturing, waste is identified through a value stream analysis, a process which identifies all activities in the value stream and identifies the specific value they add to the final product. The value analysis process then attempts to find a different, more efficient way to add the same value,

The documents, diagrams, and models produced as part of a software development project are often consumables, aids used to produce the system, but not necessarily a part of the final product.  Once a working system is delivered, the user may care little about the intermediate consumables.  Lean principles suggest that every consumable is a candidate for scrutiny.  The burden is on the artifact to prove not only that it adds value to the final product, but also that it is the most efficient way of achieving that value.

Lean Rule #2:  Minimize Inventory (Minimize Intermediate Artifacts)
In our manufacturing plant, we communicated this message:  Inventory is waste.  Why?  Inventory consumes resources.  Inventory slows down response time.   Inventory hides quality problems.  Inventory gets lost.  Inventory degrades and becomes obsolete.  The ‘benefits’ of inventory are oversold.  The ‘cost’ of inventory almost always outweighs such ‘benefits’.

The inventory of software development is documentation that is not a part of the final program.  As inventory, this documentation should be subject to value analysis.  Take requirements and design documents, for example. How much value do the really add?  How important are they to the final product?  If you compare requirements and design documents to in-process inventory, then it is striking to note that the time it takes to produce these documents probably determines the cycle time of the project.  Just as inventory must be minimized to maximize manufacturing flow, so too requirements and design documents must be kept to a minimum to maximize development flow.

There are many wastes associated with this excess documentation:  The waste of time producing the documents, waste of time reviewing the documents, and the work that goes into change requests and associated evaluations, priority setting, and system changes.  But the biggest waste of all is the waste of building the wrong system if the documentation does not correctly and completely capture the user requirements.

The best approach for minimizing intermediate artifacts is to raise the level of abstraction of documentation.  Instead of a 100 page detailed specification, write a 10 page set of rules and guidelines, and document only the exceptions.  Instead of a few inches of specifics, produce a concise 25 page matrix which summarizes the effort.

We know that users are relatively poor at envisioning the details of a system from most documents, and are even less likely to correctly perceive how it should operate in their environment until they actually use it.  Even if users could predict exactly how the system should operate at the present time, it is unlikely that the way the system is supposed to work months before it is delivered will be exactly the way users need it to work for the rest of its useful life.  All of this must be taken into account when we determine how much value these documents actually add to the final product.

Lean Rule #3:  Maximize Flow (Drive Down Development Time)
During the 1980’s we learned how to make products in hours which used to take days or weeks.  We learned that very rapid product flow resulted in very short cycle times, often one or two orders of magnitude lower than before.  During the 1990’s, e-commerce projects were often able to accomplish in weeks what used to take months or years in the traditional software development world.  Yes, in some sense they cheated.  But the bottom line is, huge amounts of useful software was deployed in the last five years with extremely short cycle times by traditional standards.

In a recent paper titled ‘Reducing Cycle Time,’[3] Dennis Frailey  proposes reducing software development cycle time using the same techniques employed to reduce manufacturing cycle time.  He suggests looking for and reducing accumulations of WIP (Work in Process).  Just as in manufacturing, if WIP is reduced, and the cycle time will be reduced.  To reduce WIP, Frailey recommends using the ‘Small Batch’ principle and the ‘Smooth Flow Principle’, concepts straight from Lean Manufacturing.

Iterative development is basically the application of these principles to programming.  The basic premise of iterative development is that small but complete portions of a system are designed and delivered throughout the development cycle, with each iteration adding an additional set of features.  The cycle time from start to finish of any iteration varies from a couple of weeks to a couple of months, and each iteration engages the entire development process from gathering requirements to acceptance testing.

Lean Rule #4:  Pull from Demand (Decide as Late as Possible)
In our video cassette manufacturing plant, we used to think that it would be ideal if our marketing department could forecast exact market requirements.  A lot of work went into sophisticated forecasting techniques to more accurately predict the future.  Then  one day we realized that we were trying to do the wrong thing.  It would not be ideal if we had a perfect forecast.  Instead, it would be ideal if we could reduce our reliance on forecasts by reducing the system response time so dramatically that the system could respond to change rather than predict it.

In a market where volatile technology requires constant product upgrades, Dell Computer has a huge advantage over it’s keenest competitors because it doesn’t forecast demand, it responds to it by making-to-order in an average of six days.  While Dell holds about six days of inventory, it’s competitors maintain six weeks of inventory.  Dell’s ability to make decisions as late as possible gives Dell a significant competitive advantage in a fast-moving market.

Software development practices which keep requirements flexible as close to system delivery as possible can provide a significant competitive advantage in a changing market.  In a volatile business environment, users are not able forecast their future needs accurately.  Freezing the design early in a software development project is just as speculative as forecasting.  Software systems should be designed to respond to change, not predict it.  In software development, as in building computers, the ability to make decisions as late as possible provides a competitive advantage.

Lean Rule #5: Empower Workers (Decide as Low as Possible)
A basic principle of Lean Manufacturing is to drive decisions down to the lowest possible level, providing both the tools and the authority for people “on the floor” to make decisions. When Toyota took over GM’s manufacturing plant in Fremont, California in 1983, it inherited workers with the worst productivity and absenteeism record in the industry. Those same workers doubled their quality and productivity record in two years. This was accomplished through formation of teams that were trained in work measurement and improvement techniques and expected to develop and continually improve their own work standards and practices.[4]

One of the problems with heavyweight intermediate documentation is that it attempts to make all of the decisions for developers, rather than giving them a set of guidelines.  In general, raising the level of abstraction of intermediate artifacts will give guidance as well as freedom to the developers as they make the detailed design and programming decisions.  It is always better to tell developers what needs to be done, not how to do it.

Developers need to understand the goal of their work and how it fits into the overall flow, what it means to meet customer requirements, and the architectural structure and GUI standards of the system.  They also need to know what they must accomplish, by when, and how to tell when it is complete.  Finally, their work needs to be made visible in short iterative cycles to provide the feedback necessary for continual improvement.

Lean Rule #6:  Meet Customer Requirements (Now and in the Future)
In his 1979 book ‘Quality is Free’, Philip Crosby defines quality as ‘conformance to requirements’.   The Standish Group study of 1994[5] noted that the most common cause of failed projects was missing, incomplete, or incorrect requirements.   The software development world has responded to this risk by amplifying the practice of gathering detailed user requirements and getting user sign-off prior to proceeding with system design.   However, this approach to defining user requirements is deeply flawed.

I  worked on one project in which the customer wanted a complex system delivered in ten months.  Time was of the essence – 10 months or bust.  And yet, being a government agency, the contract required sign-off on an external design document before internal design and coding could begin.  Several users were involved, and they dragged their feet on signing the documents.  Why?  They were concerned that they might approve something which would prove to be a mistake later on.  Since there was no easy way to change things after the design documents were signed, they took two months to approve the design.  An who can blame them?  Their jobs depended on them getting it right.  So half way into a very tight schedule, over two months of time and a lot of paper was wasted producing and getting user sign-off on design documents. 

Instead of encouraging user involvement, user sign-off tends to create an adversarial relationship between the developers and the users.  Users are required to make decisions early in the development process, and are not allowed to change their minds, even when they do not have a full concept of how the system will work or how their business situation may develop in the future.  Users are understandably reluctant to make these commitments, and they will instinctively delay decisions to as late in the process as possible.  Note that this instinct on the part of the users is in line with Lean Rule #4.

The most effective way to accurately capture user requirements is found in the iterative approach to system development.  By developing core features early and obtaining customer feedback in a focus-group demonstration of each iteration, a far more correct definition of customer requirements can be obtained.  In addition, if we accept that the requirements will necessarily change over time, we must start with the essential requirement that the system must be designed to easily adapt to changes over its lifecycle.

Lean Rule #7:  Do it Right The First Time (Incorporate Feedback)
Before Lean Manufacturing arrived at our plant in the early 1980’s, we occasionally had output of marginal quality. We would test to find the good product and rework the bad product. After understanding the “Do it Right the First Time” rule, we closed down rework stations and stopped trying to test quality into the product. Instead, we assured that each component was good at every handoff.  This involved having tests and controls at every point of manufacture to detect a drift toward out-of-spec product and stop production before any bad product was made.

“Do It Right the First Time” did not mean “Freeze the Spec”.  On the contrary, product specs changed constantly, and lean discipline meant being able to flawlessly adapt to changing market conditions.  This was accomplished through a product architecture which facilitated manufacturing change, monitoring techniques which detected errors before they happened, and tests which were designed before manufacturing began.

In 1987 Barry Boehm observed that it costs 100 times more to find and fix a problem after software delivery than to find and fix it in early design phases.[6] This observation and the “Do it Right the First Time” rule have been widely used to justify the overhead of developing a detailed system design before code is written.

The problem lies in the assumption that it is possible to generate a detailed set of documents that correctly define customer requirements, and that those requirements will not change. The fact is that requirements do change, and frequently, over the life of most systems. “Do it Right” has also been misinterpreted to mean “don’t allow changes.” In fact, once we acknowledge that change is a fundamental customer requirement, it becomes clear that what “Do it Right” requires that we provide for change.

If we want to meet customer requirements, and we acknowledge that customers don’t really know what they want at the beginning of development, then we need to incorporate a method of obtaining customer feedback during development.  Instead, most software development practices include a “Change Control Process” which makes it so difficult to respond to user feedback that developers are discouraged from asking for it.  Far from insuring a quality result, these change-resistant practices actually get in the way of “Doing it Right”.

Lean Programming employs two key techniques that make change easy. Just as Lean Manufacturing builds tests into process so as to detect when the process is broken, Lean Programming builds tests into the development process in order to ensure that when changes don’t inadvertently break the code. In fact, the best approach is to write the tests first, and then write the code. An excellent unit and regression testing capability is the best way to encourage change late in the development process.

The second technique for allowing change to happen late in development is refactoring, or improving the design of existing software in a controlled and rapid manner. When refactoring is an accepted practice, early designs can focus on the issue at hand rather than speculate as to what additional design elements will be needed. As the additional features are actually added, refactoring provides a new, simplified design to handle the new reality. When refactoring is a part of the process, we reduce speculation as to what will be needed in the future by making it easy to accommodate the future if and when it becomes the present.

Lean Rule #8: Abolish Local Optimization (Sub-Optimized Measurements are the Enemy)
In the 1980s, the biggest enemy of Lean Manufacturing was often the accounting department. We had big, expensive machines in our plant, and the idea that they should not be run at full capacity was radical, to put it mildly. We compiled daily reports of work-in-process inventory, and the accountants didn’t want these reports abandoned just because there was virtually no WIP to report.

A generation of accountants had to retire before it was “OK” to run machines below their full capacity. Designing machines for rapid changeover rather than highest throughput remains a tough sell even today. After 20 years, Lean Manufacturing is still counter-intuitive to those who lack a broad view of the enterprise.

In this context, let’s examine the role of managing scope in a software development project. Project managers have been trained to focus on managing scope, just as we in manufacturing were trained to focus on maximizing machine productivity. However, Lean Programming is fundamentally driven by time and feedback. In the same way that localized productivity optimization creates a sub-optimized overall process, so too, does focus on managing scope create a sub-optimized overall project management process.

Think about it – holding the scope to exactly what was envisioned at the beginning of a project has little value for the user whose world is changing.  In fact, it adds anxiety and paralyzes decision-making.  It doesn’t add much to the ultimate system, which will be outdated by the time it is delivered. Managing to a scope that is no longer valid wastes a lot of time and takes up a lot of space on issue lists, trade-off negotiations and ultimate fixes to the system to get things right in the end. However, as long as keeping a project within its original scope is a key project management goal, this measurement will continue to be optimized—at the expense of the overall value delivered by the project.

Scope will take care of itself if the domain is well understood and there is a well-crafted, high-level agreement on what the system will do in the domain. Scope will take care of itself if the project is driven in time buckets that are not allowed to slip. Scope will take care of itself if both parties focus on rapid development and solving the user’s problem, and adopt waste-free methods of achieving these goals.

Lean Rule #9: Partner With Suppliers (Use Evolutionary Procurement)
Lean Manufacturing did not remain in the manufacturing plant. Once the idea of partnering with suppliers was combined with an understanding of the value of rapid product flow, Supply Chain Management was born. People began to realize that it took tons of paperwork to move material between companies, and this did not add value to the product. Moreover, the paperwork cost more than one might expect, not to mention the delay in product flow that it caused. Even today, predictions of billions of dollars of savings resulting from business-to-business Web portals are based on cutting the cost of transactions required to move goods between companies.

Supply Chain Management caused companies to take a close look at their business-to-business contracts. All too often, these contracts were focused on keeping the companies from cheating each other. In addition, it was common to pit one vendor against another to assure supply and obtain the lowest cost. Again, Lean Manufacturing changed this paradigm. Deming taught that trusting relationships with single suppliers create an environment that allows optimizing the overall value to both companies.

Throughout the 1980s, companies achieved the highest quality and lowest costs in their supply chains by reducing the number of suppliers and working with the remaining suppliers as partners. The quality and creativity which resulted from collaborating supply chains was demonstrated to far outweigh the apparent (and sub-optimal) benefits that came from competitive bids and rapid turnover of suppliers. Partnering companies helped each other improve product designs and product flows. They linked systems to allow just-in-time movement of goods across several suppliers with little or no paperwork. The long-term advantages of a collaborative supply chain relationships are well documented.

Wise companies realize that traditional software development contract practices generate hidden wastes. As manufacturers discovered in the 1980s, trusted relationships with a limited set of suppliers can yield dramatic advantages. Without the adversarial relationship created by a constant focus on controlling scope and cost, software development vendors can focus on providing the best possible software for customers, fixing requirements as late as possible in the development process and providing the most value for the available money.

Lean Rule #10: Create a Culture of Continuous Improvement
When software development seems to be out of control, one response has been to increase the level of “software maturity” of the organization. This might seem to be in line with good manufacturing practice, where ISO 9000 certification and Malcom Baldridge awards are sometimes equated with excellence. However, these process documentation programs indicate excellence only when the documented process is excellent in the context of it’s use.

In many current software development projects, excellence means the ability to adapt to fast moving, rapidly changing environments.  Process-intensive approaches such as the higher levels of Software Engineering Institute's (SEI) Capability Maturity Model (CMM) may lack the flexibility to respond rapidly to change.  In a recent e-mail advisor from Cutter Consortium, Jim Highsmith highlights the tension between such heavyweight methodologies and lightweight methodologies such as Lean Programming.[7]

The question becomes, do process documentation certification programs stifle, rather than foster, a culture of continuous improvement?  Deming would probably turn over in his grave at the thought of tomes of written processes substituting for his simple Plan-Do-Check-Act approach:

• Plan:     Choose a problem. Analyze it to find a probable cause.
• Do:        Run an experiment to investigate the probable cause.
• Check:   Analyze the data from the experiment to validate the cause.
• Act:     Refine and standardize based on the results.

Iterative development allows the use of the Plan-Do-Check-Act approach within a project. During the first iteration, the hand-off from design to programming or programming to testing may be a bit rough. It’s okay if the first iteration provides a learning experience for the project team, because there are more iterations to come, so the team can improve its process. In a sense, an iterative project environment becomes an operational environment, because processes are repeated and Deming’s techniques of process improvement can be applied from one iteration to the next.

Product improvement is also possible with iterations, particularly if refactoring is used. In fact, refactoring provides a tremendous vehicle to apply the principle of continuous improvement to the programming environment.

However, we need improvements that span more than a single project. We must improve future project performance by learning from existing ones. Here again, Lean Manufacturing can point the way. During the 1980s, a set of practices summarized in the ten rules of Lean Manufacturing were adopted widely across most manufacturing plants in the West. These practices then spread to service organizations, to logistics organizations, to supply chains, and beyond. They have withstood the test of time across multiple domains.

Following the simple rules of Lean Manufacturing has brought dramatic improvements to every industry in which they have been applied. These same rules can and should be applied to software development projects. The resulting Lean Programming practices will lead to the highest quality, lowest cost, shortest lead time software development possible.
_______________

Appendix 1:  Summary of W. Edwards Demming’s 14 points

   1. Create consistency of purpose.
   2. Adopt a win-win philosophy.
   3. Don’t depend on mass inspection; build quality in.
   4. Don’t award business based on price; minimize total cost; build long-term relationships of loyalty and trust with a single suppliers.
   5. Constantly improve the system of production, service, planning, etc.
   6. Train for skills.
   7. Provide leadership:  help people do a better job.
   8. Drive out fear and build trust so everyone can do a better job.
   9. Break down barriers between departments; abolish competition and build a win-win system of cooperation.
  10. Eliminate slogans, exhortations and zero defect targets; the cause of the bulk of problems lie in the system, and are beyond the power of workers to correct.
  11. Eliminate quotas, numerical goals and Management by Objectives; substitute leadership.
  12. Remove barriers that rob people of joy in their work; abolish the annual rating or merit system.
  13. Educate and improve individuals.
  14. Involve the entire organization.

There are many summaries of Demming’s 14 points, which he modified throughout the years, in the spirit of continuous improvement.  The above summary is based on his last version of the 14 points.[8].
_________________

References

[1] The Machine That Changed the World : The Story of Lean Production, by Womack, James P., Daniel T. Jones, and Daniel Roos, New York: Rawson and Associates; 1990

[2] Strategy as Simple Rules, by Eisenhardt, Kathleen M and Donald N. Sull, Harvard Business Review, Volume 79, Number 1, January 2001, pp 107- 116

[3] Reducing Cycle Time, by Frailey, Dennis, Software Development Magazine, August, 2000

[4] Time-and-Motion Regained, by Paul Adler, Harvard Business Review, January-February 1993 pp 97-108

[5]Charting the Seas of Information Technology – Chaos, by The Standish Group International, 1994

[6] Industrial Software Metrics Top 10 List, by Boehm, Barry, IEEE Software, Volume 4 Number 5, September, 1987, pp 84-85

[7] E-Projects in India, by Jim Highsmith, e-Project Advisor, Cutter Consortium's e-Project Management Advisory Service, March 1, 2001.

[8] Gone But Never Forgotten, by Brad Stratton, editor, Quality Progress Magazine, March 1994
______________

Annotated Bibliography (Chronological)

Quality Control Handbook, Joseph M. Juran, originally published in 1951, now in its forth edition
Considered the standard reference in the field of quality. Like Demming, Juran consulted mainly in Japan during the 1950’s and 60’s.
Managerial Breakthrough, Joseph M. Juran, originally published in 1964
Widely ignored when it was first published, this book is now considered a landmark treatise on continuous improvement.
Quality is Free, by Philip Crosby, New York:  McGraw-Hill, Inc. 1979
This is the book we used to launch the TQM program in our plant.  Demming never liked the zero defects approach advocated in this book, and it contains little about statistical quality control.  However, many corporate executives got the quality message from Phil Corsby.
Study of ‘Toyota’ Production System from Industrial Engineering Viewpoint, by Shigeo Shingo, Osaka, Japan, Shinsei Printing Co. Ltd.  1981
The title of this book is an indication of the quality of the translation, but that was not a barrier to our plant.  We studied the book cover-to-cover and got our introduction to Lean Manufacturing from this book.
Toyota Production System, Practical Approach to Production Management, by Yasuhiro Monden, Norcross, Georgia, Industrial Engineering and Management Press.  1983
This is the classic book on Lean Manufacturing.
Zero Inventories, by Robert W. Hall, Homewood, IL, Dow Jones-Irwin 1983.
Robert Hall, a professor at Indiana University, understood the concept of Just-in-Time earlier than most academics.  This book is still considered the definitive work on JIT.
A Revolution in Manufacturing, the SMED System, by Shigeo Shingo, Cambridge, MA, Productivity, Inc.; Originally published as Shinguru Dandori in 1983, English translation 1985.
The title of this book is an indication of the quality of the translation, but that was not a barrier to our plant.  We studied the book cover-to-cover and got our introduction to Lean Manufacturing from this book.
The Goal, by Eliyahu M. Goldratt, First Edition Published in 1984, Second Revised Edition Published in Great Barrington, MA, 1992
This book is a business novel. It is the easiest book to read for an introduction to Lean Manufacturing and is definitely a classic.  You will find this book on the reading list of most Operations Management courses.  Goldratt went on to develop the ‘Theory of Constraints’ and write several related business novels.
Out of the Crisis, by W. Edwards Demming; 1986
This is the book in which Demming outlined his famous 14 points.
The Demming Management Method, by Mary Walton and W. Edwards Demming; 1988
Probably the best book there is on Demming and his management approach.
Toyota Production System – Beyond Large Scale Production, Taiichi Ohno, published in Japanese in 1978 and in English in 1988 by Productivity, Inc.
This is an explanation of JIT by the inventor. The English version arrived after JIT had become widespread in the US.
The Machine That Changed the World : The Story of Lean Production, by Womack, James P., Daniel T. Jones, and Daniel Roos, New York: Rawson and Associates; 1990
This landmark book about the Toyota Production System was the first to associate the term ‘Lean’ with manufacturing.
Lean Thinking, by Womack, James P., and Daniel T. Jones, Simon & Schuster; 1996
A follow-up on the story of Lean Production, this book extends the concept of Lean throughout the enterprise.  It shows how the "lean principles" of value (as defined by the customer), value stream, flow, pull, and perfection can be applied to all areas of the enterprise.  (Software development is implicitly included.)

Originally Published in Software Development Magazine, May and June, 2001.

Screen Beans Art, © A Bit Better Corporation

Component-Based Software Development

Around 1800, Eli Whitney proposed manufacturing rifles with interchangeable parts, instead of crafting each rifle individually. Widely regarded at the beginning of mass production, the concept of interchangeable parts led to a dramatic increase in rifle production capacity while delivering the additional benefits of consistent operation and easy field maintenance of weapons.

Component-based systems represent a paradigm shift in software development similar to that of using interchangeable parts in manufacturing. A component-based system is built of standard, reusable parts that become the fundamental building blocks of future software. Component-based systems promise numerous benefits, including flexibility, scalability, and maintainability.

The promise of component-based systems has been difficult to realize. The reality is that even through the last decade, monolithic systems have dominated corporate IT. And after a few years, monolithic system become legacy systems, because they are not flexible, changeable, or easy to connect to other systems. A company will get acquired or acquire other companies or spin off divisions, and all of a sudden, the monolithic systems get in the way of progress.

During the 1990's more software by far was developed for the Internet that was developed for all the corporations in the world. But Internet software was typically developed vary rapidly, with expediency taking over where architecture once reigned. And guess what - Internet software is largely component-based. It may seem like heresy to 'true' software developers, but an Internet startup could get a shopping cart and a search module and a merchant account and a product display package from other Internet startups. There is Mapping software and travel planning software and collaborative filtering software, and when you look at it closely, these are true components. Web sites can be rapidly manufactured with 85% components, 15% glue.

Now it's time to bring components to the corporate world, and they are headed our way under the name of Web Services. Here's a quote from the January 3, 2002 CBDi forum newsletter:

Web services provide formalized separation of concerns at a usable level of abstraction and you can implement web services right now with most of the current versions of platforms and software tools. Web services are fundamentally a low-cost technology investment that easily extend what you already have to provide real business value. First look for internal applications, there are many opportunities to simplify internal integration. Second look externally at public (non secure) or private B2B applications, and make sure those security and privacy budgets address the needs of Web services. Look at the Web services that IBM and Microsoft’s customers are already delivering.

Theory of Constraints

Two decades ago, Japanese car-makers developed several new manufacturing methods, including 'Just-in-Time'. Based on the theory that most in-process inventory spends most of its time in a buffer, waiting to be processed, Just-in-Time scheduling dramatically reduced inventory, which coincidentally reduced the amount of time it took product to move through a manufacturing plant.

In the 1980's, Dr. Eli Goldratt publicized the Theory of Constraints, which improved upon Just-in-Time methodologies, and in the 1990's, Goldratt adapted the Theory of Constraints to project management in his book "Critical Chain". Based on the idea that individual estimates of project time must be padded to allow for contingencies, Goldratt demonstrated that padded project time operates in the same manner as inventory buffers to delay projects.

It's Okay to be Late
The fundamental concept behind the Theory of Constraints is that elements of a project must be scheduled based on the average time they will take to accomplish, not the maximum time they might take to accomplish. A project buffer is placed at the end of the project, which allows for some activities to be late.

Of course, it's human nature to estimate time based on the worst case scenario, not average time to complete. As soon as people are penalized for estimating average time and then not meeting the estimate, they will revert to estimating the maximum time, and project schedules will again fill up with padded time estimates. Thus, in a project using Theory of Constraints, it is important that it is 'okay' to be late.

It may be hard to buy at first, but even though it's okay for individual elements to be late, Theory of Constraints project management makes it far easier to assure that the overall project is completed on time.

Last Planner

The Lean Construction Institute has developed an effective project management technique for construction called 'Last Planner System'. This is a deceptively simple system which involves having foremen (last planners) planning activities for the immediate future, using only two rules:

1. If it can't be done, then don't plan to do it.
2. If you plan to do it, then get it done.

These two simple rules, along with a similar set of rules for the period after the immediate future, have led to enormous productivity increases in construction. For more on this visit the Lean Construction Institute.

Screen Beans Art, © A Bit Better Corporation

The Impact of Logistics Innovations on Project Management

A project is a “temporary endeavor undertaken to create a unique product or service” (Project Management Institute, 1996). Projects include building a structure, developing a product, and executing a contract. Logistics “plans, implements, and controls the efficient, effective flow and storage of goods, services, and related information from the point of origin to the point of consumption” (Council of Logistics Management, 2000). Logistics is usually thought of in connection with military or manufacturing operations. However, there can be projects involved in logistics (eg. building a bridge to move troops) and logistics involved in projects (eg. supplying material to a construction site). It is not surprising, then, that these two disciplines interact and learn from each other. In this paper, great events in logistics are examined to uncover their impact on project management.

Great Logistics Feats of Antiquity
In May of 218 BC, 29 year Hannibal led about 40,000 troops, thousands of horses, and 38 elephants over the Pyrenees and Alps from Spain to Italy, a feat which had been considered impossible. During this 15 day ‘project’, his troops built jetties and rafts for the elephants to cross the Rhone River, raided towns for provisions, and struggled through avalanche-blocked mountain passes while under attack. Many of the troops and most of the elephants were lost, but even in this weakened conditioned, Hannibal defeated the Roman army that awaited his arrival. (Encyclopedia Britannica: Hannibal)

Hannibal has been singled out as a forerunner of grand strategy in military campaigns. His superb logistics while crossing the Alps is a precursor of military logistics today, a discipline in which the US military is considered the world leader. The routine logistics of military operations are interspersed with unique, temporary ‘projects’, of which Hannibal’s crossing is a prime example. The project management challenges of crossing the Alps included fixed resources, limited time, physical obstacles and troops with mixed allegiances who were ill-suited to the task. The techniques Hannibal used to meet these challenges (planning, field engineering, rapid movement, periodic re-supply, and attentiveness to troops and animals) are standard military practice today. (Encyclopedia Britannica: Strategy)

There are other great feats of antiquity which are forerunners of today’s project management practices. Building the pyramids comes immediately to mind. The Great Pyramid in Egypt was built around 2500 BC. It took about two decades and somewhere between 20,000 and 100,000 laborers to erect. Stone was cut with crude hand tools, transported without wheeled vehicles, and put into place without lifting machinery. This “masterpiece of technical skill and engineering ability” remains “perhaps the most colossal single building ever erected on the planet”. No taller building was built until the 19th century. (Encyclopedia Britannica: Giza, Pyramids of; Building Construction)

Construction projects today are not significantly different than those which built the pyramids. An architect designed the building, materials were obtained from quarries and transported to the site, experts in various ‘trades’ prepared the materials and constructed the pyramid. Built to a fixed plan, time and resources were more or less unlimited for the pyramids (unlike today).

There is another impressive feat that has been executed from the most ancient of times and continues to this day: The Event. Perhaps it is a wedding or a funeral or a coming-of-age ceremony – just about any large gathering has always required serving a lot of food to a lot of people in a very short time. Perhaps several clans gathered once a year, and there was a woman who was particularly good at organizing such events. Planning, gathering food, attending to living quarters, cooking, serving, might be a tradition or a ritual, but the event always came off best if someone behind the scenes was coordinating it all. In some sense, the most seasoned project managers throughout various cultures and times might be the event organizers who really knew how to throw a good party on a inflexible schedule with limited resources.

From Logistics to Project Management
All of the knowledge areas of project management were are developed in these three examples of logistics from antiquity: staging a celebration, managing construction, and executing a military movement. A celebration required excellent procurement, scheduling, quality and resource management. Construction projects required superior planning, managing a fixed scope over a long time, procurement and human resource management. Military movements needed excellent execution, careful management of fixed resources, precise timing, superior human resource management and continual risk assessment.

Viewed from a different perspective, advances in logistics might be thought of as starting with project management. For instance, military logistics may start with unique projects, like ferrying elephants over a river, but once the technique is developed, it can be reused to ferry other large items over a river. Feeding a large number of relatives may be a project the first time it is done, but feeding hundreds of people a sit-down dinner in fifteen minutes is routine logistics at conventions held every day. Even the pyramids were more or less standard burial structures for a period of over 500 years. (Encyclopedia Britannica: Pyramid)

The disciplines of logistics and project management overlap in such a way that it is not always possible, or even necessary to tell them apart. Projects that are repeated and standardized may become exercises in logistics, while the first implementation of a logistical solution might be considered a project. Since both disciplines often address similar problems, there is likely to be a significant transfer of ideas from one discipline to the other.

Traditionally, logistics has been the focus of much study and innovation. Since logistics is usually applied to on-going operations, improvements in logistics usually result in increasing benefits over time. If, for example, the cost of a single car can be reduced by a dollar, then in a factory making 250,000 cars annually, a quarter million dollars will be saved each year.

The author has observed that major advances in logistics usually make their way into project management in ten or fifteen years, since the same forces that spurred the paradigm shifts in logistics are usually at work in the broader economy. Below are two examples of logistics innovations moving from manufacturing to construction in one or two decades.

Standardized Parts
Two hundred years ago, the US military wished to procure 40,000 rifles. At the time, rifles were made by skilled workers who fashioned one rifle at a time. Each rifle was different from the next, so maintenance required individually fashioned parts. Unfortunately, there were not enough skilled workers in the country to make the necessary rifles, let alone maintain them.

In 1798, Eli Whitney, the inventor of the cotton gin, proposed that he could make an incredible 10,000 rifles in two years by designing machinery (templates and fixtures) to make standardized, interchangeable parts. This was a major paradigm shift for manufacturing. It took Whitney over ten years to perfect his manufacturing system, which is regarded as the birth of the tool and die industry. Ultimately the system was successful, and the rifles in the war of 1812 were produced much faster, had higher quality, could be more easily maintained, and cost a great deal less than previous rifles. (Encyclopedia Britannica: Eli Whitney; Tool and Die Making) (Taylor, 1990)

In the 1820’s, US sawmills began producing standard dimension lumber in quantity, and in the 1830’s cheap machine-made nails became available. The first ‘balloon’ frame building is thought to be a warehouse in Chicago built in 1832. Standard construction techniques rapidly evolved: 2x4 studs placed 16 inches on center, 2x10 joists spanning up to 20 feet, stability provided by ¾” sheathing through which windows and doors were cut. Machine-made nails were easily driven into the soft wood and a building could be rapidly assembled from manufactured materials by (relatively) unskilled workers. Houses are still built this way, almost two centuries later. (Encyclopedia Britannica: Building Construction)

It’s easy to visualize how standardized, interchangeable manufacturing parts influenced the development of standardized, interchangeable building parts. The entire direction of the construction industry was heavily influenced by Eli Whitney’s rifles. Although a construction project was still a unique and temporary event, it took on many of the features of standardized manufacturing.

Mass Production
In 1908, Henry Ford introduced the Model T Ford. It was so successful that Ford had to continually invent faster, cheaper ways to manufacture the car. Over the two decades of the Model T’s life, Ford perfected the assembly process, introducing the first moving assembly line in 1913. In 1927, the River Rouge plant received just enough iron ore each morning to make a day’s worth of cars; 28 hours later the ore had become the steel in a finished car. In two decades, Ford produced almost 17 million Model T’s, displacing many horses in the process. The Model T precipitated one of the most rapid and pervasive changes of the lifestyle of common people in history. (Encyclopedia Britannica: Henry Ford)

The steel coming into Ford’s plant also moved into the building industry. During the 1920’s between World War I and the Depression, steel framed high-rise buildings came to the cities of America. As these buildings went up, specialized trades became more important, since framing, sheathing, plumbing, heating, elevators, etc. each required full time specialists. (Encyclopedia Britannica: Building Construction) The various trades required someone to sequence their activities, supply materials, assure safety standards were met, and generally manage the project. Thus the construction project manager emerged to coordinate a complex series of specialized but interrelated construction tasks, each of which contributed to the completion of the building. A well-managed construction project bears quite a few similarities to an assembly line.

Just In Time
Henry Ford’s River Rouge plant was an excellent example of Just-in-Time manufacturing, but his ideas on inventory were not widely held. After World War II, operations research developed theories on inventory management which determined optimized lot sizes, reorder points, and distribution stocking levels. In the 1970’s, a radically different theory was developed in Japan, spearheaded by Toyota Motor Corporation. (Shingo, 1981) The concept that inventory should be kept at a minimum, lots should be very small, and products should be built ‘on demand’ rather than stocked, ran contrary to the current manufacturing ‘wisdom’ in the US. However, Just-in-Time models proved to have significant advantages in capital reduction, plant throughput, quality assurance and market responsiveness.

The benefits of Japanese manufacturing techniques began to dawn on the US manufacturing community in the 1980’s. Eliyahu Goldratt’s 1984 novel, The Goal, popularized Just-In-Time concepts and introduced the ‘Theory of Constraints’. (Goldratt, 1984) This theory suggested that finding and ‘feeding’ the bottleneck workstation in a manufacturing plant would allow all other processes to hold to a steady and predictable pace. Pacing manufacturing with the ‘constraint’ workstation required the manufacturing logistics community to abandon many long-held beliefs about inventory management. But the new logistics paradigm worked so well that it rapidly became standard practice in manufacturing and distribution logistics.

If the author’s thesis that logistics innovations make their way into project management in about a decade is true, then Just-In-Time concepts or their derivatives should have impacted project management in the mid 1990’s. Sure enough, in 1997, Goldratt published Critical Chain, a novel in which he applies the Theory of Constraints to Project Management. (Goldratt, 1997)

The Theory of Constraints is useful in project management when multiple projects are competing for the same resources. Similar to its application in manufacturing, Theory of Constraints project management finds and ‘feeds’ critical path tasks and bottleneck resources. Most of the ‘common wisdom’ embodied in project scheduling techniques must be re-thought for this to work in the project environment. For instance, task durations are planned at 50% probability of completion. Thus it is expected that half of the time, tasks will be late. Indeed, it has proven to be difficult to abandon the perception that it is ‘good’ for all tasks to be completed within their estimated time.

Just-In-Time and Theory of Constraints focus on optimizing the entire flow of material or work, instead of sub-optimizing individual areas. The organizational structure and reward system in the US is often set up to reward individual effort, without much regard to (or understanding of) the impact of that effort on the ‘big picture’. Since World War II, the US economy has became increasingly globalized, and countries with different cultures and reward systems are selling products in the US. Unconstrained by ‘standard practice’, these countries consider and optimize manufacturing from a different, often broader perspective. This globalization of the economy has played a large role in bringing Just-In-Time and the Theory of Constraints to logistics.

Globalization has influenced project management also. In the electronics industry, very rapid time-to-market with ‘cast-in-stone’ release dates are required for new products, and Theory of Constraints project management is being widely applied to meet these challenges.

Lessons for the Information Industry
One of the fundamental driving forces behind paradigm shifts in logistics is an overwhelming need for lower cost and less specialized labor, which leads to standardization. This is often followed by an opposing force, the desire for customization.

Standardization
In 1800, the lack of enough skilled workers to make rifles in the face of an impending war was an unacceptable situation that demanded the invention of standardized, interchangeable parts. In the early 1900’s, the demand for inexpensive transportation that allowed ordinary people to travel great distances was a key forcing function in the emergence of mass production.

It’s easy to see this drive for standardized, interchangeable parts at work in the information industry. In 1983, the IBM PC took the world by force, rapidly becoming the standard of the industry and soon outstripping IBM’s ability to maintain control. (One can imagine that Henry Ford had to invent the assembly line to avoid the same fate as IBM.)

Standardized Internet access, embodied in standard web browser capability, has driven the on-line revolution that is sweeping the country. A single cell phone frequency in Europe has spurred European cell phone use to a far greater level than in the US, which has multiple cell phone frequencies. In the face of a huge demand for a standardized document format, Microsoft jeopardized its position by releasing incompatible versions of Microsoft® Word, while Adobe® Acrobat® filled the vacuum with a standardized ‘Portable Document Format’ (PDF).

Standardization is driven by an insatiable need which cannot be satisfied at an acceptable price by the available techniques or skilled people. Today the software industry has an enormous gap between the need for programming and the available programmers. It should be obvious to students of economic history and that the software development environment is ripe for a massive switch to standardized, interchangeable, mass-produced components.

Mass Customization
As brilliant as Henry Ford was, he missed out on a major trend that overtook the success of his Model T. After two decades, people began to get tired of black (the only color of the Model T). Copying Ford’s methods, other manufacturers figured out how to make inexpensive cars, and they started adding new features. Ford’s failure to recognize the shift in customer desires caused his company to loose its overwhelmingly dominant position in the industry. (Encyclopedia Britannica: Henry Ford) In the 1970’s the same thing happened to the US car makers – their failure to respond rapidly to the huge demand for fuel economy and high quality allowed Japanese car-makers to gain a large market share.

It is not surprising that one of the latest logistics innovations is mass customization, enabled by the information revolution. Dell Computer builds computers only after receiving an order, and ships the customized computer within days. Anderson Windows custom-builds windows to fit individual residential houses. Mass customization, a key advance in new product development and logistics, requires modular architectures and standardized components that are not assembled until individual customers place orders. Sophisticated information system support, especially in order management, is necessary for successful mass customization. (Gooley, 1998)

Products are not the only things that can be customized; the Internet has abundant examples of customized marketing, customer service, and information delivery. ASPs (Application Service Providers) are rapidly positioning themselves to ‘rent’ applications through the web. ASPs sell common applications that can be rapidly configured to meet individual customer requirements. In fact, the appearance of ASPs is probably a direct result of the pressure to produce ‘standard’ software in the face of a severe programmer shortage. ASPs success will depend on how they manage to retain the advantages of standardization while moving toward mass customization.

Software project management is also being influenced by mass customization. Instead of finalizing system requirements at the beginning of a project, it is considered good practice these days to allow users and customers to modify the requirements on-going basis as the software is being developed. The concept of continuing user feedback during software development started with the spiral life cycle, originally proposed in 1988 (Boehm, 1988), and evolved to The Unified Software Development Process (Jacobson, 1999). Common wisdom today holds that IT projects should iterate toward an increasingly competent set of capabilities through incremental builds which allow continued customization of the software during its development lifecycle.

Summary and Conclusion
Logistics innovations are not accidents. They are driven by economic forces which demand a paradigm shift to keep an undesirable situation from overwhelming the economy. Once the paradigm shift occurs, the forces that caused it continue to exert pressure on other areas. Over time, the new paradigm will make its way into project management practice.

Often the economic driving force behind innovation is an overwhelming need for something in far greater quantities at far cheaper prices than current practices allow. This drives innovation in the direction of standardization. However, once standardization becomes the common paradigm, a shift in expectations usually occurs. Customers begin to expect tailored products, without increasing the cost, of course. This drives innovation in the direction of mass customization.

Another driving force behind innovation is globalization, which brings unique perspectives to logistics and project management. A fresh look at a problem by someone who doesn’t know the ‘right’ answer can lead to out-of-the-box thinking and new approaches.

By looking for trends in economic driving forces and noting innovations in logistics, project managers can predict the forces which will drive project management practices in the future.
____________________
References
Barry W. Boehm, “A Spiral Model of Software Development and Enhancement”, IEEE Computer, Volume 21, Number 5, May, 1988, Pages 61-72

Council of Logistics Management, “Definition of Logistics”, retrieved March 29, 2000 from http://www.clm1.org/

Goldratt, Eliyahu, Critical Chain, North River Press, 1997

Goldratt, Eliyahu, The Goal, North River Press, 1984

Gooley, Toby B., “Mass Customization: How logistics makes it happen”, Logistics Management and Distribution Report, April, 1998, retrieved on March 29, 2000, from ‘Logistics Online’

Jacobson, Ivar, Booch, Grady and Rumbaugh, James; The Unified Software Development Process, Addison-Wesley, 1999

Project Management Institute, Guide to the Project Management Body of Knowledge, 1996, The definition of ‘Project’ is in Section 1.2, Page 4.

Shingo, Shigeo, Study of ‘Toyota’ Production System form Industrial Engineering Viewpoint, Japan Management Association, 1981

Taylor, David, Object-Oriented Technology, A Manager’s Guide, Addison-Wesley, 1990, Reference to Eli Whitney, Pages 85-87

Published in Proceedings of PMI Seminars; Symposium 2000; Houston, Texas, September, 2000

Screen Beans Art, © A Bit Better Corporation

A Rational Design Process – It’s Time to Stop Faking It

The literature on Object-Oriented software development processes has proposed a lifecycle which is often found to be at odds with the established software engineering processes of organizations. In particular, more established software engineering processes at higher CMM (Capability Maturity Model) levels often have deep roots in the ‘Waterfall’ lifecycle. Some of these mature processes might be evolving into legacy processes.

The emerging Software Engineering Body of Knowledge (SWEBOK) is more appropriate than the Project Management Body of Knowledge (PMBOK) as a course of study for potential project managers of software projects. PMBOK has a tendency to emphasize scope management and task decomposition, while SWEBOK focus on requirements analysis and architectural design. Recent developments in Object-Oriented software engineering assert that an emphasis on requirements rather than scope, and on architecture rather than tasks leads to superior software development processes.

Specifically, organizations should not demand detailed fixed scope, cost and schedule plans at the beginning of a significant software development effort. There is a lesson to be learn from the building industry, which allocates up to half of the overall development time to architectural design, and does not create a controlled project environment until the construction phase.

When developing e-commerce applications, requirements analysis and system architecture remain critical, but they should be expanded to a broader context. The business plan replaces a project plan in e-commerce; marketing (and even sales) drive requirements analysis; and architectural design should be broadened to include infrastructure design. E-commerce system design encompasses system-wide issues, including hardware, networks, purchased components and partnerships, as well as interfaces with back-end fulfillment and collection capabilities.

1. Introduction
The Minneapolis Star/Tribune [17] recently reported that after an international search, the Walker Art Center has selected a Swiss architect for a $50 million, 110,000 square foot expansion project. “Design details still are scarce to nonexistent…. The site plan, scheduled to be announced in four to six months, calls for a two-year design phase followed by three years of construction.”

A recent issue of Computerworld [9] reported that a grocery chain’s announcement of plans for a four-year, $250 million systems overhaul sent its stock “into a tailspin”.

Computerworld [8] also noted that a managed care organization has finally recovered from a financial disaster caused by “major computer system problems”. The company showed its first profit since 1997; posting a loss of almost $20 million last year, including $5 million spent to fix the problems.

People are confident that Minneapolis will have a spectacular new art center in five years; after all, we know how to design and build buildings. Unfortunately, announcements of new information systems do not generate the same level of confidence. After 40 years of software development, a cloud of skepticism continues to hang over promises of new and improved systems

2. Software Engineering Processes
Many, many attempts have been made to bring discipline and predictability to the software development process. There is no doubt that software engineering can benefit from a disciplined process. However, utilizing the wrong software development process can interfere with discovering and applying truly beneficial software engineering processes.

2.1 Legacy Processes – Emphasis on Scope
It is well known that in the fast moving software world, legacy systems have a tendency to prevent a company from responding rapidly to new opportunities. The limited number of large company successes in the dot-com world is but one example of this commonly held belief. This paper contends that legacy processes can be much a problem as legacy systems in developing truly extraordinary information systems.

This is not to say that software engineering should proceed in the absence of a process. But a mature software engineering process that is the wrong process can be significant handicap to producing excellent software systems. Let’s look at a hypothetical case.

To reach process capability level three, an organization determines that it is important for software project managers to have a common approach to project management. Casting about for a project management process, the organization decides that PMI (Project Management Institute) has both a Project Management Body of Knowledge (PMBOK) [19] and a certification process which can be used for all types of projects, including software development.

The organization decides that all software project managers should either hold or work toward PMP (Project Management Professional) certification. The message sent throughout the organization is that projects must be managed using processes found in PMBOK.

There are many good processes in PMBOK, but it has a significant flaw when applied to software project management. Although PMBOK claims to be independent of lifecycle, an examination of the body of knowledge indicates otherwise. With its roots in government contracting, PMBOK places very strong emphasis on determining scope, decomposing tasks into a Work Breakdown Structure, and managing each element of that structure to the scope it represents.

Here are some quotes from PMBOK to emphasize this point:

“When properly defined, the scope of the project – the work to be done – should remain constant.” (Chapter 1)
“Core planning processes….include:

• Scope Planning


• Scope Definition


• Activity Definition


• Activity Sequencing


• Activity Duration Estimating


• Schedule Development


• Resource Planning


• Cost Estimating


• Cost Budgeting


• Project Plan Development” (Chapter 3)

“A written scope statement is necessary for both projects and sub-projects…. Proper scope is critical to the project’s success.” (Chapter 5)

Chapter 5 goes on to champion the Work Breakdown Structure (WBS) and the associated work decomposition as the tool to use to establish and manage scope. The clear message is that a project without a Work Breakdown Structure is a poorly managed project indeed.

2.2 A Lesson from the Building Industry
Let’s return to the Minneapolis Art Museum and the 40% of the project which constitutes the design phase. In this project, the cost and schedule are fixed, as is the size of the expansion and its purpose (to make the art center a “gathering place”). So a vision exists, and the architect is expected to select a design and materials to realize the vision. However, during the design phase – a significant portion of the project – neither scope management nor a Work Breakdown Structure are critical to the overall success of the project. In fact, this phase is dedicated to understanding requirements and establishing the building architecture. The scope is not going to deviate significantly from the 110,000 square foot $50 million building. Beyond that, scope management and work decomposition are not dominant issues during the design phase.
It is also clear that the ultimate success of the new museum depends largely on the insightfulness of the requirements analysis and the brilliance of the architectural design. An extensive search process has already taken place to select an architect who can be trusted to be insightful when gathering requirements and brilliant when designing the building. Adding the search time to the project, note that the first half of the allotted time is spent focusing on the building’s architecture.

For the first two years, the architect will be expected manage the project, including gathering requirements and delivering the final design. A ‘project manager’ will be assigned about the time the project goes out for bids, although the architect will provide oversight and retain responsibility throughout the project.

2.3 Toward a Modern Process – Emphasis on Requirements Analysis and Architecture
Consider the developing SWEBOK (Software Engineering Body of Knowledge) document [12]. Here the primary planning process is software requirements engineering rather than scope definition. This may seem like a similar activity, but it is different in that requirements analysis is a discovery, elaboration and negotiation process that occurs throughout the lifecycle of a project.

Consider the following quotes from the Software Requirements Engineering Knowledge Area in the Iron Man version of SWEBOK: “In practice, many things conspire to make requirements engineering one of the riskiest and most poorly understood parts of the software lifecycle….All of these things mitigate against the ideal of having a requirements baseline frozen in place before development begins. Requirements will change, and this change must be managed by continuing to ‘do’ requirements engineering throughout the life-cycle of the project.”

The same document [12] notes that “The maturity of most software development organizations’ requirements engineering processes lags well behind that of their down-stream life-cycle processes.” It further notes:
“Software engineers … cannot be expected to take a list of user requirements, interpret their meaning and translate them into a configuration … that satisfies the user requirements. In most cases … user requirements are elaborated … into a number of more detailed requirements that more precisely describe what the system must do. This usually entails deploying engineering skills to construct models of the system in order to understand the logical partitioning of the system, its context in the operational environment and the data and control communications between the logical entities. A side effect of this is that new requirements … will emerge as a conceptual architecture of the system starts to take shape. It is also likely to reveal problems with the user requirements … which have to be resolved….”

The bottom line is that SWEBOK recognizes that requirements are gathered, refined, negotiated, and modified throughout the entire project, with particular emphasis on flexibility during the design phase. On the other hand, studying for PMBOK certification will give an novice project manager the impression that the waterfall lifecycle is in fact the only ‘rational’ life cycle for a software development project.

To quote from Cantor [4], “…much of the overall project management literature is better suited to construction projects than software. The science of project management was developed as a tool to aid the construction manager to plan and tract the required schedule, budget, and resources…. Building a bridge is an exercise in dependency management, managing software is primarily an exercise in content management…. Bringing the construction management mentality to software development leads to a waterfall lifecycle: …taking this rigid approach appropriate to construction adds risk to the projects.”

2.4 The Waterfall and the Project Office
The classic “Waterfall” lifecycle (see Figure 1) does not make a lot of allowances for continuing requirements analysis throughout the development cycle. The on-going nature of requirements analysis has been embodied in virtually every software development life cycle except the “Waterfall” lifecycle. (See chapter 7 of [16] for a comparison of ten lifecycle models.)

Figure 1:  The Waterfall Lifecycle

There are IT projects where the waterfall lifecycle is appropriate – an excellent example would be Y2K projects. The requirements of these projects were completely clear; requirements analysis was not very critical. Rarely did a Y2K project attempt to establish a new architecture. Fixing the date fields of all the systems in the world would be analogous to changing out all of the lead pipe in every plumbing system that exists. This takes some very skilled and creative plumbers and good project management, but you don’t need an architect or a deep analysis of what the project is supposed to accomplish.

Computerworld [7] reported in an article titled The Y2K Dividend, “The big winner, most Y2K veterans say, has been project management. … An important side effect of the new emphasis on project management has been the project office, which in many cases began as the Y2K office.” If, however, new projects which require in-depth user understanding and careful architectural design are subjected to a Y2K style of project management, these projects are could be at risk.

2.5 Object-Oriented Processes
In 1986, Dave Parnas [18] equated a ‘rational’ design process to a ‘waterfall’ lifecycle, and suggests that even though such a process is impossible to follow, perhaps we should ‘fake it’. Some years later, Booch [3] proposed that a two-dimensional set of processes is a good way to ‘fake it’. On the macro level, we manage the project using the phases that management expects, while on the micro level, and different process is going on. Booch’s motto for the macro process is “Make it so.” (from Star Trek), while his motto for the micro process is “Just do it.” (from Nike®).

Figure 2:The Unified Lifecycle

What is clear is that the software engineering community has gone to great effort to put the waterfall lifecycle behind it, while continuing to acknowledge that this may be the ideal lifecycle, but it is simply impossible to follow. Perhaps it is time to acknowledge that a software engineering process which demands a detailed scope definition to be fixed at the beginning of a project is not an ideal process, but is instead a “legacy process”.

In [15], Leffingwell hints that IEEE 830: Standard for Software Requirements Specification (1994) might need updating. In its place, he recommends using a “Modern Software Specifications Package,” which is a “logical structure” that flows from the vision document and contains an “elaboration of the various requirements for the system.” The Modern Software Specification contains a fairly detailed system design, and is produced at about the 30-40% point of a project.

The software development process is beginning to look like the one used for our art center building. Recall that from the start, there was a vision (110,000 square feet, $50 million, 5 years). In a couple of years, we can expect detailed architectural drawings and specs to go out for bids.

It is time to admit that it is not ‘ideal’ or even ‘rational’ to start with a detailed requirements definition at the beginning of a software development process; the requirements specification should be developed as on-going part of the project. If we want a good system, we must allocate a significant portion of the total time for the really important activities of the project, namely requirements definition and architectural design. Scope management and work decomposition are simply not important during this fairly large phase of the project. In fact, if they are emphasized, they will tend to impede the important work that needs to be done to lay the groundwork for an excellent system.

2.6 Developing the Architecture
Although the analogy to designing a building has been followed thus far, there are a significant differences between building software and buildings. In both fields, a model is often used to demonstrate concepts, help gather requirements, and prove the architectural design. However, while scale models of a building are throw-away items, it is common to build usable portions of a software system as part of the design process.

In fact, the iterative approach used in Object-Oriented projects recommends designing, coding, and testing architecturally significant portions of the system during the design process. This is not throwaway code; it is a usable, tested implementation of a select set of requirements. The reason for implementing portions of a system during the requirements gathering and design process is to provide design feedback and customer verification that the system architecture is sound.

Further, accepted OO project processes call for continued incremental implementation throughout the project. The second iteration incrementally refines the architecture and functionality of the first iteration. Subsequent iterations add new features and elaborate on the capability of already implemented features. Each iteration is implemented and tested and then used to provide development and customer feedback.

Leffingwell [15] notes: “It is worth pointing out that use case elaboration is not system decomposition. That is, we don’t start with a high-level use case and decompose it into more and more use cases. Instead, we are searching for more and more detailed actor’s interactions with the system. Thus, use-case elaboration is more closely aligned with refining a series of actions rather than hierarchically dividing actions into sub-actions.”

Similarly in refining the system through incremental iterations, the core system is not decomposed, but instead it is broadened. For instance, early iterations might implement the most common paths through a system, while subsequent iterations might address increasingly less likely deviations from the general flow.

2.7 Iterative Construction
When building a building, the construction phase is generally though of as the start of ‘traditional’ project management. At this point, drawings and specifications define the scope, and contractors bid on this well-defined scope. However, it is quite difficult to divorce design from implementation when using an iterative approach to software. In fact, as was mentioned earlier, requirements analysis and design continue at some level throughout the project.

However, after one or two iterations, the architecture is largely established, and the focus shifts to refining and implementing the requirements. In an OO project, the requirements are generally captured in use cases, and the process focus shifts during construction to iterative use case realization.

Iterations are not the same as phases in a ‘classic’ project. PMBOK [19] notes, “The conclusion of a project phase is generally marked by a review of both key deliverables and project performance…. These phase-end reviews are often called phase exits, stage gates, or kill points.” The deliverables are usually presumed to be complete at the end of a phase.

On the other hand, the end of an iteration is marked by acquiring enough information to move on to broader elaboration. Work products (artifacts) produced by an iteration will continue to evolve. The end of an iteration marks a change in focus, perhaps an elevation of formerly subordinate goals to a higher level, or turning attention to details that were previously tabled.

Project iterations are generally released according to a planned schedule. The scope included in an iteration usually may be modified (if necessary) to meet the schedule. Since subsequent iterations provide a mechanism for recovering scope, and since a functional system results from each iteration, OO projects tend to be managed to meet time commitments rather than scope commitments.
The iterative approach is not a new or unique concept. Most software project lifecycles employ some form of iteration. (See [16].) The problem is, many people still consider the waterfall lifecycle to be an ‘ideal’, if unattainable goal. It’s time to recognize that the software development process is fundamentally iterative, and stop trying to ‘fake it’.

3. E-COMMERECE
There is another change shaking the software world which is perhaps more profound than the evolution of software project management processes. Internet startup companies have been springing up like wildfire for the past couple of years, challenging the foundations of existing corporate systems and software development practices. A new dot-com company can generate an idea, get financing, implement a massive hardware and software system and be generating an enormous amount of business, all in the span of a few months.
How do they do it? They don’t seem to spend much time gathering and analyzing system requirements; they seem to know going in what they want to do. The startup company’s system seems to be build on purchased components and rapidly formed partnerships; internal development appears to be limited. Change is constant, from the content of the web site to the products and services offered to the infrastructure used to deliver the goods.

How is an established organization with mature software development processes supposed to compete with these upstarts? Perhaps the only way is to examine and adopt the best parts of the software development processes used in a dot-com company.

In the previous section, a case was made that requirements analysis and architectural design are the key ingredients for successful software projects. In this section, we look at the equivalent of these processes in the dot-com software development model: marketing and infrastructure design.

3.1 Practices in a Startup Company
Taking a step back, consider the mechanism through which a startup company obtains funding: the business plan. This is much more than the vision document of a project plan. The business plan starts out by identifying a market need, then makes the case that the company will be able to satisfy that need through a set of products or services which will (eventually) generate profits.

Business plans are usually quite detailed and always carefully reviewed by potential investors. They are scrutinized, questioned, refined and revised by venture capitalists, boards of directors and potential partners. Few software project plans are as thoroughly reviewed as a startup’s business plan.

It is a mistake to think that startup companies have not done requirements analysis. However, since their stakeholders are customers, they do it with marketing. Marketing is a well known discipline, and there are plenty of marketers available. A key difference between marketing and requirements analysis is the strong element of sales that is associated with marketing. If a customer doesn’t appreciate a product, there is always a sales function to foster that appreciation.

Any existing company that wants to ‘get into’ e-commerce should realize that e-commerce means selling products or services to customers. Therefore, marketing and sales are key elements of e-commerce. However, the synergy between marketing and the software development team must be high. If marketing simply tosses a laundry list of requirements at the development team, we have the worst embodiment of the waterfall process.

In a startup company, the founder (who perhaps hired the system developers) will be a very persuasive product champion. The requirements analysis process is fast and effective because it is closely linked to both the business plan and to the software development team’s motivation (eg. to their paychecks). Moreover, a startup company will generally use a Design-to-Tools lifecycle [16] for all except the unique core of the system. When coupled with an Evolutionary Delivery Lifecycle [16] this strategy allows fast initial implementation and provides a learning environment.

3.2 From Architecture to Infrastructure
The previous section hypothesized that the architectural design is the second critical success factor in a software system. This sections proposes that infrastructure is the equivalent of architecture in e-commerce environment. Successful e-commerce is dependent not only on a solid software architecture, but also on the ability to expand hardware and network capacity aggressively, and the ability to mesh seamlessly with an effective order-fulfillment and revenue collection infrastructure. How do dot-com companies come up with a workable infrastructure in just a few months?

The first thing to note is that designing exceptional infrastructures and architectures does not take lot of people, it requires a few exceptional people. This is why the Walker Art Museum did an international search for its architect. Venture capitalists will tell you that their investment decisions are based almost exclusively on the management team of a company, not the proposed product. Why is this?

In lieu of detailed project plans, investors in startup companies are looking for people who have already proven that they know how to develop a similar product or service, market it, and establish excellent operations. Since an investor is unlikely to be able to evaluate a project plan in any case, all they have to go on is the quality of the management running the business.

In a fast-moving e-commerce project, there is a great need for top notch design of the entire infrastructure, including hardware, response time under volume, and scalability; to say nothing of the software architecture. A recent Computerworld article [6] suggested that corporations can be at a significant disadvantage when designing web-based systems, because the complete infrastructure design is often not in the domain of a single person. One function may be responsible for the network while another handles servers, a third does the software development, and so on. This pigeonholing of responsibilities tends to create components that are individually optimized, to the detriment of system-wide optimization.

A further difference between dot-com infrastructure design and corporate system infrastructure design comes from the dot-com’s limited resources. A startup company is does not have the resources to develop software when an existing tool will do the job. Whether it’s a survey capability or a link to MapQuest, if the solution exists, it is not re-developed. Partner and purchase is the name of the game. On the other hand, as noted in [6], these companies are not going to shell out a lot of money for software licenses. And of course, a dot-com must develop a core capability if the company is to have any uniqueness and staying power. But dot-com companies know instinctively that re-inventing an existing wheel will sap valuable resources at best and result in an inferior wheel at worst.

So how does an established organization compete against the dot-com’s when designing the system infrastructure? First, work from a business plan rather than a project plan. Second, search for and retain top-notch infrastructure and system architects. Then be sure their charter is broad enough to allow a holistic system view. Be sure they have the charter to rapidly purchase and partner where practical. Be sure the project is adequately funded and has proper organizational sponsorship (the equivalent of venture funding). Be sure the architects and marketers are synchronized. Finally, a tight timeframe (as opposed to prescribed scope) tends to inspire creativity in the Internet environment.

3.3 Rapid Implementation
What about implementation? The pattern for rapid development of dot-com systems is well known. First, a limited deployment model is developed. For instance, the business model may be tested in a limited geographic area; or a limited number of products may be offered for sale; or a limited audience may be engaged. This ‘limited edition’ is debugged, along with its supporting infrastructure. Once the offering is stabilized, it is rapidly scaled up to the full intended audience.

The tendency to ‘go live’ to a limited audience is analogous to the beta testing common to with all software products. Since beta testing is such a common and effective practice for software products, it is a wonder that it has not been more widely adopted as a standard practice in software development projects. In fact, the Unified lifecycle of incremental iterations embodies both successive ‘builds’ and the moral equivalent of beta releases. In addition, since the early iterations are aimed at risk reduction, this process is similar to the model-then-scale implementation strategy of the dot-com’s.

4. SUMMARY AND CONCLUSION
Software projects should begin with a ‘Vision Document’ that states in general terms what the system should do and what it’s boundaries are. The vision document should clarify the business purpose of the system, along with the overall cost and timeframe which is justified and required by the business objective.

The next step is to choose the architect(s), a small group of people with proven credentials . The architectural team coordinates requirements engineering, system modeling, and architectural design. They will require significant input from users (sometimes represented by business analysts). They also need a configuration management system to tracks requirements as they are defined, elaborated, and realized. An iteration or two of the system should be developed to help define the architecture and address key risks.

This initial phase of the project should be expected to take perhaps 40% of the allotted timeframe. At it’s conclusion, a fairly detailed set of system requirements should be available. At this point, a project manager can add staff and begin to manage scope. More important, the project manager should pace the project through frequent, planned iterations that are managed to schedule.

If this seems like a risky lifecycle, consider the risks in a legacy software lifecycle. We would not consider erecting a large building without an architect. We probably wouldn’t launch a large advertising campaign without careful design by an agency. So why would we consider developing a large software system without chartering an architect to analyze the user requirements, establish a system architecture, and oversee its construction?
______________
REFERENCES
[1] Ambler, Scott, Process Patters, Building Large-Scale Systems Using Object Technology, Cambridge University Press, 1998

[2] Booch, Grady, Object-Oriented Analysis and Design with Applications, Second Edition, Benjamin/Cummings, 1994

[3] Booch, Grady; Object Solutions, Managing the Object-Oriented Project, Addison-Wesley, 1996

[4] Cantor, Murray, Object-Oriented Project Management with UML, Wiley Computer Publishing, 1998

[5] Cockburn, Alistair; Surviving Object-Oriented Projects, A Manager’s Guide, Addison-Wesley, 1998

[6] Computerworld, 1/31/2000, Dot-coms’ Newest Secret Weapon Doesn’t Have a Name

[7] Computerworld, 2/7/2000, The Y2K Dividend

[8] Computerworld, 3/13/2000, Oxford Rebounds From IT Disaster

[9] Computerworld, 3/20/2000, A&P’s 250 M IT Plan Shunned by Wall Street

[10] Fowler, Martin and Scott, Kendall, UML Distilled, Second Edition, A Brief Guide to the Standard Object Modeling Language, Addison Wesley, 2000

[11] IBM Object-Oriented Technology Center, Developing Object-Oriented Software, An Experience-Based Approach, Prentice Hall PTR, 1997

[12] IEEE Computer Society and ACM; Guide to the Software Engineering Body of Knowledge (SWEBOK), download free from web site: www.SWEBOK.org

[13] Jacobson, Ivar; Object-Oriented Software Engineering, A Use Case Driven Approach, Addison-Wesley, 1992

[14] Jacobson, Ivar, Booch, Grady and Rumbaugh, James; The Unified Software Development Process, Addison-Wesley, 1999

[15] Leffingwell, Dean and Widrig, Don; Managing Software Requirements, A Unified Approach, Addison-Wesley, 1999

[16] McConnell, Steve; Rapid Development, Taming Wild Software Schedules, Microsoft Press, 1996, Chapter 7

[17] Minneapolis Star/Tribune, Wednesday, March 22, 2000, B Section, Page 1.

[18] Parnas, David L. and Clements, Paul C.; A Rational Design Process: How and Why to Fake It, IEEE Transactions on Software Engineering, Vol. SE-12, No. 2, February, 1986

[19] PMI (Project Management Institute), Project Management Body of Knowledge (PMBOK), download free from web site: www.PMI.org

[20] Schneider, Geri and Winters, Jason, Applying Use Cases, A Practical Guide, Addison-Wesley, 1998

Screen Beans Art, © A Bit Better Corporation