The Sky Is Blue, Salesmen Lie and Projects Fail
We keep hearing about the numbers of IT projects that fail. As long as I’ve been an IT manager, even before I left the military nearly 20 years ago, I’ve heard shocking statistics about project failure rates. Gartner believes they can tell us why, but it seems to be a stubborn problem that hasn’t improved an iota, even as much information technology becomes commoditized and the discipline of project management becomes ever more sophisticated.
As managers we need to do what ever we can to make our projects succeed, well of course we do. But I’m going to argue that this “problem” isn’t one to keep you up at night, at least not because of something you read in the business press. I’m assuming your IT organization is run with a reasonable level of competence, but since you’re reading this and not out playing Ingress that seems like a fair assumption. Let me offer a couple of thoughts that help me sleep at night; and if you still have trouble ignoring the cries of Chicken Little, I will suggest a way of looking at your project portfolio that will make you comfortable with the occasional setback.
Consider the Source
Firstly, let’s remember who is telling us we have this problem: analysts, journalists and bloggers <ahem> who want to attract us to their cogent and elegant prose, and sometimes to the solutions that they or their advertisers are selling. They have an interest in convincing us that the crisis is serious and affects us. Festooned with charts and tables, most of these articles are remarkably vague. They don’t break their statistics down by the type of project or the business domain so that you can see how comparable the data are to your own project mix. Infrastructure projects have very different risks and failure probabilities from a web portal to support a new business unit, or from a green fields software product. I particularly like this InformationWeek piece because while it initially acknowledges that the statistics in these studies are suspect, they remain the premise of the article. This paper by PM Solutions, a consultancy, has some intriguing details, but most of its statistics are averages across the whole survey. 163 respondents may sound like a lot, but chances are there are only a handful whose company size, maturity and industry are comparable to yours.
And most of the time “failure” is a very big tent. A project may be considered a failure if it arrives on time and budget but is never implemented; or it is rolled out but fails to make as much as expected, or it simply isn’t adopted. It might be cut early on with few costs expended; it might be nominally cancelled but rolled into another ultimately successful project; it might be shelved, to be revived six months later. Sometimes a project will fail by delivering a poor quality product; or by providing everything planned but through unfortunate timing is obsolete on delivery. And there are legions of “failures” that deliver late and over budget – but still generate revenue or provide substantial business value. Not to mention the invaluable experience gleaned by project teams that they’ll bring to successor projects. Yet when we hear the project failed we imagine that all or most of the budgeted resources were used with nothing to show for them, no revenue, service improvement or enhanced capability.
These articles make it sound like we should worry but don’t give us enough data to judge whether or not we really need to. Is your industry more or less risky than average? Are these moribund projects the more like the integration of a newly acquired company’s back office or a roll-out of new laptops? Are these statistics coming from global banks installing SAP or from Silicon Valley start-ups building apps to boil the ocean? With only this kind of vague information, you are much better off trusting your own judgment about your project portfolio’s health.
You Know Your Portfolio Best
To make that judgment, we need a model that works for our own organization. Let’s begin by accepting that some projects will fail. To a point, this is out of our control. What was a great idea last year, or last week, isn’t today. Economic conditions change, companies alter course, technology appears or matures in its own time, chief architects fall in love with inappropriate women and move to Estonia to develop a carbon trading platform (not often, but once is too often). So some of your initiatives are going to go South and I’m not talking about off-shoring. This is creative destruction writ small, and a good thing. Realistically, it would simply be too costly to salvage every troubled project, and often pointless to do so when it’s a matter of changed external circumstances.
Accepting that there will be some level of failure, we need a model that measures the cost of failure and recognizes an acceptable rate of loss. Compose the model in three steps:
- Assess the risk vs. return profile of each one of your projects.
- Determine the collective required return from all projects in the same class, i.e. those with a similar profile.
- Calculate the total return you’re actually getting from the projects in the class.
Firstly, classify your projects based on their risk/return profile. In the figure below, I’ve illustrated a mix of projects that might appear in an organization. In my example there are four categories of project. It’s often hard to ascribe revenue to infrastructure projects but the costs are more predictable, so they tend to group in the bottom left quadrant. New software products have enormous revenue potential but concomitant risk, so appear mostly in the top right. By grouping projects with similar profiles together as a class meaningful comparisons are possible.
Project Portfolio – Categorized by Risk vs. Profit Profile
Even better, it’s possible to have some intuition within the class about what constitutes a healthy project. When the whole portfolio is aggregated as one, a comparison to the average doesn’t tell you much. 20% over budget is nothing to be ashamed of when developing a wholly new system; it would be pretty embarrassing for a server upgrade. This intuition becomes important in the next step.
Next, we compose a generalized business case for the whole class in very simple terms: a certain level of investment is to yield a certain return. If you were to have only one class, then your required rate of return would be the hurdle rate for the firm. We specify a hurdle rate for each of the project classes (ensuring the total investment in projects of all classes remains at the firm’s overall hurdle rate). For example, for regulatory compliance projects, the price of failure is significant litigation and penalty costs. To the extent that these projects enable the firm to avoid litigation and fines, they add value. If the present value of the non-compliance costs less project expenses exceeds the class hurdle rate, the project is a success. Most companies would expect something close to the risk free rate for projects that have been imposed by a regulator (and in practice might even be willing to accept a loss) so you might have a much lower hurdle rate for this class. New products and business units, on the other hand, being riskier than most other projects would require a greater rate of return. For many projects the investment is only partially in technology, so the return is not the total revenue but some proportion that represents the contribution of IT.
With the portfolio managed in classes and an expected return developed with which to assess the class, one can then look at an individual project in context with its brethren. Generally the projects in a the class will have similar challenges, time scales and returns on investment. If one of five new product projects has large overruns, that might be okay if the others are on track. If you need to cancel an infrastructure project because delays have caused the technology to be overtaken by something better, you can be sanguine about writing off sunk costs if the rest of your infrastructure projects are healthy.
Smokey Says: Only You Can Prevent Project Failure
This short essay isn’t intended to provide a portfolio management methodology. Realize only that the business press loves to cry “wolf” irrespective of how threatened your flock actually is. Given how long we’ve been hearing these alarums without the world ending, we can be sure the problem isn’t as dire as they would have us believe. Even if there is some legitimate cause for concern, there are degrees and types of failure which may not justify a declaration of defeat in your circumstances. Only those managing the project portfolio can know whether or not there’s something to worry about.
In any organization a certain level of project failure is inevitable. The key is to understand the real risk and return of any given project. This will enable you to grasp each project’s place within the big picture, and help you to respond appropriately to bumps in the road. What the pundits may deem failure could indeed be a catastrophe, but more often it’s the cost of doing business.
