Category: methodology

How corporations spin a risk into a benefit

James Maguire, managing editor of Datamation, wrote in “Indian IT Firms: Is the Future Theirs?

“In the past, companies used to award IT outsourcing contracts that were longer, 7-10 years. They would hire one firm to do it, and that firm would have subcontractors,” Ford-Taggart says. Now, big clients split up major projects and request bids on individual components. “Then they’ll say, “Look, we can have this portion done in India for 30% less.'”

This might cause more managerial headaches for the client company, but in fact it’s less risk: clients have fewer eggs in a basket with any one IT firm, so if a projects goes bad or creates cost overruns, the entire project won’t take such a big hit.

Emphasis added.

I hate this corporate executive definition of “risk”. And before you think I don’t get it, I understand what they mean but I think they’re wrong.

When I talk about reducing risk, I mean that I’m making problems less likely to occur. What the execs mean when they take this position is that they’re diversifying the accountability. They want to be able to report that while 20% of the project is at risk, the other 80% is on track.

Well sure, the other 80% can’t be used without the 20% that’s missing. But the focus here should be that we’ve got 80% of the project on time and on budget!

If you admit up front that your process is increasing management headaches, you should realize you’re increasing the likelihood of problems. You may be mitigating the potential impact, but that’s not a given.

Any mitigation strategy that seeks to reduce the impact of a failure, and does so by increasing the likelihood of failure, is probably a bad idea.

How to Kill a Project by Accident

In my last post I talked about perverse incentives in project management. Something I mentioned in passing was what happens when you don’t have positive incentives: the cases where there is simply no incentive to do the right thing. I realized there’s actually another way to get this wrong by trying too hard to get it right.

Let’s say you have just finished a project, gone into production, and it blew up. Data corruption, security problems, too slow, everything that can go wrong with a product. First you do an emergency project to fix it, then you do the after-action review to see what went wrong.

What you find is that you didn’t have good test coverage. There were whole modules that were never reviewed before going into production. It’s painfully obvious that there was a complete breakdown in the QA process.

Fixing yesterday’s problem

You’re not going to make that mistake again. You write up policies for demonstrating test coverage. You create reports to track test execution and results. You re-write employee performance expectations to align with the new methodology. (If you’ve read my stuff before, you should hear the alarm bells start ringing when you see the “m word”.)

Your next project is going exactly according to plan. Test coverage is at 90% overall, with 100% of high priority use cases covered. You’re on schedule to execute all test cases before acceptance testing starts. Defects are identified and corrected.

Then the users get it. They hate it. It doesn’t do anything the way they wanted it to. Not only that, it doesn’t even do what they asked for. How could you be so far off?

You don’t get what you don’t measure

Look back at what incentives you created. You reward doing the methodology, following the checklist. Test coverage is great, but it’s not the goal of the project. The goal is to provide something of value to the users … or at least it should be. Did you include a line in the new process that explicitly says, “Check with the users that it’s doing what they need”?

So how do you create the right incentives? Just flip the emphasis. Instead of saying an employee’s performance evaluation is 80% following the methodology and 20% client satisfaction, turn the numbers around. Your users don’t care that you followed “best practices.” They care that the product does what they need. Where is that measured in your methodology?

How to Fail by Succeeding

Dave Christiansen over at Information Technology Dark Side is talking about perverse incentives in project management, which he defines as:

Any policy, practice, cultural value, or behavior that creates perceived or real obstacles to acting in the best interest of the organization.

One class of these perverse incentives comes from the methodology police. These departments exist to turn all processes into checklists. If only you would follow the checklist, everything will work. But more importantly, if you follow the checklist you can’t be blamed if you fail.

How’s that for perverse?

A great example is that there is rarely any incentive to not spend money. Before you decide I’m out of touch with reality, notice I didn’t say “save money.” I said “not spend money.” Here’s the difference.

IT by the numbers

Let’s say you do internal IT for a company that produces widgets. Someone from Operations says that they need a new application to track defects. If you follow the checklist, you:

  • engage a business analyst, who
  • documents the business requirements, including
  • calculating the Quantifiable Business Objective, then
  • writes a specification, which is
  • inspected for completeness and format, then
  • passed on to an architect, who
  • determines if there is an off-the-shelf solution, or if it needs custom development.

At this point you’re probably several weeks and tens of thousands of dollars into the Analysis Phase of your Software Development Lifecycle. Whose job is it to step in and point out that all Ops needs is a spreadsheet with an input form and some formulas to spit out a weekly report?

Let’s put some numbers to this thing.

Assume the new reporting system will identify production problems. With this new information, Operations can save $100,000 per month. A standard ROI calculation says the project should cost no more than $2.4-million, so that it will pay for itself within two years.

Take 25% of that for hardware costs, and 25% for first-year licensing, you’ve got $1.2-million for labor costs. If people are billed out at $100/hour – and contractors can easily go three to four times that for niche industries – that’s 300 man-weeks of labor. Get ten people on the project – a project manager, two business analysts, four programmers, two testers, one sysadmin – and that’s about seven months.

If everything goes exactly to plan, seven months after the initial request you’re $2.4-million in the hole and you start saving $100,000 per month in reduced production costs. Everyone gets their bonus.

And 31 months after the initial request, assuming nothing has changed, you break even on the investment. Assuming the new system had $0 support costs.

But what if …

Way back at the first step, you gave a programmer a week to come up with a spreadsheet. Maybe the reports aren’t as good as what the large project would have produced. You only enable $50,000 per month in savings. That week to produce it costs you $4,000 in labor, and $0 in hardware and licensing.

You are only able to show half the operational savings, so you don’t get a bonus. You don’t get to put “brought multi-million dollar project in on time and on budget” on your resume.

And 31 months after the initial request, the spreadsheet has enabled over $1.5-million in operational savings.

How to Finish IT Projects Faster with Less Documentation

If you’re responsible for running an IT project you want things to be done on time and within budget. So how do you set your schedule and budget? Hopefully you define what you want to accomplish, and then ask the developers how long it’s going to take. If you’re putting out a request for proposal (RFP) you’ll have several different answers to that question. Typically the highest consideration in the selection is the total proposed cost. But really, the total time is a better choice.

Why that’s true is based not on ideas about processes, but on ideas about people.

Consultants live and die by billable hours. In the short term, they don’t have any incentive to finish their current project any faster. But in the long term, finishing faster should lead to more work as clients come to respect their ability to meet a deadline. If project managers and clients learn to value that behavior, that is.

How it Could Be

Let’s look at a production support issue for an example. Production support is completely different from most project work in one very important way: the problem is well defined. Something worked on Monday, it doesn’t work on Tuesday. Make it work just like Monday again.

For something with six-figure impact per hour of downtime – and if you think that’s an artificially-high number you’ve never worked with credit card processing – you don’t want a programmer with an impressive resume, dozens of certifications, and decades of experience with your primary programming language. You want Bob, the guy who wrote the system from scratch and demands $1k per hour with a four-hour minimum.

Once you’ve got Bob and his hand-picked team of support people, you get out of their way and let them work. Status reports might be no more than, “We’ve found the problem … We’ve identified the solution … We’re ready to test the fix … It’s live.”

How it Is

But when it comes to new development, companies play it “safe” and look for the best qualifications on paper. They hire based on keyword matching and offer rates based on industry standards for a given skill set. They require specific processes and deliverables (pet peeve: when did “deliverable” become a noun?) and status reporting becomes a significant percentage of the total budget.

Why the Difference?

There are two reasons expert teams get away with less formal process than is typical, but I can only prove one of them. The public answer business sponsors tell themselves to justify the exception to “official methodology” is that the experts have worked the methodology for so long that they can follow the same procedures without exhaustively documenting all the steps. And there is some truth to that.

But I suspect the larger reason is that experts get the work done so much faster there just isn’t enough time for documentation to build up.

The best athletes make things look easy that most people could not even do. A high jumper might clear six feet without even trying hard. Most people would never come close even with months to try.

The best IT people do the same thing. They complete projects in weeks that other people could never do. As “safe” projects drag on specifications are refined, status reports are produced, contracts are negotiated, updates are requested and provided. Meanwhile the expert team has just released to production – so it must have been a small project.

The hard part for the client is to recognize the difference between a project that went smoothly because it was easy, and one that went smoothly because the team made it look easy. But here’s the secret: You don’t really need to recognize the difference.

How to Do Better

The reason you hired someone else to do the work is because you couldn’t do it yourself. Which means you can’t accurately judge which projects really are hard, and which ones just look hard. So don’t judge the project, judge the people.

The people who seem to always be working on small, simple projects – after all, they always go quickly with no major problems – are better at execution. They will be better no matter what the project is.

Get to the point

Dave Christiansen over at Information Technology Dark Side has a good graphic representing what he calls the FSOP Cycle (Flying by the Seat of Your Pants). The basic idea is that when smart people do good things, someone will try to reproduce their success by doing the same thing that worked the first time.

The problem with trying to do this is that every time someone tries to document a “successful process”, they always leave off the first step: get smart people working on the project.

Dave outlines several reasons why capital-P Process will never solve some problems. Joel Spolsky described this same issue in his Hitting the High Notes article when he wrote, “Five Antonio Salieris won’t produce Mozart’s Requiem. Ever. Not if they work for 100 years.”

So if Process can’t solve your problem, what will? According to Dave, it’s simple:

Put a smart PERSON in the driver’s seat, and let them find the way from where you are to where you want to be. It’s the only way to get there, because process will never get you there on its own.

I’ll assume that when Dave says “smart” he really means “someone good at solving the current problem”. I could have the world’s smartest accountant and I wouldn’t want him to remove my appendix. Okay, I don’t want anyone to remove my appendix. But if I needed it done, I’d probably go find a doctor to do it. So what Dave is saying is that you’d rather have someone who’s good at solving your current type of problem, than have Joe Random Guy trying to follow some checklist.

This isn’t a complete answer, though. Some songs don’t have any high notes.

In the middle of describing why you should choose the most powerful programming language, Paul Graham writes:

But plenty of projects are not demanding at all. Most programming probably consists of writing little glue programs, and for little glue programs you can use any language that you’re already familiar with and that has good libraries for whatever you need to do.

Trevor Blackwell suggests that while that may be changing, it was still true at least until recently:

Before the rise of the Web, I think only a very small minority of software contained complex algorithms: sorting is about as complex as it got.

If it’s true that most programming doesn’t require the most powerful language, it seems fair to say most programming doesn’t require the best programmers, either.

You might notice at this point (as I just did) that Dave wasn’t talking about programming, or at least not only programming. The same principle seems to hold, though: Average people can do the most common things with the most common tools. Exceptional circumstances require exceptional tools and/or exceptional people. If only there were a way to predict when there will be exceptions …

The other problem

But let’s say we’re looking at genuinely exceptional people who have done great work. Should we ask them how they did it? After all, they’re the experts.

Well, not really. They’re only experts on what they did, not why it worked. They might have simply guessed right. Even if they didn’t think they were guessing.

Need another example of false authority? Have you ever heard someone describe a car accident, and attribute their survival to not wearing a seatbelt?

First, they don’t know that. They believe it. They obviously didn’t do a controlled experiment where the only difference was the seatbelt. Second, even if they happen to be right in this case, statistics show that it’s much more common for the seatbelt to save you than harm you.

So if you want to design a repeatable process for creating software, you can’t do it by asking people who are good at creating software.

The other other problem

One change I’d make in Dave’s FSOP diagram is in the circle labeled “Process Becomes Painful”. It’s not that the Process changes. Really what’s happening is that the project runs into some problem that isn’t addressed by the current Process.

Every practice you can name was originally designed to solve a specific problem. On very large projects, there’s the potential to encounter lots of problems, so extensive Process can simultaneously prevent many of those problems from appearing.

But attempting to prevent every conceivable problem actually causes the problem of too much time spent on the Process instead of the project.

That’s where Agile comes in. It solves the problem of too much process. Do you currently suffer from too much process? Then you could incorporate some ideas from Agile.

But make a distinction between using ideas to deal with specific problems, and adopting a whole big-M “Methodology”. Once you adopt a Methodology designed to solve the problem of too much Process, you face the danger of having too little Process.

Your expert programmers may be the best in the world at the problem you hired them for. But now you want them to do something different. And you have no way to recognize that they’re not making progress, because you’ve eliminated all the governance that came with the heavyweight Methodology.

So what’s the point, anyway?

Process is not something you can measure on a linear scale, having “more process” or “less process”. Always adding practices — even “best” practices — to your current Process is simply trying to solve all possible problems, whether you’re currently having them or not.

For each practice you have to consider what it was designed to solve, what was the point of it to begin with? Then don’t treat these practices as a checklist that must be completed every time, but follow the principles behind them.

There’s a line in the movie Dogma that I think describes this really well. Just substitute “process” for “idea” and “Methodology” for “belief”:

You can change an idea. Changing a belief is trickier. Life should be malleable and progressive; working from idea to idea permits that. Beliefs anchor you to certain points and limit growth; new ideas can’t generate. Life becomes stagnant.

Meet the new boss, same as the old boss v2

Sometimes you read something that you can’t summarize without losing a lot. I just can’t find any extra words in this post, so here it is in its entirety:

1) Whatever language is currently popular will be the target of dislike for novel and marginal languages.

2) Substitute technology or methodology for language in #1. In the case of methodology, it seems a straw man suffices.

3) Advocates will point to the success of toy projects to support claims for their language/methodology/technology (LMT).

4) Eventually either scale matters or nothing matters. Success brings scale. An LMT is worthy of consideration only after proving out at scale.

5) Feature velocity matters in early stage Web 2.0 startups with hyperbolic time to market, but that is only a popular topic on the Web for the same reason Hollywood loves to hand out Oscars.

6) Industry success brings baggage. Purity is the sign of an unpopular LMT. The volume of participants alone will otherwise muddy the water.

7) Popularity invites scrutiny. Being unfairly blamed for project failure signals a maturing LMT; unfairly claiming success, immature LMT. Advocates rarely spend much time differentiating success factors.

8 ) You can tell whether a LMT is mature by whether it is easier to find a practitioner or a consultant. Or by whether there is more software written *with* or prose written *about* the LMT.

9) If you stick around the industry long enough, the tech refresh cycle will repeat with different terminology and personalities. The neophytes trying to make their bones will accuse the old guard of being unable to adapt, when really we just don’t want to stay on this treadmill. That’s why making statements like “Java is the new COBOL” are ironic; given time, “N+1 is the new N” for all values of N. It’s the same playbook, every time — but as Harlan Ellison said of fiction, every story has already been told, but nobody was listening the first time.

10) Per #9, I could have written this same post, with little alteration, ten, twenty or thirty years ago. It seems to take ten years of practise to truly understand the value of any LMT. Early adopters do play the important role of exploring all the dead ends and limitations, at their cost. It’s cheaper to watch other people fail, just like it hurts less to watch other people get injured.

11) Lisp is older than I am. There’s a big difference between novel and marginal, although the marginal LMTs try to appear novel by inserting themselves into every tech refresh cycle. Disco will rise again!

12) If an LMT is truly essential, learning it is eventually involuntary. Early adopters assume high risks; on the plus side they generate a lot of fodder for blogs, books, courses and conferences.

13) I wonder if I can get rich writing a book called Agile Lisp for Web 2.0 SOA. At least the consulting and course revenue would be sweet. Maybe I can buy an island. Or at least afford the mortgage payments on a small semi-detached bungalow in the Bay area.

14) It requires support from a major industry player to bootstrap any novel LMT into popularity. The marginal LMTs often are good or even great, but lack sponsors.

15) C/C++ remain fundamental for historical reasons. C is a good compromise between portability and performance — in fact, a C compiler creates more optimal code than humans on modern machine architectures. Even if not using C/C++ for implementation, most advocates of new languages must at least acknowledge how much heavy lifting C/C++ does for them.

16) Ditto with Agile and every preceding iterative methodology. Winding the clock back to waterfall is cheating. I’m more sophisticated than a neanderthal, but that won’t work as a pick up line.

17) Per #13, I don’t think so, because writing this post was already a chore, let alone expanding the material to book length. Me an Yegge both need a good editor.

This covers the technology pretty well. All he left out was the reason so much is coming back.