SocialStartups.com

Disclaimer: I have no idea if this works. I thought of this last week as I was walking around with my friend Bre and decided to post it here to get feedback. There are many people who have had far more software development management experience than I have, and I’m very interested to know if they think this idea could work at all.

Summary

Prototype Driven Development (PDD) is a software development methodology that believes that by continuously creating, improving, and interacting with prototypes, software will be launched faster, with higher quality, and with a stronger focus on the customer. This is accomplished by using cross-functional teams to continually iterate on prototypes of gradually increasing fidelity while constantly absorbing input from the team and others.

Waterfall and Scrum

What is Waterfall?

The waterfall method of software development is a monolithic system created at a time when building and testing software was costly – punch cards had to be made or hardware had to be redesigned, so it was critical to figure everything out early in the process because any feedback could be gathered.

The waterfall process looks like this:

At each step, if a problem is discovered or the context of the project has changed, control of the project is routed back to the top of the waterfall.

See http://en.wikipedia.org/wiki/Waterfall_model for a more in-depth discussion.

The traditional waterfall process of software development doesn’t work because it:

1. is very slow and heavy weight (months to complete projects)
2. silos creativity between product, design, and development
3. expects impossible levels of foresight from product (and at times, design) to avoid going backwards and revising all of the documentation
4. creates lots of documentation that never gets read once the project is complete since maintenance fixes end up changing or adding to the requirements and design
5. does not provide sufficient progress visibility for senior management.

What is Scrum?

Scrum is the idea that by squeezing the development cycle down from 2-4 months to 2-4 weeks, the development process is less prone to estimation errors and more likely to ship higher quality code faster. The team works off of a backlog of features, with each feature small enough to be accomplished in a matter of days or hours. By reducing the size of a feature, Scrum teams can more quickly react to changes in prioritization and provide more visibility for management into what is done vs. not yet finished. The team has a daily standup, where each person outlines what they did yesterday, what they are doing now, and what blockers they might have. Each person also updates a document describing how much effort is remaining for each of the tasks assigned to them.

The Scrum process looks like this:

See http://en.wikipedia.org/wiki/Scrum_(development) for a more in-depth discussion.

Agile methodologies like Scrum don’t work because they end up like waterfall, just on a smaller scale. Instead of 2 months each of requirements, design, development, and testing, you have 2 weeks of each. This is because there is typically a “cycle zero”, where product and design do a fair amount of planning before launching into the first sprint. This cycle zero is basically the first two steps of the waterfall method. Agile does, however, address some of the problems of waterfall:

1. is very slow and heavy weight (months to complete projects) Medium weight as the team can react from sprint to sprint (2-4 week cycles)
2. silos creativity between product, design, and development Product, design, and development meet for a scrum standup every day, so there is frequent communication. Requirements are still created upfront by product and design, outside the Scrum process.
3. expects impossible levels of foresight from product (and at times, design) to avoid going backwards and revising all of the documentation Foresight is still expected from product / design, but at least the cycles are shorter (2-4 weeks) instead of months.
4. creates lots of documentation that never gets read once the project is complete since maintenance fixes end up changing or adding to the requirements and design No documentation is needed because of frequent communication. The time and effort spent writing documentation is funneled into making better product
5. does not provide sufficient progress visibility for senior management. Senior management, at the end of every sprint, theoretically can see a demo of what was worked on. “Potentially shippable code” is the phrase Scrum uses.

Prototype Driven Development

PPD puts usability and adaptability first. By constantly iterating off of a prototype and folding in input from the team, internal users, and usability tests, a product can be created quickly and with high quality.

With PDD, 1-3 engineers meet with a product manager and a designer to kick off a project. The team sits down and discusses the general problem and spends some time (no more than two hours) coming up with the general framework of a solution. It doesn’t matter that the solution isn’t optimal and most details haven’t been figured out. Often times this vague idea comes along with a pencil / whiteboard sketch of the UI.

The team disperses. The developers go and implement the most barebones prototype of the idea – black and white, fake data, no CSS, stubbed code, just enough to start a conversation. The product manager and designer start working on prioritized user stories, interaction design, etc.

The next day, the team meets and presents their work to each other. The devs show off their crude prototype. The product manager presents what she think are the most important activities the product needs to support. The designer lays out his ideas for layouts, user flows, etc. Each idea is vetted by the group and consensus is established. Maybe the product changes direction or maybe one particular use case becomes more important. The team then goes back and iterates again, improving the prototype or design or requirements.

This meeting happens every day at the same time. Each day, there is more clarity than the previous day because the crude prototype continually evolves into a more mature product. Each day, the team is interacting with each other and with the prototype, adjusting course as necessary. Each day, senior management can come in and interact with the prototype and react to it. At any point, the call can be made to lock down the feature set and drive towards launch.

The key point here is that prototype driven development forces everyone involved to focus on the primary activity being modeled by the feature. There isn’t time at the beginning to fuss about colors or CSS since there is only one day to make the prototype a bit more accurate. Each day, the team is evaluating the usability of the prototype, trying it out or asking others to evaluate it. Only when the key activities have been fleshed out will attention turn to the details.

To follow up on the five failures described above, here’s how PDD scores:

1. is very slow and heavy weight (months to complete projects) Medium weight as the team can react from sprint to sprint (2-4 week cycles) Super lightweight process that can adapt daily
2. silos creativity between product, design, and development Product, design, and development meet for a scrum standup every day, so there is frequent communication. Requirements are still created upfront by product and design, outside the Scrum process. No silos because everyone is contributing to the creative process every day.
3. expects impossible levels of foresight from product (and at times, design) to avoid going backwards and revising all of the documentation Foresight is still expected from product / design, but at least the cycles are shorter (2-4 weeks) instead of months. The team is course correcting every single day, so the cost of trying something is very low and philosophical arguments are minimized.
4. creates lots of documentation that never gets read once the project is complete since maintenance fixes end up changing or adding to the requirements and design No documentation is needed because of frequent communication. The time and effort spent writing documentation is funneled into making better product No documentation, other than an interactive prototype that is available immediately.
5. does not provide sufficient progress visibility for senior management. Senior management, at the end of every sprint, theoretically can see a demo of what was worked on. “Potentially shippable code” is the phrase Scrum uses. Senior Management immediately has a prototype to evaluate and react to.

Discussion

PDD requires great people. The product manager has to have a relentless focus on the primary goals of the product to avoid feature creep. The designer must be available every single day and be willing to create and throw away many iterations of their work. The designer must be able to work quickly, in low fidelity, and in an environment with ambiguous requirements. The developer must have high levels of comfort with prototyping code, stubbing out code / faking data, and a willingness to edit and throw out lots of code. PDD is like collaborating to write a comic book – there will be lots of editing of the storyline, the graphics, and the words themselves.

Prioritization is outside the scope of PDD. It is assumed that prioritization has occurred, and the team is being tasked with executing on a prioritized project.

Where is QA? I’m not sure. Flickr doesn’t have QA. I think QA can sit side-by-side with the developer and do some variant of Test Driven Development – http://en.wikipedia.org/wiki/Test_driven_development. In this situation, the QA engineer is writing test cases for the specific feature of the prototype the SDE is currently implementing. Since requirements can adjust day to day, the QA engineer needs to be as flexible as the rest of the team in editing their work daily.