Pre-condition #1: Identity

Facebook, with the release of social plugins, has officially announced that they believe they are the Internet's digital identity system*. How so? Their new social plugins — Like buttons, Personalized widgets, site-wide toolbars — all assume that you are logged into Facebook all the time. No one has ever made an assumption like that before, and it dramatically changes the game. Also, it's scary because it's probably true.

[Update: My old boss Ted reminds me that Amazon Associates widgets also assume you are logged in, to show you personalized ads. IMO, it's more pernicious now because Facebook is approaching 500M users. If you exist on the Internet, you are probably on Facebook.]

Pre-condition #2: Traffic

Everyone wants traffic, and nobody has more of it than Facebook. So when Facebook announced "If you install our bug into your system, we'll send you traffic", publishers celebrated. Of course CNN wants their links in Facebook Newsfeeds. More traffic = more pageviews = higher CPMs to charge advertisers. Thanks to 20th century capitalism, everyone is chasing short-term metrics. Classic tragedy of the commons.

Scary Result #1: Ad network domination

Given pre-condition #2 (traffic), Facebook Like buttons are going to be everywhere. All of the top sites will have them, and most of the medium-sized sites will too. Who wouldn't want more traffic in exchange for adding one simple line of code, so simple anyone could do it? 

What does this mean? Facebook is going to have a window into every important website on the internet. You went to Victoria Secret? Facebook knows (see Pre-condition #1). You then went to Gawker? Facebook can show you ads for Victoria Secret products on Gawker because it knows you were just there. It's called ad re-targeting, and it's the most effective innovation online ads have seen in a while. And no one will be able to do it better than Facebook.

Scary Result #2: We know what you buy

There's been lots of talk about Facebook Credits – a Facebook created currency that allows you to buy real or virtual goods using your credit card, PayPal, etc. What if you could get a real Facebook credit card? Facebook would then be able to attain the Holy Grail of Brick & Mortar retailing – tying together what you are buying with your digital identity (again, pre-condition #1). Right now, millions (billions?) of dollars are spent by small businesses, trying to get you in the door – Happy Hour specials, Buy One Get One Free, Yellow Pages ads. But if you come in and buy something, they have no idea who you are, and therefore have no way to encourage you to come back**. 

A Facebook credit card is different. They know exactly who you are when you buy that special pair of Louboutins. And they can use that information to show you better ads or product recommendations. Remember, Amazon and Netflix give you great recommendations because they have more data, not better algorithms. And because Facebook sees financial benefits from these alternative streams, the Facebook credit card can have better rewards / lower rates than anything else, and they have the scale to do it. No wonder Blippy raised $11.2M at a $46.2M valuation.

Scary Result #3: We know where you are

f8 attendees had RFID chips implanted in their badges. To check-in, you simply swipe your badge against a kiosk. What if your Facebook credit card had an RFID chip in it? This should scare the shit out of Foursquare, Gowalla, and Bump. Imagine I walk into a cafe and the owner says, press your card against this pad to check-in; 10th check-in gets you a free coffee. No more fumbling for your iphone, waiting for the GPS / cell tower triangulation, and looking like an idiot. You don't even have to take the card out of your wallet – wave your whole wallet over the reader and you're in. Or, just buy something.

And don't think Facebook doesn't have the scale or cash or ambition to create a Point of Sale system; that'll just close the loop even tighter.

Why this scares the shit out of me

Facebook is filled with really, really, really smart people. And they've shown an incredible ability to innovate at large scale, with an ambition that is unmatched. So what happens when we give up our privacy in exchange for 5% off? What happens when Facebook knows more about the economy's transactions than Visa? What happens when Facebook is watching you closer than Google?

Data is everything in the 21st century. Who ever has more data wins the Internet, and I don't trust Facebook with that kind of scale and power.

————–

*You might remember the single sign-on wars of years past: Microsoft Passport, YahooID, OpenID, then later Facebook Connect / Twitter Connect / OAuth.

**The closest thing are loyalty punch cards you get at cafes or pencil & paper mailing list signups.

Public Timeline

I’m starting with the public timeline because, while it’s not the sexiest thing on this list, it’s critical for new users as a solution to the cold start problem. The public timeline is the first place new users will look for new content. It’s also how they will determine if they wish to join your community – the size, the tone, the cultural-norms, and the freshness of your community is easily communicated via the public timeline. And, of course, it’s also where new users can find other interesting users, which leads me to my next pattern.

Asymmetrical Follow

This has been written about before, so I won’t say much. Asymmetrical follow means I can follow the updates of a user without their permission. They, in turn, could follow me back but it’s not required. This allows for the kind of preferential attachment characteristic of scale-free networks, and it’s scale-free networks that are primed for viral propagation. When the goal is content distribution powered by network effects, it just doesn’t matter if you actually know the person or not. All that matters is if the person if a reliable source for interesting content. This is why social networks have failed to become anything more than social networks. Remember this graph from Brad Horowitz?

Only 1% of a community are content creators

Most people’s social networks aren’t large enough to contain more than a handful of super-star content creators. I believe the average Facebook account has 300 users. 1% of that is 3 – not enough to be a valuable source of content. If you think Dunbar’s number is a more accurate group size – which I do not with regard to online communities – then you’re only left with 1 or 2 people (1% of 150 = 1.5). Yes, you can argue that Facebook has done very well for itself living off of soap-opera like content – who is dating whom, who got drunk at what party, what was she wearing?!?) but there are natural limits to that growth. Their commenting feature will help drive page views, but there isn’t a whole lot of new value being created there.

Newsfeed

Nothing shocking about this. Now that your users have gone out and followed their friends and other interesting users, their homepage should now be the newsfeed that aggregates all of that content into one place. But it’s not sufficient just to include the stream of content. You need to also show who contributed that content (and, in the case of re-blogging, the other hands it passed through). Why? Because that’s how you can evaluate the people you are following (and, with re-blogging, discover new people to follow).

Re-blogging

I’m a huge re-blogging fan. It’s the engine that drives the content diffusion through the network. The concept is simple: if someone I’m following shares something interesting, I can easily push that same interesting piece of content out to everyone who is following me while providing proper attribution. Someone who follows me can do the same, and again, and again. This is really powerful, as the people who follow me are most likely not the same as the people I follow. Re-blogging provides a transport device for great interesting content to travel through connected components of the network (which are generally much larger than your immediate social network).

Social Proof

While judging the content someone shares is a decent proxy for evaluating whether or not to follow that person, social proof can help. Show how many followers the person has. Better yet, calculate their influence (like PageRank does for websites). Or, show how many favorites their posts have accrued or re-blogs their posts have had. It’s a quick and easy way for users to ascertain the reputation of a user in the context of your site.

Bonus: APIs and RSS

I promised five, but here’s an extra. Use APIs and RSS to amplify your power. Provide APIs so that others can build tools that extend your reach. Publish RSS feeds so that users can incorporate your interesting content into their existing routines. Make use of other companies’ APIs to publish your content out to them (like publishing to Twitter). It’s very difficult to create new habits, but it’s much easier to go where your users already are (Facebook, Twitter, Google Reader in my case).

Like I said at the start, I’ll be back soon with some commentary on how well (or not) various sites are implementing these ideas. Off the top of my head, I’m thinking about Blip.fm, bit.ly, and Soup.io. I’d love to talk about Etsy, but I feel like it wouldn’t be appropriate.

I was chatting with my friend Dan Nelson today after bumping into him at the Cornell Music Department library. He’s currently a Ph.D student at UPenn studying musical composition. He threw an idea at me and, after some collaboration, we came up with something akin to a Flickr for composed music.

It would work like this:

Everyone who registers gets a tumblog. (Side note: I love Tumblr. I thought about apologizing to everyone who keeps hearing me implement ideas with Tumblr, but I’m not going to. They deserve it for a job well done.)

Composers will use their tumblogs to publish 2 things. One, an mp3 of their composition. Two, a PDF of the score. Composers are encouraged to tag their content for improved discovery.

Everyone can use their tumblr dashboard to follow composers they like and heart music they like. Everyone can also use their tumblogs to re-blog music they particularly like and post about their experience with the music they’ve discovered.

Like Flickr, people can search for music or composers. Like Flickr, you can explore the most interesting compositions. And most importantly, like Flickr, you can pay to have a hard copy printed, bound, and mailed directly to you.

Your market is anyone who buys sheet music – basically every school and private music teacher in the country (and internationally). Sure, some people would just print out the PDFs for free, but I bet enough schools and teachers would pay for the nicely published and bound parts and score to make a profit. Like Chris Anderson has been saying since he wrote the Long Tail, you can be successful even if only a minority of your users pay.

If successful, what we will have done is broken the pre-Internet strangehold of the major publishing houses and academic elitism on composed music, similar to what Vimeo is doing for video and Etsy is doing for handmade goods. Something I thought about when I was at Etsy that applies here is, “An audience for every artist.” We can create a forum for quality composed music with a long tail. We can use social aggregation instead of editorial fiat to discovery great music. We can empower people to try their hand at composition even if they are a one-hit-wonder.

If anyone is interested in helping Dan build this company, please contact me – david.lifson at gmail – and I’ll pass your name along. It’s super simple, engineering-wise; Tumblr’s API is free and easy to use, storing the data and metadata is easy, slap a search index on the data, and you’re done. All that would be left is to set up a business partnership with a printing company like Subito Music, and the task of getting the word out and building up a community. And really, how many startups these days come with a business model, not to mention a highly targeted audience and a trove of user preference data.

And here’s another:

And one more:

The rest of the NYT article outlines how far eBay has fallen. eBay had short-term focus (quarterly results) and fell into the innovator’s dilemma because they were held hostage by the hostility and vested interests of their sellers. Consider this quote, via Om Daily:

When you’ve got buyers needs in one ear and sellers screaming their needs in the other ear every day, you find yourself constantly tested – do you have enough persistence, enough patience, and enough customer focus to succeed?

Tags: amazon, ebay

Doc Searls Weblog · The Live Web

Yes, yes, yes, and yes. The web is a medium for activities – for collaboration and participation. When I read (or hear or watch) something, I want to know who created that content. I want to respond to it (as I am now with this post). I can publish this post out to my friends via a variety of methods thanks to RSS, so that my content can go to where my friends’ are, and not forcing them to come to me.

It’s perfect that at the end of Doc Searls’ blog post he links to Umair Haque’s post called How to Build a Next-Gen Business Now. Read it. Understand it. Expand the pie for everyone.

Summary

Prototype Driven Development (PDD) is a software development methodology that believes that by continuously creating, improving, and interacting with prototypes, software will be launched faster, with higher quality, and with a stronger focus on the customer. This is accomplished by using cross-functional teams to continually iterate on prototypes of gradually increasing fidelity while constantly absorbing input from the team and others.

Waterfall and Scrum

What is Waterfall?

The waterfall method of software development is a monolithic system created at a time when building and testing software was costly – punch cards had to be made or hardware had to be redesigned, so it was critical to figure everything out early in the process because any feedback could be gathered.

The waterfall process looks like this:

At each step, if a problem is discovered or the context of the project has changed, control of the project is routed back to the top of the waterfall.

See http://en.wikipedia.org/wiki/Waterfall_model for a more in-depth discussion.

The traditional waterfall process of software development doesn’t work because it:

1. is very slow and heavy weight (months to complete projects)
2. silos creativity between product, design, and development
3. expects impossible levels of foresight from product (and at times, design) to avoid going backwards and revising all of the documentation
4. creates lots of documentation that never gets read once the project is complete since maintenance fixes end up changing or adding to the requirements and design
5. does not provide sufficient progress visibility for senior management.

What is Scrum?

Scrum is the idea that by squeezing the development cycle down from 2-4 months to 2-4 weeks, the development process is less prone to estimation errors and more likely to ship higher quality code faster. The team works off of a backlog of features, with each feature small enough to be accomplished in a matter of days or hours. By reducing the size of a feature, Scrum teams can more quickly react to changes in prioritization and provide more visibility for management into what is done vs. not yet finished. The team has a daily standup, where each person outlines what they did yesterday, what they are doing now, and what blockers they might have. Each person also updates a document describing how much effort is remaining for each of the tasks assigned to them.

The Scrum process looks like this:

See http://en.wikipedia.org/wiki/Scrum_(development) for a more in-depth discussion.

Agile methodologies like Scrum don’t work because they end up like waterfall, just on a smaller scale. Instead of 2 months each of requirements, design, development, and testing, you have 2 weeks of each. This is because there is typically a “cycle zero”, where product and design do a fair amount of planning before launching into the first sprint. This cycle zero is basically the first two steps of the waterfall method. Agile does, however, address some of the problems of waterfall:

1. is very slow and heavy weight (months to complete projects) Medium weight as the team can react from sprint to sprint (2-4 week cycles)
2. silos creativity between product, design, and development Product, design, and development meet for a scrum standup every day, so there is frequent communication. Requirements are still created upfront by product and design, outside the Scrum process.
3. expects impossible levels of foresight from product (and at times, design) to avoid going backwards and revising all of the documentation Foresight is still expected from product / design, but at least the cycles are shorter (2-4 weeks) instead of months.
4. creates lots of documentation that never gets read once the project is complete since maintenance fixes end up changing or adding to the requirements and design No documentation is needed because of frequent communication. The time and effort spent writing documentation is funneled into making better product
5. does not provide sufficient progress visibility for senior management. Senior management, at the end of every sprint, theoretically can see a demo of what was worked on. “Potentially shippable code” is the phrase Scrum uses.

Prototype Driven Development

PPD puts usability and adaptability first. By constantly iterating off of a prototype and folding in input from the team, internal users, and usability tests, a product can be created quickly and with high quality.

With PDD, 1-3 engineers meet with a product manager and a designer to kick off a project. The team sits down and discusses the general problem and spends some time (no more than two hours) coming up with the general framework of a solution. It doesn’t matter that the solution isn’t optimal and most details haven’t been figured out. Often times this vague idea comes along with a pencil / whiteboard sketch of the UI.

The team disperses. The developers go and implement the most barebones prototype of the idea – black and white, fake data, no CSS, stubbed code, just enough to start a conversation. The product manager and designer start working on prioritized user stories, interaction design, etc.

The next day, the team meets and presents their work to each other. The devs show off their crude prototype. The product manager presents what she think are the most important activities the product needs to support. The designer lays out his ideas for layouts, user flows, etc. Each idea is vetted by the group and consensus is established. Maybe the product changes direction or maybe one particular use case becomes more important. The team then goes back and iterates again, improving the prototype or design or requirements.

This meeting happens every day at the same time. Each day, there is more clarity than the previous day because the crude prototype continually evolves into a more mature product. Each day, the team is interacting with each other and with the prototype, adjusting course as necessary. Each day, senior management can come in and interact with the prototype and react to it. At any point, the call can be made to lock down the feature set and drive towards launch.

The key point here is that prototype driven development forces everyone involved to focus on the primary activity being modeled by the feature. There isn’t time at the beginning to fuss about colors or CSS since there is only one day to make the prototype a bit more accurate. Each day, the team is evaluating the usability of the prototype, trying it out or asking others to evaluate it. Only when the key activities have been fleshed out will attention turn to the details.

To follow up on the five failures described above, here’s how PDD scores:

1. is very slow and heavy weight (months to complete projects) Medium weight as the team can react from sprint to sprint (2-4 week cycles) Super lightweight process that can adapt daily
2. silos creativity between product, design, and development Product, design, and development meet for a scrum standup every day, so there is frequent communication. Requirements are still created upfront by product and design, outside the Scrum process. No silos because everyone is contributing to the creative process every day.
3. expects impossible levels of foresight from product (and at times, design) to avoid going backwards and revising all of the documentation Foresight is still expected from product / design, but at least the cycles are shorter (2-4 weeks) instead of months. The team is course correcting every single day, so the cost of trying something is very low and philosophical arguments are minimized.
4. creates lots of documentation that never gets read once the project is complete since maintenance fixes end up changing or adding to the requirements and design No documentation is needed because of frequent communication. The time and effort spent writing documentation is funneled into making better product No documentation, other than an interactive prototype that is available immediately.
5. does not provide sufficient progress visibility for senior management. Senior management, at the end of every sprint, theoretically can see a demo of what was worked on. “Potentially shippable code” is the phrase Scrum uses. Senior Management immediately has a prototype to evaluate and react to.

Discussion

PDD requires great people. The product manager has to have a relentless focus on the primary goals of the product to avoid feature creep. The designer must be available every single day and be willing to create and throw away many iterations of their work. The designer must be able to work quickly, in low fidelity, and in an environment with ambiguous requirements. The developer must have high levels of comfort with prototyping code, stubbing out code / faking data, and a willingness to edit and throw out lots of code. PDD is like collaborating to write a comic book – there will be lots of editing of the storyline, the graphics, and the words themselves.

Prioritization is outside the scope of PDD. It is assumed that prioritization has occurred, and the team is being tasked with executing on a prioritized project.

Where is QA? I’m not sure. Flickr doesn’t have QA. I think QA can sit side-by-side with the developer and do some variant of Test Driven Development – http://en.wikipedia.org/wiki/Test_driven_development. In this situation, the QA engineer is writing test cases for the specific feature of the prototype the SDE is currently implementing. Since requirements can adjust day to day, the QA engineer needs to be as flexible as the rest of the team in editing their work daily.

Four hundred and twenty eight posts ago, I started a thread on the Etsy forums. The thread was intended to have two purposes. One was to simply get a sense of the community and introduce myself to them. Second, I wanted to get some better intuition as to how sophisticated Etsy sellers (the majority of forum posters) are about running their business and the e-commerce business generally. So I asked a basic prioritization question – would you rather us make search better or fix a bug that would occasionally reset the page views counter on your item listing pages? (For those curious, the views system is stored entirely in a cache, and when the cache gets full and a record gets evicted, the page view number resets to 0. Clearly, the system was not engineered to be used in this manner.) My follow up was, if fixing the view system is not a priority, would you rather we get rid of it entirely or keep it broken.

The danger is to get lulled into an urgency to please. When hundreds of users are demanding a feature, you may feel compelled to acquiesce and build the requested feature. Before you do that, stop and consider Henry Ford: “If I did what people said they wanted, I would have built a faster horse.” (or something like that.) Customers are excellent gauges as when something is wrong, but can be extremely misleading about both what exactly is wrong and how it should be fixed. Furthermore, customers do not (or should not) have better visibility than you do into strategic goals, key business metrics, engineering resources, etc. They don’t have your long-term vision nor your understanding of complex dependencies. So don’t jump the gun. Listen, follow the comments to the source, and solve the root of the problem.

I considered Tim O’Reilly’s suggestion of investing in pieces of an “Internet Operating System”, which could be the answer although they’d have to fight Amazon, Google, Cisco and others for the bragging rights. Requirement (1) no, (2) yes.

I considered gaming, hardware, healthcare, social networking, and others…. but Microsoft is always involved there with mixed success. Is there anything left?

I’ve got a crazy suggestion. And yes, your own personal blog is the perfect place for crazy suggestions. So here it is – Microsoft should work to be the #1 destination site for vertical searching of the “organic web” (I just made that phrase up). The organic web would be defined as information that is continually changing. One example is airline ticket prices. Another is real estate, and another is classifieds. Microsoft should go out and develop / acquire any company who currently has the following properties: (1) The relevant data changes continuously, (2) The site is a leading player in their vertical, and (3) search is the main user activity on the site. Examples I can think of are Farecast (they bought this one), Craigslist (good luck there), and Redfin. With insider access to the data, Microsoft could provide superior search experiences to Google. Microsoft could then create a search portal that would be the first place everyone would go to search for data in these areas. Google’s crawlers can only go so fast – if Microsoft could provide a “real-time” search engine customized to a particular vertical, they could differentiate themselves in a very powerful way.

That’s it, keeping it short tonight since I’ve got a meeting in 10 hours with the CEO, COO, a few others.

Found this via Paul Kedrosky’s Weekend Reading post.

Really interesting stuff – given that people are really bad at making correct stock market predictions, the more information they know, the worse they perform.

Behavioral economics studies these phenomena, and firms are counting on it.

I wonder if this is true in other areas as well, like deciding on how to price items for sale on Etsy.

Tags: behavioraleconomics, pricing

Amazing, I was just writing up something like this yesterday. I said:

For good measure, you could also add in GPhone data – who is in my address book, what I’m saying over SMS and phone conversation (transcribed into text via a service like Jott), and my location. The only part (and I admit it’s a crucial part) that I don’t understand is how Google will benefit by “out-opening” Facebook. My guess is that more data = googly goodness. Google will know more about you if you take Google with you, or bring the places you visit to Google.

Blogged with Flock

Tags: Google, social networks, platform