SocialStartups.com

Customer forums are always interesting, particularly for e-commerce sites. There is something about staying at home and being bored that ultimately leads people to “window shop” online, which leads them to socializing with other people who are doing the exact same thing. The range of personalities is wild and wildly interesting.

Four hundred and twenty eight posts ago, I started a thread on the Etsy forums. The thread was intended to have two purposes. One was to simply get a sense of the community and introduce myself to them. Second, I wanted to get some better intuition as to how sophisticated Etsy sellers (the majority of forum posters) are about running their business and the e-commerce business generally. So I asked a basic prioritization question - would you rather us make search better or fix a bug that would occasionally reset the page views counter on your item listing pages? (For those curious, the views system is stored entirely in a cache, and when the cache gets full and a record gets evicted, the page view number resets to 0. Clearly, the system was not engineered to be used in this manner.) My follow up was, if fixing the view system is not a priority, would you rather we get rid of it entirely or keep it broken.

The danger is to get lulled into an urgency to please. When hundreds of users are demanding a feature, you may feel compelled to acquiesce and build the requested feature. Before you do that, stop and consider Henry Ford: “If I did what people said they wanted, I would have built a faster horse.” (or something like that.) Customers are excellent gauges as when something is wrong, but can be extremely misleading about both what exactly is wrong and how it should be fixed. Furthermore, customers do not (or should not) have better visibility than you do into strategic goals, key business metrics, engineering resources, etc. They don’t have your long-term vision nor your understanding of complex dependencies. So don’t jump the gun. Listen, follow the comments to the source, and solve the root of the problem.

While on the subway heading back to Brooklyn - I had gone to see Iron Man at Union Square, it was great - I was thinking about Microsoft. I was trying to imagine what exactly Microsoft could do that 1) doesn’t have an entrenched player, and 2) they might be able to be successful at. Search the Google way, in my opinion, does not satisfy either requirement, even if they bought Yahoo. So what else?

I considered Tim O’Reilly’s suggestion of investing in pieces of an “Internet Operating System”, which could be the answer although they’d have to fight Amazon, Google, Cisco and others for the bragging rights. Requirement (1) no, (2) yes.

I considered gaming, hardware, healthcare, social networking, and others…. but Microsoft is always involved there with mixed success. Is there anything left?

I’ve got a crazy suggestion. And yes, your own personal blog is the perfect place for crazy suggestions. So here it is - Microsoft should work to be the #1 destination site for vertical searching of the “organic web” (I just made that phrase up). The organic web would be defined as information that is continually changing. One example is airline ticket prices. Another is real estate, and another is classifieds. Microsoft should go out and develop / acquire any company who currently has the following properties: (1) The relevant data changes continuously, (2) The site is a leading player in their vertical, and (3) search is the main user activity on the site. Examples I can think of are Farecast (they bought this one), Craigslist (good luck there), and Redfin. With insider access to the data, Microsoft could provide superior search experiences to Google. Microsoft could then create a search portal that would be the first place everyone would go to search for data in these areas. Google’s crawlers can only go so fast - if Microsoft could provide a “real-time” search engine customized to a particular vertical, they could differentiate themselves in a very powerful way.

That’s it, keeping it short tonight since I’ve got a meeting in 10 hours with the CEO, COO, a few others.

Om Malik wrote an interesting post about twitter pricing yesterday, but I think he’s a little off. I don’t blame him, considering his background is not computer science. And besides, it started a really interesting conversation. Before we start talking about Twitter pricing plans, we need to come to an agreement about what technically is hurting Twitter. Ideally, scaling issues should be orthogonal to your business plan; if you are successful, lots of people use your product, and that’s a good problem to have. Generally, you don’t want to tax your best users.

So on to the technology. Here’s the clue that we’ll start with:

Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency’s sake, Twitter was built with technologies and practices that are more appropriate to a content management system.

From Twitter’s post on architecture and the problems they are facing 

When I read “content management system”, I’m thinking “blogging platform”. My guess is that Twitter is built to be a massively multi-user blogging and blog reading system - every user gets a blog to publish posts with and a blog reader to aggregate the posts of their friends. Considering Evan Williams was the founder of Blogger, I think it’s pretty reasonable.

So if you think of it that way, then the obvious way to architect the system is publishing via RSS and aggregating via RSS. When you write a new tweet, your message gets stored in the database. (Yes, shoving all of that data into a database is a really difficult engineering problem in itself. Assuredly they will partition across multiple databases if they don’t already.) The massive pain comes in when pulling in what your friends’ tweets are. Let’s talk through how it works. Your twitter homepage is acting like an RSS reader, so first it will lookup all of the feeds it needs to check - all of the people you follow. Then, for every person you follow, an RSS feed will be read or generated. The resulting set of RSS feeds will be merged back together and sorted chronologically. The result is your Twitter homepage.

Notice here that this is what is called a “pull” or “poll” model - you are checking for new posts whether there are new posts or not. This can generate a ton of unnecessary load on servers and databases, not to mention network traffic costs. With the advent of Twitter applications, these applications are constantly polling Twitter to see if there is anything new to publish. Ping, ping, ping. All to see if there is something new afoot.

Which brings us around to pricing. It is not, as Om suggested, Scoble’s fault for having 25,000 people following him. The cost is not sending one of his messages 25,000 times. No, actually it’s Scoble’s fault for following 21,000 people and constantly checking for new tweets from those people. It’s also the fault of power users like him using applications that aggressively use the Twitter API to check for new tweets - most likely the same people who use those applications are following large numbers of people.

As with all scaling problems, the first idea is “cache more!”. And sure, you can cache the heavy Twitter producers. But Scoble isn’t following just the big twitter users - he’s following everyone he can, because that’s how he believes he can get an edge on news and trends. Can the long tail be cached? Doubtful - there are too many users who fall into that category. Can you charge those who follow more than, say, 1000 people? Maybe $10 a month for every thousand people you follow, with the first 1,000 free? That could work, but it’s risky. Would Scoble, in the face of paying $210 a month, permanently switch to Pownce? Or Friendfeed if they built a twitter clone? How many would follow?

The solution, of course, is to do exactly what Twitter says they are doing - switch to a different model and scale horizontally (”throw more machines at it”). I’m interested to see how it turns out for them.

How Thinking Costs You - washingtonpost.com

Found this via Paul Kedrosky’s Weekend Reading post.

Really interesting stuff - given that people are really bad at making correct stock market predictions, the more information they know, the worse they perform.

Behavioral economics studies these phenomena, and firms are counting on it.

I wonder if this is true in other areas as well, like deciding on how to price items for sale on Etsy.

Tags: behavioraleconomics, pricing

I think the best innovation from twitter was messaging with @username. If I remember right, Twitter didn’t support that at first - it was a grassroots invention by Twitter users that was picked up and officially supported. Facebook did something similar even before Twitter: when you wrote a post on your Facebook blog (haven’t seen much uptake there), you can choose from your friends list which friends are mentioned in the post. Kinda kludgy solution though, since you have to scroll through hundreds of people and click a bunch of checkboxes.

It’s obvious from the evolution of @replies on Twitter that this is something very organic and natural to humans. The internet is not a solitary vacuum; all software is social.

So here’s a thought: let’s bring @replies to the rest of the web. Whether I’m writing a post on my blog or commenting on a Flickr photo or sharing an item on Google Reader, I should be able to use @username. This serves two purposes.

1. Who is being referred to?

One is to give everyone reading your comment to understand who you are talking to. This is a basic tenet of face-to-face group communication - you turn to a specific person in the group, address them by name, and speak. Sometimes, like at a big dinner party, you might not know all the guests, leaving you guessing as to who is whom and what their background is. On the internet, we can do better. By linking to some kind of profile, the comment reader can read up on who is being pulled into the conversation and better understand context.

2. Who is referring to you?

Here’s something the internet can do that can’t happen in real life - being able to read the record of all conversations that made reference to you. Twitter does this with their “Replies” page. Why not off Twitter as well?

How #1 could be implemented

This is a really difficult engineering problem, and I won’t pretend like I’ve got all the answers. So I’ll do my best. There are a number of existing web sites that vend OpenID accounts, including Yahoo, Blogger, and LiveJournal. Here’s the list. All of these services support some kind of “Profile” page, where the user can publish information about themselves. So we have a decentralized way of naming people (OpenID) and we have a way to lookup information about that person (hosted profile). So what’s missing is browser support for interpreting the @reply markup.

What’s that you say? No one is going to use awkward OpenID URLs to name people? You’re right. So, browsers will also need hooks into your Address Book, so that they know which “John” you are referring to. This could have the same auto-complete UI that email clients already support - as soon as you start typing @John, a small drop down appears next to your cursor showing the various people you know who match “John”. You pick the right one, and the markup is entered for you, linking to John’s profile.

How #2 could be implemented

The last bit of this is discovering all the places people are referring to you. This is tricky, and the two ideas I have have weaknesses. One idea uses another open technology called XMPP, the Jabber protocol. Here’s how it could work. When your browser publishes “@John”, it will use XMPP to send a message to John’s OpenID server notifying John of the reference to his name. When John logs into his OpenID-supporting service of choice, he can be shown all of the messages that have been pushed to him.

The other idea is for the OpenID server to support an HTTP POST whose payload would be the URL where the reference was made. The OpenID server would log all traffic to that special URL and pass it on to John once he logs in.

Thoughts?

Anyone have thoughts on this? Obviously to big (some might call it “unlikely”) changes need to happen. First, browsers need to add support for OpenID based @name markup. Second, browsers need to know how to send XMPP messages (or, invoke a hidden URL hosted by the OpenID server, which might be easier.) Lastly, OpenID servers need to process these incoming messages and present them to the user in some helpful way.

Naturally, I imagine there are a host of security concerns to work through, especially with browsers pushing URLs around. Still, I think this would create a very interesting social ecosystem. What do you think?

As part of my new job at Etsy, I’ve been amassing a gigantic list of every idea anyone (customers included) has had on a way to improve Etsy. The point is not to cull from the list a handful of good ideas. Instead, we’re going to look at everything and try and tease out what’s causing people to react the way they are. Many times people will ask for a feature that solves an immediate problem, but they’re taking aspirin, not making a lifestyle change. You can get yourself into trouble doing this if you’re not careful - “quick, let’s redesign EVERYTHING” - but it can be very instructive.

I think Data Portability is a symptom of a bigger problem, which is that people expect to be able to project their world view onto their Internet experience. I can interact with my social network in real life without restriction, so why not on the internet? There are so many other examples beyond Social Network data. To be fair, I think Data Portability is great. I was a supporter of it at Amazon, and I really appreciate the work Chris Saad and folks are doing. I even lurk in the Data Portability google group and read the threads. So this is not a shot at them.

Umair Haque writes the Bubblegen blog which is inspiring once you decipher what he’s saying. At first glance, he can occasionally sound like a raving genius, creating large gaps in his logic where it clearly makes sense to him but no one’s following. But his core message has been the same for years, way before anyone else understood - help your users create value for themselves and the network effects will reward you. Think on that. This is not about sucking every last penny out of them via precise ad targeting. This is about good beating evil. His writing had a big influence on me and my decision to join Etsy.

So this is how I see Data Portability fitting in. Ignore, for the moment, the libertarian Silicon Valley echo chamber where everyone is demanding their data. The average person doesn’t care which company stores their data. The thought never crosses their mind. Instead, it’s all about what I can do today and what do I want to do tomorrow. People want their social network data consumed by other applications because it creates value for them. People want their attention data - well, actually they have no idea what “attention data” means - so that the experience they carry in their heads as memories are reflected back into the services they use. People want their purchase data from Amazon piped into Netflix (and vice versa) so that each site stops recommending the same damn movie.

Data Portability, of course, enables all of this, which is why it’s such a great project. But let’s not delude ourselves into thinking mainstream America is concerned about some evil corporation having a stranglehold on their data. All that matters is if people leverage the data in their minds within their computing experience.

Tomorrow I’ll write about the need for a universal identity reference service. Facebook began it with “List the people you reference in your post” and Twitter continued it with @username. It’s brilliant and has far-reaching effects, I think.

For now, have a good night.

Gotta love a word like “normalcy”, and Wikipedia for having an entry on it.

So many things have happened since my last post, so I’ll cut right to it.

1. I left Amazon to go work for Etsy.

Here’s the story. When I was at Amazon, I ran a lunch group and email list called tech chatter. I thought of it a month into my employment at Amazon, and pitched it to my then-boss Aileen Sandridge. (Useless trivia fact #1: her younger sister Manny is friends with my friends from college, and would come party at my house. I didn’t know any of this until I moved to Seattle. Also, Aileen’s husband was the bar raiser on my interview loop for Amazon. Like I said, useless, but damn it, it’s my blog, I can do what I want.) By June 2006, I got my head on straight enough to start it, so Community Tech Chatter was born. We’d talk about whatever was interesting in the community space outside of Amazon. (Useless trivia fact #2: Community Tech Chatter was later renamed WWD Tech Chatter, to include all of World Wide Discovery, the now non-existent organization which had been run by the inspiring Kim Rachmeler.) Around December 2006, I stumbled upon Etsy and fell in love with it. “What amazing items!” and “A tag-based catalog!”.

Ever since I started in community, I always wanted to, one day, be able to do what Kim did so well - manage Community and Personalization at an e-commerce company. Why? Because I love delighting people and fostering strong relationships. I blame my parents. I knew that I was probably decades away from doing so at Amazon, and likely a decade away from doing so at a smaller company. So I kept trucking along, working as a software developer in Community and then, seizing a rare opportunity, became the product/program manager for Personalized Recommendations - a group that was as well known, accomplished, and mystical as any.

So I did that for 6 months and loved every minute of it. I felt like, for someone my age and experience-level, I had happened upon the perfect job. It’s true. I got to think up cool stuff all day long, and help other people build it. I worked normal hours, had dinner with my lovely girlfriend every night, and got paid to dream.

One February morning, Lisa asked me what I wished I was doing right now that I wasn’t doing. I think she wanted me to say something like, “Walking along the beaches of the Amalfi Coast with you”, but instead I said “Run Community and Personalization at Etsy”. I didn’t say Amazon because, really, they would never take me seriously. Nor should they - I’m not qualified. I said Etsy because, other than Amazon, Etsy was the only other company I’d come across that really excited me. I then said, “But I’m several years away from applying, I’m still way too inexperienced.” My two year anniversary of employment at Amazon was a month away.

Lisa wouldn’t hear it. “Go apply! If that’s what you want, you should do it.” And so I did - I sent an email to work@etsy.com, cold call, saying I wanted to work on Discovery features at Etsy. I didn’t expect much considering the lack of a job posting and my lack of experience. Instead, they asked me a few questions over email, interviewed me in person, and voila, I’m hired! Time to move to Brooklyn, I guess.

2. Road Trip!

I was gonna fly. Really, nothing made more sense. It’s quick, it’s comfortable (if you go first class), the movers would take care of everything. But no, Lisa really, really, wanted to drive. I have to thank Mr. John Thimsen, my sixth and final manager at Amazon, for convincing me to go. (For those of you Amazonians keeping score at home, here’s the count: 1) Aileen, 2) Russell, 3) Ian, 4) Ted, 5) Eva for a few weeks, 6) John. Not bad for 2 years.) It was incredible. We drove Seattle to New York, crossing WA, ID, MT, WY, SD, MN, WI, IL, IN, OH, PA, NJ, and finally NY. You can find pictures here: http://flickr.com/photos/dlifson. We saw all kinds of wildlife ranging from prairie dogs to bison, elk to mountain goats, bighorn sheep to deer. We had homemade pie in Hixton, WI (Blue Collar Cafe, highly recommended. Try the habanero flavored sweet jellies, too). We watched the sun set on Mt. Rushmore. We had an incredible dinner in Minneapolis. We got to see our dear friend Jake Kalberer, who is making his own way back to normalcy.  We drove 3500 miles in 10 days, mostly driving in the day. One tip - don’t bother taking the northern route until after Memorial Day; too many parks and roads are closed. Maybe another time we’ll see Glacier National Park, Lewis and Clark Caverns, Yellowstone National Park.

photo credit wallyg, via Flickr

3. Quick, get a sublet!

My lease for my North Park Slope (Brooklyn) apartment doesn’t begin until June 1st, which gave me the pleasure of having a truly authentic “Welcome to New York, buddy” experience. First, the apartment I found on Craigslist while in my Minneapolis hotel room turned out to be a woman’s living room, complete with ancient and uncomfortable sofa bed. Rejecting that for the sake of my body, I’ve been sleeping on a 1.5″ thick camping mattress and some blankets. Turned out to be a good idea - one night I was reading in bed and discovered a small red insect crawling over the pages of my novel. Yes sir, my first NY bedbug. I think I forgot to mention the rent was $1425. Did I mention the great location?

No NY moving experience is complete without sordid tales of relocation pain. After signing my lease and handing over $10,500 up front (Jesus!), I ask if I can move in early since she’s vacating the apartment in a few days. She tells me I can move in the week of May 26th, at no cost. Great! I phone my moving company, Star Moving, and pass along the good news. This week, I’m notified that the driver left Seattle on Saturday. Hmm, 10 days before the 26th. It could work, since I was told to expect 7-13 days delivery time. On Tuesday the 20th, I get a call from the driver, saying he’ll be in Brooklyn on Thursday (the 22nd) and would like to schedule a delivery time. What? 5 days? That’s not good.

I dash back to my computer and fire off an email to my landlord. Could I move in earlier? She comes back the next day with, no, actually you can’t move in any earlier than may 30th. Disaster. What can the driver do? Circle Manhattan Island for 8 days? Thankfully, Kathy Martindale over at Star Moving stepped in and saved the day. Instead of charging me the $1500 minimum fee for putting my stuff back into storage, she took care of the whole thing at no cost to me! Did I mention how great Star Moving’s customer service was? Still, fingers are crossed because it’s not the 30th yet and I still don’t have my stuff or keys to my new apartment.

Where was Lisa in all of this? She and I had decided it was best for her to go to Italy to visit her friend Eliana, who was going through some tough times and really could use a friend. Lisa and Eliana are soulmates, platonic variety. Plus, it was cheaper to buy her a non-stop round-trip ticket to Milan ($787) than find a 1 month sublet for two people since she’d be staying and eating for free at Eliana’s family’s house. Boy, do I miss her. I must say, distance does make the heart grow fonder.

photo credit Amit Gupta, via Flickr

4. Etsy, reincarnate

Amazing. That’s the word that ran through my head so many times my first week at Etsy. Sometimes in wonder, sometimes in astonishment. Like I’d heard so many times, things change quickly in a startup. What I was hired to do (own the buying experience on Etsy) was not what needed to be done, so I stepped into that void. So now, for better or worse, I am managing all of product management at Etsy, meaning I am the owner of the Product Roadmap and Product Prioritization. Subject to approval by the CEO and COO, of course. And now I’m expected to manage people, which is kind of like being thrown into a tsunami and expecting to learn how to swim and surf.  And you know what, it’s awesome. I can do this. And all the while, we’re going to revolutionize e-commerce.

5. The future, and beyond!

Right. So I’m going to force myself to post regularly, because I’m convinced that writing regularly will make the writing I need to do for work come easier. I don’t know what I’ll be writing about, but I’ll do my best to keep it relevant.

In the meantime, some Etsy bits:

1. I’m using Summize to track all twitter messages containing the word Etsy, so if you want to leave me a surprise, drop me a line.

2. Lisa and I gave away all of our plates, bowls, and glasses before moving. I’m really excited about the idea of buying a whole new set on Etsy, either pre-made or via Alchemy. Alchemy, by the way, is brilliant and a game changer.

3. My dear friend Abby Taylor is knitting me a “thinking cap” that I can put on when I need to think. It’s going to say “thinking” on it. I love the handmade lifestyle and I’m constantly impressed by what people can make with their own talents and creativity. I had originally put the request out via Alchemy and received 9 bids. Pretty good!

4. Maria Thomas, our COO, just started on Monday, and she’s fantastic. She is the voice of experience, wisdom, and authority that Etsy needed. This train is just getting started.

5. I’m looking to hire another product manager and a UI designer. If you know anybody, send them my way please!

OK, I’m done. See you tomorrow.