My comment is still in moderation, so in the spirit of the real-time Web, here it is:

Tom, great post. The problem is spot on. The solution makes one key assumption — that you can’t approach the problem with pre-computation.

That’s true for a service like Hunch which sees a tiny amount of data about me, especially when I first show up. Given how little you know about me, you have two choices: either you pre-compute info in an impossibly large space, which is impractical, just as you describe, or you do the type of real-time processing which is much more effective. So far so good.

But that’s not necessarily the best way to approach the problem from the standpoint of someone who had a lot of data about me, e.g., Google or Facebook or even Amazon. The set of Internet-connected humans is small from a computational standpoint and the meta-data trail we leave is growing at a much slower rate than compute/storage. Pre-computing starting with 100 dimensions doesn’t work. Pre-computing starting with a few billion humans works really well, if you have a lot of data on the humans.

This is one of the fundamental advantages Amazon, FB, GOOG and others have compared to point services such as Hunch. It’s not a fair fight. So you have to innovate like crazy to compensate. Rock on!

People who do data analysis and machine learning have learned one thing through experience (can someone claim it as their Law?): to solve a complex problem with little data you need fancy technology but you may be able to solve the same problem with much simpler technologies if you have a lot of data.

So, the big guys have a fundamental advantage. However, there are ways to even out the playing field. It starts with users owning their data and giving it to a trusted third party that can do the same type of pre-computation the big guys can do. Then you put the right access control mechanisms allowing users to share this info with third party services like Hunch.

Facebook already does something like this through F8. Facebook apps get access to a lot of valuable information not available through other means. But Facebook doesn’t share any really interesting pre-computed analytics, at least not right now. They are the smartest of the big guy bunch so far. With the exception of the the string of privacy faux pas, I’ve been consistently impressed with their strategy.

It felt strange to be on the opposite coast from the team as they were going through the final checklists but I guess this is what I get for co-founding a company in the Bay Area. I’d do it again with the right partner, though. Jason Shellen has been great to work with as a founder, partner and CEO. His vision for expanding blogging and, more broadly speaking, self-expression on the Web to an ever broader audience of contributors is powerfully present in the initial release. The team has done a fantastic job integrating the feedback from our private release in the fall into a neat service with great consistency of purpose.

A: you win the cool CFO award
C: you carry the spirit
G: you are funnier than your brother
J: now comes the truly exciting part
M: let’s do that mountain ride again
RF: your design makes me wish for your cooking
RK: thanks for pushing it over the hump
W: I’ll assemble furniture with you again
Z: where were you earlier?

Go Plinky!

I see three generations of SEO optimization in the wild that build upon each other. Below are their distinguishing characteristics.

• SEO 1.0: human-driven SEO. This is primarily about people tweaking content and linking structure. The main distinguishing characteristic of an SEO 1.0 company is that there are people on staff who spend a portion of their time optimizing content for SEO. The best example is probably About.com, with its 750+ experts and dozens of editors. The experts and editors choose which topics to write on and know how to write to drive rank. They know how to format titles and what words they should avoid and what’s the “right” length of an article, etc. They also know how to build a strong link internal structure, albeit mostly by hand. Our own TechTarget and THCN are pretty expert at SEO 1.0. For Internet marketing, HubSpot is a great resource which helps many companies get on the SEO 1.0 train. The biggest problem of SEO 1.0 is scale–it takes humans to optimize content.
• SEO 1.5: adding UGC for scale. What you may not be able to do with a dozen or a few hundred experts you may be able to do with lots of contributors. Blogging sites are the main examples. They key here is to get to scale and avoid “poisonous” SEO–the type of content that actually can drive your rank down.
• SEO 2.0: machine-driven SEO. If SEO 1.0 is about the human touch, SEO 2.0 is about software crunching through content databases. What SEO 2.0 lacks in human skill it gains in scale. The key is having good meta-data. WordPress.com is a good example. Although the content is UGC, nobody is teaching WP bloggers how to write for SEO. However, the smart folks at Automattic are taking the meta-data from blog posts, including but not limited to titles, tags, linking and visit histories, and using them to generate both new pages (for example, using the SEO tag) and new links in content (as in the auto-generated possibly related posts at the bottom of blog posts). E-commerce sites are great candidates for SEO 2.0 because products carry a lot of meta-data allowing new pages to be built by brand, price and other features. Many also do SEO 1.0 through on-staff experts/marketers writing about the products and SEO 1.5 by adding (or syndicating) reviews. Another good example is EveryZing, run by my friend Tom Wilde, which can add meta-data to media assets (podcasts, videos) and make them search-friendly by doing speech-to-text. Yet more examples are what BitPipe (now owned by TechTarget) or Scribd are doing.
• SEO 2.5: content enhancement. I’m noticing an increasing effort, particularly with UGC, to use software tools to enhance the quality of content  that is produced as opposed to try to add value around it (such as through additional reading links, etc.). The idea is simple: the better the content and the more relevant links in it, the more interesting it is for the search engines and potentially for visitors. One player in this space is Zemanta (which Fred Wilson recently invested in), whose widget I’m staring at now as I write this post. I like what these guys are doing. At Plinky, we are taking a very different approach with the same goal–help people create better UGC.
• SEO 3.0: it’s all about back links. If you have a lot of content with good meta-data you can create an amazing internal linking structure within the sites you control. But you need to have back links to drive rank. Domain squatters for years have built link farms across the domains they own and have tried to hide the coordinated nature of their linking from search engines. Then came the comment and review spammers who essentially poisoned much of that pool of UGC for search. On the legitimate side, businesses have struck content partnerships that increase back links for years. But it wasn’t until the last couple of years that many companies approached the problem of building legitimate, value-added back links with scale and automation in mind. We first saw this through site templates (MySpace or WordPress, for example) and widgets and the back links to their creators. I am now seeing a next generation of startups innovating in this area. The delicate balance is to add tons of back links that add value to visitors as opposed to look like a link farm.

The majority of SEO experts and consultants built their fame doing SEO 1.0. Some have experience with SEO 1.5 and can suggest manual widget distribution strategies to help with SEO 3.0. These people tend to be focused much more on content than on software. In my own experience and in talking to heads of marketing and CTOs at Polaris portfolio companies I get the sense that very few SEO gurus have any experience with building the foundational software systems that drive SEO 2.0-3.0. This is a big problem. It is not easy to build an SEO-optimized content management + publishing system driven by a large content database with largely automated editorial processes. If you want to take your business to scale, you have to think about your SEO strategy and architect your service for this from the ground up. This is not something you can bolt on. Also, unless you already have an SEO-knowledgeable architect on your team, expect to spend a long time finding one and consider building an SEO advisory board to augment the knowledge/experience of the person you’ll end up with.

The next part in the series will look at the economics of “free” traffic.

Clay’s main point is that much social surplus is trapped in countless hours of passive media consumption.

So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project–every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in–that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it’s a back-of-the-envelope calculation, but it’s the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that’s 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, “Where do they find the time?” when they’re looking at things like Wikipedia don’t understand how tiny that entire project is, as a carve-out of this asset that’s finally being dragged into what Tim calls an architecture of participation.

Think of the participation gap as the opportunity cost of passive as opposed to active, participatory engagement. Not all passive engagement is bad and not all participatory engagement is good but it is hard to argue that there is an imbalance.

Aaron makes a solid point that web sites are not movies. Movies are fixed content. They are what they are. All the tweaking has been done prior to launch during editing (after test screenings and other ways to gather feedback). Movies are also easy to grasp–a couple of hours in a dark room, you get the point, you go home. Most Web sites are constantly evolving. They could also be deep and take much longer than two hours to explore.

If your product is like a Holywood movie then perhaps the Holywood launch might work for you. Truth be told, most of the sites that 37signals launches are clean, simple, easy to understand and of pretty high quality.

For others, Aaron’s suggestion is to go with the Gmail model of creating artificial scarcity. Aaron doesn’t use the term himself but quotes mojombo:

Artificial scarcity is a great technique to generate excitement for a product while also limiting growth to a rate that won’t melt your servers. We worked through a huge number of problems and early users gave us some of the ideas that have defined GitHub. By doing a Hollywood launch, things would have been very different and I am convinced, very much worse.

I’m a big fan of artificial scarcity as a way to gain leverage. I don’t think startups use it enough. The broader concept can be applied to many situations in addition to how to launch web sites.

I learned about artificial scarcity first in my highschool economics class. Apparently, in the 80s the US government threatened Japanese car makers with tariffs because they saw their vehicles as threats to Detroit manufacturers. The Japanese, being both smart and courteous, replied along the lines of “Oh, we are so sorry. We understand your concerns. No need to impose tariffs. We will impose voluntary export restrictions to lower the quantity of Japanese vehicles entering Europe.” The result was that consumers ended up paying more for Japanese cars, just as they would have under a tariff. However, the additional revenue went into the bank accounts of the Japanese auto-makers as opposed to the US government coffers. (For the econ geeks, supply became perfectly inelastic at the voluntary export restriction level, leading to a jump in vehicle price and profit/vehicle.)

Not everyone is a fan of articifical scarcity as it, perhaps unfairly, advantages businesses. Some more discussion on the topic here and here.

Designers underestimate the thresholds for an effective site. Respondents consider a site “effective” when visitors are satisfied with respect to enjoyment, can find information somewhat easily, and never get lost in the site. By at least one point on a five-point scale, visitors have higher expectations for effectiveness than do designers. Nonprofit organizations believe that effective sites do not have “information gaps between what visitors want and what the site provides” and that visitors are at least “somewhat satisfied” with their sites. Designers should give greater consideration to overall effectiveness, thereby reducing the chance of failure for a user to find the information they seek.

Designers are overly optimistic about visitors’ ability to maintain orientation. In the survey, the ability to maintain orientation was defined as visitors’ ability to know “where they are, where they can go next, and which pages are related.” About 70% of designers believe that visitors are almost always able to maintain orientation. That drops to about 30% when non-profit organizations express their view. In contrast, only about 10% of visitors report being able to almost always maintain their orientation. Fewer than 5% report that they tend to get lost frequently. Said another way, your visitors don’t know your site as well as you do, so make sure it is obvious how to find information through meaningful menus, prompts, and not too much clutter.

Worth taking a look at the full report.

We are looking to bring a couple more people on the team at this point. Check out the Plinky blog for details.

I got an email today from a friend who read the VentureBeat story about Ryan coming on board and asked something along the lines of “why are you hiring such a high-level guy so early on?” Making interactive design both feel and workgreat is a daunting task. There are many sites out there that look cool but do terribly at achieving their business objectives, e.g., getting someone to buy a product or generating a ton of click-throughs. All form. no function. On the flip side, there are many sites that are great businesses but are very light on design and, many times, emotional appeal. All function, no form. Striking not just good but optimalbalance between form and function requires a level of understanding of both the philosophy behind the user experience and the driving forces behind the business as well as ongoing fine-grained experimentation/iteration. Ryan is just the person we need to get this right.

• James Lamberti, SVP, Search & Technology, comScore
• Steve Rubel, SVP & Director of Insights, Edelman Digital
• Don Springer, President & CEO, Collective Intellect

Debra Aho Williamson from eMarketer, who was the moderator, didn’t have to ask too many hard questions as the panelists were more than happy to disagree with each other. There were three key issues:

• The difference between measurement and insight
• Approaches to measurement
• The role of standards

“Measurement doesn’t necessarily lead to insight” was a point Steve Rubel made a few times. He was implying that much of the new-fangled data collection and measurement focused on social & conversational media brought little value and insight to brand marketers. That may be true in general but in fast-changing environments it is rather difficult to say a priori what data one must collect and what analysis one must perform to achieve insight. Insight tends to be serendipitous and in my experience I’ve found that serendipity tends to correlate well with data availability. It’s hard to see what you can’t look at.

comScore’s Lamberti was emphatic that measurement doesn’t need to  be reinvented. Although conversational media and traditional media are rather different on a number of axes–reach vs. engagement campaigns, large vs. low volume, targeting value vs. targeting advocates, pushed messaging vs. dialog, etc.–the fundamental measurement is the same. Don Springer’s take was that one does, in fact, need fundamental advancements in measurenment in order to track key metrics such as number of conversations, share of voice and sentiment change. I think I fall on the side of Springer here. What’s the equivalent of sentiment change measurement in traditional Web analytics? Tracking star ratings or equivalents is insufficient since they won’t tell you want a blogger is thinking about a product or brand.

Steve Rubel and Don Springer really got into it when they discussed standards. Steve’s take was that w/o standards much of the measurement conversations are meaningless as they are about apples vs. oranges. At one point Don said “standards stifle innovation.” Both positions have truth in them but that’s topic for a longer post. I have to catch a flight back to Boston.

Update: the panel was also covered by PaidContent.