When Vertical Search Died and No One Cared
Ok, Albert’s succinct and brilliant 5 words commentary is probably all you really need to read, but what the hell, if you prefer the long winded, convoluted, overly analyzed version, read on! (this shows exactly why I’m never dreamed of publishing anything my whole life, not even in my high school yearbook - and which Asian kid didn’t write for their yearbook? well atleast I played the violin)
In 2005, vertical search engines reached almost the epic hype proportion of B2B . Everywhere you turn, a vertical search engine was launched. . . one for video, another for blog, some for code snippets, others for medical information, others for houses. (I wont stump on their graves by linking to them).
Back then, I called bullshit on the whole thing for a slightly different reason than Tom Evlin . . . but both of us agree that the majority of them will fail. Which was highly controversial given that many heavy weights (Fred, Danny, and Jupiter) believed in its promise.
Well, the majority of vertical search engines followed the trajectory of the once mighty technorati blog search engine - vanquished by Google in one effortless scoop . . . killed by Google’s increasing indexing speed, ever expanding indexing capabilities (size & scope). . . and not the least of which, the quickly unbuzz worthy but hugely successful “OneBox.”
One of the main short comings of the Google machine was the speed in which it picks up new content . . . it used to be close to 2 weeks before any given page would be indexed. Today, for some content it could be in hours if not shorter. Google added new hardware and tied its indexing frequency algorithm to something like a pagerank. Many vertical search engines had hoped to win by focusing on categories where content relevancy is closely tied to timeliness . . . it turned out to be a dead end.
Other search engines tried to go “semantic” on Google by extracting meta data out of webpages and offering additional filtering and attributing functionalities. As it turned out, users only wanted to type once and click on the results. No one wanted to spend more than 2 minutes fiddling with drop downs and other filtering options.
Furthermore, despite, its public stance against the semantic web (perhaps simply a strategic posturing to stay ahead of competition), Google brilliantly used OneBox to extract information from webpages and presented in context of its more traditional search results. Video results is a great example. It used to be only only “oneboxed” with a few lines on the top of the search results and now its fully integrated into the SERP. (I wanted to throw in a Lord of a Ring reference here but decided against it)
Not all vertical search engines have failed though. Those that does not rely on web based data for its index has done pretty well (house hunting sites) - however they have turned into more of a traditional database + portal than a classic search engine. Others have relied on crawling the “deep web” where google bots do not/cannot visit to differentiate itself. One such category is travel search engines (where a lot of news came out last week). I’m, however, very very skeptical of the financial results of Kayak and SideStep - I have reasons to believe that a significant portion of their revenue have nothing to do with travel or search. (I’ll write about it later when I have time to do some screen caps).
So whats the final take away? Give the Google PM and the engineering team responsible for OneBox a huge raise and promotion. These guys fended off the biggest threat to Google’s paid search golden goose since Goto.com+Inktomi and didn’t even get enough respect to be poached by Benchmark to be “EIR’s”. (ok, maybe there hasn’t been many legitimate threats to Google so maybe its not that big of a deal . . . oh ya, and no, Facebook is not a threat to Google . . . yet )
