Today’s Default Search Landscape

by Mike Levin SEO & Datamaster, 07/03/2006

OK, at long last, I’m going to be updating my super-search-page. The premise of the page, which has served as my homepage for nearly 8 years, is that to stay sharp in the field of SEO, there is nowhere better to look than at the search engine results themselves. Instead of metasearchers (which were very popular in the days I created it), or even position trackers (which only record a SERP number of doubtful accuracy), it is always much better to look at the search results themselves, and even “view source” to look at how they’re tracking things.

The tracking usually is through a redirect when you click on a SERP. This is the easiest way for engines to know which results you’ve chosen. Over the years, I’ve seen all sorts of different methods attempted. Due to the power of DHTML and the document object model, I’ve even seen JavaScript functions invoked on the “onclick” event of links that otherwise did not appear to be tracked. Ajax brings this to even new heights, allowing tracking merely of what you mouseover without any click being necessary. This opens the door to some nasty cross-site-scripting vulnerabilities (XSS), but that’s a subject for another article.

With the value of looking directly at search results established, the problem is that doing so with your benchmark keywords at numerous sites is time consuming. That’s where my simple idea of bookmarking benchmark keywords first occurred, and it’s amazing to me that more people don’t do it. You become familiar with the state of the search engine landscape by looking at the results of many engines for the same keywords over time. Yes, people have automated tracking tools that detect the changes in the SERPs, but there’s always something to be said for looking at the results directly.

Anyway, it’s been almost a year since my last update, and I’ve been very remiss about adding blog search tools to my page. I was always of the feeling that these only appealed to a very small “in-crowd”, and most blogs are found through the default search anyway. But now, I’m 2-years into the PR business, and truly appreciate the “early radar” effect of Technorati and the others, which tell you when a story’s going to break within hours of the blog post being made, rather than days. So, this next update will include the blog search tools.

So, the big changes since my last update (both in reality, and in my thinking) is that the “main” search for Ask is now clearly Ask, and almost no one remembers or uses Teoma. Another big change is the obvious importance of Baidu, as being nearly as big as Google, Yahoo or MSN. The blog search tools are more important (because I’m blogging a lot more?). And I’m now thinking that I may want to incorporate some of the “product” search tools that function as the unofficial “fourth and fifth” most common used search boxes: eBay and Amazon. It’s not pure search engine optimization issues, but if you want the full picture of how your benchmark keywords are directing traffic, you have to look at  these search results too.

Gigablast, OpenFind, Walhello, and Wisenut all seem to be working well, but whehter or not they are all still considered original-source search engines or not, I’m not sure anymore and will have to double-check. There are some new players on the scene that I would like to include, that have their own original data, like Hakia and Snap. There’s some Web 2.0-ish search engines, like Eurekster. Amazon’s A9 search maybe should be included, but its not original data, and they don’t use standard querystring parameters, making it difficult to integrate into my search tool. And on a similar note, Alexa’s search seems to be more comprised of original data than it used to be.

Directories continue their slide in performance. Google’s BigDaddy update seems to have subdued the importance of all the Open Directory Project clones. And Open Directory itself (DMOZ) appears to be crumbling under its own weight–Time Warner really dropped the ball on that one. The spirit of DMOZ seems to be shifting over to Wikipedia, which fixes the Open Directory editor class cast system by making everyone an editor. ODP continues to have significant influence in controlling titles, such as with MSN, but is no longer a “must-have” SEO criteria. Yahoo still is. So, directories will still have a place on the super search page. I need to check if Thunderstone still affects anything.

LookSmart… hmmmm. I have no idea. I’ll have to check what’s up with LookSmart, Zeal, and they’re WiseNut too, aren’t they?

OK, the old timers. Is there any new data at sites like Lycos, AltaVista, HotSpot, Excite or the others? I think not. It’s all rehash of Yahoo or Google data, so they can get their cut of that $7 billion dollar keyword search industry. And of course, there’s the ever-popular AOL Search, which rehashes Google, but ads some awesome added features with its video search. Hmmm. Video. And pictures. No! I have to keep my page focused on the “default” searches.

OK then how about the dark horse PPC competitors? Kanoodle, Ah-Ha, Enhanced Interactive, FindWhat, and the like? Very little idea. Will double-check each, and look for new players.

Then, there’s the ever-present but never discussed InfoSpace, that really owns a heck of a lot of search properties, including old favorites, such as WebCrawler, DogPile, MetaCrawler and the slightly more obscure Web Catalog. They also powered Excite there for awhile. InfoSpace is worth watching, because as far as I can tell, they have their own original data, though Web Catalog (previously Thunderstone). Have to double-check that. InfoSpace searches are also the data source for the very popular WordTracker app.

And finally, there is the endless remixing of the data in metasearchers. Metasearching is so friggin easy and common that you really have to do something special to differentiate yourself. Two that have stood out over the years are Vivisimo with its clustering, and Kartoo with its flashy Genie.

Oh, and finally, finally, there is About.com, which is joined now by Wikipedia as content-heavy sites that drive enough traffic in their own right to have their respective search tools included. MySpace would be in this category as well, but their default search tool appears to be Yahoo-driven, which puts them in the category of AOL, as important as a potential high-volume traffic mover, but not original data.

Things I’m well aware of but am deliberately ignoring in order to keep the super search page focused include picture-search, where Riya seems to be the most interesting thing at the moment, but not absorbing enough new imagery at nearly a fast enough rate to keep it interesting on a daily basis. Also in this category are Flickr, PhotoBucket and Webshots. In fact, there are so many, it would deserve its whole own page.

Foreign search usually falls under this category as well. If some trends emerge that are too big to not include, like Baidu, I include them. But I can’t include every search engine specializing in single countries or languages. Google and Yahoo are so popular worldwide, that things have to reach a pretty big level to break into my search page. Open Find and Viola have both done that in the past, and I do have to survey the international scene to check what’s most important right now.

Same goes for specific news search, shopping search (beyond the big 2 of eBay and Amazon), local search, video search, and social bookmark/social network search. My super search page is focused on “the most important” searches, followed up by dark horse trends that are either influential, or I believe could displace the 800lb gorillas under the right circumstances. I cannot ferret out every little “vertical” search, as Danny Sullivan would put it, or the page would lose all usability.

OK, with that out of the way, I’m off to check my facts, dissect and update. I’m thinking once I get it right this time, I may move the super search page over to HitTail to give it an extra little boost of exposure. Rapid, direct viewing of many search results is a rudimentary and too-often overlooked process.