Ajax Scroller & The Searchstream

by Mike Levin SEO & Datamaster, 10/16/2005

In case you’re wondering, yes I just added the Matrix-like scrolling of my search log data to my site. As readers of my site know by now, I’m on a mission to associate leading-edge white hat SEO with the PR industry, and to claim as large a portion of that market for my employer, Connors Communications, as the field goes mainstream. Right now, as John Battelle points out in his book, The Search, SEO has something of a Wild West feel to it right now. There are a lot of snake oil salesmen in the SEO space right now, and everyone who wants the easy-to-understand business model—both from the service provider and customer side—have turned to paid keyword campaigns, aka AdWords and Yahoo Search Marketing (once GoTo, once Overture). Yahoo should have perhaps kept the Overture name, because when making a blanket statement about paid search, you used to be able to say AdWords/Overture. Now, you have to say AdWords/Yahoo Search Marketing. Sure, they get the company name into the statement, but it is less powerful. I liked the concept that you were making an overture towards the searchers. It was almost as strong as AdWords.

Anyway, back to the scroller. Part of our special sauce in SEO is a sort-of access and control of the client’s full search log data that is rare in an SEO company, much less a PR firm. You either need to shuttle massive log files back and forth, which may be spread across multiple servers, or you need to install JavaScript code, but that usually sends the data to the servers belonging to Urchin, ClickTracks, NetTracker, WebTrends or some other Analytics company. The problem here is that you can only run their reports against the data, and they can’t necessarily store it forever. When you need to extract the fine detail in order to get hyper-competitive from a natural optimization standpoint, the data just isn’t there. Connors takes a different approach, and has formulated an ultra-efficient way to keep your search log file data forever. What you’re seeing at the top of my blog pages is the dataflow, or the clickstreams leading to my site in particular. I could do an app like this with any of Connors’ clients, if they were willing to expose such data. It’s nice from a “creating buzz” standpoint, and I may do it with Connors’ site itself.

So, how can I keep the scroller going virtually forever? Ajax, of course! Yes, a PR firm in NYC that has Ajax programming capabilities. It may not be 100% clear at the moment why PR firms with such technical capabilities is such an advantage, but just as with any emerging industry, it will become clear over time. Then, everyone will wonder what things were like before. That’s already happening with search marketing in the form of AdWords. It will eventually happen, but on a much smaller scale, with the merging of PR and SEO, and the introduction of the equivalent of the AdWord killer app and business model for the PR. It is little coincidence that PageRank, named in a double entendre after Larry Page, is often abbreviated as PR.

Since the scrolling app is Ajax, I intermittently fetch new data without a page reload. It’s an example of how iframes are still useful even in light of Ajax. I had to decide how chatty vs. chunky the communication was to be. I also had to decide whether I was going to shuttle the data back and forth as pure data or display-ready HTML fragments. I went with fairly chunky, pre-formatted payloads to minimize the load on the client, and to minimize chatter. You can load one of these pages and keep it loaded all day, and it barely shows in our performance logs, fetching only about 50 records at a time once a minute-or-so. I’m using one of the available DHTML text scrollers, so Ajax doesn’t really factor into the movement very much. In fact, the most difficult part of the project was timing the Ajax refresh with the last bit of the chunk scrolling off the screen. I’m sure I have browser compatibility issues to hammer out. I’ve been checking it on FireFox and MSIE as I went.

I particularly like the connection to the way the Matrix screens work. The idea of watching the dataflow and actually gleaning meaning, which while perfect for a SciFi movie, seemed so unlikely and geeky to the average person. But the clickstream established by search data is much different, because of the “keyword rivers” they establish. I’ve been watching these keyword rivers scroll past me for ages knowing I was privy to a unique perspective. People get too hung up on reports and the clickstream to understand the fascinating nature of the searchstream. With Ajax, I saw the perfect opportunity to share this experience with the world at large, by scrolling it past you. If you do refreshes, it may be a little repetitive, because my search hits are not (yet) really coming in that fast.

It gets interesting by NOT doing a refresh (more contrary Web-logic, thanks to Ajax) and just watch them scroll past you. You are sitting there, watching the database of intentions, on my little slice of the Web. You will NEVER see the same exact search twice. They may look similar, but they are all a little bit different, usually in the parameters. You can click on any of the lines and reproduce the search experience of the original visitor. You may not see the exact same results due to geotargeting and datacenters. But if you do find me, you can click on the result, and on the next pageload, you’ll see your search term at the top of the list!