Mike Levin SEO

Future-proof your technology-skills with Linux, Python, vim & git... and me!

Cometh The Super-Curator - How SEO is Changing

by Mike Levin SEO & Datamaster, 09/13/2012

Under the “new” SEO model, you will either need to become an Agent with a high reputation score, or be able to rally other Agents with high reputation scores. But your reputation will only make you authoritative and able to impact search rankings in your areas of expertise—how the Circles you belong to have been named (tagged), and the subject-matter on which you regularly write, and have been rewarded with +1’s. This has not happened yet, but the overwhelming evidence is that it will.

So you need to either BE the genuine article, or know how to motivate the behavior of others who are the genuine article in the field you’re addressing. This will lead to whole new ranking manipulation schemes. However, there will be string incentive for these “Agents” (really just Authors or a company proxy) to not put their reputations at risk over inappropriate endorsements, because a lifetime of reputation could be irrecoverably lost in a few seconds of Google penalization. It’s the Google+ profile that would be penalized—not the site. So you’re putting your reputation at risk with every endorsement.

So what to do if not some fancy modernized version of link development using people? Well, the idea is we’re now in a truer meritocracy. Huh? Simply, if your product is good, then your audience will find and reward you. You will ascend in the ranks naturally. Your exposure will be truly earned, and there is no manipulation. It’s what PageRank and the BackRub algorithm should have been, but without identities on the table and reputations on the line, Webmasters were unaccountable—and with such unaccountability came corruption.

Google’s family jewels—the Link Graph—is both easily corruptible and becoming increasingly easy to reproduce. Storage and bandwidth speed aside, crawling the Internet is increasingly easy. Recent experiments show you can pick up about a billion pages in a week with relatively little computing resources. Plus, the function and necessity of such a crawl is increasingly in question when web publishers WANT their content to be found and will go out of their way to tweet and publish it into the stream.

Really, information infrastructure is Google’s new family jewels—things you hear labeled as data-centers and BigTable and the App Engine. They are the mechanics behind crawl and search, GMail, Google Docs and every other Google service. They can be adapted in creative new ways—most notably of late, Google+, in preparation for a brave new world in which identities, publishing histories and content interaction (comments, +1’s, etc.) play a huge role. Almost all new noteworthy content is going to be tossed into one of the monitored announcement streams, and not actually require a crawl to be discovered.

Stuff that requires a crawl to pick up is probably the least monetize-able content—stuff that the author barely cares if it gets found and so doesn’t care to announce. Even if you don’t tweet or otherwise announce your content, modern publishing platforms such as WordPress still tend to “ping” centralized content announcement systems with something called PubSubHubbub. If all else fails social media-wise, your very content management system is trying to step in as a stream-pinging safety-net. Usually these features are labeled under settings as promote or market my content.

And so, “old school” SEO becomes increasingly relegated to overflow on searches that are not deemed to be of enough monitory value to warrant serving an ad, and also have no content that has gone into the stream and been voted-up by either someone you know nor by someone of high expertise and reputation in that field. Old-school SEO is the backfill. It’s like Inktomi in the old days of GoTo.com. Search is a necessary evil that leads you away from advertisers who will otherwise pay for your visit.

In short, search resulting from a crawl is for hard-core research. It’s for people ready and willing to dig deep into search results and do lots of query refinement. It’s for people looking to solve problems or surface obscure information. This is also unfortunately where long-tail spammers do their best work, because competitiveness drops off and you can spin out thousands or millions of pages and use them to go fishing for exact-match 4 or 5-word queries. In that sort of space, many of the rules about PageRank don’t apply. The more precise and obscure the search, the more on-page factors and light link manipulation work to capture top search positions.

But how were all these millions of long-tail targeting spam pages monetized? The Google AdSense network! Erik Schmidt himself called the Internet a cesspool. This is after fifteen years of Google beating the hell out of primary publishers with a net traffic arbitrage system favoring tech-savvy content aggregators (clever thieves), and forcing everyone who lacked that savvy to buy their traffic with AdWords. Keep that up for a decade. Google built their brand. Then they started swimming upstream with the quality of advertisers they dealt with and percentage of your budgets that they commanded.

Oh yeah—then when you don’t need it anymore, start dialing back AsSense spam incentive, clean up the cesspool to make yourself more attractive to deal with by the big-budget advertisers of the world, and offer more products to let you get into that real estate that for so many years you would have needed to hire magical shaman with a computer science degree to maybe sorta have a chance of getting into. The largest prizes out there to be won are gradually being taken out of the column labeled: “Achievable through SEO” and into the column labeled: “You gotta pay Google”.

Now, Google can’t make changes like that all at once. It’s got to be gradual over a period of years. There was a time when Google measured how far they could push the envelope on their homepage in the HTML byte count of the page. This turned out to be key to their early success—providing contrast to the other Portal-like sites competing with them at the time, like Lycos, Excite and Go, putting increased focus on “just search” and how much better those results were. Those days are LONG gone, but because Google users are not locked-in customers, Google must make changes in tiny measured steps so as to not drive everyone away. Fortunes can change overnight.

And so that’s what we’ve been seeing since about October of 2008 when Schmidt came out with that statement. A seldom discussed anymore Google algorithm update called Vince provided a safe-harbor for brands, but little did anyone know how much they were going to need it, because Vince was soon followed up by the Panda and Penguin algorithm tweaks. Functionally, Vince provided a safety-box for big brands to sit in while Panda and Penguin wreaked havoc on everyone who didn’t trip-off the big-brand indicators.

The result is a much cleaned-up search result space, and a cleaner canvas upon which to paint a new picture of Google—one that takes much more advertising. All these anti-spam measures and new product offerings are a result of Google realizing that spammers are eating their lunch, and they need to create more and better products for advertisers that occupy that same space. It’s reasonable on many levels, but rolls back the level playing field in which clever individuals can make larger grabs for audience than major publishers.

However, the underlying “organic” or “natural” search results isn’t really a pristine canvas. No matter how much Google adjusts thresholds in its spam-catchers, clever spammers can always fly just below that threshold—so, the strata of results go: Advertisers, Spammers, Meritorious content. Google needed to find a way to promote meritorious content above the spammers, and that means dialing back the influence of the old Link Graph, and dialing up the influence of a new breed of content curators who get their super-powers from a game that is very difficult to corrupt.

Why is Google+ difficult to corrupt? Because much of what these profiles are doing is transparent. It’s establishing a history of +1’s, published articles, membership in Circles, and probably much more. It takes as much work to build up one of these online reputations as it does a real-life reputation. In fact, they will probably be merging to be one and the same in much of the world participating in the information age. Look at Wikipedia for some parallels. Okay, editors aren’t completely transparent, but the edit histories are and the number of eyeballs looking at it is massive. Some bad articles and information may get through, but you’re talking about all the time in the world to clean it up.

We haven’t seen the impact of AgentRank yet on search results, but you can bet things are going in that direction, because the alternative is so awful. And so, the new strata of results will go: Advertisements, Meritorious content as per super-curators, Spammers, The Old Web and unrecognized gems.

So what about the old web and those unrecognized gems of the days of yore that somehow still manages to get hosted? And what about new content that gets produced but is not promoted in any way whatsoever, but should still be findable? Product documentation? Laws? Public domain texts? Well, much of this will be provided explicitly through APIs, and even more will get marked up with modern HTML5 and schema.org microdata, making it not really part of the old web.

But how does such content compete? Well, the answer is it doesn’t have to be competitive. Such content is probably targeting 3, 4 or 5-word combinations, and therefore a much smaller audience, and probably less profitable. The playing-field gets more level as the rewards diminish. Oh, the long-tail is also strong where the rewards are big, but the market is small and product sales-cycles are long, and lots of obscure research is involved—but that’s the subject for another article.

And so there you have it. All search traffic being arbitrated through Google gets sorted out a little differently. They’re being smarter about how much of that traffic they can get paid for, and what’s left over can’t really be allowed to go to hell the way it was back in the days of Portals when search was a necessary evil. Google won’t let that happen, so they’re fixing what now must be viewed as the back-fill and overflow to advertising with a system that much favors reputable authors ascending to be super-curators over the invisible-handed spammers.