Mar 4, 2011

SEO is very sophisticated (and.... we're back!)

Sorry all for the 2-month posting hiatus. I've been travelling and job searching for much of the past 10 weeks, so decided to put the blog on hold for a bit. Posting shall now recommence, with planned contributions from other authors (if I can convince them).

To get started, I want to mention a new phenomenon that I've begun to see in my inbox. Every few weeks I get an email that looks like this:

Hi Jake,

My name is -----------, and I'm writing in regards to your site. I'm currently working with, and we recently published an article entitled "10 Most --------- ----------- Computer Labs In the Country". You can review the article here: (http://-------------/computer-labs-in-the-country/)

If you find that this resource would be of interest to your audience, please feel free to share it with them at your discretion.

Either way, I'm glad to have come across your blog. If there's anything else on our site that interests you, please feel free to let me know. Thanks again for the great content!

----------- -----

So I'm both amazed and a bit confused. The site in question advertises online colleges and universities, and it's clearly (mildly) spammy. I've dashed out all of the identifying information in order to avoid giving them what they are fishing for, which is a boost in their search engine rank. (If you'd like to check out the site, it's http colon slash slash onlinecolleges dot net)

This is quite plainly the practice of Search Engine Optimization (SEO), known to be a rising trend on the web. The New York Times recently published two articles about SEO, and Slate also put out an entertaining piece. The main idea is that web traffic is driven by search queries, and for a large number of search terms it's relatively easy to game the ranking algorithm of Google, Yahoo, etc. -- as long as one can manage to get several links to the desired site embedded in enough relevant pages scattered around the web. So let us observe: the blog you are currently reading contains the word "computer", "online" and "learning" many many times -- hence it's a prime target boosting one's ranking for queries such as "online university".

So what about this surprises me? Up until now I was under the impression that SEO manifests itself like all other spam we see. A company hoping to boost the ranking of site X pays an SEO firm to spread links to X around the web via content farms, unmoderated blog comments, etc. -- but all of these practices can be done via scripting and web crawling, i.e. it's all automated. I've actually done some work on detecting such "web spam", i.e. hosts and pages generated solely for the purpose of optimizing search rankings. When this research was published nearly 4 years ago I and my coauthors emphasized a key observation: "good" sites rarely link to "spammy" sites. This makes detecting spammy sites relatively easy, since they don't get a lot in-links from trusted sites. That is, sites where SEO companies can easily embed links can be quickly discovered and tagged as untrustworthy sources for ranking.

But what we have here is different. It appears that an actual individual, a human being, looked at this blog, determined my email address and emailed me to request a link. The hope here, of course, is that I'd be so flattered by this personal request that I might actually advertise their site. This suggests a rather different economic model from the usual approach to spam -- indeed, should we even think of this as spam? It also hints at the value of an endorsement from a trusted site, as the individual who contacted me had to invest nontrivial human time to find the blog and make the request, with a likely low chance of having me bite. Another email I received 2 months ago was even more explicit:
One more thing, are you interested in receiving a payment for adding a link on one of your pages? The link will go to an education site.

My budget is $100 and can pay via PayPal. Let me know if you are interested and I can follow up with an email or a phone call.

(I ignored the request, although for $250 I might have considered it...)

This new approach, actually paying for endorsements, will likely prove very challenging for search engines in the future. Classifying spammy links is easy when they are all coming from a small set of nefarious pages/hosts. It looks like SEO has gotten quite a bit more sophisticated, however, as it now manages to get links and endorsements embedded in otherwise genuine content.

Do any of you out there have similar stories about link requests?