Canonical Domains, HTTPS, and Sundry Malarkey of a Geek Nature

We’re doing some work for The Yorkshire Linen Company. (normally I wouldn’t be so crass as to mention their name, but I have a special reason for doing so that I’ll get round to in a minute). Because we’re clever, we’ve found an unusual technical side-effect of their website set-up that is harming their Google rankings. It’s an illustration of unintended consequences but also a textbook case of measures you can take to protect your website revenues from the capriciousness of Google’s algorithms. It also shows why you need ongoing specialist support to capture and fix the kind of problems that frankly many web companies couldn’t even explain (hint hint!).

The problem started in an unexpected way: the taking of online orders. Normally, if you sign up to an online payment provider such as Protx or Worldpay the actual taking of credit card details happens on their server rather than yours.

If you’re watching carefully next time you’re buying something online, you might notice that the last steps of your purchase take place on https://www.protx.com, rather than http://www.mikesgreatshoes.com where you started out buying your patent leather fetish shoes. The reasons for this are boring and technical but crucially relate to the security of your transaction.

Increasingly, these payment providers are offering you the opportunity to carry out the payments on your own server. However, to do this you need a secure certificate. This sounds like a piece of paper, but is actually a digital certificate that demonstrates that your server scrambles up any data it holds in such a way that prevents anyone from hacking in and getting your credit card details. It also comes with a ’paper trail’ that means that any fraud that does occur can be traced back somewhere. When you see a website address beginning with HTTPS, that means it is on a secure server.

All boring and abstract enough, until this latter method was enacted on Yorkshire Linen. Rather unexpectedly, this gave Google the opportunity to index the pages of the site on both the HTTP and HTTPS version of the site. So the duvet covers were normally available at http://www.yorkshirelinen.com/show-products/2/, but Google was able to access the same page alive and well at https://www.yorkshirelinen.com/show-products/2/. It’s like finding an identical twin of the main website.

And this gives Google a problem: which version of the page to keep in its index?

For Whatever Reason (and I personally can’t think of anything reasonable) Google busily went through more or less the entire website, added the secure version of every page to its index and removed the normal URL. Whoops!

Now Google has what is known as a ’canonical’ domain for every website. We own various versions of our own domain, such as www.ninexb.com, www.firecrestonline.com and so on, but our canonical domain is 9xb.com. Occasionally, Google finds that it can index a page on one of the other domains we hold, and when that happens it loses all the power of our main domain. And when that happens, that page loses much of its ranking potential.

So all of a sudden, that mean that a few of the keywords the Yorkshire Linen company were ranking for have momentarily vanished. Great!

So what can you do about it?

Firstly, a large part of YLC’s online sales actually come from their extremely well optimised Google product feed. Thanks to this, sales remain bouyant despite this momentary glitch in their regular organic SEO performance. This means the Yorkshire Linen Company are protected to a degree from the vagaries of such technical matters.

Secondly, we’re letting Google know that the HTTPS version of the site is not canonical. Any visitor to any page on the site that shouldn’t be part of the secure ordering process will be silently and unobtrusively shuffled off to the regular version of that page on the regular domain name.

Thirdly, this redirect is also accompanied by a 301 response header. That’s geek talk for “we’re telling Google to remove the incorrect version of its page permanently from its index, and use the correct version instead.”

Finally, we’re writing this blog article. This includes this link - which actually goes to the incorrect version of the homepage at the HTTPS version of the address. Because our site is spidered pretty much daily, Google will find that link, follow it to the Yorkshire Linen site, discover that the secure version of the domain has been moved to the regular version and begin to re-index the site properly.

All of which will mean that inside a few days, the Yorkshire Linen Company will be back on track and the tills will be ringing as rapidly as they were before this problem ever happened.

Today’s Moral?

If your web development strategy doesn’t include specialist SEO knowledge and ongoing marketing review, this kind of problem can easily occur without there being an obvious remedy. Many digital agencies can only supply the knowledge necessary to develop a website without having the deep understanding of Google necessary to remedy or protect your website from these problems at all. If your website is your revenue stream, then saving a couple of grand on your development costs or marketing won’t look so clever if it all comes crashing to a halt and no-one can tell you why.