SEO Case Study: Our Migration to HTTPS

As you might have noticed, over a month ago Link-Assistant.Com's website has migrated from HTTP to HTTPS.

The migration was no piece of cake, I have to say, but all the effort has truly paid off — we went through the process with no harm to our rankings and organic traffic. And now, as all the dust has settled after the migration, I decided to sum up our experience in this case study, full of practical tips and advice.

Hopefully, it will help those of my fellow SEOs who aren't willing to start another "I migrated to https and positions dropped" thread on Inbound or Reddit. Yet, I want to warn you before you read further. This is not an ultimate-and-bullet-proof kind of recipe; things may get screwed up when you don't expect it. And I can only guarantee that it's worked for us, and is likely to work for you.

A short intro for those new to HTTPS:

HTTPS is a protocol for secure communication between a user and a website. It makes it harder to eavesdrop on the information that you send and receive over the Internet. HTTPS is a new standard meant by default these days, as every managed hosting provider ensures end-to-end encryption.

Should you bother if you have a website and it runs on HTTP? The short answer is "yes". And the longer version unfolds into three key points:

• HTTPS provides more security and thus better user experience for website visitors;
• Some SEOs believe that it benefits keyword rankings;
• Every now and then the guys at Google would invent a new trick to make you finally migrate.

"From the end of January with Chrome 56, Chrome will mark HTTP sites that collect passwords or credit cards as non-secure".

But is there a catch? You bet. Switching a website to HTTPS is a lot of work, and done wrong, it can easily become an SEO disaster.

If you regularly read through the SEO-related news feeds, you couldn't help noticing a bunch of sad stories about people losing keyword positions, traffic, business, home, and finally their families after they've migrated to HTTPS.

Well, is there a way to avoid the apocalypses? I have some good and bad news here for you:

1) First, you will inevitably lose some of your positions and traffic during the HTTPS migration;
2) Second, they will restore in 1-2 weeks, provided you follow all the migration best practices.

So let's see what these best practices are and how they helped us keep https://link-assistant.com rankings safe.

Migrating to HTTPS (the Link-Assistant.Com's way):

1. Determine the 'as is' state and make a plan
    1.1 Get under the hood and draft your website's structure
    1.2 Create a keyword ranking report
2. Get the team on board
3. To stage or not to stage?
4. The SEO part
    4.1 Before going live
    4.2 Social media metrics
    4.3 After going live
5. The Results

1. Determine the 'as is' state and make up a plan

1.1 Get under the hood and draft your website's structure

A good plan will always save you some time for a water cooler chat with colleagues. But when we're talking a project of http://link-assistant.com's size, I can say for sure — starting out without a proper plan is a failure guarantee.

Just imagine a website with tens of thousands of pages that's been live for over ten years, and that has undergone countless redesigns and structural changes during its existence!

Its core functionality is built around the Yii framework that handles the main content area and the News section. Besides that, we have pages that belong to the legacy website, also managed by Yii, and a legacy blog powered by Wordpress. All these are accompanied by a forum running on top of phpBB, and a customer support area powered by Kayako.

The list is quite impressive already, but we're not even halfway through. To make things worse, even though some areas of the website share the same CMS, they still have independent html templates (for example, the affiliate area). And as a result, each of them needs to be treated as an independent migration unit!

I could go on and on with all the snags I came across while planning this migration, but I'd better stop for security reasons ;) Instead, here's a fragment of the organizational chart I used for my migration.

The takeaway:

With HTTPS migration, planning is what can make it or break it. And even if you have a small 10-page website written in pure HTML, I do recommend you to get under the hood and draft its structure. While you do this, think about things that may stop working properly once you migrate to HTTPS. These may include:

• External scripts
• Forms
• Support chat
• Downloadables
• Payment gateway
• External APIs
• etc.

Pay attention to the technologies or CMSs underlying each area of your website. You need to know how to handle protocol-related settings for each of them. For example, if you have WordPress-based sections, note that you'd need to change Wordpress Address (URL) and Site Address (URL) in WordPress' general settings.

Aim at several outputs of your planning activity:

• A website structure chart that will help you visualize each and every structural element on your site;
• A list of tasks related to every CMS and technology;
• A checklist to use while testing your site after the migration.

1.2. Create a keyword ranking report

As a part of my preliminary activities, I also created a new project in Rank Tracker with all our target keywords that we normally promote and monitor, as well as the keywords that showed up in the Search Analytics report in Search Console.

I grouped the keywords by relevant website areas (using Rank Tracker's tags), and created a scheduled rank checking task, so that Rank Tracker would update our rankings daily:

Thanks to these preparations, by the time of migration I've already collected enough historic ranking data to benchmark my post-migration rankings against.

And more to that, thanks to the fact that all the keywords were grouped by website areas, I was able to monitor how this or that area of the website performed in search. Extremely convenient to instantly spot where the problem lies!

The takeaway:

1) Make sure to check rankings for your target keywords at least once before you switch to HTTPS. So that you're able to compare the rankings before and after the migration.

2) Assign every keyword to a specific area of your website, so that if some of your rankings suddenly drop, you'll know where to look for the problem.

3) Schedule ranking checks to run daily at least for two weeks after the migration, to be able to instantly react if something goes wrong.

2. Get the team on board

An HTTPS migration is obviously more of a technical task than an SEO one. And although HTML, GIT, NGINX, and PHP acronyms involved in the process don't make me feel perplexed, I'm persuaded that an SEO should stay away from coding.

Luckily, I managed to get a system administrator and a web developer on my team. The former was responsible for setting up the staging environment and migrating from the stage to our production server. And the latter was working in close collaboration with me — I was creating tasks and ensuring quality afterwards.

This let us streamline and speed up the process, since everyone was involved in the tasks they were good at.

The takeaway:

If you manage simpler projects or your website is running on a widespread CMS, chances are that you can handle the migration on your own.

But let's be honest — even the least experienced web developer is likely to be a better coder than the most knowledgeable SEO. So getting the right specialists on board is likely to save your personal and your company's time and minimize the potential damage.

3. To stage or not to stage

When planning an HTTPS migration you can choose one of these options to deploy the changes:

• Deploy to the production website directly. (dangerous)
• Deploy to the development environment and then roll out the changes to the production website. (safer)
• Deploy to the development environment, then roll out the changes to the staging environment, and finally merge the staging and the production environments. (the safest)

A staging environment is a mirror of the production environment. In other words, it's a mirror of your website. The staging environment is normally not accessible to search engines or users. It's only used to simulate the production website's behavior without interrupting its normal operation.

Taken the complexity of https://link-assistant.com, I decided to roll out a fully functional staging environment. It was a 100% mirror of the production website. And after the stage was ready, we stopped making any changes to the production website code (we froze it, in other words) until the migration date.

The staging server was configured to respond via HTTPS. All the changes related to the HTTPS migration were implemented and tested on the staging server first.

Looking back at the work done and the number of bugs we had to fix, I can say that this was the one and only way to handle the situation.

The takeaway:

Setting up a staging environment may be time- and money-consuming, but you'll be rewarded with smoother and stress-free experience during the migration.

4. The SEO part

4.1. Before the deployment to the production

Once the stage environment was ready we started making SEO-related changes.

a) Fixing the mixed content issues.

Mixed content occurs when you move a website to HTTPS, but some of its resources are still being loaded over HTTP (most often this will happen to your images, videos, CSS and JS files).

These insecure elements become loopholes that let attackers break into your otherwise "secured" website. So using a single insecure resource makes your whole page insecure. And, more to that, trying to protect their users from hacker attacks, many modern browsers prevent the insecure elements from loading (so your page will simply not load properly) or show "insecure content" warnings to your visitors (so your page may not load at all).

So, basically, my mixed content campaign boiled down to a simple mantra — "HTTPS everywhere":

• All src attributes of images and scripts must either point to absolute HTTPS URLs or to relative paths (in case of internal resources).
  <img src="https://example.com/image.png">
  <script src="/example.js">
• Iframes must be loaded via the HTTPS URLs or relative paths.
• All internal links must either point to absolute HTTPS URLs or relative path.
  <a href="https://example.com/">
  <a href="/index.html">
• Tags for Open Graph and Twitter Cards should point to HTTPS URLs
  <property="og:image" content=" https://example.com/image.png ">
  <name="twitter:image" content=" https://example.com/image.png ">
  <meta property="og:url" content=" https://example.com/">
• Your XML sitemap and RSS feed should also contain only HTTPS URLs.

The takeaway:

Checking all those issues for dozens of thousands of pages was a heck of a job. So my key takeaway was that I needed a tool to automate the whole process. And this is exactly how the idea came up to add a mixed HTTP/HTTPS content checker to our WebSite Auditor. Hope this tool will save you, guys, from what I've been through :)

b) Taking special care of the canonical tags.

Canonical tags are something your developers most likely have very little idea about (and neither did mine). So, before I knew where we were, he made all canonical tags point to the same pages where the tags were placed. And applied the change across the whole website:)

Fixing didn't take much time, but I surely was to be more specific with process tasking.

The takeaway:

Your web development team is not supposed to know everything about SEO. And it is your job to give clear instructions on the SEO-related stuff to the non-SEO team members.

c) Eliminating unnecessary redirect chains.

Another thing I had to do was to revise the redirects to make sure our HTTPS migration is not producing unnecessary redirect chains like this:

example.com 301 -> http://www.example.com 301 -> https://www.example.com

Excessive redirects slow down the website, and the increased load time, in its turn, impacts user experience and may have a negative influence on keyword rankings.

The takeaway:

With HTTPS migration, redirect chains emerge most unexpectedly. You've had page A redirected to page B five years ago, and have surely forgotten about that. Now, redirecting page B to its HTTPS version, you get 2 unnecessary redirects instead of 1. Not a big deal if it happens on one page. But what if it happens across the whole 200K-page website?

And since checking your redirects one by one will take you ages, I strongly recommend using tools like Screaming Frog and WebSite Auditor.

4.2. Social media counters

I was very determined to preserve our vanity metrics after the migration, yet this task turned out to be tricky.

In the News section of https://link-assistant.com, we have four social buttons. These are Facebook, G+, Twitter, and LinkedIn. More to that, there's a comments section, which is powered by Facebook social plugin.

G+ and LinkedIn turned out to be completely HTTPS friendly (their counters showed correct data both on HTTP and HTTPS versions of the same page), while Twitter and Facebook caused us some pain.

As you probably know, Twitter doesn't provide an official share counter anymore. At Link-Assistant.com, we rely on the NewShareCounts.com data. However, the counter treats the HTTP and HTTPS versions of the same page as different pages. With the default setup, the share counter would return zero for all the older posts.

After some research, we found out that the NewShareCounts.com plugin has a nifty data-url="" parameter that allows querying API for an arbitrary URL. So we tried adjusting the plugin code. For all the posts that had been published before the migration, the plugin returned share counts for the HTTP URLs, instead of the HTTPS versions where it was actually installed.

This trick worked well for about a week after the migration. Then the Twitter counter started to show zero on some pages. My guess was that Twitterbot started re-crawling the pages, which screwed up our sharing counts.

However, another (and final!) solution came up pretty fast. The property="og:url" should also point to the old HTTP URL. Voila! We cracked the Twitter counter.

If you want to preserve your Facebook share counts, you are supposed to follow this Facebook's instruction. Now, how do you like that? According to the instruction, you must let Facebook crawler access the old HTTP page after the migration. Why bother with HTTPS and security at all then?!

So, we had to think of a more elegant, effective, and secure solution. Which was to cache Facebook share counts and have them hardcoded into the old pages. Sure thing, this option has its downsides, but at least it keeps the website secure and gives users the idea of how popular the content was (and it hasn't caused any problems so far.)

The takeaway:

After you migrate to HTTPS, share counters on all the new pages you create will work fine, just the same as with HTTP.

However, trying to preserve the vanity metrics your pages earned while being served via HTTP is way more difficult. So, to my mind, if your older content doesn't have a lot of shares to boast about, it might be easier to simply remove the share counters from it altogether.

However if this option is not for you, I don't mind your making use of the tiny hacks I've outlined.

4.3. After the deployment

There're three rules that I follow when we're about to upload important website changes to the server:

1) Do not deploy important changes to the production server on Friday!

2) NEVER EVER DEPLOY important changes to the production server ON FRIDAY!

3) Make sure robots.txt doesn't block the content that should normally be available to search engines.

The first two rules are really important when you transfer to HTTPS because there's a bunch of things that need to be fixed immediately. And these things won't wait till Monday, or even till tomorrow.

Below are the tasks I performed after the migration.

1) I rebuilt my Website Auditor project and made sure that all the pages return 200 HTTP status code.

2) I got over the quality assurance checklist that I mentioned in the first part of the case study.

3) Once I made sure that the website functioned properly, I double-checked mixed content and redirect chain issues (as described above).

4) The next step was to create a new web property in Search Console for the HTTPS version of the website. I ported all the settings that I had for the HTTP version to the new property, like:

• Sitemaps;
• URL parameters;
• Disavow file.

Note that there's no need to submit a change of address for moving your site from HTTP to HTTPS. And you may add the new HTTPS version to the webmaster's tools of Bing, Yandex, etc. if you consider this necessary.

The takeaway:

The moment your new HTTPS website goes live is not the time for champagne splashes, but for careful testing and instant bug-fixing.

5. The results

I'm going to show you a couple of screenshots now that prove that migrating to HTTPS, when done right, doesn't have to bring any damage.

Let's get started with the pinnacle and Holy Grail of SEO — organic traffic, and traffic from Google. Organic traffic looks super boring. No changes at all, it's a totally monotonous curve:

Google organic traffic looks predictably boring as well, and it almost mirrors the overall organic traffic. It's not surprising though.

The following graph was taken from the Search Console account of the new HTTPS property. It shows the number of impressions. As you may notice, they completely restored within a month.

Over the first month, Google crawler was very hungry. Just take a look at the number of pages crawled per day:

Keyword rankings were more volatile than usual during the first week. But they stabilized eventually. Low search volume keywords were more unstable than high-volume short-tails. And during the transition time, both HTTP and HTTPS versions of the same pages were from time to time seen in the search results for various queries.

Over to you

So, this is how our HTTPS migration went through — without much hustle and surprise. Sorry if it wasn't melodramatic enough, with no disastrous traffic drops and magic recovery afterwards :)

Hope some of you will find it helpful. And please feel free to share your own stories in the comments (hopefully, with a couple of screenshots to illustrate your results!).

As always, I'm ready for your questions and rotten tomatoes :)

---