Is an HTTPS migration guide necessary?
As Dojono has it's own easy to use, fully AMP compliant, high speed CMS, we spend a lot of time migrating websites from other platforms such as Wordpress and Drupal to ours. Migrations attract a lot of planning. Our migration process is comprehensive. We have a machine process that is capable of scraping the original site, extracting images and other media, grabbing page content and meta data so that our content team has a very small amount of manual work to do to complete the migration. Migrations generally go somewhere between smoothly and very smoothly. It's all about plan, plan and plan some more.
Historically, as part of our standard Website Audit, we mechanically advise customers to upgrade to HTTPS if their site is not using it already. We have discussed the reasons for migrating to HTTPS previously, and have tended to leave customers to it - advising them to adhere to the Google's best practices advice on the matter... after all, it's a simple process, so what can go wrong?
Failing to carry this process out properly can cause serious and potentially lasting damage to your rankings. Issues can take one or more of the following forms;
- the entire site or specific pages can become inaccessible,
- the entire site or individual pages can be considered duplicate,
- internal redirects can occur,
- multiple internals redirects can occur,
- circular redirects can occur.
If you think that doesn't look too bad then you are mistaken. A migration between HTTP and HTTPS by Google's own admission will result in a slight drop in your traffic and rankings whilst their systems deal with the change, if you do it properly. Google says: "If you migrate your site from HTTP to HTTPS, Google treats this simply as a site move with URL changes. This can temporarily affect some of your traffic numbers. See the site move overview page to learn more". If you don't do it properly then you will damage your rankings, although how bad that damage is only doing it incorrectly can answer.
The positives are, that an orderly migration, with planning and testing can make this a trivial process.
Planning your migration
Planning is key! The amount of planning, however, depends on the size of your website and the technology you use. There are a few things to verify before you start a migration;
Just by way of a recap, the migration of a site from HTTP to HTTPS makes use of a process called redirection. In essence it tells a client (a web browser or search engine) that the URL they have visited has moved. This can take a variety of forms, broadly comprising two categories, temporary and permanent. Temporary redirects are designed to indicate that the URL being visited has moved, but the new location should not be stored (cached) for any extended duration as it may change again shortly - this is commonly known as a "Temporary redirect" or a "302 redirect", 302 being the HTTP response code assigned by the server to the response when the new location is returned. The next category is a "Permanent Redirect", also known as a "301 Redirect", again 301 being the HTTP response code assigned. This tells the client that the new location has changed permanently and will not change in the near future and that it may as well cache the new location for future reference, removing the need to check back very frequently.
During the migration process you need to establish how you have linked internal pages. There are 4 options;
- relative URLs. These are URLs that point to another page based on where this page is. The location of the linked (target) page may not be the same if linked to from other pages in the site. These URLs can safely be ignored for the purposes of this discussion, although should not be used as a general rule if at all possible. Examples of relative URLs are
- absolute URLs. Absolute URLs are URLs that remain consistent across the entire site. These are good and require no attention for the purposes of this discussion... or mostly any similar discussion and should be used in preference to any other sort of internal link if at all possible. Examples of an absolute URL might be
- protocol-less URLs. These are fully qualified URLs that do not contain the protocol. The protocol part of the URL is the bit that goes in front of the
:colon in the URL, so in most cases, and for the purposes of this discussion, is either
https. Protocol-less URLs usually look like
//smeloans.co.uk/how-it-works/. These are commonly used when the website creator or website creation software does not know if the site will run as HTTP or HTTPS and make use of a relatively modern browser functionality that substitutes missing information from the target URL from the current URL. Which means in this example if you were on
httpthen the browser would use
httpand after your migration to HTTPS the exact same logic will use
httpsinstead, removing the need to change anything in the webpage content in any way.
- fully qualified URLs. Fully qualified URLs generally contain a minimum of the protocol and domain name. If it is the case that your site uses fully qualified URLs then pay particular attention to the next section, as this is the main use case. Examples would be
/how-to-create-a-positive-work-environment/. *These URLs will need to be updated throughout your website**.
How does your website store URLs?
This is a bit of a tedious but relevant question, especially as many CMS employ very bad internal linking policies. For example, the Wordpress core is very clean. It makes a habit of using absolute URLs, making the technical side of the migration process as simple as installing your certificate and then telling Wordpress that you are now using HTTPS. The problem, in the case of Wordpress (and other CMS') starts with the use of non-core elements such as widgets, plug-ins and visual editors... and content creators not adhering to best practice. In these situations fully qualified URLs start to get injected into the pages in a variety of places as previously described. These are then stored in the database making them very hard to find and to fix. Additionally, as is the case with Wordpress and Joomla, configuration files are also commonly written to disk (as opposed to stored in the database) which increases the difficulty of the task.
How do I update my URLs?
If you find yourself in this lamentable pickle, and don't be embarrassed as it isn't your fault, then you have a few options;
- manually go through your entire site, updating URLs as you go. This is obviously a slow and error prone approach which does not scale well. If you have a site with a limited number of pages, media assets, etc then it could be an option.
- bone up on your SQL and
awkskills. For the technically capable among you this is a relatively trivial task to find all the files and database entries making reference to
http://your-site-name...and updating them to
https://your-site-name...or preferably, to an absolute URL as described above so that any future migrations and updates do not require a repetition of this process.
- use a migration tool. Many CMS' like Wordpress, Drupal and Joomla provide backup and restore, site clone and/or site migration tools that are specifically designed for this purpose. If you are using one of these CMS then we would recommend such a tool, although we don't recommend any specific tool. Bear in mind this process can take time to run so your website may experience downtime or you might need to migrate to another server in the process, with all the implications that entails.
Once we are happy that the technical migration of the site has been completed we need to make sure that we can test and measure properly.
Redirecting from HTTP to HTTPS
You will need to redirect all resource requests from HTTP to HTTPS. This step is frequently executed badly, often leading to a loss of traffic (direct, organic and other) which then results in indexation problems. At a high level, assuming your domain was smeloans.co.uk, your redirect rules should redirect;
Redirect rules should be
301 redirects, telling Google and others that the change is, well, permanent. Redirects should also maintain the path in the URL. For example,
http://www.smeloans.co.uk/about-us/ should go directly to
https://smeloans.co.uk/about-us/ and not to
https://smeloans.co.uk/... seems obvious we know, but a mistake like this will boost your home page through the roof for a short while before your entire website vanishes from SERPs. Redirects should not chain, for example, we don't want
http://www.ithinkfinance.com/contact-us/ going to
http://ithinkfinance.com/contact-us/ and then to
How you implement your redirects very much depends on the technology you use. It may be Wordpress which requires a configuration change, it might be Apache via the use of
mod_rewrite and a
.htaccess file or by configuring cloudFront, HAPRoxy or a myriad of other platforms and technologies.
Once you have implemented your redirect strategy then test and test and test and test some more.
Check your website by visiting the HTTP version. In our example,
http://multimonthloans.co.uk/. The first thing we want to check is that the URL in the address bar says
https://multimonthloans.co.uk/. If it does then great, first step successfully completed. The next check is, has the page drawn properly? If the page does not look right, maybe the styling is wrong or images and other resources are missing, indicating an internal linkage problem. The reason we suggest using Chrome is that Chrome will, by default, refuse to load resources from an HTTP address when referenced by a page served over HTTPS. To find out which particular assets are failing to load or are being blocked, access the Console via the Developer Tools interface, which can be accessed by right clicking on the page and selecting Inspect. The Console will list these and any other errors generated by the page.
Checking individual links
There are a lot of online tools such as redirect-checker.org and http://redirectcheck.com which will check your redirects are working. The main purpose of these tools is to both check that the redirect is working and to show you the detail around the process. The detailed information can be used to ensure there are no double redirects in place (as described in the second paragraph of the Redirecting from HTTP to HTTPS section). Testing what redirects are being generated for a specific URL will allow you to verify your work. You can use this process to spot check URLs that are important or might have caused you some issues in the past.
A more comprehensive check
For a fuller audit of your site you might consider using a tool like ScreamingFrog or Visual SEO. Although the unlicensed versions of this software often have a 500 page limit it is generally enough to gain some confidence that what you have done is correct.
See what Google sees
It is imperative that you use the testing tools provided by Google to ensure it can see your new pages. There are two key tools that we would strenuously urge you to use; the first is the Fetch as Google tool which is part of GSC which tells you if Google can access the page and if there are any issues that arise as a result. We also suggest, especially if your site is part of the Mobile First Indexing programme, that you use the mobile friendly tester. This will check your page and let you know if Google thinks it is mobile friendly or not and if not what issues it has detected. If these two tools are happy and render the page as expected then you can be confident that your internal resource linking is working.
Testing SERPs entries
Once you have managed to navigate all of these steps then the final test will be to make sure those valuable SERPs entries are all working. In order to test, locate your website using a search term you know is ranking or, alternatively, use the
site: advance Google search operator, such as
site:multimonthloans.co.uk which will list all the pages Google has indexed for that site. Check a few links and ensure they are arriving at the correct location, that the page is drawing properly and that the URL in the browser address bar starts with
https. This step is important as Google will still be holding your site's HTTP addresses in its database for a while and failure to redirect properly will result in a drop in organic visitors and ultimately that page's ranking.
Just to demonstrate this point a little further. In our example (right) we show a SERP entry for the phrase "No credit check loans", a keyword our example website MultiMonthLoans, is ranking for in p2. Notice that Google shows the URL (the green text next to the red arrow) as http://multimonthloans.co.uk/..., however, as the site has been upgraded to HTTPS clicking that result will go to the HTTP link before being redirected to the correct HTTPS link. This means that;
- people are getting to the HTTPS version of the site - so you are not losing any organic traffic,
- Google's crawler, when it visits your site next, is being redirected to the HTTPS version so it knows to start replacing the HTTP entries with the HTTPS entries,
- Google understands that the HTTP version is no longer valid so no duplicate content penalties will apply.
Google Search Console
Make sure you have GSC configured properly. There is often resistance to Google's advice about loading all the various flavours of your property onto GSC, if for no other reason than it often results in clutter that can be a little annoying at first. It is only once you have gone through a migration such as this that you gain an understanding as to why Google has issued this guidance. The purpose of adding all 4, yes 4! (HTTP, HTTPS, www and the naked domain (non-www)), properties to GSC (and we do this as soon as we launch or take possession of a client's site) is that once Google is aware of the various flavours it can record data for them and you can see, in the fullness of time, if there is activity for a property when there shouldn't be. By way of clarification, if you have both HTTP and HTTPS properties in GSC before you undergo a migration, then you should see activity move from one to the other, leaving the HTTP one empty in due course. If the migration has not been successful then you will continue to see traffic, indexed pages and other activity on the HTTP property which will allow you to identify omissions and correct them.
Access GSC and remove perfidious sitemaps before submitting the new one. Make sure that whatever mechanism you use to generate your sitemap (CMS' like Wordpress do this automatically) that it uses HTTPS and not HTTP as we don't want to send Google to the original HTTP site, just to be redirected to the HTTPS version.
Monitoring your progress
In addition to the usual external monitoring processes you have in place to check server availability, response times, page validity and the like, you are also advised to check the other related properties in GSC, which were discussed in the Google Search Console section above. Checking the Coverage and Performance reports for these properties, ensuring there is no new data, confirms that the redirect directives are working correctly. Any data appearing in these reports after the migration indicates that the process has not been completed properly and that there are still resources available on the HTTP address which Google is accessing. As previously discussed, this is undesirable and potentially damaging and should be addressed as a priority.
It is important to secure your website, even if you are not collecting sensitive information because failing to do so can still give third parties access to information about your visitors that you and they may not want. Google shares this view and rewards websites that take the security of its visitors seriously. This is furthermore demonstrated by the fact that Google's Chrome browser no longer shows a green padlock for secured websites but instead shows a warning for sites that are not secured. We expect Chrome to increase the severity of this notification over time, ultimately showing the same pop-up you might see if a certificate is invalid, expired or the site has violated a Safe Browsing rule. If you have not secured your site then we advise you review our guide, use our tools to check progress and convert your website as soon possible.