By Lucy Kirkness | October 2018
If you’re here I can assume that you understand the importance of keyword research, and why it is essential for achieving marketing success. Rather than going through detailed instructions, I will go through our methodology for our current keyword research process.
Your keyword file may look slightly different depending on which keyword research tool you primarily use. Whether you use Ahrefs, SEMRush, or another tool, you will want to have these basic columns in your file.
Ahrefs (our main tool of choice) has some extremely useful data, which you can use for your analysis, and include in your file. I will quickly summarise these metrics below.
As search volume is not indicative of true traffic potential of a keyword, we have to look at the potential clicks from this search volume. Some searches result in clicks on multiple results, and some might not result in any clicks at all. This is especially true when people find their answer in a featured snippet or people also answer box, without needing to click through for more information. Ahrefs ‘Clicks’ metric reflects the total number of clicks on the search results that people perform per month while searching for a keyword.
Clicks Per Search (CPS) shows how many different search results people click on average after performing a search on a keyword. Knowing the CPS of a keyword is crucial for understanding the traffic potential of pages that rank below the #1 result. The higher the CPS — the more chances that you’ll get some nice traffic even at a slightly lower position like 4 or 5.
Return Rate is a relative value that illustrates how often people search for this keyword again. It doesn’t show the exact number of ‘returns’ and is only useful when comparing keywords with each other. A return rate of 2, does not mean that people search for this keyword twice. It just means that they search for it quite often. The metric is there to give you a general idea that people search for a keyword with a return rate of 2.2 a bit more often than a keyword that has a return rate of 2.0, and 1.06, for example.
Parent topic determines if you can rank for your target keyword while targeting a more general topic on your page instead. To identify Parent topic Ahrefs take the #1 ranking page for your keyword and look for the best keyword that this page ranks for.
No matter how much search volume, or how high the keyword difficulty metric, the best way to determine the difficulty of ranking for a particular keyword is to analyse who ranks on the front page of search engines. It’s also important to determine how the front page looks, and whether it contains any featured snippets. The SERP Features metric shows you what the first SERP looks like for the target keyword, i.e. whether it contains Adwords top ads, Top stories, Related questions etc.
A good place to start is by rolling up your sleeves and trying to think of keyword ideas on your own. It’s typically easier to think of broader topics than long-tail keywords, but it’s good to begin with some brainstorming before digging into all of the research data. Try to think about other niche topics, product names, brand names etc and you should be able to come up with a good list.
I prefer to use Ahrefs for this process, although we do also use SEMRush. If you have an Ahrefs account, you can save a lot of time by using this tool. Start with your seed keyword and type it into the Keywords Explorer tab.
You will now be looking at the Ahrefs keyword dashboard, which will look like this.
In order to validate your ideas, you can navigate through the various reports in the left hand column of the dasboard:
If you’re doing keyword research for a blog or resource centre, the new ‘Questions’ report will be very useful. Simply export the reports and combine into one file. Copy and paste into the original keyword file you set up with our recommended columns.
Once you have your basic list of keywords you can continue to gather data. We use several approaches for expanding our seed list of keywords and collecting more data.
If you have an existing website, then you have a head start with your keyword research as there will be plenty of data on keywords you already have visibility for. The best way to find this data is by leveraging Google Analytics and Google Search Console.
The screenshot below is taken directly from Google Search Console.
The more established your site is, the more data you will have to play with, and the more potential keywords you will find.
You can use keyword modifiers and scraper tools to continue expanding your list of keywords. Keyword Shitter (!) is a free tool that scrapes many of the sources that people use every day to search, including Google, YouTube, eBay, etc.
If it makes sense for your website and content vertical, i.e. a property marketplace with multiple locations, recipes, or other topics, you could use a tool such as Merge Words to quickly expand you list.
SEMRush is one of the most valuable search marketing tools for competitor analysis, as it provides highly valuable organic and paid keyword and traffic data. The paid search keyword data in particular is very useful. You can see exactly what keywords your competitors are bidding on, as well as the costs, giving you a good indication of the potential profitability of each keyphrase. Similarly, you can use Ahrefs for competitor keyword research.
Using marketplaces such as Amazon and eBay are great for finding potential keywords with a high commercial intent, ideal for eCommerce businesses. Amazon is the biggest eCommerce store online and many studies have shown that people looking to buy something, start their search with Amazon. Let’s see what this would look like for our example.
Another marketplace which could prove to be very useful for product based sites is Pinterest. You’ll find some great ideas by looking at the ‘suggested search terms’.
We also want to find relevant keywords that are related to our primary topic. One of my favourite tools for discovering related questions or concepts is Answer the Public.
Once you have collected as many relevant keyword ideas as possible, it’s best to run all keywords through the same tool to collect keyword metric data. Whilst none of the tools are 100% accurate, by using the same tool you will be able to make accurate comparisons. We use Ahrefs, however you may want to use SEMRush (or their API), or Google Keyword Planner.
Now that you have a populated keyword file, it’s time to start cleaning it up. There are a few basic steps that we go through:
Depending on which tool you used, you may have a bunch of unnecessary columns, such as Ad groups, Currency, Number of results, Trends etc. At this stage, it’s best to remove any of the clutter. See the ‘prepare keyword file’ section for our recommended columns.
The first thing we want to do with our keyword ideas list is remove any duplicates we may have generated throughout the research phase. You can do this in Excel by simply selecting the data range, clicking on the ‘Data’ tab, and selecting ‘Remove Duplicates’ from the Data Tools group. Or, using Google Sheets and a ‘Remove Duplicates’ add-on.
When carrying out keyword research for an eCommerce business, we ideally want to be looking at a mature SERP, i.e. where there is demand and commercial intent. Therefore, we would recommend removing any keywords with a CPC less than ~£/$1. You will need to assess your keyword footprint to decide on the best baseline CPC (it may be more or less).
My go to tool for this is Term Explorer. Whilst not perfect, the tool allows us to collect some very important competitive data, including:
If you only have a small seed list, I’d recommend running a 'Bulk Keyword' job, where you can generate up to 1,000 keyword ideas using the Free Account. Or, from 10,000 to 90,000 keyword ideas on Paid Plans.
Otherwise, jump straight into using the 'Keyword Analyser' tool. Once your project has finished, simply click download CSV and enjoy all of this juicy data to add to your keyword file.
You may notice that search volumes are based on Google Adwords data. You can either use this data across the board, or do a quick VLOOKUP to pull in search volume data from your previous data collection, e.g. if using SEMRush or Ahrefs.
If you have access to SEMRush API, you can go ahead and import search volume data using this script:
The next step, and arguably the most important, is to group the keywords into ‘buckets’ according to their user intent. The 4 buckets I use are:
This is when a searcher is looking for a predetermined destination. Think of people who still type a domain name into the search box.
This probably covers the largest bucket of keywords, and is generally representative or users looking for a quick answer. These are people looking for a phone number, directions, 'how to' information, or even a piece of recent news.
This is where the searcher is looking to gain information to help them inform a buying decision, even if they do not convert; this is the gathering of information that has the potential to lead to a sale.
These are queries where the searcher is looking to make a purchase, find a place to make a purchase, or complete a task. These can range from queries looking to purchase online, look up the address of a store, to signing up for a service.
This process can be done manually, where you go through each keyword and tag them appropriately. Or, you can automate the whole process by using an Excel macro. If you don’t know any Visual Basics, or don’t particularly want to learn, then you can buy Nick Eubanks ready-made search intent macro-enabled Excel template. Instructions on how to use it here.
With a few tweaks to the tagging criteria, you can auto populate your keyword research spreadsheet with the searchers inferred intent. Now we can start understanding the semantics behind our list of keywords, which will aid our analysis in choosing our target keywords and mapping these to different content targets in the purchase cycle.
Or, you can use a tool called Sentinel, also built by Nick Eubanks and his team, who says:
Sentinel is a keyword intelligence tool me and my team created to help automatically take all the data you just gathered, automatically tag for intent based on your specific keyword vertical and the modifiers therein, and then automatically prioritize which keywords you should be going after based on rank potential.
Due to the nature of eCommerce websites, we are particularly interested in keywords which have commercial intent, or are transactional.
Once your keywords are grouped by intent, it is much easier to start mapping these keywords to your site pages. As a general rule of thumb, we use this strategy to map keywords to different page groups in your site structure.
Product pages - transactional intent keywords
Category pages - commercial investigation keywords
Blog posts - Informational keywords
Site info pages - navigational keywords
Now you have all the data you need to make directional decisions in terms of prioritizing your keyword lists and then mapping intent. The presentation of this data will also allow you to spot quick wins for optimisation. For example:
Look for pages ranking for keywords where the keyword is not being used in the meta title, domain, URL, or on-page content – these opportunities can usually be won by implementing basic on-page keyword targeting.
I hope you have found the article useful. Please do share your thoughts, as well as any tips you might have for improving the process.