Uncovering The New Behaviors Reshaping Keyword Research

We brought together a panel of search gurus to show us how they focus on user intent and the search behaviors that drive high-quality traffic to the right pages.

This hour-long webinar explored how to improve your keyword research and deliver better organic performance, with first-hand accounts of how search pros are finding new ways to drive stronger SEO performance.

Our host for the day was Ari Nahmani (AN), CEO and Founder of Kahena. On the panel, we were joined by Matthew Capala (MC), CEO and Co-Founder of Alphametic, and Gerald Murphy (GM), Senior SEO Business Manager at Similarweb.

Here, we recap the key topics, questions, and answers discussed, and share the what, why, and how behind modern keyword research, with practical examples drawn from decades of experience. And a cheeky bonus: Take away tips and tricks that will help you boost your SERP ranking, uncover keyword opportunities, and champion SEO as the hero of your digital marketing strategy.

Keyword research process

Q. What does your keyword research process look like right now?

MC: We use something called ‘The Pareto Principle of SEO.’ Vireo was an Italian economist who discovered that 80% of land and wealth in Italy was owned by 20% of the population. And this 80/20 rule (Pareto principle) is present in almost all walks of life, especially in digital marketing and SEO.

The first thing to identify is: what are the 20% of keywords that drive 80% of business value. You can use page search data to establish what non-branded keywords are converting. Second, we map them to where we rank to form a transactional keyword strategy, to drive business value. If we only focus on this 20%, we can make a big business impact in terms of revenue.

Next, look at which website pages produce the highest conversion rates. Then, you know the keywords mapped to these pages will (likely) produce the highest upside, whether that’s leads or revenue. We then head to the SERPS for these keywords to see what the top results look like.

Once you’ve discovered your ‘keyword winners’ (those 20% keywords), you should undertake some form of a user search behavioral study.

In a recent study by Brian Dean, around 1800 Google user behavior sessions were analyzed. Here’s what they found:

• 23% of people used auto-complete, something that’s extremely important to know.
• The average search session takes 76 seconds to complete
• Half of all search sessions are finished within 53 seconds.
• Within nine seconds, 50% of people clicked on the link
• 14 seconds was the average time it took from searching to clicking on the link

Search behavior has changed dramatically over the years; back in the day, you would have a very quick time span between searching and clicking on the blue links because that’s all you would see.

Sometimes, complexity can drive marketers to inertia. I always divide keywords and content into two intent buckets: transactional/commercial and informational/navigational. Once I identify a small group of keywords to focus on, I’ll run deep user studies. Here’s a quick example.

Let’s say you’re ranking 1-5 on a keyword (we call these ‘keyword winners’ taking 60%-ish of SERP clicks); your strategy may be more defensive as the goal is to maintain your ranking. Next, we look at our ‘keyword sleepers’ ranking in position 5-10. You may be getting a 2-3% click-through rate from positions 7-8, so, moving to position 2 can have a huge impact. Then, at a very strategic level, we develop what we like to call persona-driven SEO conversion funnels. This is an exercise where we start with a buyer persona, then map keywords to these buckets; we also map it to content and specific conversions. This is how we prioritize and organize our keyword strategy.

We use Rank Ranger Enterprise 360 software. I’m a Rank Ranger Ambassador. So, as someone who already likes and uses the product, I’m very excited about the integration with Similarweb. Because even if you’re not doing Google ads, you can look at what type of unbranded keywords your competitors are buying. This gives you a glimpse into the transactional keywords the category leaders are purchasing.

AN: Keyword research must begin with a process of discovery. If you already have a lot of data, whether that’s paid search data, conversion, or transaction-related data, it’s important to use these as indicators to suggest the sorts of search queries that result in hard business. I’ve heard of people starting out their keyword research simply using instinct alone to pick and choose keywords. But it’s about understanding the customer at a very core level, and you need to get that from multiple sources. And I’m not just talking about data, you need to have conversations with the sales, customer support, and marketing teams. Consider things like:

• What do we understand about the customer?
• What are the pain points?
• What are they trying to solve?

We use Google Keyword Planner as one of our primary sources, and we get real data because we run PPC campaigns. But, there’s a wealth of other data that we mash it up with, including Rank Ranger–a Similarweb company. Combining this data allows us to identify and contextualize high-volume keywords. It’s all about different data sources, including human ones.

GM: So, the thing that really underpins all of this is psychology. We should think about what people are actually going to do when they search for this term. And this takes us into the behavioral aspects of it, not just what the searcher is looking for. So, thinking about what somebody would go on to do after the search matters too. In its simplest form, keyword research has three stages.

First is the ‘why’ – which is the underlying psychology for the search. You might also need to consider pieces of branded content, to help someone better understand your product after they’ve made a purchase. Additional content can be created to support your backlink, off-page strategy, and social search.

Second is the ‘get,’ which is all of those data sources. This is where first-party and third-party data is highly valuable. So, if you’ve got a website and you’ve got a search bar in it, then you’ve got access to the onsite search of your branded website. And that’s really powerful. If I was working for a retailer now, that would be one of my go-to sources of information because I want to know and find out what products people are searching for on my website, that they know my brand already has an association with. This actually goes back into the ‘why’, and you can see how they interlinked with each other.

Finally, something Matthew mentioned earlier is the SERP features. But this is where we look at the optimization of that content.

Critically, one thing we should avoid is creating content for the sake of it. Do something different. Analyze who the top-ranking websites are, look at what they’re doing, and consider what they’re not doing too. Do your SWOT analysis, and consider what your content needs to look like based on that. At the end of the day, websites that rank high are essentially in the engine’s eyes for your particular location, and the best piece of content on the web for that particular term. You then want to take the time to create better, more relevant, more useful content for that particular term.

AN: It’s not just about the search volume, the competitiveness, or even the intent, but in understanding what the core strategy is, and making sure everybody is aligned.

Keyword strategy and tactics over time

AN: There have been a lot of changes in our ecosystem in the last few years. We have completely changed how we look at keyword research, at least when it comes to what we believe the predictive outcomes may be as a part of that strategy. For example, 5 or 10 years ago, we would add some seed words, get some data on the search volume, then categorize these into transactional, informational, or navigational. We may break it up by page type, or whether it could be used in a blog post or service page, or consider whether it’s a category or product target. Then, we’d optimize, and based on our ranking, we would assume a certain click-through rate; with these, we could model potential leads or revenue. That’s how we used to do things.

Today, we focus more on understanding the SERPs. The SERPs have changed so dramatically and continue to change. So, whether it’s paid shopping ads or even organic shopping listings going mobile-first. The SERPS look and feel are far from what they used to be. This impacts expectations around CTRs and more. So aside from analyzing the data, we seek to understand both the user intent and what Google understands the intent is. Now, if we can rank for specific terms, we ask; does that serve our strategy, and would we actually be able to achieve what we want?

Just one example of that is trying to rank for a high-volume, generic term. Even as SEOs, we get caught up in looking at the numbers. But if you actually look at the SERPs, Google is doing everything in its power to divert you or make you refine the search or click on an ad or something visual. So, these traditional blue links will have a much harder time ranking. Potentially, this results in us considering going after a long-tail strategy – which, as you know, comes with added complexities.

GM: When you look back, if you were asked this question 10 years ago, things like seed terms and individual keywords being exported into excel would have been commonplace. Whereas now, it’s about flipping that around and actually looking at trends, particularly at URLs and sub-folder level activities. We’re maturing in the sense of going at a higher level and not actually getting bogged down to the individual keyword phrase and seed keywords. But critically, we are now starting to challenge technology, which is why we here at Similarweb acquired Rank Ranger.

People need a range of different metrics to analyze – things like how many phrases or keywords does this particular page rank for? Which is something you get on the Similarweb platform – and it’s really to start helping plant those seeds. Then, you can drill down to view things like keyword difficulty, keyword intent, search volume, and all of those other vital data points. It’s about bringing everything together, really helping them think deeply about their website and performance, and considering what other topics they need to succeed.

AN: When you consider previously, we would just look at search volume and a few competitiveness metrics; it’s virtually impossible to scale that to look at every individual and then keep up with those changes over time. Using Rank Ranger, I can filter things in and out to see the competitive terms that matter and which are competing with search ads. So, ranking number one may be way down the page or many swipes on the mobile device. Whereas maybe some of these chunky, middle, or long-tail terms, even if they’re transactional, actually don’t have as many SERP features that we would need to compete with.

GM: SERP features are becoming real-time, and a subset of them adapt to real-world scenarios. So, in terms of how things have changed and how we work today, these kinds of technological advancements really make a difference. For instance, the ‘People Also Ask’ feature can change by the minute, as it changes and reacts to real-time data. Rank Ranger allows us to track the latest SERP features, and real-time access showing the scale of the next big SERP feature, combined with our zero-click data, is a match made in heaven.

MC: As a hands-on SEO practitioner for over 15 years, I’ve seen a massive change in how keyword research is done. I built my first website in 2006, and the type of strategies and metrics we used are still important, but they’re almost like a prerequisite. During COVID, I was studying and writing a book; and I predicted that modern SEO is really about understanding and predicting human psychology. Why? Because if you reverse engineer Google’s algorithm, they are trying to be more human; and they use artificial intelligence and machine learning to help them achieve this.

There was a study undertaken by Similarweb that analyzed 12 trillion keywords. Because of new SERP features, it took an average of 10 seconds for someone to click a link. For instance, a user would start with a very broad keyword, and then based on auto-complete and related questions, they start narrowing. This very behavior has brought us new SEO metrics to consider when doing keyword research.

As an industry, we have become fixated on rankings and traffic, but we really need to ask if we’re building trust and if we’re getting it right with our content. It’s important because these factors could lead to a drop in position on a given keyword.

AN: Fundamentally, we have to ensure content answers the intent of our user’s query. Are we surprising and delighting them? It’s not enough to just rank but to stay there and have a sustainable ranking, you need to make sure that content really fulfills the query, by answering and informing. Engagement of a page results in better rankings.

Key Takeouts

• When you think about keyword research, don’t think about the process as a one-off exercise but as an ongoing process.
• Identify what the 20% of keywords are that drive 80% of business value.
• It’s not just about data – it’s about talking to customers, sales, support, and marketing to find the pain points – then cross-referencing with your keyword data sources.
• Organize your keywords into the right groups, and understand what personas and landing pages are connected to each.
• Don’t create content for the sake of creating content. Do something that’s different, have a look and analyze the top-ranked websites, and look at what they are doing AND what they are not doing.
• Focus on building trust, and ensure your content answers the intent of a user’s query.

Importance of search intent

Q. How important do you rate user/search intent as a metric in your keyword research process, and what types of intent do you focus on when creating content for different stages of the funnel?

GM: Search intent is the backbone of keyword research, and it’s often called the buyer or consumer journey. For each important business unit, you should map a buyer’s journey that considers awareness, consideration, and decision. Think about segmenting your keywords into those three stages, which will start to reveal key insights into how many blog posts, for example, are ranked in each of those stages, how many of your competitor’s pages currently rank for them, etc.

Doing this then allows you to reverse engineer the situation. You can use Similarweb to analyze key URLs and see what other keywords they’re ranking for. To do this, you input a rival’s URL into the Ranking Distribution element of Similarweb Digital Marketing Intelligence to reveal what keywords they rank for.

AN: We’ve been looking at scraping or accessing non-traditional data sources and then piping them through tools like Rank Ranger. For example, if you’re in B2B and you have a platform like Gong for call recording, you can anonymize the transcripts, then build word clouds to help you better understand customer pain points. Reviews are another non-traditional data source that you can use for the same purpose. Look both at pain points and what it is people seem to know you for. For instance, if we’re a project management software company, and people are raving about using us to build Gantt charts, you need to consider if you have a page for that. So, instead of simply going after ‘project management software as a term, you go deeper by understanding what customers are actually looking for.

Onsite search is another great example of a non-traditional data source. You can use it to see what people are searching for and help them find it more logically – this could be a change to the navigation or the creation of an entirely new page.

Aggregating these terms facilitates better customer connections by serving content that matches their needs.

Scaling intent categories

Q. How do we scale, categorize, or do better in modern SEO concerning scaling the intent categories of our keyword research?

MC: According to WordStream, 50% of search queries contain four or more words. So, as we consider assigning intent to different keywords, and you talk about long-tail keywords, there’s a big play in the long-tail game. Google predicts a search query to better understand it; because around 15% of all the keywords entered into Google are never seen before terms. So, the freshness of keywords in your research is important.

Here’s a basic example of this in practice. If you watch Netflix, people always search for ‘best Netflix shows.’ If you had a page that ranked, to keep it in position, you must update it with the latest titles, for example, Squid Games. This is like a Wikipedia approach to keyword research, where if you are targeting upper funnel keywords, you need to ensure that you explore the topic thoroughly and you have all the freshest terms in there.

We incorporate social insights into our keyword research, Twitter mining is key and works really well. Quite often, when it comes to these never seen before keywords, search behavior actually initiates from social. If an influencer is talking about a product or using a certain term, all of a sudden, people go to Google and start searching for it. So, you can mine data from Twitter, then use Wikipedia for associations once you’ve identified your root keyword. Then, you go down into related topics and start building your content strategy to rank for more related keywords. There is a link between social and SEO that needs to be explored regarding trending topics.

AN: The idea of trending and freshness is super important. Whether you’re an SEO or a marketer, your ability to inform and surprise stakeholders by saying, ‘it’s not just important for SEO, it’s important for the business is a game changer, as it can inform real-life product and business decisions.

A study of Gen Zers between 18 and 24 showed that 40% prefer to use TikTok and Instagram or TikTok and YouTube as primary search engines. This idea of social networks influencing Google search, breakout terms, and trending terms is really interesting.

GM: Similarweb’s Keyword Generator Tool takes a base phrase or seed word, and generates lists of keyword suggestions; sourced across Google, YouTube, and Amazon. You can use a range of metrics and insights to assess and prioritize this list.

With it, you can uncover phrase-match keywords question queries, trending keywords, and related keywords. Helping you generate highly-relevant keywords, at scale to inform your strategy, save time, and give you rapid access to the intel you need to succeed.

Key Takeouts

• Search intent is the backbone of keyword research.
• Segment your keywords into three stages: awareness, consideration, and decision.
• Use non-traditional data sources for inspiration – call transcripts, reviews, etc.
• Facilitate better connections with customers by serving up content that matches their needs.
• Incorporate social insights into your keyword research.
• Trending and freshness are super important.

Impact of search features

Q. How do SERP features affect keyword research and implementation strategies?

MC: We work with a lot of beauty and ecommerce brands. In the US, when you search for a keyword like ‘makeup’, you’ll see Sephora as number one. They are the market leader in terms of an ecommerce beauty site. And right below them on the SERP, you have ‘people also ask.’ So, if you are a smaller beauty site and you’re trying to get to the top 10, you should create a page and answer one of these specific questions. Do it well, and you can appear on a very expensive, very competitive transactional commercial intent keyword within the ‘people also ask’ section.

Cross-engine keyword research is a must. There’s Bing and Yahoo. But there are other places you should be looking at too. Google Trends, Quora, YouTube, Instagram, and TikTok are all great places to research a topic, see what types of keywords and questions people are asking, and what the influencers are talking about. For ecommerce, Amazon is a huge source of data and a great place to mine transactional keywords from. Within Similarweb, you have Shopper Intelligence, where you can see a collection of insights from various sources. All of these places can feed into your commercial intent keyword strategy.

AN: In summary, use multiple data sources, categorize them appropriately, understand the customer intent, and the psychology, then build a strategy around that. Also, consider trends and freshness. Most of the searches on other platforms will eventually make their way back to traditional search.

Key Takeouts

• Use SERP features to inform your keyword and content strategy.
• Cross-engine keyword research is a must.
• Some SERP features update in real-time – stay alert to changes to stay relevant.

Wrapping up

Outdated methods and tools hamper your progress. Without access to fresh data that’s powered by true online behaviors, your keyword strategy will stagnate. To achieve success, you need a strategy and tools that allow you to adapt, track, and optimize at a moment’s notice.

Similarweb gives you actionable marketing intelligence insights, inspired by actual human intent and actions. Making keyword research effortless, timely, and always on-point.

by Liz March

Digital Research Specialist

Liz March has 15 years of experience in content creation. She enjoys the outdoors, F1, and reading, and is pursuing a BSc in Environmental Science.