Why keyword research still matters

There is a persistent idea floating around that keyword research is a relic of the "ten blue links" era. That Google has gotten smart enough to figure things out on its own. That you should just write good content and the rankings will follow.

This is half true and entirely dangerous.

Google processes 8.5 billion searches daily, roughly 99,000 every second. About 15% of those queries have never been searched before. The search engine has gotten better at understanding intent and matching it to content, yes. But it still relies on the words people type. And if you do not know what those words are, you are guessing.

Here is the part that makes keyword research more relevant than ever: 70% of all search traffic comes from long-tail keywords, yet only 0.0008% of keywords exceed 100,000 monthly searches. The gap between what most people assume "the keywords" are and what actually drives traffic is enormous. Without research, you will target the obvious head terms your competitors already own and miss the long-tail queries where your content could rank within weeks.

The SEO landscape in 2026 adds another layer. Google's AI Overviews now appear in roughly 13 to 30 percent of queries, and they reduce organic click-through rates for the top position by up to 58%. Not every keyword is equally affected, though. Local queries, transactional searches, and complex subjective questions still drive strong organic traffic. Keyword research is how you tell the difference.

The 8-step keyword research workflow

The workflow below synthesizes the approaches used by practitioners at agencies like Siege Media, Ahrefs, and Backlinko into a repeatable system. It accounts for AI-era dynamics that most published guides still ignore.

Step 1: Start with audience research, not tools

Before you open Ahrefs or any other tool, map your target audience's pain points, the language they use, and where they search. Talk to your sales team. Read customer support tickets. Browse Reddit threads and Quora questions in your niche. The goal is to build a mental model of the person behind the search query.

Categorize future keywords by where they fall in the buyer journey: awareness (they know they have a problem), consideration (they are evaluating solutions), and decision (they are ready to buy). This classification will drive content format decisions later.

Step 2: Generate seed keywords

Brainstorm 3 to 5 core topic pillars related to your business. Then expand each pillar using keyword frameworks:

• "Best CRM software" or "Best running shoes reviews"
• "HubSpot vs Salesforce" or "Mailchimp alternatives"
• "How to fix a leaky faucet" or "Email marketing guide"
• "Project management tools for remote teams"
• "Ahrefs pricing" or "SEO audit cost"

Source seeds from Google Keyword Planner, competitor websites, customer questions, Google Autocomplete suggestions, YouTube search suggestions, and industry forums. Ten to fifteen solid seeds per pillar is enough to fuel the next step.

Step 3: Expand and discover with tools

Pull keywords from multiple sources. No single tool provides complete data. Ahrefs maintains a database of 28.7 billion keywords across 200+ countries. Google Keyword Planner provides the only data sourced directly from Google. Each tool has blind spots, so cross-referencing gives you a more complete picture.

Pay special attention to "Also Rank For" and "Also Talk About" reports in Ahrefs, as these reveal the semantic landscape around your seed terms. These reports often surface keywords you would never think of on your own.

Step 4: Analyze and prioritize

For each keyword, evaluate five factors: search volume, keyword difficulty relative to your domain rating, traffic potential (which matters more than raw volume), CPC as a commercial intent signal, and whether your planned content matches what actually ranks in the SERP.

For newer sites with a Domain Rating under 30, target keywords with KD below 20. Sites in the DR 30 to 50 range can compete for KD 20 to 40. Established sites above DR 50 can aim higher. These are guidelines, not rules. A comprehensive piece on a low-competition topic can outperform what the difficulty score suggests.

Step 5: Validate against the SERP

This step separates effective keyword research from the mechanical kind. For every priority keyword, Google it in an incognito window and examine the actual results:

• Do AI Overviews appear? If yes, organic CTR will be significantly lower.
• What content types rank? Blog posts, product pages, tools, or videos?
• What is the Domain Rating of the top 5 results?
• Which SERP features are present? Featured snippets, PAA boxes, shopping results?

If the SERP is dominated by content types you cannot or will not create, move on. Google is telling you what it wants for that query. Arguing with the SERP is a losing strategy.

Step 6: Cluster and map to topics

Group related keywords into topic clusters built around pillar pages. The simplest method: if two keywords show more than 60% of the same URLs in their top 10 results, they belong on the same page. If they share less than 30%, they need separate pages. Assign one primary keyword plus 3 to 5 supporting keywords per page.

Ahrefs makes this faster with its Parent Topic feature. For each keyword, it identifies the #1 ranking page and finds the keyword sending it the most traffic. All keywords sharing the same Parent Topic get clustered together. In Keywords Explorer, click "Clusters by Parent Topic" for instant grouping of up to 10,000 keywords.

Step 7: Create content briefs

Every brief bridging keyword research and content creation should include: primary keyword with volume and difficulty, 3 to 5 secondary keywords, suggested H2/H3 headings from SERP analysis, target word count based on competitor analysis, meta title under 60 characters, meta description between 120 and 160 characters, internal linking targets, and the unique angle your piece brings to the topic.

Step 8: Monitor and refresh quarterly

Keyword research is not a one-time project. Set quarterly reviews at minimum. Monitor AI Overview expansion into your vertical, track SERP feature changes for your core keywords, and feed new high-impression queries from Google Search Console back into your clustering workflow. Keywords that did not trigger AI Overviews six months ago may now have them.

Ahrefs Content Gap analysis

Content Gap analysis is where you find keywords your competitors rank for that you do not. This is one of the highest-ROI activities in keyword research because you are targeting terms that are already proven to drive traffic in your niche.

The process: navigate to the Competitive Analysis tool in Ahrefs, set it to "keywords," enter your domain, add 1 to 3 competitor domains, and click "Show keyword opportunities." Toggle "Main positions only" to filter out image packs and sitelinks.

Then refine the results. Exclude competitor brand names. Set minimum volume to 20+. Set maximum KD to 30 for quick wins. Require at least 2 competitors ranking in the top 10 for stronger signal. For each resulting keyword cluster, check whether you have existing content using a site: search. If yes, expand that content to cover the missing subtopics. If no, plan new content.

Run content gap analysis quarterly or before every major content push. The backlink profiles of competing pages also surface link building opportunities you can pursue simultaneously.

Google Search Console workflows

GSC provides ground-truth data that no third-party tool can replicate. It shows actual Google impressions and clicks, not estimates. Here are the four workflows that produce the most actionable results.

High impressions, low CTR (quick wins)

In Performance, enable all four metrics and set the date range to the last 3 months. Sort Queries by impressions, highest first, and look for queries with high impressions but low CTR. These indicate unoptimized title tags, weak meta descriptions, or poor intent match. The fix is usually rewriting your SERP listing to be more compelling. A title tag change can lift CTR by 20 to 30 percent with zero content changes.

Striking distance keywords (positions 8 to 20)

Filter Position greater than 7 and less than 21, then sort by impressions. These keywords are on the edge of page one. Small optimizations like adding 2 to 3 internal links from high-authority pages, expanding thin content sections, or updating with fresh statistics can push them into the top 5.

Question keyword discovery

In the Queries tab, use the regex filter ^(who|what|where|when|why|how|can|does|is|are|do|will|should) to surface all question-based queries. These are ideal for FAQ sections, H2/H3 subheadings, and featured snippet targeting. Questions also perform well in Google's People Also Ask boxes.

Cannibalization detection

Select a keyword and check which URLs receive impressions for it. If multiple pages compete for the same query and intent, they are splitting your ranking signals. Consolidate with 301 redirects or differentiate the intent each page serves.

Free tools that deliver real value

You do not need a $99/month subscription to start keyword research. These free tools cover most of what smaller teams and solo practitioners need.

Google Keyword Planner gives you the only volume data sourced directly from Google, though it shows ranges instead of exact numbers for non-advertisers. Enter 1 to 3 seeds or paste a competitor URL to extract their keyword themes. The "Low" competition label refers to ad competition, not SEO difficulty, but the two often correlate.

Google Trends is indispensable for seasonal analysis. Use 5-year timeframes to spot recurring patterns and 12-month views for emerging trends. The "Rising" related queries marked "Breakout" (5,000%+ growth) surface opportunities before they show up in keyword tools. Always publish seasonal content 2 to 3 months before the peak.

AnswerThePublic generates branching question trees from Google's People Also Ask data across Who/What/Where/When/Why/How categories, plus prepositions and comparisons. Limited to 3 free searches daily, but each search reveals dozens of content ideas organized by intent type.

AlsoAsked scrapes real-time PAA data and generates multi-level mind maps showing how questions connect to each other. First-level questions branch into second and third levels, revealing the complete topical landscape Google recognizes for any subject.

Keyword Surfer is a free Chrome extension that overlays volume estimates, CPC, related keywords, and estimated domain traffic directly on Google search results. Zero friction because it works during normal browsing. It does not measure keyword difficulty, so pair it with a paid tool for competition analysis.

Search intent classification

Google now prioritizes user intent over exact keyword matches. If your content does not match the intent behind a query, it will not rank, no matter how well optimized it is. Intent classification has become the most important step in the entire keyword research process.

Commercial investigation keywords are often the most valuable in conversion-focused SEO because users are close to purchasing but still open to influence. These keywords convert at significantly higher rates than informational queries while facing less competition than pure transactional terms.

For any keyword you are serious about, use the 3 Cs framework: check the Content Type that dominates the SERP (blog posts vs. product pages vs. videos), the Content Format (listicles vs. how-to guides vs. landing pages), and the Content Angle (what hooks the top results use). If your planned content does not match what Google already rewards for that query, you need to rethink your approach or pick a different keyword.

Keyword clustering and topic mapping

Google's core updates over the past three years have consistently rewarded sites that cover subjects thoroughly rather than targeting individual keywords in isolation. Organized content clusters drive roughly 30% more organic traffic and hold rankings 2.5x longer than standalone pieces. Clustering also prevents keyword cannibalization, the problem where multiple pages on your site compete for the same query and end up splitting ranking signals.

There are three practical approaches, ranked by accuracy:

SERP-based clustering is the most reliable. It groups keywords based on actual search result overlap. If Google ranks the same URLs for two keywords, those keywords share intent and belong together. The standard threshold: keywords sharing 30% or more of URLs in their top 10 results belong on the same page. Tools like Keyword Insights, SE Ranking, and Keyword Cupid perform this analysis automatically.

Ahrefs Parent Topic method is the fastest. For each keyword, Ahrefs identifies the #1 ranking page and finds the keyword sending it the most traffic. That becomes the Parent Topic. All keywords sharing the same Parent Topic get clustered together. It is less accurate than full SERP-based clustering because it only considers the top result, but the speed tradeoff works well for initial exploration.

Manual clustering gives you maximum nuance but does not scale. Google each keyword in incognito, compare the SERPs, and group those with overlapping results. Practical up to about 200 keywords before it becomes tedious.

Translate your clusters into a hub-and-spoke site architecture. A pillar page covers the core topic comprehensively (typically 2,500+ words) and links to all cluster pages. Each cluster page goes deep on a specific subtopic, linking back to the pillar and to related cluster pages. Keep important cluster pages within 2 to 3 clicks of the homepage.

How AI Overviews are changing strategy

This is the section most keyword research guides still get wrong, either by ignoring AI Overviews entirely or by panicking about them. The reality is more nuanced and more actionable than either extreme.

AI Overviews now reach 2 billion monthly users worldwide and appear in roughly 13 to 30 percent of queries, depending on the study. Google has been calibrating aggressively, pulling coverage back from ~25% to ~16% and then expanding again. The impact on organic CTR is real: Ahrefs found that AI Overviews reduce the CTR for position #1 by up to 58% as of late 2025. Seer Interactive's analysis across 3,119 queries found organic CTR for queries with AI Overviews dropped from 1.76% to 0.61%.

But here is the nuance. About 84% of AI Overviews appear for informational queries. Local queries are virtually untouched, at just 0.01%. Transactional and branded searches remain largely unaffected. This creates a clear strategic hierarchy for keyword selection.

Five practical adaptations for your keyword research:

1. Prioritize query types less likely to trigger AIOs. Local-intent keywords, transactional queries, branded searches, and complex subjective questions requiring personal experience are your safest bets for traditional organic traffic.
2. Optimize to be cited within AI Overviews. 92.36% of AIO citations come from top-10 organic results. Structure content with clear summaries, descriptive headings, lists, tables, and statistics with attribution. 85% of cited pages were published in the last two years, so freshness matters.
3. Track citation visibility alongside rankings. Being cited in an AI Overview can drive 35% more organic clicks than ranking without a citation. Add "AIO present" as a column in your keyword tracking spreadsheet.
4. Invest in experience-based content. Original research, first-person case studies, and authentic user experiences are hard for AI to replicate and increasingly valued by Google's E-E-A-T framework.
5. Monitor continuously. AIO visibility is not static. Keywords triggering overviews today may stop triggering them next quarter, and vice versa.

Meanwhile, ChatGPT hit 800 million weekly active users in early 2025, and 24% of Americans now reach for it before Google. The citation patterns differ dramatically from traditional rankings: 89% of ChatGPT citations come from URLs ranking position 21+ in Google. This means your keyword strategy increasingly needs to account for multiple discovery surfaces, not just Google's page one.

The metrics that drive decisions

Search volume: useful but overrated

Most SEO tools derive their volume data from Google Keyword Planner, which uses roughly 80 logarithmically distributed values repeated across millions of keywords. It also groups close variants together, hiding long-tail variations. Ahrefs supplements this with clickstream data from millions of real users to provide more granular estimates. An AuthorityHacker study found Ahrefs had the best traffic estimation accuracy among six tools, with an average discrepancy of 22.5%.

The takeaway: volume data from any tool is an estimate, not a fact. Different tools can show 38.5%+ differences for the same keyword. Use volume as a directional signal, not a precise target.

Keyword difficulty: what the numbers mean

Ahrefs calculates KD (0 to 100) as a weighted average of referring domains to top-10 ranking pages, plotted on a logarithmic scale. KD 50 is genuinely hard, not "medium." It requires exponentially more backlinks than KD 25. When you hover over KD scores in Ahrefs, it shows estimated backlinks needed for page one.

A KD score is a pure backlink metric. It does not factor content quality, brand authority, topical relevance, or whether your content format matches the SERP. Treat it as one signal among many, not a go/no-go decision point.

Traffic potential vs. raw volume

Ahrefs' Traffic Potential metric shows the total organic traffic the #1 ranking page receives from all keywords it ranks for, not just the target keyword. A keyword with 500 monthly searches might have traffic potential of 7,800 because the top page ranks for thousands of related terms. The average #1 result ranks for over 1,000 keywords. This makes traffic potential a far more accurate predictor of actual website traffic than raw search volume.

CPC as a commercial intent proxy

CPC represents what advertisers pay per click for a keyword. Higher CPC correlates with higher commercial value and conversion likelihood. A keyword with $8 CPC carries significantly more business value than one with $0.50 CPC, even if the search volumes are similar. Ahrefs calculates organic traffic value by multiplying estimated organic clicks by CPC, showing what that traffic would cost if you bought it through paid advertising.

Common mistakes to avoid

Ignoring search intent. This is the most damaging mistake. A luxury hotel targeting "cheap hotels" drives traffic that will never convert. A B2B SaaS company ranking for an informational query with a product page gets high bounce rates and no signups. Fix: analyze the SERP for every target keyword and match your content type to what Google rewards.

Targeting keywords above your weight class. A new site pursuing "running shoes" (KD 90+) against Nike and Amazon has zero realistic chance. Build topical authority with lower-difficulty terms first, then work your way up. The quick-win approach: for every competitive keyword you want eventually, publish 5 pieces targeting easier related terms first.

Chasing volume while ignoring business value. A keyword with 50,000 monthly searches and a 0.1% conversion rate may be worth less than one with 500 searches and a 5% conversion rate. Evaluate traffic potential, business relevance, and conversion intent together.

Keyword cannibalization. Multiple pages targeting the same keyword and intent split your ranking signals. Moz themselves had three pages competing for "keyword cannibalization" with none ranking above position 8. Detect overlap in Ahrefs using the "Multiple URLs Only" filter in Organic Keywords, then consolidate with 301 redirects or differentiate each page's intent.

Treating keyword research as a one-time activity. AI Overviews appear and disappear unpredictably, competitor content changes, and seasonal patterns shift. Run full refreshes quarterly and check GSC monthly for new high-impression queries you are not targeting.

Not searching beyond Google. YouTube is the second-largest search engine. Reddit is the #1 most-cited domain across AI search platforms. TikTok is increasingly used as a search engine by younger users. Your keyword strategy needs to account for every platform where your audience searches, not just Google.

Templates and scoring frameworks

The keyword research spreadsheet

Your spreadsheet should include these columns at minimum: Keyword, Topic Cluster, Monthly Search Volume, Keyword Difficulty, CPC, Search Intent (I/C/T/N), SERP Features Present, Current Ranking Position, Target URL, Content Type Needed, Priority Score, Secondary Keywords, Funnel Stage, Traffic Potential, and Notes. Advanced teams add Trend Direction, Business Relevance Score, and Content Status.

Notable free templates worth downloading: Backlinko's multi-tab Google Sheets template (with seed brainstorming, source tabs, and evaluation criteria), HubSpot's color-coded template with intent mapping, and Asana's project management-oriented keyword research template.

The weighted composite scoring framework

Assign each keyword a priority score using this formula:

Priority Score = (Business Value x 0.40) + (Ease of Ranking x 0.35) + (Traffic Potential x 0.25)

Rate each factor on a 1 to 10 scale. New sites should weight "Ease of Ranking" highest. B2B companies with long sales cycles should weight "Business Value" highest. Ecommerce sites should weight "Traffic Potential" highest. Adjust the weights to match your situation.

The effort vs. impact matrix

Plot keywords on a 2x2 grid. High-impact and low-effort keywords are "Quick Wins" that you should tackle first. High-impact and high-effort terms are "Strategic Bets" that require careful planning and dedicated link building. Low-impact and low-effort keywords are "Fill Gaps" for when you have bandwidth. Low-impact and high-effort keywords should be avoided entirely.

The Keyword Golden Ratio for new sites

The KGR formula: (Number of Google results with the keyword in the title) divided by (Monthly Search Volume for keywords under 250 volume). A KGR of 0.25 or lower indicates strong quick-rank potential. This method works best as a supplementary tactic for new sites targeting long-tail keywords. It should not be your only strategy, but it can produce fast wins while you build authority for more competitive terms.

Author

Michael Timi

Partner & Marketing Manager, eMac Media

Drives strategic partnerships and revenue growth through high-impact marketing initiatives, business development, and lead generation.

Editor

Princess Pitts

Director of Communications Strategy, eMac Media

Specializes in editorial strategy, content governance, and brand communications at scale.