AhrefsBot: IP Addresses, How to Block It, and Complete Guide

AhrefsBot is the web crawler that powers Ahrefs — one of the most widely used SEO toolsets. If you have seen “AhrefsBot” in your server logs, analytics, or firewall alerts, this guide covers everything you need to know: what it does, its IP addresses, how to verify it, and how to block it if you choose to.

What Is AhrefsBot?

AhrefsBot is Ahrefs’ primary web crawler. It continuously scans the internet and collects data that powers Ahrefs’ SEO tools, including Site Explorer, Keywords Explorer, Site Audit, Content Explorer, and Rank Tracker.

The crawler visits your pages, reads their content and link structure, and stores this information in Ahrefs’ index — one of the largest databases of live backlinks, with over 14 trillion known links.

AhrefsBot also feeds data to Yep.com, Ahrefs’ own search engine. So even if you do not use Ahrefs yourself, the bot may still visit your site to index it for Yep search results.

What Data Does AhrefsBot Collect?

URLs and internal link structures
Page titles, headings, and on-page content
Anchor texts from both internal and external links
HTTP status codes and redirect chains
Meta tags (robots, canonical, hreflang)
Response times and page load metrics

This data appears in Ahrefs tools when users look up your domain. If you use Ahrefs to monitor your own backlinks, your site’s data accuracy depends on how well AhrefsBot can crawl it.

AhrefsBot User-Agent String

AhrefsBot identifies itself in HTTP requests with this user-agent string:

Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)

The version number may change over time. The consistent identifier is AhrefsBot — this is the string you use in robots.txt rules and server configurations.

There is also a related crawler called AhrefsSiteAudit, which is used specifically when Ahrefs users run Site Audit on their own websites. This is a separate bot with its own user-agent.

AhrefsBot IP Addresses

Ahrefs publishes the IP address ranges used by AhrefsBot. If you need to verify that traffic is genuinely from AhrefsBot (and not a spoofed user-agent), check the source IP against these ranges.

Known AhrefsBot IP Ranges

AhrefsBot crawls from IP addresses in several ranges. The primary ranges include:

54.36.148.0/24
54.36.149.0/24
54.36.150.0/24
195.154.122.0/24
195.154.123.0/24
195.154.126.0/24
195.154.127.0/24

Ahrefs also maintains additional IP ranges that may change over time. For a real-time lookup, you can use our Ahrefs Bot IP Address Checker to verify any IP against the current published ranges.

How to Verify Real AhrefsBot Traffic

Fake bot traffic sometimes uses the AhrefsBot user-agent string to disguise itself. To verify that traffic is genuinely from Ahrefs:

Check the IP address against the published ranges above
Run a reverse DNS lookup: The hostname should resolve to a domain ending in .ahrefs.com or .ahrefs.net
Run a forward DNS lookup: The hostname from step 2 should resolve back to the same IP address

Example verification using command line:

# Step 1: Reverse DNS
host 54.36.148.1
# Should return: 1.148.36.54.in-addr.arpa domain name pointer crawl-54-36-148-1.ahrefs.com

# Step 2: Forward DNS
host crawl-54-36-148-1.ahrefs.com
# Should return: crawl-54-36-148-1.ahrefs.com has address 54.36.148.1

If the reverse and forward DNS lookups do not match, or the hostname does not end in ahrefs.com, the traffic is spoofed. Block it at the firewall level.

How to Block AhrefsBot

If you do not use Ahrefs and prefer that the bot does not crawl your site, you have several options. Choose the method that fits your server setup.

Method 1: robots.txt (Recommended)

The simplest and most reliable method. AhrefsBot respects robots.txt directives.

User-agent: AhrefsBot
Disallow: /

Add these lines to your robots.txt file. AhrefsBot will stop crawling your site within a few days once it reads the updated file.

To block AhrefsBot from specific directories only:

User-agent: AhrefsBot
Disallow: /admin/
Disallow: /private/
Allow: /

Need help generating a robots.txt file? Use our free robots.txt generator.

Method 2: .htaccess (Apache)

For Apache web servers, you can block AhrefsBot at the server level:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} AhrefsBot [NC]
RewriteRule .* - [F,L]

This returns a 403 Forbidden response to any request from AhrefsBot. Unlike robots.txt, this actively blocks the connection rather than relying on the bot to respect a directive.

Method 3: Nginx

For Nginx servers, add this to your server block:

if ($http_user_agent ~* "AhrefsBot") {
    return 403;
}

Method 4: Cloudflare Firewall Rules

If your site uses Cloudflare, you can block AhrefsBot through the dashboard:

Go to Security → WAF → Custom Rules
Create a new rule
Set the condition: (http.user_agent contains "AhrefsBot")
Set the action: Block

Method 5: Block by IP Range

If you want to block at the network level regardless of user-agent (for example, if you see spoofed traffic), you can block the AhrefsBot IP ranges in your firewall:

# iptables example
iptables -A INPUT -s 54.36.148.0/24 -j DROP
iptables -A INPUT -s 54.36.149.0/24 -j DROP
iptables -A INPUT -s 54.36.150.0/24 -j DROP

How to Slow Down AhrefsBot

If you do not want to block AhrefsBot completely but want to reduce how many resources it uses, you can add a Crawl-delay directive:

User-agent: AhrefsBot
Crawl-delay: 10

This tells AhrefsBot to wait 10 seconds between requests. Increase the number for less frequent crawling. AhrefsBot honors Crawl-delay directives.

You can also adjust the crawl rate through Ahrefs Webmaster Tools at ahrefs.com/webmaster if you have verified your site there.

Should You Block AhrefsBot?

This depends on your situation.

Reasons to Allow AhrefsBot

You use Ahrefs: Blocking the bot makes your own site data less accurate in Ahrefs tools
Competitors monitor you: If AhrefsBot cannot crawl you, your backlink data in Ahrefs will be stale — but so will your competitors’ view of your site
Yep.com indexing: AhrefsBot feeds into the Yep search engine. Blocking it removes your site from Yep results
Minimal resource impact: AhrefsBot is well-behaved and typically uses minimal server resources

Reasons to Block AhrefsBot

Server resource constraints: On very small or shared hosting, any crawler adds load
You do not use Ahrefs: If you do not use the tool and do not care about your data in it, there is no benefit to allowing it
Privacy concerns: You prefer to minimize how much data third parties collect about your site
Seeing spoofed traffic: If fake AhrefsBot traffic is causing issues, blocking the IP ranges can help

For most websites, allowing AhrefsBot is the better choice. The crawler is well-behaved, respects robots.txt and Crawl-delay, and the data it collects helps you (or your SEO team) monitor your site’s performance.

AhrefsBot vs Other SEO Crawlers

AhrefsBot is one of several SEO crawler bots you might see in your logs:

Bot	Company	Respects robots.txt	Crawl-delay
AhrefsBot	Ahrefs	Yes	Yes
SemrushBot	Semrush	Yes	Yes
DotBot	Moz	Yes	Yes
MJ12bot	Majestic	Yes	Yes
Googlebot	Google	Yes	No (uses own rate limits)
bingbot	Microsoft	Yes	Yes

All major SEO crawlers respect robots.txt. The Crawl-delay directive is the most effective way to manage their impact on your server.

Common AhrefsBot Issues and Solutions

AhrefsBot is ignoring my robots.txt

It can take several days for AhrefsBot to pick up robots.txt changes. If it persists after a week, verify that your robots.txt file is accessible at yourdomain.com/robots.txt and contains the correct directives. Syntax errors (wrong capitalization, missing line breaks) can cause rules to be ignored.

I see AhrefsBot but the IP does not match known ranges

This is likely spoofed traffic using AhrefsBot’s user-agent string. Verify using reverse DNS lookup as described above. If the IP does not resolve to ahrefs.com, block it at the firewall level — not in robots.txt, since fake bots ignore robots.txt.

AhrefsBot is crawling too aggressively

Add a Crawl-delay: 30 directive for AhrefsBot in your robots.txt. You can also register your site in Ahrefs Webmaster Tools and adjust the crawl rate there.

I blocked AhrefsBot but still see it in Ahrefs data

Blocking AhrefsBot prevents it from crawling your pages directly, but Ahrefs can still collect data about your site from external sources — other sites that link to you, for example. Backlink data will still appear even if the bot is blocked.

Frequently Asked Questions

Is AhrefsBot harmful?

No. AhrefsBot is classified as a “good bot.” It respects robots.txt, honors Crawl-delay directives, and identifies itself honestly in HTTP requests. It does not attempt to exploit vulnerabilities, scrape private data, or bypass access controls.

Why is AhrefsBot crawling my site so much?

AhrefsBot’s crawl frequency depends on your site’s size, how many external links point to it, and how frequently your content changes. Large sites with many backlinks get crawled more often. Use Crawl-delay to manage the frequency.

Does blocking AhrefsBot affect my Google rankings?

No. AhrefsBot is completely separate from Googlebot. Blocking AhrefsBot has zero impact on your Google Search rankings. It only affects how your site appears in Ahrefs’ tools and the Yep.com search engine.

What is the difference between AhrefsBot and AhrefsSiteAudit?

AhrefsBot is the main crawler that scans the entire web. AhrefsSiteAudit is a separate crawler that only runs when an Ahrefs user initiates a Site Audit on their own verified website. If you see AhrefsSiteAudit in your logs, someone is auditing your site through Ahrefs.

How do I allow AhrefsBot but block other SEO bots?

In your robots.txt, explicitly allow AhrefsBot while blocking others:

User-agent: AhrefsBot
Allow: /

User-agent: SemrushBot
Disallow: /

User-agent: MJ12bot
Disallow: /

Where can I find the latest AhrefsBot IP addresses?

Use our Ahrefs Bot IP Address Checker for real-time verification. You can also check Ahrefs’ official documentation at ahrefs.com/robot for the most current information.