Blocking countries and AI crawlers


Your protection area gives you three ways to keep unwanted visitors away from your site:

  • Countries: block or check visitors by the country they're in.
  • Bot blocking: block kinds of bots by how they identify themselves.
  • Block AI crawlers and other services: block particular services, which stillorigin recognises by their official IP addresses.

Bot blocking and Block AI crawlers and other services both deal with AI crawlers, but they work differently and are set separately. Many sites use both.

Countries

On the Countries tab, you can add countries for your site. For each country, choose:

  • Block: visitors from that country can't open your site.
  • Captcha Protection check: visitors from that country complete a quick browser check before your site opens (see "The browser check your visitors may see").

A country you add starts in log mode, so it's only recorded at first. When you're happy with what it would do, switch it to Enforce. You can add up to 250 countries. No countries are set by default.

Countries your host has set for your site are shown under Set by your host, marked Blocked by your host, Checked by your host or Logged by your host. You can't change those.

Verified Identified traffic, such as search engines, and addresses your host trusts are never blocked or checked by country.

On the Blocks tab, country blocks show as Country rule: followed by the country's name. Country checks show as Browser check: country rule. Country rules still in log mode show as Country rule in log mode: followed by the country's name.

If your host manages country blocking for you, you'll see:

Your host manages country blocking for your site.

Contact your host if you'd like it changed. Any countries you had already added are kept, and come back if your host allows changes again.

Bot blocking

Bot blocking is on the Settings tab. It blocks bots by how they identify themselves (their "user agent"). Choose any of:

  • AI crawlers: bots that collect pages for AI models and AI search, such as GPTBot, ClaudeBot and CCBot.
  • SEO and marketing crawlers: bots from SEO tools, such as AhrefsBot, SemrushBot and MJ12bot.
  • Scripts and HTTP libraries: requests from command-line tools, code libraries and headless browsers. Take care: this can also block uptime monitors and other services that connect to your site.

Nothing is selected by default. Search engines such as Google and Bing are never blocked. Bot blocking always blocks, even while your Website protection rules are in log mode.

On the Blocks tab, these show as AI crawler (bot blocking), Script or HTTP library (bot blocking) and SEO crawler (bot blocking).

If you don't see Bot blocking on Settings, your host manages it for you. Contact your host if you'd like it changed.

Block AI crawlers and other services

Block AI crawlers and other services is on the Settings tab. It blocks services that stillorigin identifies by their official IP addresses, such as AI crawlers, on your site.

  • Services your host blocks are shown ticked and marked Blocked by your host. You can't change those, but you can add blocks of your own.
  • If you choose to block a search engine, you'll be asked to tick I understand, block them first. Blocked search engines can't read your site, so your pages may drop out of search results.
  • Click Save blocks to apply your changes.

Services you block, or that your host blocks by default, see the stillorigin block page with "Reason: this automated service is not allowed." Services your host blocks for all its sites don't see a page at all, because their connection is refused. These blocks always apply, even while your Website protection rules are in log mode.

If you don't see Block AI crawlers and other services on Settings, your host manages it for you. Contact your host if you'd like it changed.

Services your host blocks by default

Your host may block some services on your site by default. If so, you'll see them on Settings under Blocked by default by your host, each with an Allow button. You don't need any extra permission to allow them.

Before you allow a service, you'll see a warning and have to tick I understand, allow access. Outside services can send a lot of traffic, which can slow your site down, so check your hosting terms first. If you change your mind, click Block again.

stillorigin doesn't block any services by default. Your host chooses which ones to block.

Fake crawlers

Some bots pretend to be a well-known service, such as a search engine, but don't come from that service's official addresses. Your host decides whether these are blocked, and you can't change it. On the Blocks tab they show as Fake crawler.

Block reasons

On the Blocks tab, these blocks show as:

  • blocked by default: (service name): a service your host blocks by default.
  • blocked by owner: (service name): a service blocked under Block AI crawlers and other services.
  • Fake crawler: a bot pretending to be a well-known service.

Was this answer helpful?

Still need help?

Our friendly support team are ready to offer assistance with any issues you may be encountering.
Click the button below to open a ticket:
Open Ticket

 WordPress Hosting

Fast hosting for WordPress
Experience the best in Australian WordPress hosting with lightning fast servers, built-in caching, and performance tools.

 Build Your Website

Sitejet Hosting
Build your site fast with a drag and drop editor with no coding required. 140+ quality, templates to get you started.

 Register Domains

It all starts with your domain name
Find the perfect domain and register now with our competitive pricing on all extensions.

 Web Hosting

Fast, local, secure hosting
Full featured hosting on cPanel with multiple server locations around the country.
« Back