> For the complete documentation index, see [llms.txt](https://ae-docs.keywordrush.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ae-docs.keywordrush.com/getting-data-reliably/blocking.md).

# When a shop blocks access

Most shops work fine. Plenty of stores are perfectly happy to be read — you paste a product link and it just works. Whether you'll ever run into a block depends entirely on the store; it's a possibility to be aware of, not something that happens everywhere.

{% hint style="info" %}
Affiliate Egg reads live pages because these shops offer no API and no product feed to pull from. If a shop **does** publish a product feed, importing it with [Content Egg's Feed module](https://ce-docs.keywordrush.com/modules/feed-modules) is cleaner and never gets blocked — reach for that first when it's an option. Affiliate Egg is for everything else.
{% endhint %}

Here's what's going on: Affiliate Egg makes a real request to the shop each time it searches for a product or refreshes a price, much like a browser. Most shops don't mind. Some, though, run anti-bot protection or limit how many requests one visitor can make — cross that line and the shop starts refusing requests, usually with a **503** or **403** error.

## The kinds of blocks you might hit

* **Rate limits.** Sending too many requests too quickly — from bulk-adding products or frequent price updates — can get your server's IP temporarily blocked. These usually clear on their own within about 24 hours.
* **Anti-bot walls.** Some shops (often behind Cloudflare and similar) challenge automated visitors with captchas or JavaScript checks. These don't clear by waiting.
* **A few shops won't cooperate at all.** A small number are locked down tightly enough that reliable, unattended reading isn't realistic.

None of this is the norm — but when it happens, here's what to do.

## What actually helps

* [x] **Use direct product URLs**, not keyword search — the single biggest improvement. Far fewer requests, and none of the fragile search pages.
* [x] **Go slow.** Add products a few at a time, and avoid aggressive price-update schedules.
* [x] **Ask the shop to whitelist your server's IP.** You're sending them sales, so many merchants are happy to — and asking permission is good form either way. Your host can tell you the server's IP address.
* [x] **Use a scraping service** for the harder cases. Your requests route through the service's rotating IPs (and can render JavaScript), which gets past most rate limits and many anti-bot walls. It's paid, but it's what makes difficult shops workable. See [Scraping services](/getting-data-reliably/extractor-settings.md).
* [x] **Set custom cookies** for shops that need a region or session cookie to show prices. See [Custom cookies](/getting-data-reliably/customcookies.md).

## What won't help

* **Waiting out an anti-bot wall.** Captchas and JavaScript challenges don't clear on a timer — you need a scraping service or a whitelisted IP.
* **A custom parser.** A parser is for shops whose data *can't be read* (see [Is your shop supported?](/supported-shops-and-custom-parsers/supported-shops.md)); it doesn't change whether a shop lets you in. If a shop blocks you, the fix is a scraping service or a whitelist, not a parser.