Anyone has experienced high load of traffic from AI Bots / Scrappers recently?

Question

Anyone has experienced high load of traffic from AI Bots / Scrappers recently? I'm trying to understand the size of the problem.

I know that Cloudflare and others have started offering such services like blocking this kind of traffic.

For people like us, setting up those tools is easy peasy, however for non-technical folks might be a headache.

I have already setup this for a few people and was wondering if there is an opportunity to productize the service.

I will really appreciate any insights.

Martin Llano · Accepted Answer

Thanks Ben! Appreciate the honest feedback!

Ben Katz · Answer

My $0.02? I don't care if AI bots scrape my entire website. Gives me a better chance of ranking for some query on ChatGPT. 

Google Search as it is today will die over the long term anyway - you can already see them putting AI-generated results at the top of the page ahead of their actual search results.

So this is a non-issue for me, even a positive. No sense in fighting the wave, embrace it :)

Go to Homepage	`g` `h`
Go to Done Todos	`g` `d`
Go to Questions	`g` `q`
Compose a New Todo	`n`
Go to Search	`/`
Show this dialog	`?`

Anyone has experienced high load of traffic from AI Bots / Scrappers recently?

Keyboard Shortcuts