Back
Similar todos
got someone scraping my site, but they're rotating their IP's....
stop logging 404’s from crawlers 
#wip
stop logging 404’s from crawlers #wip
bug #162 crawl timeout problem on some sites 
#apercite
recrawl site 
#rewardsflights
fix site down cause of bots and scrappers (to many logs broke up my db) 
#lab
fixed crawler bug 
#sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler for 
#teacherjobs
take down 
#interiorai cause thousands of IPs hammering my server
used cursor/claude to rewrite a crawler for weegee into a new format.
It was insanely fast. It had some things wrong but due to good e2e tests I spotted it immediately and could correct it.
fixed crawler bugs #sportstechjobs
fixed crawler bugs #sportstechjobs
fixed crawler bugs #sportstechjobs