Back
Similar todos
Restart movies scraping task that failed last night due to excessive timeouts #life
See similar todos

No replies yet

Fix movies scraper because it broke overnight #life
See similar todos

No replies yet

restrict number of concurrent scraping tasks a user can have #infinityxml
See similar todos

No replies yet

Fix movies scraper because data format changed overnight #life
See similar todos

No replies yet

Update scraper, now storing movie director, quality and still #life
See similar todos

No replies yet

restrict size of scraping task #infinityxml
See similar todos

No replies yet

improve scraping script
See similar todos

No replies yet

Add retry delay to apartment scraper because website goes down every night #life
See similar todos

No replies yet

speed up scraping script
See similar todos

No replies yet

Finish a web scrapper that make the process of adding bulk new quality replays in a matter of 3 clicks #demtovideo
See similar todos

No replies yet

refactor rescue_on calls on scrapers and add other common errors that should be retried automatically #reva
See similar todos

No replies yet

I just pushed an update that: Has 3,000 URL max per scrape; Prints to sheet in batches of 10 as it scrapes; Will continue scraping even if you close sheets, log out, etc.; Will continue scraping past the previous 6 minute timeout restriction #infinityxml
See similar todos

No replies yet

relaunched the scraping machine #thecompaniesapi
See similar todos

No replies yet

Fix rate limiting issues
See similar todos

No replies yet

add scraper service to grab video urls #watchr
See similar todos

No replies yet

wait for rate limit #therapistai
See similar todos

No replies yet

experiment with auto-retrying limit stops #rush
See similar todos

No replies yet

#slackdiscord Add failsafe on rate-limit so I know a) when to try again and b) to avoid getting timed out for eternity
See similar todos

No replies yet

Notice Watchdog is still getting blocked by X on occasion even though I have much better resiliency now with auto refresh. Investigating hijacking the requests so I can see the X-Rate-Limit-Remaining header to know when to start scraping again instead of refreshing blindly #watchdog !private
See similar todos
see #remoteok recover a bit after fixing the scrapers that were broken for 10 months
See similar todos

No replies yet