Back
Similar todos
increase timeout for scan of websites that have a lot of servers behind them 
#httpscop
getting crawled at weegee.ch - by aggressive crawlers which brought the site down. Gonna do the simplest thing by blocking the IPs. As a crawler myself I know that this usually causes enough troubles to stop a crawler.
fixed bug in crawler 
#sportstechjobs
fixed bug in #sportstechjobs crawler
fixed bug in #sportstechjobs crawler
fixed bug in #sportstechjobs crawler
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bug #sportstechjobs
fixed crawler bugs #sportstechjobs
fixed crawler bugs #sportstechjobs
fixed crawler bugs #sportstechjobs
recrawl site 
#rewardsflights
fixed cron job for crawlers
fixed bug #sportstechjobs crawler
update search timeout to differ on mobile 
#pkgstats