Back
Similar todos
rewrote one of the running website scrapers to use playwright so that the JS bits get rendered
🔨 working on docker performance tasks on my hosting setup #mylife
Successfully moved scraper from reading json from the puppeteer output to getting a POST request after the process is done! 💪 (less problems with messy output :S) #bagsoup
worked on job website tests on one of the scrapers
more scraper work and I finally figured out how to work around a few issues I couldn't quite wrap my brain around.
worked on how to test/keep a scraping project updated more easily
🛠 moved the two last crons running on my laptop to a remote server (they were using puppeteer to control a chromium instance) #life
check why scrapping stopped to wprk #capgo
re-worked one of my job scrapers and trying to decide if I should keep it or keep moving forward on it
The chromiums mini server seems to be working fine so I'm gonna start using it for all scraping tasks, will see... 🫣 took me 2 hours to setup because chromiums refused to listen on public ip no matter what I'd do and ended using socat #servers
make run scrapping for a bit and try to debug it
Work on automated web scraping #pagesonpages
write first prototype automated web scraper #pagesonpages
wrote a job scraper because why not?
late night hacking on a scraper project and mostly getting the UI updated