Back
Similar todos
working with Pandas (Python) to update a few existing scrapers. It's really, really nice but I ran into a bug trying to pull URLs that I don't want to think about right now.
finished out my day updating scraper-proxy to support css and xpath selectors along with a few features I haven't seen anyone do yet.
started switching some nasty scraping code project over to Pandas because it works so damn well
🐼 reading up on the Python Pandas library since it makes scraping HTML tables easy peasy
worked on two new #jobs scrapers using RSS
working on some a few scrapers for pulling in data vs. just using sheets #daysuntillatenight
working on some scraping for data analysis #linkhandy
finally finished scraper and data looks good. a few hard refactors but I have a pretty good handle on spatula now.
fixed Pandas issue and I have better, cleaner data now
worked on how to test/keep a scraping project updated more easily
Prepare scrapers code for batches and pagination #mrscraper
worked on a scraper service tonight to update it for a few common area layouts and the general collection system
built two more article scrapers using spatula. It's really nice.
pulled in a new csv scraper data for #daysuntillatenight
build more scrapers #pagesonpages
refactored a scraper app to use django-q more effectively
added scraping from pages #scrapebook
late night hacking on a scraper project and mostly getting the UI updated
⬆️ upgraded the running website and looked at refreshing data scrapers that pull in race info
#smumods done scraping hundreds of a Single Page App website. Learnt loads. Ask me anything on scrapers if you want to.