Back
Similar todos
finally finished scraper and data looks good. a few hard refactors but I have a pretty good handle on spatula now.
🍔 built a project scraper over lunch that I don't hate. I used sqlite-utils with mixed results. might just port it back to django and call it a day
pushed a bunch of importer changes and started on a layer import/data tracking structure #conf
working with Pandas (Python) to update a few existing scrapers. It's really, really nice but I ran into a bug trying to pull URLs that I don't want to think about right now.
built two more article scrapers using spatula. It's really nice.
worked on some more Python project automation to sync up a bunch of random changes
started switching some nasty scraping code project over to Pandas because it works so damn well
worked on a new email plugin setup for #concord using Python pluggy. It's still abit too trippy to use, but it's interesting
found a nice offline copy for pypi data so diving into that to figure out what makes sense to import in #djangopackages
try new importer #rewardsflights
finished Unsplash importer, but the results are not amazing #lfk
🔨 Updates scraper-proxy tool and rewriting/refactoring from the ground up to handle structured data
working on some scraping for data analysis #linkhandy
worked on how to test/keep a scraping project updated more easily
pulled in a new csv scraper data for #daysuntillatenight
working on some a few scrapers for pulling in data vs. just using sheets #daysuntillatenight
adds a few category pages scrapers using spatula
more scraper work and I finally figured out how to work around a few issues I couldn't quite wrap my brain around.
quick fix for importer #rewardsflights