Back
Similar todos
wrote some failing tests and ended up fixing several pypi bugs #djangopackages
found a nice offline copy for pypi data so diving into that to figure out what makes sense to import in #djangopackages
finished pypi refactor of #djangopackages
finished pypi tests and deployed fixes #djangopackages
fixed a #djangopackages bug and started working on adding pypa/packaging support vs. rolling my own
📝 Worked on #djangopackages to rewrite/augment how package states are pulled from PyPI to help improve the overall accuracy of the data.
🚜 refactored pypi stats generation script to show a progress bar and to throttle self #djangopackages
overhauled another scraper/processor with much better tests and fixed several non-obvious bugs. I should have written tested upfront.
worked on how to test/keep a scraping project updated more easily
started adding some tests to make fixing random license and other bugs easier #djangopackages
finished a few python based wikipedia and google news scripts for fetching content #mylife
working with Pandas (Python) to update a few existing scrapers. It's really, really nice but I ran into a bug trying to pull URLs that I don't want to think about right now.
updated scraper logic to fix tags and author information along with some easy to update tests to make this much easier to find/fix next time around
🚜 Refactored several private python packages to standardize on a new testing/install setup #revsys
Work on getting tests data #oneclicktest
finally finished scraper and data looks good. a few hard refactors but I have a pretty good handle on spatula now.
worked on another data importer using the new python spatula library. It's not bad, but proving to be a bit more difficult to get my 10+ year old project synced than I'd like
🔨 Updates scraper-proxy tool and rewriting/refactoring from the ground up to handle structured data
worked on some more Python project automation to sync up a bunch of random changes
I fixed another issue in #speechzap and wrote the first tests to gradually start controlling the technical debt.