improved duplicate detection #happenlist

Continuing work on my tech tree. Today first things #happenlist

i just spent half an hour writing a prompt to a feature. This is my job now #happenlist

Some changes to data capture after real world tests #happenlist

Some operational work, taking new algo for a ride #happenlist

working on algo change resulting in major cost reduction in data extraction #happenlist

Worked on algo improvement #happenlist

Further work on deep data extraction argh I want it to be done #happenlist

Improved deep data extraction #happenlist

working further on data extraction v3. a data extraction v4 is already clearly visible from here but i can finish v3 without regression wrt v2 #happenlist

fixed a bit of code bugs in deeper data extraction #happenlist

worked out bugs with new deepened data extraction mechanism #happenlist

added a devtool to the #happenlist backoffice

refactor and test scaffolding for algorithm expansion #happenlist

fixed kafka tunnel for local testing #happenlist

retry jobs button #happenlist

parts of admin panel became much more insightful, more likable a status board that makes immediately clear what still needs to be done #happenlist

small improvement to deeper data collection #happenlist

worked a bit on deepening the indexing algo #happenlist

Did a release and started on the nextgen crawler that will really increase coverage #happenlist