From the outside it looks like every site is its own special case, so adding a site to a feed really means writing a small crawler for that site. I don't think it can be solved for ANY site from scratch. If I were building this, I'd limit the source list (a local newsletter probably needs a few dozen sources, not the whole web) and write a dedicated bot for each site. With AI help that part isn't hard anymore. Then it becomes a question of support: noticing when a site changes its layout and fixing that one bot.
From the outside it looks like every site is its own special case, so adding a site to a feed really means writing a small crawler for that site. I don't think it can be solved for ANY site from scratch. If I were building this, I'd limit the source list (a local newsletter probably needs a few dozen sources, not the whole web) and write a dedicated bot for each site. With AI help that part isn't hard anymore. Then it becomes a question of support: noticing when a site changes its layout and fixing that one bot.