@onerinas, no, it is to get the new content on any site when it's published.
I need it to get new content across all kinds of web and social channels so that #heynews can curate the best stories for a user's newsletter.
Some common issues I face:
- Websites without an RSS feed
- Websites that block bots from scraping their pages
- Websites blocking IPs that are not residential or from a specific geo
I'm using RSS.app to resolve these issues, so HeyNews can follow all kinds of sources.
From the outside it looks like every site is its own special case, so adding a site to a feed really means writing a small crawler for that site. I don't think it can be solved for ANY site from scratch. If I were building this, I'd limit the source list (a local newsletter probably needs a few dozen sources, not the whole web) and write a dedicated bot for each site. With AI help that part isn't hard anymore. Then it becomes a question of support: noticing when a site changes its layout and fixing that one bot.
Late to this one, but the HeyNews newsletter runs on beehiiv and it's held up well for us.
Disclosure: We build tools that work with any of these platforms.
- Automate curation and writing, in your voice: heynews.co
- Turn sponsor ads into native copy: adapt.heynews.co
- Compare newsletter tools: www.bestnewslettertools.com
- Rank your newsletter on a public leaderboard, from $1: inboxbid.lol
No newsletter yet? We'll help you build one: schedule.heynews.co/discovery…
API requirement is the hard part. It should work automatically within the app.