Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I’m looking for a program that runs as daemon, downloads Atom and RSS feeds at least every minute, and provides a paginated Atom feed of all entries of all feeds through a simple HTTP server. It should also retain deleted entries. A useful extra would be to be able to label feeds, as in Google Reader, so that each label gets its own aggregate feed.

If you don’t know of such a program or know that it probably doesn’t exist, please let me know if you think there are any principle hurdles, because if not, I would queue it up as a project of my own.

The use cases are similar to that of services like Google Reader. I follow a larger number of RSS and Atom feeds and want to avoid having to regularly synchronize the list between multiple computers and my smartphone. On my smartphone, I also want to save traffic. In addition to those use cases, I need a high update rate, so I can react in a timely fashion, and to catch entries that are quickly deleted again, and I don’t want to have to leave one of my computers running non-stop either.

share|improve this question

2 Answers 2

up vote 1 down vote accepted

For about eight months (until mid-August) I used Tiny Tiny RSS with the slight hacks described above, so thanks again for that answer! However, I never actually needed the powerful web interface the API and many of its other great features—what I did need at some point was the ability to manipulate the HTTP request headers (to insert cookies and authentication keys), to send request through proxies, to manipulate the XML before parsing it, etc.

I ended up writing my own application—the Resyndicator—which I’ve been using productively for a little of one month now. I focused on making as little assumption as possible about the kinds of data sources, resyndication queries, and transformations people might want to use, so if something is not easily subclassable (e.g., requires copy-&-pasting of code) that’s likely a bug.

So far I’ve implemented a base class for fetching feeds (anything feedparser can parse) and for pulling in streams from the Twitter streaming API. The user can then use SQLAlchemy filter statements to specify which fetched entries should be aggregated into which resyndicated feeds. It also supports publishing to PubSubHubbub.

The program is still pretty raw at this point (esp. since I also created my own feedgenerator fork), but I’m working on it whenever I have some free time. I hope it helps someone!

share|improve this answer

I haven't used it personally, but it looks like Tiny Tiny RSS might do what you want. It's a web app (Apache + PHP + PostgreSQL) that can generate re-feeds based on various criteria.

share|improve this answer
    
Thanks. I must’ve overlooked that resyndication feature. I installed Tiny Tiny RSS on my staging server. It provides everything I need and a lot more (strange choice of a name). Only the smallest update interval is 15 minutes. I had to edit backend.php to add the 1-minute option to the menus. In include/rssfuncs.php I had to adapt the DAEMON_SLEEP_INTERVAL and remove a check for updates by a different process. The check tests whether the feed has been updated within the last five minutes, and if so, assumes that another process must have updated it and has it skipped. –  Telofy Dec 27 '11 at 21:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.