A Web2py planet (aggregator) using feedparser and rss2

Based on the web2py interactive examples.

In model db.py add a feed table containing feeds URLs:

db.define_table("feed",
    Field("name"),
    Field("author"),
    Field("email", requires=IS_EMAIL()),
    Field("url", requires=IS_URL(), comment="RSS/Atom feed"),
    Field("link", requires=IS_URL(), comment="Blog href"), 
    Field("general", "boolean", comment="Many categories (needs filters)"),
    )

In controller default.py, add a planet function to render a basic page (fetching feeds with feedparser):

def planet()
    import datetime
    import re
    import gluon.contrib.rss2 as rss2
    import gluon.contrib.feedparser as feedparser

    # filter for general (not categorized) feeds
    regex =  re.compile('web2py',re.I)

    feeds = db(db.feed.id>0).select()

    entries = []

    for feed in feeds:
        # fetch and parse feeds
        d = feedparser.parse(feed.url)
        for entry in d.entries:
            if not feed.general or regex.search(entry.description):
                # extract entry attributes
                entries.append({
                    'feed': {'author':feed.author,'link':feed.link,'url':feed.url,'name':feed.name},
                    'title': entry.title,
                    'link': entry.link,
                    'description': entry.description,
                    'author': hasattr(entry, 'author_detail')
                         and entry.author_detail.name
                          or feed.author, 
                    'date': datetime.datetime(*entry.date_parsed[:6]) 
                })

    # sort entries by date, descending
    entries.sort(key=lambda x: x['date'],reverse=True)

    now = datetime.datetime.now()

    # aggregate rss2 feed with parsed entries
    rss = rss2.RSS2(title="Planet web2py",
       link = URL(r=request,c="default",f="planet"),
       description = "planet author",
       lastBuildDate = now,
       items = [
          rss2.RSSItem(
            title = entry['title'],
            link = entry['link'],
            description = entry['description'],
            author = entry['author'],
            # guid = rss2.Guid('unkown'),
            pubDate = entry['date']) for entry in entries]
       )

    # return new rss feed xml
    response.headers['Content-Type']='application/rss+xml'
    return rss2.dumps(rss)

Remeber to create some feed (using the database appadmin), and then call planet function!

Working examples can be found at:

• web2py sample planet
• postgresql spanish tentative planet

Full source code of complete examples published at google code project: planet-web2py