Capturing Feedburner RSS

[Spenser]

For a project I'm working on, I'm having to write several Perl scripts to run daily to take in RSS feeds from several software related sites. I wrote--or rather, cut and paste together--the following script. It works just fine on several RSS feeds (e.g, MySQL's Perl DBI Forum: http://forums.mysql.com/feed.php?51,type=rss). However, it doesn't work with sites that use Feedburner (e.g., ONlamp.com). There seems to be something different about their feed or XML structure.

I'm able to use the script (Re: Writing a simple RSS feed 'grabber' with XML::Parser.) that demerphq posted to dump the feed. The dump file doesn't look too different from that of other sites. When I run the script below, I get "Found 0 entries". Somehow I'm not quite getting the root node, which from the dump file looks like it shold be //entry. Does anyone have experience in capturing Feedburner's RSS feed? Can anyone tell me where I'm going wrong? Thanks in advance for any useful advice.

#!/usr/bin/perl -w

use strict;
use URI::Escape;
use LWP::Simple qw/get/;
use XML::LibXML;
use XML::XPath;
use XML::XPath::XMLParser; 
    
my $xml_url = 'http://feeds2.feedburner.com/oreilly/perl';
    
print "Getting RSS from O'Reilly Network - ONlamp.com \n";
my $xml_file = get($xml_url);
my $parser = XML::LibXML->new;
my $source = $parser->parse_string( $xml_file );
    
print "Getting nodes & counting them \n";
my @entries = $source->find('//entry')->get_nodelist;
my $count_entries = @entries;
    
print "Found $count_entries entries. \n";
    
foreach my $entry (@entries){
   my $title = $entry->find('title')->string_value;
   my $author = $entry->find('author')->string_value;
   my $content = $entry->find('content')->string_value;
   my $link = $entry->find('link')->string_value;
   my $published = $entry->find('published')->string_value;

   print "Saving entry for $author in MySQL \n";
   &save_to_mysql($title,$author,$content,$link,$published);
}

exit;

[ikegami]

There seems to be something different about their feed or XML structure.

That is an Atom feed (not an RSS feed) as indicated by the use of a {http://www.w3.org/2005/Atom}feed element for the root.

RSS is a poor spec. It requires guesswork by the reader. Atom is a very similar spec that addresses the issues of RSS. Most feed readers can handle both.

When I run the script below, I get "Found 0 entries".

XPath //entry looks for descendant nodes named entry in the null namespace. Atom feeds contain no such nodes. You need to look for entry nodes in the http://www.w3.org/2005/Atom namespace. See the notes for the findnodes sub. They apply to find as well.

Might I suggest XML::Atom::Syndication::Feed

Update: Added third paragraph.

[Spenser]

Yes, you're both right. It's an Atom feed and I need XML::Atom. Here's the script again with the appropriate modules for future reference or the use of others. This works smoothly. Thanks for the insights.

#!/usr/bin/perl -w

use strict;
use XML::Atom::Feed;
use XML::Atom::Entry;
    
my $xml_url = 'http://feeds2.feedburner.com/oreilly/perl';
my $feed = XML::Atom::Feed->new($xml_url);

foreach my $entry ($feed->entries()) {
   my $title = $entry->title();
   my $author = $entry->author()->name();
   my $content = $entry->content()->content();
   my $link = $entry->link()->href();
   my $published = $entry->published();
   
   &save_to_mysql($title,$author,$content,$link,$published);
}

exit;

In case anyone else uses this bit of code, notice that I used the keys from Feedburner's feed. For example, to get the author's name, I had to use the key of author, then the key under that of name. These come from the feed file and are not functions from the perl modules.

[Corion]

That's easy. The ones working are RSS feeds. The nonworking examples are Atom feeds. If you claim they "don't look too different", you need to look closer.

[n3toy]

++ On the accepted solution and updated code. This will be helpful to Monks who stumble across this node in the future.