Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer


by Ctrl-z (Friar)
on Jun 29, 2005 at 09:38 UTC ( [id://470942]=sourcecode: print w/replies, xml ) Need Help??
Category: Audio Related Programs
Author/Contact Info Ctrl-z
Description: Utility to resynchronise iTunes with tracks orphaned on an iPod, or alternatively do a full iPod backup. Big caveats - see POD (err, not iPod) before using


=head1 NAME

iPod-backup: reclaim tracks orphaned on an iPod 


    perl -do MODE -out DIR -mnt DRIVE [-top DIR]

    -out    Output directory for any reclaimed files
    -mnt    iPod mount path or drive
    -top    iTunes top-level directory
    -do     perform one of the following operations:

            stat    print iPod vs iTunes statistics (default)
            sync    reclaimed tracks not in iTunes
            bak     backup full iPod


Man, dont even get me started on a rant about Apple software...

When disk files are lost, iTunes helpfully does nothing then silently 
removes them from your iPod next time you plug it in. While the latter
can be avoided by disabling automatic sync'ing, there is no way to 
import tracks on the iPod back into iTunes...ho hum

This script checks your iPod's database against iTune's and imports an
tracks not available in iTunes back onto the harddisk.

It can also do a full backup of the iPod in a human friendly manner, i


rather than Apple's quirky 



This is quick and dirty programming...use at your own risk.

In particular, the iPod database is a binary format. Rather than caref
reverse engineer by hand, I just weild the Swiss Army Chainsaw and hac
+k off 
what I dont need. Your mileage may vary, as they say.

Only tested on Windows XP. Linux requires 2.6 kernel or a FAT32 format
iPod. You would probably be better off looking at Gnupod for that.


use strict;
use utf8;

usage() if @ARGV % 2;

my %ARGS  = @ARGV;
my @KEYS  = (undef, "Name", "Path", "Album", "Artist", "Genre", "Kind"
+, undef, "Comments");
my $TYPE  = qr/\.(?:mp3|aux)/i; 
my @STAT  = ("sync'd","iPod","iTunes");
my $IPOD  =  1;
my $SYNC  =  0;
my $DISK  = -1;

my $mode          = $ARGS{-do}  || "stat";
my $ipod_temp     = $ARGS{-out} || "./ipod-temp";
my $ipod_mount    = $ARGS{-mnt} || usage();
my $itunes_root   = $ARGS{-top};

my $ipod_itunesdb = "$ipod_mount/iPod_Control/iTunes/iTunesDB";
my $ipod_sysinfo  = "$ipod_mount/iPod_Control/Device/SysInfo";
my $ipod_music    = "$ipod_mount/iPod_Control/Music";
my $itunes_lib    = "$itunes_root/iTunes Music Library.xml";
my $itunes_music  = "$itunes_root/iTunes Music";

die "'$ipod_itunesdb' does not exist\n"     unless -f $ipod_itunesdb;
die "'$itunes_root' is not a directory\n"   unless !$itunes_root || -d
+ $itunes_root;
die "'$itunes_lib' does not exist\n"        unless !$itunes_root || -f
+ $itunes_lib;

if($mode eq "stat"){
    stat_ipod($ipod_itunesdb, $itunes_lib);
elsif($mode eq "bak"){
    import_ipod($ipod_temp, $ipod_itunesdb);
elsif($mode eq "sync"){
    import_ipod($ipod_temp, $ipod_itunesdb, $itunes_lib);

sub usage
    my $src = fslurp( fopen($0) );
    print "\n", $1,"\n\n",$2,"\n\n"
        if $src =~ /=head1 NAME\s*?(.*?)\s*?=head1 SYNOPSIS\s*?(.*?)\s
+*?=head1 DISCUSSION/s;
    exit 0;

# Print shared and unique tracks on iPod and iTunes
sub stat_ipod
    my $ipod  = shift || die "Error: No path to iTunesDB\n";
    my $local = shift || die "Error: No path to XML library\n";
    my $dup   = {};
    my ($disk, $pod, $syn);

    $ipod  = parse_itunesdb($ipod);
    $local = parse_itunes_library($local);

    $dup->{ $_->{Artist}." - ".$_->{Album}." - ".$_->{Name} }++ foreac
+h @$ipod;
    $dup->{ $_->{Artist}." - ".$_->{Album}." - ".$_->{Name} }-- foreac
+h @$local;

    foreach(sort keys %$dup){
        my $s = $dup->{$_};
        printf "%-70s %-6s\n", (length $_ < 70) ? $_ : substr($_,0,67)
+."...", $STAT[$s];
        ($s == $DISK) ? $disk :
        ($s == $IPOD) ? $pod : $syn += 1;

    my $format = "%-6s: %5d / %-5d\n";
    print "\n\n";
    printf $format, "iPod",  $pod,  scalar(@$ipod);
    printf $format, "iTunes",$disk, scalar(@$local);
    printf $format, "Sync'd",$syn,  scalar(@$ipod)+scalar(@$local);

# Import tracks on an iPod
# If $local is provided, only imports those not currently in iTunes
# else, imports the lot for backup
sub import_ipod
    my $dir   = shift || "./ipod-temp";
    my $ipod  = shift || die "Error: No path to iTunesDB\n";
    my $local = shift;

    my $dup   = {};
    my @err   = ();
    my $prog  = 0;
    my $totl  = 0;

    $ipod  = parse_itunesdb($ipod);
    $local = parse_itunes_library($local) if $local;
    $local = [] unless $local;

    print "(ipod)  scanned tracks: ",scalar(@$ipod),"\n";
    print "(local) scanned tracks: ",scalar(@$local),"\n";

    $dup->{ $_->{Artist}."/".$_->{Album}."/".$_->{Name} } = $_    fore
+ach @$ipod;
    $dup->{ $_->{Artist}."/".$_->{Album}."/".$_->{Name} } = undef fore
+ach @$local;
    (defined $dup->{$_}) ? $totl++ : delete $dup->{$_} foreach(keys %$

    mkdir($dir) unless -d $dir;

    foreach(sort keys %$dup)
        my $track = $dup->{$_} or next;
        my $path  = $track->{Path};
        my $ext   = $1 if $track->{Path} =~ /\.(\S+)$/;

        my $in    = "$ipod_mount$track->{Path}";
        my $d1    = "$dir/$track->{Artist}";
        my $d2    = "$d1/$track->{Album}";
        my $out   = "$d2/$track->{Name}.$ext";

        my $ui    = $track->{Artist}." - ".$track->{Album}." - ".$trac

        mkdir($d1) unless -d $d1;
        mkdir($d2) unless -d $d2;

        printf "%-11s %-65s", ++$prog."/".$totl, 
               (length($ui) < 60) ? $ui : substr($ui,0,60)."...";

        $@ ='';
            fcopy($in, $out) unless( -f $out && (stat($in))[7] == (sta
+t($out))[7] );
            print "ok\n";
            push @err, "$out: $@";
            print "--\n";

    print STDERR "\nErrors:\n";
    print @err ? join('\n', @err) : "none", "\n";

sub parse_itunesdb
    my $f = fopen(shift);
    my $s = fslurp($f);
    my $fields;

    my @records = ();
    my $record  = {};

    foreach( split(/mhod/, $s) )
        my ($eor, $t, $k);
        next unless length $_;

        s/\000\000+/ /sgo;

        # this doesnt work...???
        # s/^(..(.).(.).....)//;
        # $c = escape($1);
        # $l = chr($2);
        # $t = $K[ord($3)];

        $t = substr($_, 0, 10);
        $t = $KEYS[ord(substr($t, 4, 5))];
        $_ = substr($_, 10, length $_);        

        next if !$t;

        if($t eq "Path")
            $_ = $1 if m/^(.*?$TYPE)/;
            $_ =~ s/:/\//sg;

        # TODO: Some final checks for non-printable pathnames...

        $record->{$t} = $_ ;
            $record->{Artist} = "Unknown"  unless defined $record->{Ar
            $record->{Album}  = "Unknown"  unless defined $record->{Al
            push @records, $record         if     defined $record->{Na
            $record = {};
    return \@records;

sub parse_itunes_library
    my $lib = fopen(shift);
    my $ok  = 0;

    my @records = ();
    my $record  = {};

        last  if m/^\s*?<key>Playlists<\/key>/;
        $ok++ if m/^\s*?<key>Tracks<\/key>/;
        next unless $ok;

            $record->{$1} = xml_decode($3) ;
            $record->{Artist} = "Unknown"  unless defined $record->{Ar
            $record->{Album}  = "Unknown"  unless defined $record->{Al
            push @records, $record         if     defined $record->{Na
            $record = {};
    return \@records;

# decode filename urls in XML
sub url_decode 
    my $str = shift;
    $str =~ tr/+/ /;
    $str =~ s/%(..)/pack("C",hex($1))/eg;
    return $str;

# TODO: decode XML-entities in fields
sub xml_decode 
    my $str = shift;
    $str =~ s/\&\#(\d+)\;/chr($1)/esg;
    return $str;

sub fcopy
    my $in  = shift;
    my $out = shift;
    my $buf = shift || 1024;
    my $fin  = fopen($in);
    my $fout = fopen(">".$out);
    my ($b, $r, $w);
    while($r = sysread($fin, $b, $buf))
        $w = syswrite($fout, $b, $r);
        last if $r < $buf;

sub fopen
    my $path = shift or die "fopen: no path\n";
    my $f;
    open $f,$path or die "$path: $!";
    binmode $f;
    return  $f;

sub fslurp
    my $f    = shift;
    local $/ = undef;
    return <$f>;
Replies are listed 'Best First'.
Re: ipod-backup
by shiza (Hermit) on Oct 25, 2005 at 22:25 UTC
    Well, re-installed the OS on my wife's computer this past weekend. Her iPod was basically her backup for her music because I thought (as insane as it sounds) that iTunes could suck the music back in after the install. It doesn't.

    Yesterday evening I hear screams coming from our office. iTunes apparently doesn't like a dirty iPod and decided to clean it up which entailed deleting 7 Gigs of music and photos. Thanks Apple!

    I immediately had my wife call Apple to see if they could help her restore her data. Well, they wouldn't even answer a question until we gave them a credit card number?! Keep in mind it's still under a 1 yr warranty. Thanks again Apple!

    My next step was to do some data recovery. I d/l some software and ran it. It found the missing songs and I restored them to my hard drive. Whew, i'm a hero now! Wrong. The restored files are all screwed up. Mis-labled songs, overlapping songs, etc...

    CTRL-Z, do you think your script can help me with my situation?

    P.S. - IMO Apple purposely weaved this inconvenience into iTunes...LAWSUIT!? I'm just kidding about the lawsuit, but seriously, why leave out simple and obviously needed functionality. If they didn't do it on purpose, maybe they need to re-structure their QA dept. I'm betting it was purposeful though.
      The restored files are all screwed up. Mis-labled songs, overlapping songs, etc...

      On my iPod, the tracks are stored in a seemingly random grouping in several folders - like some kind of hash table. The track names are truncated, but the mp3 data is not "overlapped" (is that what you meant?). Maybe your iPod used the Apple HFS+ filesystem? If that is the case, all I can do is point you at the Gnupod/Linux kernel 2.6 references in the POD. Sorry.

      If the mp3 data is ok, then you might be able to use the import_ipod function to read the binary iTunesDB restored off the ipod. Just point $ipod_itunesdb at the restored iTunesDB. The original caveats still apply though...

      Check out the Gnupod link - it is written in perl, by people who have actually studied the iPod. They could be able to advise you further.

      Good luck, man. /msg me if you need more help.

      time was, I could move my arms like a bird and...
Re: ipod-backup
by Anonymous Monk on Apr 24, 2006 at 14:05 UTC
    Very cool, could I request fleshing out the TODO's though? I half hacked the non-printing character TODO but it's still a bit wonky (my code).
      Hi Anonymous Monk,

      Perhaps if you posted the code you have somone may help you with it.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: sourcecode [id://470942]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (3)
As of 2024-04-20 13:40 GMT
Find Nodes?
    Voting Booth?

    No recent polls found