Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Copy XML file, write new attributes and move the old files.

by Nevamonk (Novice)
on Jan 26, 2015 at 08:00 UTC ( [id://1114493]=perlquestion: print w/replies, xml ) Need Help??

Nevamonk has asked for the wisdom of the Perl Monks concerning the following question:

Hello guys,

I seek your wisdom regarding the copying of an XML-file. Basically, this script will read incoming XML files, matches them to a database, where some ID's may need to be overwritten, but i need to keep the old data file to. So i was thinking of copying the file and matching the new file to database, overwriting the attributes in both name and value. Now how do i copy an XML file and bind it to a variable?

use warnings ; use strict ; use XML::LibXML; use DBI ; use File::Copy qw(copy); my $xmlInputFile = ""; my $dbConnection =""; my $parser = XML::LibXML->new(); my $databasename = "test_db"; my $hostname = "localhost"; my $username = "rrdtool"; my $password = ""; my $elemNameID; my $elemID; my $elemValueID; my $elemValueList; my $xmlOutputFile; my $matchRefFile; #=========================================# # SUB-Routines # #=========================================# sub connectDatabase { my $bConnected = 0; $dbConnection = DBI->connect( "dbi:mysql:database=$databasename;ho +st=$hostname", $username, $password ); if ( $dbConnection ) { $bConnected = 1; } else { print "There was an error connecting to : $databasename => $DB +I::errstr\n"; } return $bConnected; } #=========================================# sub WriteToLog { my ( $logline ) = @_; open ( LOGFILE, '>>RRDTool_translate_test.log'); print LOGFILE $logline; close ( LOGFILE ); } #=========================================# sub createNewXmlFile { #open my $matchRefFile, '<', ""; #binmode $matchRefFile; # drop all PerlIO layers possibly created + by a use open pragma #my $doc = XML::LibXML->load_xml(IO => $matchRefFile); #open my $xmlOutputFile, '>', 'xmlOutputFile.xml'; #binmode $xmlOutputFile; # as above # $doc->toFH($xmlOutputFile); # or #print {$xmlOutputFile} $doc->toString(); copy $xmlInputFile, $xmlOutputFile; rename $xmlOutputFile, 'xmlOutputFile.xml'; } #=========================================# sub matchNodeDB { if ( 1 == connectDatabase) { my $sqlSelectQuery = ( "SELECT IF NOT EXISTS * FROM XMLelementen + WHERE name ='$elemNameID' ELSE INSERT INTO XMLelementen VALUES ('$el +emNameID')"); my $query_handle = connectDatabase->prepare ( $sqlSelectQuery ); $query_handle->execute(); $query_handle->bind_columns(undef, \$elemID); return $elemID; } else { WriteToLog ("failed to get / insert $elemNameID from Database.\n +"); die (WriteToLog ("exitting on failed matchNodeDB sub for [$elemN +ameID]\n Exitting...\n")); } } #=========================================# sub WriteToFile { open ( OutputFile, '>>$xmlOutputFile'); $elemNameID->set_att(p => $elemID); $elemValueID->set_att(p => $elemID); WriteToLog ("Overwriting attribute ID's for $matchRefFile"); close ( OutputFile ); } #=========================================# sub escapeQuotes { my ( $inString ) = @_; # Escape single quotes $inString =~ s/\'/\\'/g; # Escape double quotes $inString =~ s/\"/\\"/g; # Escape the @-symbol $inString =~ s/\@/\\@/g; return $inString; } #=========================================# sub readXmlFile { my ( $xmlDir, $xmlInputFile ) = @_; WriteToLog ( "Parsing: $xmlInputFile"); my $matchRefFile = $parser->parse_file ("$xmlDir/$xmlInputFile"); createNewXmlFile ( $xmlDir, $xmlInputFile, $xmlOutputFile ); my $qryElemName = ("/mdc/md/mi/mt"); my $qryElemValue = ("/mdc/md/mi/r"); for my $manElemList ( $matchRefFile->findnodes ( $qryElemName )) { my $elemNameID = $manElemList->getAttribute( "p"); foreach ($elemNameID) { my $elemValueList = ( $matchRefFile->findnodes ( $qryElemValu +e )); my $elemValueID = ( $elemValueList->getAttribute ( "p" )); if ($elemNameID == $elemValueID) { matchNodeDB ($elemNameID); WriteToLog ("Binding [$elemID] to [$elemNameID]"); WriteToFile ( $elemID, $elemNameID, $elemValueID, $xmlOutp +utFile ); } } WriteToLog ( "\n"); } WriteToLog ( "\n"); } #=========================================# + sub readXmlDir { my ( $xmlDirectory ) = @_; opendir (DIR, $xmlDirectory ) or die $!; while ( my $xmlFile = readdir( DIR ) ) { next unless ( -f "$xmlDirectory/$xmlFile" ); next unless ( $xmlFile =~ m/\.xml$/ ); WriteToLog ( "Matching: [$xmlFile] to database.\n"); readXmlFile ( $xmlDirectory, $xmlFile ); } closedir ( DIR ); } #=========================================# ## < MAIN ENTRY > WriteToLog( "RRD Tool Translate test Version: $version ( $copyright )\ +n"); WriteToLog( "Matching $xmlInputFile to Database: $databasename \n"); if ( 1 == connectDatabase() ) { readXmlDir( "/home/rrdtool") } else { WriteToLog ( "No Database connection established, exiting") }

Replies are listed 'Best First'.
Re: Copy XML file, write new attributes and move the old files. (bind to a variable)
by Anonymous Monk on Jan 26, 2015 at 08:05 UTC

    Now how do i copy an XML file and bind it to a variable?

    What does that mean "bind to a variable"?

      Hey sorry, struggling with the website. I think i overdid it abit codwise. With bind to a variable i mean to have the copied file "xmloutputfile" assigned to a variable to have it available for me to use in other subs.
Re: Copy XML file, write new attributes and move the old files.
by chacham (Prior) on Jan 26, 2015 at 14:47 UTC

    A couple comments on the SQL. One, is that syntax valid? I don't know mysql, but "SELECT IF NOT EXISTS" is not valid SQL. EXISTS() returns a binary value and thus must be used in a WHERE clause or CASE expression. SELECT ... WHERE EXISTS() is the standard method. If that is an IF statement instead, the SELECT would appear later: IF EXISTS() THEN SELECT ... ELSE INSERT....

    Two, the query is using variables to put create a dynamic SQL statement. Dynamic SQL is generally considered insecure. But, it can be easily avoided via placeholders and passing the values in the execute() statement.

      not really what the question was about, i mainly posted the entire code to not confuse anyone, cq give all the information possible, the SQL would be debugged next.

      but then again, this is very helpful information, so thanks alot for taking the time to help a newbie out. i will look into it.

Re: Copy XML file, write new attributes and move the old files.
by pme (Monsignor) on Jan 27, 2015 at 07:54 UTC
    Hi Nevamonk, welcome to the monastery!

    It seems to be that you process RRD's XML dump files. You may consider using perl modules to access RRD directly, eg. RRD::Simple

      thank you good sir =D

      I haven't really looked at the RRD handlers since the data delivered by the server is XML by default, and since this script is used before RRD will even come into the picture, i just stayed with XML::libXML.

      But! i have more of these issues, so i will definitly take a look at the RRD options, thank you for your help!

      hold on, i think i misunderstood the statement.

      they are indeed RRD dumpfiles, but the server providing the dump is beyond my control, so therefor i have to work with the xml files.

      and my apologies for the confusion

Re: Copy XML file, write new attributes and move the old files.
by poj (Abbot) on Jan 26, 2015 at 11:48 UTC
    What is the schema for the XMLelementen table ?
    poj

      id (prim key, auto increment)

      name (VARCHAR(255)

        In that case, here's a working script to demonstrate the basics of what I think you are trying to do. poj
Re: Copy XML file, write new attributes and move the old files.
by sundialsvc4 (Abbot) on Jan 26, 2015 at 17:50 UTC

    For any application such as this, I would suggest that you not alter any of the existing data in the incoming XML file in any way.   Instead, perhaps add a second top-level element to the XML file which describes exactly what you did:   the identity of each element that was replaced, the old and new values (or at least the old ones, since the new ones are elsewhere), and a date/time stamp in the top-level element indicating exactly when this was done.   Then, if you wish, it moves the file ... taking care not to overwrite any existing copy that might already be there, but instead to add a new one beside it.   (If you plan things right, existing utilities like Unix logrotate can be pressed into service if you need to clean-up sufficiently old copies.)

    If the program is run again, it has the option of looking for the existence of this new, sentinel element, to detect if it has already been run.   Or, it might add “yet another” top-level element to describe the outcome of this new run.   My point is that the entirety of this strategy is, by design, non-destructive.   None of the original data is lost, and the program records its own activities in a way that is also non-destructive to any previous records.

      thank you for your reply.

      that's exactly the point, the old data needs to be stored.

      there will be a clean up script in place as soon as this one is working.

      ofcourse the logs and print statements will dissapear, that is purely for debug purposes.

      i like your idea of copying and moving the files, i will try that as soon as possible, thanks again.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1114493]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (5)
As of 2024-03-29 07:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found