Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

HTTP PUT Request with separate values in csv

by StayCalm (Novice)
on May 16, 2018 at 13:30 UTC ( #1214656=perlquestion: print w/replies, xml ) Need Help??
StayCalm has asked for the wisdom of the Perl Monks concerning the following question:

Hallo People. I have one question about HTTP PUT Request with separate values in csv. In this code example I compare hashes, but for my PUT and POST request I need separate values.
For example, I have two output files from different Systems:

output1.csv (name, ip) - Primary system

TEST_1,10.56.7.80 TEST_3,10.66.78.10 TEST_4,10.66.81.9 TEST_2,10.67.9.12


output2.csv (id,name,ip) - Secondary System
01,TEST_1,10.56.7.80 02,TEST_3,10.66.251.9 03,TEST_5,10.66.81.9

My result should be:
  • I do nothing with Test1 (because it is already in System 2)
  • I should update Test3 (because now I have a different ip address)
  • I should update Test5-->Test4 (because now I have a different names with the same ip adress)
  • I should add Test2, because I do not have it in the secondary System
  • use strict; use warnings; use feature qw(say); use autodie; use XML::Twig; use LWP::UserAgent; use HTTP::Headers; use HTTP::Request; use File::Slurp; use JSON -support_by_pp; use LWP 5.64; use MIME::Base64; use IO::Socket::SSL; use File::Slurp; my ($file_1, $file_2) = ('output1.csv', 'output2.csv'); open my $fh, '<', $file_1 or die "Can't open $file_1: $!"; my %first = map { chomp; split /\s*,\s*/ } <$fh>; open $fh, '<', $file_2 or die "Can't open $file_2: $!"; my %second = map { chomp; (split /\s*,\s*/)[1,2] } <$fh>; #Create a user agent object my $ua = LWP::UserAgent->new(ssl_opts=> { SSL_verify_mode => SSL_VERIFY_NONE(), verify_hostname => 0, }); foreach my $name (sort keys %first) { if (not exists $second{$name}) { say "Devices should be added: $name"; next; } if ($first{$name} eq $second{$name}) { say "Match found $name, $first{$name}"; } else { say "UPDATE need be done for $second{$name}";

    Till this place my code works. I compare my hushes and I see messenges in shell.
    Now I want to make PUT (Update) request for these devices, those are nor identic

    Idee ist to open separate both of files and to add to Secondary the ipaddress or names from Primary System

    To make it this way I Need to take ip(or name) from the device from output1.csv and to make PUT request to secondary System (Update)

    open ( my $input_1, '<', 'output1.csv' ) or die $!; while ( <$input_1> ) { chomp; my ($name_1, $ip_1) = split /,/; (my $id, $first{$name}, $second{$name}) = split /,/; my $xml = XML::Twig -> new -> parsefile ( 'example.xml' ); $xml ->set_pretty_print('indented_a'); open ( my $input_2, '<', 'output2.csv' ) or die $!; while ( <$input_2> ) { chomp; $xml -> root -> set_att('name', $name_1); $xml -> get_xpath('//ipaddress',0) -> set_text($ip_1); my $uri="https://hostname:9060/ers/config/networkdevice/$id"; my $req = HTTP::Request->new('PUT', $uri, [Accept=>'application/vnd.com.cisco.ise.network.networkdevice.1.1 ++xml', Content_Type=>'application/vnd.com.cisco.ise.network.networkdevic +e.1.1+xml; charset=utf-8'], $xml->sprint); $req->content($xml->sprint); $req->authorization_basic("user", "user"); #Pass request to the user agent and get a response back #Pass request to the user agent and get a response back my $res = $ua->request($req); #Check the outcome of the response if ($res->is_success) { print $res->status_line, "\n"; } else { print $res->status_line, "\n"; } } } } }

    example.xml - it ist the structure of XML data for update(PUT Request)


    If I want to change Attribute, I add it in this file and make HTTP Request(POST, PUT) Wenn I start my code, I get this message:

    UPDATE need be done for 10.66.251.9 404 Not Found 404 Not Found 404 Not Found 404 Not Found 404 Not Found 404 Not Found Match found TEST_1,10.56.7.80

    But actually I Need this one :

    UPDATE need be done for 10.66.251.9 UPDATE need be done for 10.66.81.9 200 ok 200 ok Match found TEST_1,10.56.7.80 Devices should be added: Test2

    Replies are listed 'Best First'.
    Re: HTTP PUT Request with separate values in csv
    by Veltro (Pilgrim) on May 16, 2018 at 16:52 UTC

      I just have to say it is very difficult to read your code. Maybe you can indent it properly. People would be much more inclined to try and help you. Also, are we supposed to know what is in 'example.xml'. Posting complete code listings is also not very useful. Try to simplify your question. Reading your code I am absolutely clueless in what you are actually try to do.

      Nevertheless to help you on your way a little bit, try to focus on the actual problem. What is the problem? Is the problem in the part that examines the two csv files or is it in the server request. As a matter of fact that is where I would look first.

      Create a small sample code for testing and once you have that part working, then start working on the rest of your code.

      For example

      I suggest you use strict and warnings, it will help you identify problems in your code much sooner.

      use strict ; use warnings ;
      my $xml = XML::Twig->new()->parsefile('example.xml') ; $xml-> ... ; $xml -> get_xpath('//ipaddress',0) -> set_text("<IP HERE>") ;

      Write a test to check if the previous instruction worked. I don't know, are these of HTML::Element XML::Element? If so then print $xml->as_text() print $xml->as_XML could work. Once you know this worked, then move on:

      my $uri="https://hostname:9060/ers/config/networkdevice/<ID HERE>"; my $req = HTTP::Request->new( ... $req->content($xml->sprint) ; # Strange, why is there another $xml->sp +rint here? $req->...

      Now try to find more ways to introspect $req and see if it contains everything you want. Try to do a request and see if it works:

      my $res = $ua->request( $req ) ; # Check the outcome of the response if ($res->is_success) { print $res->status_line, "\n"; } else { print $res->status_line, "\n"; }

      edit: forgot to add the Twig parsefile instruction.

        Sorry,I've changed my posting. And about your question:
        I think my Problem is to separate my hush in right way:

        (my $id, $first{$name}, $second{$name}) = split /,/;

        There is no Problem with the server
        I've tested it already with this code:

        open ( my $input_1, '<', 'output1.csv' ) or die $!; while ( <$input_1> ) { chomp; my ($name_1, $ip_1) = split /,/; my $xml = XML::Twig -> new -> parsefile ( 'example.xml' ); $xml ->set_pretty_print('indented_a'); open ( my $input_2, '<', 'output2.csv' ) or die $!; while ( <$input_2> ) { chomp; (my $id, $first{$name}, $second{$name}) = split /,/; $xml -> root -> set_att('name', $name_1); $xml -> get_xpath('//ipaddress',0) -> set_text($ip_1); my $uri="https://hostname:9060/ers/config/networkdevice/$id"; my $req = HTTP::Request->new('PUT', $uri, [Accept=>'application/vnd.com.cisco.ise.network.networkdevice.1.1 ++xml', Content_Type=>'application/vnd.com.cisco.ise.network.networkdevic +e.1.1+xml; charset=utf-8'], $xml->sprint); $req->content($xml->sprint); $req->authorization_basic("user", "user");

        It works, but it works wrong!This code updates every device (also identical devices) And I have this message.After Update it tries to do smth(I do not understand what)and I become 400,405 Http Status. I also get warnings:

        UPDATE need be done for 10.56.7.80 200 OK 400 Bad Request Use of uninitialized value $id in concatenation (.) or string at My_Fi +le line 65, <$_[...]> line 3. 405 Method Not Allowed Use of uninitialized value $id in concatenation (.) or string at My_Fi +le line 65, <$_[...]> line 4. 405 Method Not Allowed 400 Bad Request

        10.56.7.80 example ip address

        Line 65: my $uri="https://10.66.1.16:9060/ers/config/networkdevice/$id +";

          OK, but you are still not trying to do what I suggested.

          Minimize your code and create much smaller pieces of test code so you will be able to learn how to debug your own code.

          I'll give you another example what I am talking about. In the next piece of text code I have found multiple problems, so play around with it a little bit and try to fix it yourself:

          use strict; use warnings; use Data::Dumper; my ($file_1, $file_2) = ('output1.csv', 'output2.csv'); open my $fh, '<', $file_1 or die "Can't open $file_1: $!"; my %first = map { chomp; split /\s*,\s*/ } <$fh>; print Dumper( \%first ) ; open $fh, '<', $file_2 or die "Can't open $file_2: $!"; my %second = map { chomp; (split /\s*,\s*/)[1,2] } <$fh>; print Dumper( \%second ) ; foreach my $name (sort keys %first) { if (not exists $second{$name}) { print "Devices should be added: $name\n"; next; } if ($first{$name} eq $second{$name}) { print "Match found $name, $first{$name}\n"; } else { print "UPDATE need be done for $second{$name}\n"; open ( my $input_1, '<', 'output1.csv' ) or die $!; while ( <$input_1> ) { chomp; print " input_1 = $_\n" ; my ($name_1, $ip_1) = split /,/; print " (1) $name, $ip_1\n" ; (my $id, $first{$name}, $second{$name}) = split /,/; print " (2) $id, $first{$name}, $second{$name}\n" ; } } }
    Re: HTTP PUT Request with separate values in csv
    by poj (Monsignor) on May 16, 2018 at 19:31 UTC

      How does this statement

      My result should be: I do nothing with Test1 (because it is already in System 2)

      reconcile with your expected result ?

      UPDATE need be done for 10.66.254.1

      poj

        Was a mistake, I changed

          Consider dividing the task into 2 steps so that you can debug each independently. First identify the updates required and store them in arrays ;

          #!/usr/bin/perl use strict; use warnings; use LWP::UserAgent; use XML::Twig; use autodie; use Data::Dumper; my @filename = ('output1.csv', 'output2.csv'); my %file1 = () ; open my $fh, '<', $filename[0]; # using autodie while (<$fh>){ chomp; my ($name,$ip) = split /\s*,\s*/; $file1{$name} = $ip; } close $fh; #%file1 = ( # TEST_1 => '10.56.7.80', # TEST_3 => '10.66.78.10', # TEST_4 => '10.66.81.9', # TEST_2 => '10.67.9.12', #); my %file2 = () ; my %ip2name = (); open $fh, '<', $filename[1]; # using autodie while (<$fh>){ chomp; my ($id,$name,$ip) = split /\s*,\s*/; $file2{$name} = [$id,$ip]; $ip2name{$ip} = [$id,$name]; } close $fh; #%file2 = ( # TEST_1 => ['01','10.56.7.80'], # TEST_3 => ['02','10.66.251.9'], # TEST_5 => ['03','10.66.81.9'], #); print Dumper \%file1,\%file2,\%ip2name; # determine updates and additions requires my @updateIP =(); my @updateName=(); my @add = (); foreach my $name (sort keys %file1) { my $ip1 = $file1{$name}; if ( not exists $file2{$name} ){ # check if ip exists if ( not exists $ip2name{$ip1} ){ print "ADD name : '$name' '$ip1'\n"; push @add,[undef,$name,$ip1]; } else { my $id2 = $ip2name{$ip1}[0]; my $name2 = $ip2name{$ip1}[1]; print "UPDATE NAME change id '$id2' name from '$name2' to '$nam +e'\n"; push @updateName,[$id2, $name, $ip1]; } next; } my $ip2 = $file2{$name}[1]; if ($ip1 eq $ip2) { print "MATCHED device '$name', '$ip1'\n"; } else { my $id2 = $file2{$name}[0]; print "UPDATE IP change '$name' id '$id2' ip from '$ip2' to '$ip1' + \n"; push @updateIP,[$id2,$name,$ip1]; } } print Dumper \@add,\@updateIP,\@updateName;

          When that is working add on implementing the changes

          # read XML template my $xml = XML::Twig->new->parse( \*DATA ); $xml->set_pretty_print('indented_a'); #Create a user agent object my $ua = LWP::UserAgent->new( ssl_opts=> { # SSL_verify_mode => SSL_VERIFY_NONE, verify_hostname => 0,} ); # apply updates my $uri = "https://hostname:9060/ers/config/networkdevice/"; my $header = [ Accept => 'application/vnd.com.cisco.ise.network.networkdevice.1.1+x +ml', Content_Type => 'application/vnd.com.cisco.ise.network.networkdevice +.1.1+xml; charset=utf-8']; for (@updateIP){ my ($id,$name,$ip) = @$_; print "----- Updating $id, $name to $ip\n"; $xml->root->set_att('name', $name); $xml->get_xpath('//ipaddress',0)->set_text($ip); my $req = HTTP::Request->new('PUT', $uri.$id, $header, $xml->sprint) +; $req->authorization_basic("user", "user"); print $req->as_string; # testing #my $res = $ua->request($req); #if ($res->is_success) { # print $res->status_line, "\n"; #} else { # print $res->status_line, "\n"; #} print "-----\n" } __DATA__ <device name="example"> <ipaddress>0.0.0.0</ipaddress> </device>

          Note, I have only shown the ip updates, the additions and name changes you need to complete yourself

          poj

    Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Node Status?
    node history
    Node Type: perlquestion [id://1214656]
    Approved by marto
    Front-paged by kcott
    help
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others imbibing at the Monastery: (5)
    As of 2018-11-13 02:14 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?
      My code is most likely broken because:
















      Results (149 votes). Check out past polls.

      Notices?