Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Seekers of Perl Wisdom

( #479=superdoc: print w/ replies, xml ) Need Help??

If you have a question on how to do something in Perl, or you need a Perl solution to an actual real-life problem, or you're unsure why something you've tried just isn't working... then this section is the place to ask. Post a new question!

However, you might consider asking in the chatterbox first (if you're a registered user). The response time tends to be quicker, and if it turns out that the problem/solutions are too much for the cb to handle, the kind monks will be sure to direct you here.

User Questions
Convert CSV file to XML file using Perl?
2 direct replies — Read more / Contribute
by laknarc
on Jul 03, 2015 at 21:10

    I have already posted this question as Anonymous monk. Please delete my previous thread. I have registered and created this question again. we have a CSV file which need be to converted to XML using Perl or Unix shell scripting. I was able to build this XML using SQL and Korn shell script. However, SQL/shell script is running for long time. Hence, I'm considering to write a Perl to generate this XML file. Basically need to build this XML hierarchically by avoiding the duplicate values. Please take a look into the CSV and XML file below. But the both SQL and Korn shell script takes 15 minutes to create the XML for just 6000 records. Please let me know if there is any other efficient way to process this CSV file. this CSV file contains total 20 million records. Need to build this XML file within 60 to 120 minutes. Appreciate your help. Input data below:-

    RLTP_ID,RLTP_NAME,PROD_ID,PROD_NAME,CUST_ID,CUST_NAME,ACC_ID,ACC_NAME, +ACC_BALANCE,TXN_ID,TXN_AMT,ACC_TYPE,ACC_CODE,TXN_CODE 10,Phil,1,Personal,2,Fixed,3,Savings,3000,4,500,X,YY,11 10,Phil,1,Personal,2,Fixed,3,Savings,3000,4,500,X,YY,12 10,Phil,1,Personal,2,Fixed,3,Savings,3000,4,500,X,ZZ,11 10,Phil,1,Personal,2,Fixed,3,Savings,3000,4,500,X,ZZ,12 10,Phil,1,Personal,2,Fixed,7,Savings,3000,8,500,X,AA,11 10,Phil,1,Personal,2,Fixed,7,Savings,3000,8,500,X,AA,12 10,Phil,1,Personal,2,Fixed,7,Savings,3000,8,500,X,AA,13 10,Phil,1,Personal,2,Fixed,7,Savings,3000,8,500,X,BB,11 10,Phil,1,Personal,2,Fixed,7,Savings,3000,8,500,X,BB,12 10,Phil,1,Personal,2,Fixed,7,Savings,3000,8,500,X,BB,13

    Output XML

    <transactiondetails> <rltp_id>10</rltp_id> <rltp_name>Phil</rltp_name> <product> <prod_id>1</prod_id> <prod_name>Personal</prod_name> <customer> <cust_id>2</cust_id> <cust_name>Fixed</cust_name> <account> <acc_id>3</acc_id> <acc_name>Savings</acc_name> <acc_balance>3000</acc_balance> <acc_type>X</acc_type> <acc_code>ZZ</acc_code> <acc_code>YY</acc_code> <transaction> <txntrack> <txn_id>4</txn_id> <txn_amt>500</txn_amt> <txn_code>11</txn_code> <txn_code>12</txn_code> </txntrack> </transaction> <transaction> <txntrack> <txn_id>5</txn_id> <txn_amt>500</txn_amt> <txn_code/> </txntrack> </transaction> <transaction> <txntrack> <txn_id>6</txn_id> <txn_amt>500</txn_amt> <txn_code/> </txntrack> </transaction> </account> <account> <acc_id>7</acc_id> <acc_name>Savings</acc_name> <acc_balance>3000</acc_balance> <acc_type>X</acc_type> <acc_code>AA</acc_code> <acc_code>BB</acc_code> <transaction> <txntrack> <txn_id>8</txn_id> <txn_amt>500</txn_amt> <txn_code>11</txn_code> <txn_code>12</txn_code> <txn_code>13</txn_code> </txntrack> </transaction> </account> </customer> </product> </transactiondetails>
Stopping script execution from within the code like Python's pdb
4 direct replies — Read more / Contribute
by schrodinger
on Jul 03, 2015 at 09:23
    I am a newcomer to Perl from Python. I am trying to debug an API call's Perl code, which is run via uWSGI. Currently, I'm using logs and `say`s to debug my code, but ideally, I would like to invoke the debugger at arbitrary locations in the code. In Python, I can do,
    import pdb pdb.set_trace()
    And the debugger will be invoked, pausing the script at the location of the set_trace() call. I'm sure there's an equivalent in Perl, but I'm just not able to find it. Please help :)
Another one perl LWP quastion
3 direct replies — Read more / Contribute
by m4merg
on Jul 03, 2015 at 07:25

    Hi, Monks!

    I'm trying to use LWP::UserAgent to get content from URL. I've succeed, but after some time URL just banned my IP (at least i think that they did, because I could not access to site at all). So i decided to use proxy, but i still can't get to this URL. Here is the code im using for this:

    my $ua = new LWP::UserAgent; my $proxy = "http://112.137.164.232:3128"; $ua->proxy(['https'], $proxy); $ua->agent('Mozilla/5.0'); my $links = "http://sci-hub.org/"; my $req = new HTTP::Request GET => $links; my $stuff = $ua->request($req); print "Content-type: text/html\n\n"; print $stuff->content;

    Using this code i can get content from another URL (for ex. google.com), but cant get it from the desired one (sci-hub.org). Also i actually can get access to the desired URL through web browser using this proxy (though i can't without proxy as they banned me), but can't get it from perl script:

    Can't connect to sci-hub.org:80 (Connection timed out) LWP::Protocol::http::Socket: connect: Connection timed out at /usr/sha +re/perl5/LWP/Protocol/http.pm line 41

    So, proxy is correct in general and code is correct in general (at least i think so), but for the desired URL it doesnt work. So what is the problem?

    Sorry if I duplicate another topic, but i've searched for the solution of my problem hard (on this site also), used a lot of different ways and didnt find anything usefull at the end and thanks in advance for any help.

"Type mismatch" Error While Using Win32:OLE for MS Outlook
3 direct replies — Read more / Contribute
by dushyant
on Jul 03, 2015 at 03:30

    Hi Monks,

    I am tring to download email attachments from Microsoft Outlook using Win32::OLE. But I stuck at initial level I don't even able to get outlook folder list.
    Please find below code I am using and Error I got.

    I am using,
    32 bit Straberry Perl ( perl 5, version 20, subversion 1 (v5.20.1) built for MSWin32-x86-multi-thread-64int),
    OS (64 bit) = Windows 7 Enterprise Service Pack 1
    Microsoft Outlook 2013

    Error :

    D:\Perl_Project\OutLook>test1.pl
    Win32::OLE(0.1712) error 0x80020005: "Type mismatch" in METHOD/PROPERTYGET "GetDefaultFolder" argument 1 at D:\Perl_Project\OutLook\test1.pl line 9.
    Can't call method "Folders" on an undefined value at D:\Perl_Project\OutLook\test1.pl line 15.
    D:\Perl_Project\OutLook>perl -v

    #! D:\Straberry\perl\bin\perl -w use strict; use Win32::OLE qw/in/; #use Win32::OLE::Const 'Outlook.Application'; #use Win32::OLE::Variant; my $OL = Win32::OLE->GetActiveObject('Outlook.Application') || Win32:: +OLE->new('Outlook.Application') or die "$!\n";; my $NameSpace = $OL->GetNameSpace("MAPI"); my $Folder = $NameSpace->GetDefaultFolder("olFolderInbox"); #print ref($Folder->{Items}) . "\n"; foreach my $subfolder ( in $Folder->Folders) { printf "%s\n", $subfolder->{Name}; }

    Thanks.
Perl expect : send command executing multiple times
1 direct reply — Read more / Contribute
by ryalagikar
on Jul 03, 2015 at 03:11

    I am writing perl-expect script, in script am trying to login to remote machine then sudo su to another user and then execute two commands. Everything is working fine but the first command is executing multiple times.

    my $ssh = Net::OpenSSH->new("$user:password\@$host"); $ssh->error and die "Unable to connect $host " . $ssh->error; my ($pty,$pid) = $ssh->open2pty({stderr_to_stdout => 1}, 'sudo su - us +er1') or die "Failed to attempt"; my $expect = Expect->init($pty); $expect->log_file("expect.pm_log", "w"); my $result = $expect->expect(3, ["\[\r\n]?\[^\r\n]+\[%#>\$] \$" ,sub { shift->send("da +te\r"); exp_continue;}], [qr/Day/ , sub { my $self = shift; $self->send("touch +test.txt\r"); $self->soft_close();}] )or die "**** Failed ** ";

    Here 'date' command is executing multiple times(2...20times).

    Suggestions Please

missing values when using Parallel::ForkManager
1 direct reply — Read more / Contribute
by plagent
on Jul 03, 2015 at 01:51
    I used Parallel::ForkManager to scan the motifs in DNA string pairs. The script is the following:
    foreach my $id (1..100) { PWM: for my $pwm (@ciona_pwm_array) { $pm->start and next PWM; ($count) = count_pwm_num($pwm,$seq_1,$seq_2); $pm->finish; } $pm->wait_all_children; }
    There are three parameters for the function count_pwm_num. The first one is $pwm, which is from list @ciona_pwm_array and is the motif feature. The second and third are DNA strings. The function is to calculate the motif number (how many?)using motif feature $pwm in the DNA strings. The aim of the function count_pwm_num is to report the motif number from both strings. The following table is the report from the program. However, I calculated the same DNA strings 100 times ( the first for-loop) and used same motif list (@ciona_pwm_array, the second for-loop), there is always missed motif(s) in the report. For example, row a (first row), the program reported all six motif number from A-F. row b missed the number of the motif A (because they were same DNA strings, the output should keep the same). row c missed the number of motif B.row d missed the number for motif A and B. It seemed the missed motif numbers were randomly selected. So my question is: why count_pwm_num randomly skip some $pwm and do not perform the function of count_pwm_num?

    program result table:

    id	A	B	C	D	E	F
    a	1	1	0	1	1	0
    b		1	0	1	1	0
    c	1		0	1	1	0
    d			0	1	1	0
    e	1	1	0	1	1	0
    ..
    
    Expected result
    
    id	A	B	C	D	E	F
    a	1	1	0	1	1	0
    b	1	1	0	1	1	0
    c	1	1	0	1	1	0
    d	1	1	0	1	1	0
    e	1	1	0	1	1	0
    ..
    
Howto: Comparator for B-Tree in Kyoto Cabinet
2 direct replies — Read more / Contribute
by Kraeze
on Jul 03, 2015 at 01:49

    Dear Monks,

    It is my first time asking a question here.

    Background: I have used the perl api for Kyoto Cabinet, given here: http://fallabs.com/kyotocabinet/. I opted to use the B Tree implementation and would like to write my own comparator. As the keys are sorted lexicographically by default, it does not suit my keys.

    My keys are stored as intervals e.g. "1-50", representing the start and end positions of bases in a biological sequence. Hence to order them I would compare only the first integer.

    The only information I have found in the API is the following paragraph:

    "The default record comparator is the lexical ordering function. That is, records in the B+ tree database are placed in the lexical order of each key. If you want to use another ordering, call `tune_comparator' to set a functor which implements the ordering function." Ref: Tuning the File Tree Database @ http://fallabs.com/kyotocabinet/spex.html

    Question:How do I write my own comparator in perl? Thank you in advance.

Aggregating the column based on the common column values
2 direct replies — Read more / Contribute
by kanskr
on Jul 03, 2015 at 00:08

    My data is like below

    Username | Roles | Type |date |SR NO|Remarks| abc|admin |added | 01072015 abc|developer |deleted |01072015 abc|deploy |added |01072015 xyz |admin |deleted |01072015 xyz| deploy|deleted|01072015 cdf|deploy|added|01072015

    and i need to do the following a) i need to group based on the user, roles and concatenate the roles filed with the delimiter "," b) I need to create the output as below.

    Username |roles added |roles deleted |date username |roles_added |roles deleted |date abc |admin,deploy |developer |01072015 xyz ||admin,deploy |01072015 cdf |deploy||01072015

    In order to achieve this,am using the code below

    use strict; use warnings; use diagnostics; use Data::Dumper; open(FIL,"report.txt") or die("$!"); my %k=(); while (my $line=<FIL>) { next if $. < 2; my ($user,$roles,$type,$dt,$empty1,$empty2)=split(/\|/,$line); push @{$k{$user}{$type}}, $roles; } my @names=(sort keys(%k)); foreach ( @names) { if ( @{$k{$_}{Added}} ne ''){ print "${k{user}}\n"; print join ',', @{$k{$_}{Added}}; print "\n"; } if ( @{$k{$_}{Deleted}} ne '') { print join ',', @{$k{$_}{Deleted}}; print "\n"; } }

    The script fails with below error Can't use an undefined value as an ARRAY reference at report3.pl line 34, <FIL> line 23. at report3.pl line 34 Please help me how i can generate the output as expected. Many thanks

[SOLVED] Ordering DBIx::Class joined searches by timestamp
No replies — Read more | Post response
by LunarCowgirl
on Jul 02, 2015 at 22:59

    Is it not possible to order joined searches by timestamp with DBIx::Class? I have a SQLite table with created and updated columns that are of the type timestamp. I have a search:

    sub get_latest_by_category { my ( $self, $id, $rows ) = @_; return $self->search( { 'category.id' => $id, 'me.published' => 1, }, { join => { 'artist_categories' => 'category' }, order_by => { -desc => 'me.created' }, rows => $rows, } ); }

    It returns the data sorted by the name column in ascending order, which is the first column after the primary key, rather than by the created column in descending order. I can order by created in non-joined searches, and if I change me.created to another column in the above search, it'll return the results in the correct sort order and direction. It just won't do it with a joined search ordered by a timestamp column.

    A DBIC_TRACE shows that it's reading the query correctly "ORDER BY me.created DESC". Is there something special about this type of search? Or if this is a limitation, is there an easy way around it?

    UPDATE: Ah, never mind. I discovered where the error was. It was a problem with the test database.
Max of 23 and 27 is 23?
6 direct replies — Read more / Contribute
by luxgladius
on Jul 02, 2015 at 13:35
    Here's the output of an actual debug session. Can anybody explain this?
    DB<10> p $#l 27 DB<11> p $#F 23 DB<12> p max($#l, $#F) 23 DB<13> p max(27,23) 27 DB<14> p 0+$#l 27 DB<15> p max(0+$#l, $#F) 27 DB<16> p max($#l, 0+$#F) 23
    So the second-to-last line tells me the solution, but I have no clue what the actual problem is. For what it's worth, I'm using 64-bit Activestate Perl 5.20.

    Edit: The max is from List::Util.


Add your question
Title:
Your question:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others cooling their heels in the Monastery: (6)
    As of 2015-07-04 06:12 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









      Results (57 votes), past polls