Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Seekers of Perl Wisdom

( #479=superdoc: print w/ replies, xml ) Need Help??

If you have a question on how to do something in Perl, or you need a Perl solution to an actual real-life problem, or you're unsure why something you've tried just isn't working... then this section is the place to ask. Post a new question!

However, you might consider asking in the chatterbox first (if you're a registered user). The response time tends to be quicker, and if it turns out that the problem/solutions are too much for the cb to handle, the kind monks will be sure to direct you here.

User Questions
Testing private methods a no-no?
3 direct replies — Read more / Contribute
by nysus
on May 20, 2016 at 10:22

    I'm new to testing. I've got a test that uses a call to a private method like so:

    # Check that module loads data my $xyz_events = $checker->_get_event_data('arg'); my $is_hash_ref = ref $xyz_events eq 'HASH'; is $is_hash_ref, 1, 'returns a hash ref';

    I'm thinking it's OK to break the rules a bit since I'm just running tests. It also seems like it makes writing tests easier by testing small components of the module. But I worry it's a sign that I'm going about testing in the wrong way. What is the opinion of the wiser Monks about this?

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

DBI::CSV using a variable to request columns
3 direct replies — Read more / Contribute
by Sandy_Bio_Perl
on May 20, 2016 at 10:21

    Hello Monks. I would like to use a variable "$columns" to describe the columns I wish to extract. Perl will not allow me to store, for example, $column = "sid = $row->{sid}"; because it demands that I declare $row

    I have pasted the troublesome code below. Thank you, in advance for your perls!

    use strict; use warnings; use Text::CSV; use DBI; use Data::Dumper; # Connect to the database, (the directory containing our csv file(s)) my $dbh = DBI->connect ("dbi:CSV:", undef, undef, { f_dir => ".", f_ext => ".csv", f_encoding => "utf-8", RaiseError => 1, }) or die $DBI::errstr; # Output using sql query my $query = (qq(select * from newCsv.csv WHERE gender ='male' AND geno +type ='a' )); # my $columns = "sid = $row->{sid} \tgender = $row->{gender}\n"; # per +l is seeking declaration of $row my $queryResult; my $sth = $dbh->prepare($query); $sth->execute; while (my $row = $sth->fetchrow_hashref) { my $col= "sid = $row->{sid} \tgender = $row->{gender}\n"; # cannot ch +ange this to my $col=$columns $queryResult = $queryResult.$col; } $sth->finish(); $dbh->disconnect(); print "Result \n\n$queryResult";
convert columns into matrix and get population count?
2 direct replies — Read more / Contribute
by fasoli
on May 20, 2016 at 10:05
    Dear Monks,

    I am looking to find a way to convert a 2 column / 100,000 line file into a matrix. In this file, the columns represent X and Y values.

    What I want to get is the following matrix, like so:

    If I have x coordinates x1, x2, x3 and y coordinates y1, y2, y3, I want to get all possible combinations, like x1,y1, x1,y2, x1,y3, x2,y1, x2,y2, x2,y3, x3,y1,vx3,y2, x3,y3 - in the form of a matrix but I can't format it properly for this post :'(

    I also want to get a population, which would be a count of how many times the coordinate (for example) (56,6 , 85,5) occurs and divide this by 100,000.

    A few questions as I don't know how to begin:

    1. the coordinates have 10 decimal points and I'd like to maybe round them up at 2, is there a way to do that before calculating the matrix?

    2. is there a Perl module specifically for matrices? Does this http://docstore.mik.ua/orelly/perl/advprog/ch02_02.htm seem like a link that would help me? Also I tried some things from here https://www.biostars.org/p/60546/, specifically I tried this one-liner

    perl -lane '$cnt{$_}++; END { foreach $key (keys %cnt) {print "$key\t$cnt{$key}"} }' input.txt

    do you think it could be doing what I want? I know it's not giving me any population details but I'm not even sure if this code is for combining all coordinates?

    IThe reason why I need this is to be able to plot a heat map later on with gnuplot.

    Also another question for a potential gnuplot/Perl specialist here, is there a way for gnuplot to maybe "interpret" or count the occurrences on its own, without me having a population column or something like that in the file? Maybe if the input coordinates are already in the form of a matrix, gnuplot is able to count them and push this result into a heat map, representing the fractional population of the data?

    I have a feeling that it might be a matter of "simply" having loops going through all entries or something like that? I would appreciate any hints/information as I don't know how to begin, I don't expect you to try and write the code for me but some guidance would be more than welcome.

    Thank you all very much in advance, any information would be greatly appreciated.

portMIDI wrapper for Perl5
2 direct replies — Read more / Contribute
by perltux
on May 20, 2016 at 09:10
    I just found out that somebody has written a Perl6 wrapper for the portMIDI library, see here:
    https://github.com/Perl6-Noise-Gang/Audio-PortMIDI/
    The original library is here:
    http://portmedia.sourceforge.net/portmidi/

    Since I would very much like to be able to use portMIDI with Perl5 I would like to know how hard it would be to port the Perl6 portMIDI wrapper to Perl5.

    I only have average Perl5 programming experience, but have zero experience with Perl6 and have never written wrappers for C libraries (and don't have C/C++ progamming skills).
Help stop brute force
4 direct replies — Read more / Contribute
by Anonymous Monk
on May 20, 2016 at 08:51
    The last four months I have been noticing a large amount of bots trying to brute force wordpress on domains that do not run wordpress.
    They use a very unknown Perl web framework that has a login of its own (with absolutely no brute force protection in any way unless you count captcha), but since the brute force was just for wordpress the Perl system's are left unscathed.

    The first course of action was to ban it from the sites, that was easy. But after a few months of reviewing the logs and finding this IP constantly trying the same thing. Well, that made me a little mad so I started redirecting this IP to a 34 terabyte file. That slowed it's requests to about 8 per day instead of 50 a day.
    When I searched the IP and find out more information. There was a few reports on the reason why they try to brute force wordpress is to hold the website for ransom.

    I don't expect any of your answers to be like mine. How to slow brute force with spam. But by redirecting the IP to another server with a huge file did increase the servers resources that where being attacked.

    Ideally i would like a solution that could slow the brute force requests and reduce the load the attacks have on the servers.
    Could sleep(9000000); work better?
    Could captcha in the login form be reliable enough to stop brute force ?
find the coverage of sequence in a particular range
3 direct replies — Read more / Contribute
by Anonymous Monk
on May 20, 2016 at 07:04
    I have a sequence file seq.txt which gives each sequence of length 150bp at the specific position.
    seq.txt position range seq 46 1..1524 ----------------- 832 1..1524 ----------------- 1008 1..1524 ----------------- 1407 1..1524 ----------------- 2360 2052..3260 ------------------ 2967 2052..3260 ------------------ 403 1..1524 ----------------- 800 1..1524 ----------------- 2986 2052..3260 ------------------ 3170 2052..3260 ------------------
    I want to find out how much sequence is covered within a particular range. For example range 1...1524 is reported with 6 positions 46, 403, 800, 832, 1008, 1407 where there is an overlap between 800 and 832. The output should give (150+150+32+118+32+150+150) = 782 With my script I am trying to create an array which stores 1 to 1524. Making the array 1 if it contains the position+150 else the array is set to 0. But there is a problem with the loop I think which is not giving the correct output. Any help or idea will be appreciated. My script is:
    use strict; use warnings; my $i; my $j; my $seq; open my $m, '<', 'count_try.txt' or die 'Cannot open count_try.txt'; while ($seq = <$m>) { chomp $seq; my (@range) = split(/\t/, $seq); my ($fm, $to) = split(/\.\./, $range[1]); my $r = $range[0]; for ($i = $fm; $i <= $to; $i++) { $j = $r + 150; if ($i >= $r and $i <= $j) {$range[$i] = 1;} else {$range[$i] = 0;} print $range[$i]."\t"; } } close $m;
Hex-matching Regex pattern in scalar
3 direct replies — Read more / Contribute
by CliffG
on May 20, 2016 at 05:24
    Hello Monks, I need some guidance. I am trying to write a little subroutine that takes before and after values and creates a substitution expression, so I can replace non-displayable characters in a file with displayable ones for example. However I cannot make this work with hex values; what I'm attempting looks like this (which replaces hex 'e9' with char '\x65'):
    my $pre = '\xe9'; my $post = '\x65'; my $re = qr/s\/$pre\/$post\/g/; my $path = 'C:\Scripts\Working2'; my $fileSep = "\\"; my $file = 'Users_0.xml'; my $tempFile = 'C:\Scripts\Working2\Users2.xml'; if (!open(IF, "<$path$fileSep$file")) { die("Could not open file $path$fileSep$file $!"); } if (!open(OF, ">$tempFile")) { die("Could not open file $tempFile $!"); } while($str = <IF>) { $str =~ $re; print OF $str; } close IF; close OF;
    I know the substitution '$str =~ s/\xe9/\x65/g;' works just fine but parameterising it is the problem. Straightforward for char strings but seemingly less so for hex ones ... Can someone please advise?
Changing "filename" of eval-ed code
2 direct replies — Read more / Contribute
by perlancar
on May 20, 2016 at 04:18

    I use quite a bit of generated code in my program (meaning Perl code that is assembled in a string then eval-ed), for example, when generating argument validator code using Data::Sah.

    This turns out to be rather annoying when debugging warning/error messages. Perl only shows the filename as "(eval NUMBER)", for example:

    perl -wE'eval "1+1; 1"; eval "\n; 1+1; 2"' Useless use of a constant (2) in void context at (eval 1) line 1. Useless use of a constant (2) in void context at (eval 2) line 2.

    It would be slightly more helpful if there were a way to change the filename into something more descriptive, so the warn/error message can be something like:

    Useless use of a constant (2) in void context at (validator code for X + generated by Foo/Bar.pm line 234) line 1. Useless use of a constant (2) in void context at (validator code for Y + generated by Foo/Baz.pm line 99) line 2.
Convert CSV to XML
3 direct replies — Read more / Contribute
by andreas1234567
on May 20, 2016 at 02:58
    Dear Monks,
    I am tasked to convert CSV to XML and have turned to the not-updated-in-15-years XML::CSV. It has a number of issues - lack on support for accented characters and poor error handling are some of them. What would you recommend for converting CSV into a custom, possibly rather complex XML format?
    --
    No matter how great and destructive your problems may seem now, remember, you've probably only seen the tip of them. [1]
dbi::CSV works with text, but not numbers
3 direct replies — Read more / Contribute
by Sandy_Bio_Perl
on May 19, 2016 at 17:35

    Dear Monks, I thought I had stumbled across a great module that would allow me to apply sql queries to a spreadsheet saved in csv format. The queries work well when the cells of the csv file contain text, but not when they contain numbers, producing the error message "Use of uninitialized value in concatenation (.) or string at"

    I have attached the relevant bits of my code below. Any help would be hugely apprec1ated. Thank you

    # Connect to the database, (the directory containing our csv file(s)) my $dbh = DBI->connect ("dbi:CSV:", undef, undef, { f_dir => ".", f_ext => ".csv", f_encoding => "utf-8", RaiseError => 1, }) or die $DBI::errstr; # Output the name and contact field from each row my $sth = $dbh->prepare (qq(select * from newCsv.csv)); $sth->execute; while (my $row = $sth->fetchrow_hashref) { print "row = $row->{sid} \tgender = $row->{gender} \t\tgenotype += $row->{name} \n"; ## this works well all cells contain only text print "test group = $row->{tGroup} \n"; ## creates 'use of unini +tialized value in concatenation (.)... etc. tGroup contains numbers } $sth->finish(); $dbh->disconnect();

Add your question
Title:
Your question:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others meditating upon the Monastery: (8)
    As of 2016-05-24 19:51 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?