Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Meditations

( [id://480]=superdoc: print w/replies, xml ) Need Help??

If you've discovered something amazing about Perl that you just need to share with everyone, this is the right place.

This section is also used for non-question discussions about Perl, and for any discussions that are not specifically programming related. For example, if you want to share or discuss opinions on hacker culture, the job market, or Perl 6 development, this is the place. (Note, however, that discussions about the PerlMonks web site belong in PerlMonks Discussion.)

Meditations is sometimes used as a sounding-board — a place to post initial drafts of perl tutorials, code modules, book reviews, articles, quizzes, etc. — so that the author can benefit from the collective insight of the monks before publishing the finished item to its proper place (be it Tutorials, Cool Uses for Perl, Reviews, or whatever). If you do this, it is generally considered appropriate to prefix your node title with "RFC:" (for "request for comments").

User Meditations
XS: How to call another Perl function/method with same arguments
1 direct reply — Read more / Contribute
by etj
on Jun 09, 2024 at 11:48
    Recently in PDL I was reimplementing often-used Perl functions in XS for raw speed gainz. A problem I dealt with was effectively dispatching to another Perl method in some circumstances with the same Perl arguments, as efficiently as possible.

    I'll leave out how it took a couple of days of experimenting to figure this out; that wasn't due to the dispatching technique being wrong, but a simple logic error a bit further up the function. If there's any value to mentioning that here, it's to check your assumptions (possibly with printf) when things don't go to expectations.

    The obvious way to do this, according to https://perldoc.perl.org/perlcall, is to pass the G_NOARGS flag. However, that is (somewhat obscurely) documented as not being a good idea to use with perl_call_method. The technique I settled on, permalinked at https://github.com/PDLPorters/pdl/blob/cd5ffff1b85ff81effefc2085067e9d71801efd6/Basic/Core/Core.xs#L199-L220:

    SP -= items; PUSHMARK(SP); SPAGAIN; /* these pass this set of ar +gs on */ int retvals = perl_call_method("new", G_SCALAR); SPAGAIN; if (retvals != 1) barf("new returned no values"); RETVAL = POPs;
    For completeness, this is the full text of the function:
    SV * topdl(klass, arg1, ...) SV *klass; SV *arg1; CODE: if (items > 2 || (!SvROK(arg1) && SvTYPE(arg1) < SVt_PVAV) || (SvROK(arg1) && SvTYPE(SvRV(arg1)) == SVt_PVAV) ) { SP -= items; PUSHMARK(SP); SPAGAIN; /* these pass this set of ar +gs on */ int retvals = perl_call_method("new", G_SCALAR); SPAGAIN; if (retvals != 1) barf("new returned no values"); RETVAL = POPs; } else if (SvROK(arg1) && SvOBJECT(SvRV(arg1))) { RETVAL = arg1; } else { barf("Can not convert a %s to a %s", sv_reftype(arg1, 1), SvPV_n +olen(klass)); } SvREFCNT_inc(RETVAL); OUTPUT: RETVAL
    EDIT to explain the magic a bit more: SP is the local copy (which Perl does for efficiency when you repeatedly push arguments) of the global current "Perl stack pointer" (as I am calling it here): the address of ST(items-1) for the next function called, i.e. the top of the current stack frame. When you PUSHMARK an address, that will be ST(0) for the next Perl function that gets called. items for that next function will be the global "Perl stack pointer" minus that next function's ST(0) (plus 1). SPAGAIN copies the global "Perl stack pointer" into SP - I am carefully not saying "copies back", because of the faintly tricky stuff done here.

    Breaking the code into individual statements, with comments for each:

    SP -= items; /* make the local SP point at our ST(0) */ PUSHMARK(SP); /* make the next function's ST(0) be that */ SPAGAIN; /* reset our local SP to the global "top of current stack" */ int retvals = perl_call_method("new", G_SCALAR); /* call method so has + right stack frame */ SPAGAIN; /* set our local SP to the global "top of current stack" */ if (retvals != 1) barf("new returned no values"); /* use croak in non- +PDL code */ RETVAL = POPs; /* set the SV* to the return value */
    You may think, as I have just realised in writing this, that the first 3 lines do unnecessary work, since PUSHMARK(SP - items) would valuably replace them. You would be right! And I have just done this. However, leaving this example as it is still seems valuable (and the generated machine code is likely to be extremely similar), so I am doing so.
a nifty utility script from chatgpt
4 direct replies — Read more / Contribute
by Aldebaran
on Jun 09, 2024 at 04:18

    Hey monks,

    I'm backing up files with chatgpt and perl. I got a pretty good utility program out of it (them, whatever). It's a utility to sort pics and convert .heic to .jpg. I happen to be looking for a particular picture I took in 2021, so this is really helpful. Typical output:

    Moved and renamed: /home/fritz/Pictures/6.pics/2022-12-08-220201009.mp +4 -> /media/hogan/175F-DC61/Organized_Pics/2022/12/2022_12_08_5.mp4 Skipping: Wallpapers (not a file) Processing file: Screenshot from 2022-10-16 16-58-24.png File path: /home/fritz/Pictures/Wallpapers/Screenshot from 2022-10-16 +16-58-24.png Failed to delete original file /home/fritz/Pictures/Wallpapers/Screens +hot from 2022-10-16 16-58-24.png: No such file or directory Moved and renamed: /home/fritz/Pictures/Wallpapers/Screenshot from 202 +2-10-16 16-58-24.png -> /media/hogan/175F-DC61/Organized_Pics/2022/10 +/2022_10_16_13.png Processing complete! Number of files created: 960 fritz@laptop:~/Documents$

    Source:

    #!/usr/bin/perl use strict; use warnings; use File::Find; use File::Path qw(make_path); use File::Copy qw(move); use POSIX qw(strftime); # Define the source and target directories my $source_dir = '/home/fritz/Pictures'; my $target_dir = '/media/hogan/175F-DC61/Organized_Pics'; # Check if the source directory exists unless (-d $source_dir) { die "Source directory $source_dir does not exist.\n"; } # Create the target directory if it doesn't exist unless (-d $target_dir) { make_path($target_dir) or die "Failed to create target directory $ +target_dir: $!\n"; print "Created target directory: $target_dir\n"; } # Hash to store file type counts my %file_types; my $file_count = 0; # Subroutine to process each file sub process_file { if (-f $_) { my ($ext) = $_ =~ /\.([^.]+)$/; $ext = lc $ext if defined $ext; # Convert to lowercase if (defined $ext && $ext =~ /^(jpg|jpeg|png|gif|bmp|tiff|heic| +mp4)$/) { print "Processing file: $_\n"; my $file_path = $File::Find::name; print "File path: $file_path\n"; my $mod_time = (stat($file_path))[9]; my $date = strftime "%Y_%m_%d", localtime($mod_time); my ($year, $month, $day) = split('_', $date); my $dest_dir = "$target_dir/$year/$month"; unless (-d $dest_dir) { make_path($dest_dir) or die "Failed to create destinat +ion directory $dest_dir: $!\n"; print "Created destination directory: $dest_dir\n"; } my $count = 1; my $new_file_name; if ($ext eq 'heic') { $new_file_name = "${year}_${month}_${day}_$count.jpg"; while (-e "$dest_dir/$new_file_name") { $count++; $new_file_name = "${year}_${month}_${day}_$count.j +pg"; } my $converted_file_path = "$dest_dir/$new_file_name"; my $convert_command = "heif-convert $file_path $conver +ted_file_path"; system($convert_command) == 0 or die "Failed to conver +t $file_path: $!\n"; unlink $file_path or warn "Failed to delete original f +ile $file_path: $!\n"; print "Converted and moved: $file_path -> $converted_f +ile_path\n"; } else { $new_file_name = "${year}_${month}_${day}_$count.$ext" +; while (-e "$dest_dir/$new_file_name") { $count++; $new_file_name = "${year}_${month}_${day}_$count.$ +ext"; } move($file_path, "$dest_dir/$new_file_name") or die "F +ailed to move file $file_path: $!\n"; unlink $file_path or warn "Failed to delete original f +ile $file_path: $!\n"; print "Moved and renamed: $file_path -> $dest_dir/$new +_file_name\n"; } $file_count++; } else { print "Skipping file: $_ (not an image)\n"; } } else { print "Skipping: $_ (not a file)\n"; } } # Find and process files in the source directory print "Starting to process files in $source_dir...\n"; find(\&process_file, $source_dir); print "Processing complete! Number of files created: $file_count\n";

    What bells and whistles might we coach it to add?

    Cheers from my own private Idaho,

SO and AI
8 direct replies — Read more / Contribute
by stevieb
on May 20, 2024 at 02:53

    Stack just monetized all of the data that we, as public advocates of free information, provided.

    Is Perlmonks going to do this? If it is, I want all of my content removed immediately. I do not agree to the knowledge I've learned from those before me and I've subsequently shared being sold to anyone without due attribution.

    If Perlmonks plans on selling its user data to anyone, I outright refuse to take part, and want my data to be excluded entirely.

    -stevieb

Bizarre copy of ARRAY etc - solved
No replies — Read more | Post response
by etj
on May 13, 2024 at 17:07
    A simple XS function in PDL, firstvals_nophys, was giving "panic: attempt to copy freed scalar", but only if called on a complex-valued ndarray. The aim of this post is to appear when a despairing XS programmer googles that message, and give them another thing to check. When constructing a test to capture this, another message that appeared was "Bizarre copy of ARRAY". This is the old text of the function:
    void firstvals_nophys(x) pdl *x PPCODE: if (!(x->state & PDL_ALLOCATED)) barf("firstvals_nophys called on +non-ALLOCATED %p", x); PDL_Indx i, maxvals = PDLMIN(10, x->nvals); EXTEND(SP, maxvals); for(i=0; i<maxvals; i++) { PDL_Anyval anyval = pdl_get_offs(x, i); if (anyval.type < 0) barf("Error getting value, type=%d", anyval +.type); SV *sv = sv_newmortal(); ANYVAL_TO_SV(sv, anyval); PUSHs(sv); }
    The problem was that the ANYVAL_TO_SV macro was, only for complex-valued data, calling a Perl function to create a Math::Complex object (well, a subclass thereof because the overloads were wrong). That obviously uses the top of the stack, including writing values into it, and reading values out of it, including mortal ones that then got freed because they were done with. Therefore, the function was returning with some garbage on the stack, but the last value was correct.

    The solution was simply to do a PUTBACK after the PUSH, which moves the top of the stack above data we care about. The new text of the function with that:

    void firstvals_nophys(x) pdl *x PPCODE: if (!(x->state & PDL_ALLOCATED)) barf("firstvals_nophys called on +non-ALLOCATED %p", x); PDL_Indx i, maxvals = PDLMIN(10, x->nvals); EXTEND(SP, maxvals); for(i=0; i<maxvals; i++) { PDL_Anyval anyval = pdl_get_offs(x, i); if (anyval.type < 0) barf("Error getting value, type=%d", anyval +.type); SV *sv = sv_newmortal(); ANYVAL_TO_SV(sv, anyval); PUSHs(sv); PUTBACK; }
    The commit that fixed this is at https://github.com/PDLPorters/pdl/commit/68389413537c6ea7ed85f121a580b3b008ba82a6. The docs on how to call a Perl function from C are at https://perldoc.perl.org/perlcall, with a full explanation of PUSHMARK, PUSH*, PUTBACK, and (after the call) SPAGAIN, and maybe POP* and maybe then PUTBACK.
Debugger issue solved (two years ago)
2 direct replies — Read more / Contribute
by talexb
on May 06, 2024 at 20:57

    I upgraded a machine to the latest Ubuntu and got Perl 5.34.0, which included a problem with the debugger:

    DB<4> v + Undefined subr +outine &DB::cmd_l called at /usr/share/perl/5.34/perl5db.pl line 6034 +. at /usr/share/perl/5.34/perl5db.pl line 6034. + DB::cm +d_v("v", "", 56) called at /usr/share/perl/5.34/perl5db.pl line 4798 DB::cmd_wrapper("v", "", 56) called at /usr/share/perl/5.34/pe +rl5db.pl line 4311 DB::Ob +j::_handle_cmd_wrapper_commands(DB::Obj=HASH(0x55e85838c150)) called +at /usr/share/perl/5.34/perl5db.pl line 32 00 + DB::DB + called at report_warnings.pl line 56 Debugged program terminated. Use q to quit or R to restart, + use o inhibit_ +exit to avoid stopping after program termination, h q, h R or h o to get additional info. DB<4>
    My friend Google brought me to this page that had links to the necessary patch to perl5db.pl on github, bringing the script from version 1.60 to 1.60_01.

    And, of course, the patch worked just fine. What a great community. Thanks for the patch!

    PS: Ugh, sorry -- the problem was that the v command that I use a lot (Where am I? Oh, there I am!) crashed the debugger. The patch solves that problem.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

The Virtue of Laziness
4 direct replies — Read more / Contribute
by jo37
on May 04, 2024 at 17:57

    Dear Monks and Nuns,

    lately I came across an issue with dereferencing array refs. It looks like there is some hidden "lazy deref" when an array ref is used in a foreach loop, compared to the usage as a sub argument. Consider these two subs that do nothing but die. The main difference is the array dereference as an argument to foreach or map.

    use experimental 'signatures'; sub map_die ($ar) { eval {map {die} @$ar}; } sub for_die ($ar) { eval { for (@$ar) { die; } } }
    Benchmarking is amazing:
    use Benchmark 'cmpthese'; my @arr = ((0) x 1e6); cmpthese(0, { map => sub {map_die(\@arr)}, for => sub {for_die(\@arr)}, }); __DATA__ Rate map for map 1257/s -- -100% for 1664823/s 132352% --

    Then I remembered the "lazy generators" from List::Gen and gave it a try. There is some progress, but it cannot come up to foreach.

    use List::Gen 'array'; sub gen_die ($ar) { eval { &array($ar)->map(sub {die}); } } cmpthese(0, { map => sub {map_die(\@arr)}, for => sub {for_die(\@arr)}, gen => sub {gen_die(\@arr)}, }); __DATA__ Rate map gen for map 1316/s -- -93% -100% gen 18831/s 1330% -- -99% for 1662271/s 126174% 8727% --

    Wouldn't it be nice to have some kind of "explicit lazy dereferencing" in Perl?

    Update May 10, 2024: Added the signatures feature as suggested by Danny in Re: The Virtue of Laziness.

    Greetings,
    🐻

    $gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$
Imager support for PNG, JPEG and GIF on macOS Sonoma
1 direct reply — Read more / Contribute
by Anonymous Monk
on Apr 20, 2024 at 22:51
    Installing our beloved Imager on macOS Sonoma with support for PNG, JPEG and GIF involves some pain. Here are the results of my struggle so others won't have to. Do this after installing Imager (requires Homebrew):
    brew install pkg-config
    
    brew install libpng
    pkg-config --cflags libpng
    cpan Imager::File::PNG
    
    brew install jpeg
    pkg-config --cflags libjpeg
    cpan Imager::File::JPEG
    
    brew install giflib
    cpan Imager::File::GIF
    
    GIF: Test code failed: Can't link/include 'gif_lib.h', 'stdio.h', 'errno.h', 'string.h', 'gif'...
    ! Configure failed for Imager-File-GIF-0.98. See /Users/you/.cpanm/work/1713652269.80239/build.log for details.
    
    cd /Users/you/.cpanm/work/1713652269.80239/Imager-File-GIF-0.98
    
    perl Makefile.PL -v --incpath=/opt/homebrew/include --libpath=/opt/homebrew/var/homebrew/linked/giflib/lib
    
    make
    make test
    make install
    
    For some reason the following didn't work with the cpan client:
    o conf makepl_arg "LIBS=-L/opt/homebrew/var/homebrew/linked/giflib/lib INC=-I/opt/homebrew/include"
    
    PS - pkg-config can't find giflib so the paths were found like this:
    sudo /usr/libexec/locate.updatedb
    
    locate giflib
    /opt/homebrew/var/homebrew/linked/giflib
    
    locate gif_lib.h        
    /opt/homebrew/Cellar/giflib/5.2.1/include/gif_lib.h
    /opt/homebrew/include/gif_lib.h
    
CPAN autobundle fail
2 direct replies — Read more / Contribute
by Anonymous Monk
on Apr 10, 2024 at 13:50
    I was trying to autobundle an old perl setup but cpan was just sitting there failing to contact mirrors (cpanm user so that mirror list was probably ancient). This inspired me to visit https://www.cpan.org/SITES.html where it says www.cpan.org don't do mirrors anymore. Created the autobundle like so:
    cpan -M https://www.cpan.org -a
[OT] The Long List is Long resurrected
2 direct replies — Read more / Contribute
by marioroy
on Apr 06, 2024 at 01:23

    Recently, I tried NVIDIA's nvc++ compiler and noticed a compile time regression (> 40 seconds). That's a long time. I reached out to NVIDIA. Though, no resolution from their side. Some time later, I saw another regression upgrading GCC from version 13 to 14. It seems that std::mutex regressed.

    That spawned a chain-reaction. I logged-appended the March and April 2024 events to my summary page.

    A spinlock mutex class resolved both issues, further improving performance. I reached out to Greg, author of the phmap C++ library. The LLiL challenge is interesting. He shared tips plus the string_cnt struct, accommodating fixed-length and variable-length keys.

    Clang performance regression due to GCC 14
    phmap: Ability to reset the inner sub map
    string_cnt struct commit by Greg

    eyepopslikeamosquito's llil3vec.cpp inspiration (scroll down the page), for making the vector version llil4vec fast (though gobbles memory for unlimited length strings), is the reason efforts to making the map variants catch up, while keeping memory consumption low. There are several map variants. The llil4hmap, llil4emh, and llil4umap demonstrations compute the key hash_value once only, and stores it with with the key.

    llil4map sub maps managed by the C++ library, using phmap::parallel_flat_hash_map
    llil4map2 memory efficient version, vector of pointers to the map key-value pairs
    llil4hmap sub maps managed by the application, using phmap::flat_hash_map
    llil4emh sub maps managed by the application, using emhash7::HashMap
    llil4umap sub maps managed by the application, using std::unordered_map

    Greg Popovitch added a clever new type string_cnt, supporting fixed-length and long strings without recompiling. See enhanced llil4map found in his examples folder, also memory efficient. Note: I beautified the include directives in my demonstrations, matching your version for consistency.

    I glanced through the long thread by eyepopslikeamosquito. At the time, we were thrilled completing in less than 10 seconds. More so below 6 seconds. Fast forward to early 2024, the llil4map demonstration completes in 0.3 seconds, due to linear scaling capabilities (all levels parallel). The llil4vec example may run faster, but stop improving beyond more CPU cores.

    We reached the sub-second territory, processing three input files.

    Limit 12 CPU Cores

    $ NUM_THREADS=12 ./llil4map big{1,2,3}.txt | cksum llil4map (fixed string length=12) start use OpenMP use boost sort get properties 0.311 secs map to vector 0.055 secs vector stable sort 0.095 secs write stdout 0.036 secs total time 0.500 secs count lines 10545600 count unique 10367603 2956888413 93308427 $ NUM_THREADS=12 ./llil4vec big{1,2,3}.txt | cksum llil4vec (fixed string length=12) start use OpenMP use boost sort get properties 0.170 secs sort properties 0.061 secs vector reduce 0.017 secs vector stable sort 0.065 secs write stdout 0.047 secs total time 0.362 secs count lines 10545600 count unique 10367603 2956888413 93308427

    No Limit

    $ ./llil4map big{1,2,3}.txt | cksum llil4map (fixed string length=12) start use OpenMP use boost sort get properties 0.101 secs map to vector 0.052 secs vector stable sort 0.115 secs write stdout 0.029 secs total time 0.298 secs count lines 10545600 count unique 10367603 2956888413 93308427 $ ./llil4vec big{1,2,3}.txt | cksum llil4vec (fixed string length=12) start use OpenMP use boost sort get properties 0.203 secs sort properties 0.088 secs vector reduce 0.024 secs vector stable sort 0.103 secs write stdout 0.029 secs total time 0.449 secs count lines 10545600 count unique 10367603 2956888413 93308427

    Many thanks eyepopslikeamosquito for being there. I promise no more messages for a while. You inspired C++ in me. Next is Taichi Lang.

    Greg Popovitch shared a tip for releasing memory immediately, during "map to vector". Thank you. That was helpful also, for llil4umap.

    Blessings and grace,

Google Research releases... PERL!
3 direct replies — Read more / Contribute
by salva
on Mar 20, 2024 at 10:32
Changes in MooX::Role::Parameterized
No replies — Read more | Post response
by choroba
on Mar 17, 2024 at 16:12

    What is it good for?

    If youíve never worked with MooX::Role::Parameterized or MooseX::Role::Parameterized, you might wonder what is a parameterized role at all?

    Roles are used when you need to share behaviour among several classes that donít have to be related by inheritance. Normally, a role just adds a bunch of methods to the class that consumes it (thereís more, you can for example specify which other methods the role expects to already exist).

    A parameterized role makes it possible to provide parameters for the consumed role. This way, you can adjust the behaviour for each consuming class.

    The old syntax

    The standard syntax to apply a role to a class before version 0.100 of the module was to use the apply class method:

    # My/Role.pm package My::Role; use Moo::Role; use MooX::Role::Parameterized; role { my ($params, $mop) = @_; $mop->has($params->{name} => is => $params->{is}); }

    # My/Obj.pm package My::Obj; use Moo; use My::Role; 'My::Role'->apply({ name => 'size', is => 'ro', });

    If we now created an object $o using my $o = 'My::Obj'->new(size => 2), we could get the value of the attribute size using the $o->size getter: the role created a new read-only attribute size for us.

    The old experimental syntax

    What I didnít like about applying a role to a class the old standard way was it wasnít declarative. You could easily overlook it as a block of code happening at runtime, while the meaning of the code was This is how a role is consumed. Therefore, I used the alternative experimental syntax:

    package My::Obj; use Moo; use MooX::Role::Parameterized::With 'My::Role' => { name => 'size', is => 'ro', };

    It's part of a use clause, so itís clear that itís happening at compile time.

    The new syntax

    I promoted one of my side-jobs to a full-time job recently. They gave me a new computer where I had to install all my code base to start working on it 8 hours a day instead of a couple a month.

    Imagine my surprise when the code stopped with an error:

    Can't locate object method "size" via package "My::Obj" at ./run.pl line 37.

    Line 37 was where I called $o->size!

    When installing the dependencies for my code, the most recent version of MooX::Role::Parameterized was installed from CPAN (0.501). The experimental syntax is no longer documented and as I found out, doesnít work anymore.

    The old non-experimental syntax still works, but thereís a new syntax, too. It uses the with keyword that looks like the one that can be used to consume a Moo::Role, but if we first use MooX::Role::Parameterized::With, it can also accept parameters for the role application.

    package My::Obj; use Moo; use MooX::Role::Parameterized::With 0.501; with 'My::Role' => { name => 'size', is => 'ro', };

    Moreover, we should change the definition of the role, too. Parameters should be predeclared using the parameter keyword (similarly to MooseX::Role::Parameterized), and they can be then accessed via getters instead of peeking inside a parameter hash reference.

    package My::Role; use Moo::Role; use MooX::Role::Parameterized 0.501; parameter name => (is => 'ro'); parameter is => (is => 'ro'); role { my ($params, $mop) = @_; $mop->has($params->name => is => $params->is); }

    Note: Published to my blog, too.

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
Moved from 'Today I Learned' (2024-02-14 02:31:10)
No replies — Read more | Post response
by Discipulus
on Feb 13, 2024 at 21:31
    perl -MData::ICal -MData::Dump -e "my $cal = Data::ICal->new(filename => $ARGV[0]); dd @{$cal->entries};" sample.ics
Call for Speakers for the 2024 Carolina Code Conference is open until April 15th
1 direct reply — Read more / Contribute
by brightball
on Jan 09, 2024 at 14:35

    A polyglot conference for all who code!

    Just wanted to drop by and let everyone know that our Call for Speakers has opened for 2024. We're growing this year, doubling from 150 to 300 attendees and a 2 day event in beautiful Greenville, SC.

    https://blog.carolina.codes/p/happy-new-year-call-for-speakers

    If you'd like a look at our speakers from last year, we had a solid line up headlined by keynotes from Charles Nutter (jRuby) and Bruce Tate (7 Languages in 7 Weeks, active with Elixir) for a total of 15 speakers on a single track.

    https://blog.carolina.codes/p/announcing-our-2023-speakers

    We utilize a staggered schedule throughout the day with alternating 30 minute (moderate) and 10 minute (lightning) talks between keynotes to showcase a variety of topics to the audience while keeping everyone engaged. We're committed to maintaining the single track to ensure that our speakers have the entire audience available. There's nothing worse than being selected for a conference and then ending up in a room where nobody shows up to your talk.

    All talks are professionally recorded and published.

    Please let me know if you have any questions. It's an excellent opportunity to showcase cool Perl things to a broad audience.

Have you ever lost your work?
9 direct replies — Read more / Contribute
by harangzsolt33
on Jan 08, 2024 at 12:22
    Recently I was writing a simple script (maybe 200 lines total), and I did something quickly, which I forgot what it was. But it deleted my script completely without trace!!! I may have accidentally pressed a bad key combination in Notepad2 or I don't know what happened. I ran the script. It ran without errors. It was finally working the way I wanted it. So, I closed the editor. And originally, I saved it on my desktop, and now it was gone. It wasn't in the Trash bin either. It was completely gone! How did it disappear in a flash? Have you ever had a similar experience where you worked on something and it mysteriously disappeared without a trace?

    I have a program called Recuva which runs on Windows and looks for deleted files and tries to restore them. But it did not find the file I was looking for. I was really surprised, because usually if you just delete something and then immediately go to Recuva, it will find that file. And chances are it may still be able to restore it. But it couldn't even locate the file. So, my next thought was I'm going to open HxD which is a hex editor which can view files, memory, or disks. I selected the main hard drive, which is 2TB. And I thought, how am I going to find this script? I thought of a unique line which is something that appears in my script that likely does not appear anywhere else, and I typed that into search. I thought, this will take forever. But no! It found it within 1 minute, and I was able to salvage my script! It's a miracle!!! This is one reason why you don't want to encrypt your hard drive. Lol

    ( I guess, this post could be formatted as a question for vote, but I'm not sure. It's just something I thought of. Not really Perl related or PerlMonks website related, so I wasn't sure where to write this. )

Speed of simple pattern count. A comparison
1 direct reply — Read more / Contribute
by rsFalse
on Jan 06, 2024 at 21:53
    Hello,

    I've just played with various ways to count 1-2 letter patterns in the longer string, and compared speed. If the task is to count the number of exact substring in non-overlapping manner, then there are many ways how to do it. And if to count a single ASCII character, then there are even more ways.
    Some ways (e.g. functions/chop and functions/chomp) destruct the target string, so I did the copy of it every time.
    Used several perl versions including up to 5.38.2, year 2023.

    UPDATE. After jwkrahn comment (Re: Speed of simple pattern count. A comparison) I added variants with functions/index and functions/rindex. Also I added functions/substr variant, which simply takes by one character (I did no included this variant with 2 character long pattern, because it also needs to increase position if I search for non-overlapping matches).

    Count a single character variations and speed:
    #!/usr/bin/perl use strict; use warnings; use Benchmark 'cmpthese'; my $target = 'abc' x 1e4; cmpthese(-1,{ 'y' => sub { my $m = 0; $m = $target =~ y/a//; }, '=()=' => sub { my $m = 0; $m = () = $target =~ m/a/g; }, 'while' => sub { my $m = 0; $m ++ while $target =~ m/a/g; }, '(?{})' => sub { my $m = 0; $target =~ m/a(?{ $m ++ })(*F)/g; +}, 'split_grep' => sub { my $m = 0; $m = grep $_ eq 'a', split '', $target; }, 'split_by' => sub { my $m = 0; $m = -1 + split 'a', 'x' . $target . 'x'; }, 'chop' => sub { my $m = 0; my $target2 = $target; $m += ( 'a' eq chop $target2 ) while length $target2; }, 'chomp' => sub { my $m = 0; my $target2 = $target; local $/ = 'a'; 0 while chomp $target2 and ++ $m or chop $target2; }, 'index' => sub { my $m = 0; my $pat_len = length 'a'; my $pos = -$pat_len; $m ++ while -1 < ( $pos = index $target, 'a', $pos + $pat_len +); }, 'rindex' => sub { my $m = 0; my $pat_len = length 'a'; my $pos = -1 + length $target; $m ++ while -1 < ( $pos = rindex $target, 'a', $pos - $pat_len + ); }, 'substr' => sub { my $m = 0; $m += ( 'a' eq substr $target, $_, 1 ) for 0 .. -2 + length $t +arget; }, });
    OUTPUT:
    perl-5.38.2 ========== Rate split_grep (?{}) substr chop chomp =()= while i +ndex rindex split_by tr / y split_grep 214/s -- -4% -23% -36% -47% -68% -70% +-77% -79% -99% -99% (?{}) 222/s 4% -- -20% -34% -45% -67% -69% +-77% -78% -98% -99% substr 279/s 30% 26% -- -17% -31% -59% -61% +-71% -73% -98% -99% chop 336/s 57% 51% 20% -- -16% -50% -53% +-65% -67% -98% -99% chomp 403/s 88% 81% 44% 20% -- -40% -44% +-58% -61% -97% -99% =()= 673/s 214% 203% 141% 100% 67% -- -6% +-29% -34% -95% -98% while 718/s 235% 223% 157% 114% 78% 7% -- +-24% -30% -95% -98% index 948/s 342% 326% 240% 182% 135% 41% 32% + -- -8% -93% -97% rindex 1028/s 380% 362% 268% 206% 155% 53% 43% + 8% -- -93% -97% split_by 14354/s 6599% 6359% 5045% 4170% 3466% 2032% 1899% 1 +415% 1297% -- -59% tr / y 34600/s 16047% 15470% 12301% 10192% 8496% 5039% 4718% 3 +551% 3267% 141% -- perl-5.32.0 ========== Rate split_grep (?{}) substr chomp chop =()= while r +index index split_by tr / y split_grep 163/s -- -16% -45% -46% -53% -68% -78% + -83% -83% -99% -100% (?{}) 194/s 19% -- -34% -36% -44% -62% -73% + -79% -79% -99% -99% substr 296/s 82% 52% -- -2% -14% -42% -59% + -68% -68% -98% -99% chomp 302/s 85% 55% 2% -- -13% -41% -59% + -68% -68% -98% -99% chop 345/s 112% 78% 17% 14% -- -32% -53% + -63% -63% -98% -99% =()= 511/s 213% 163% 72% 69% 48% -- -30% + -46% -46% -97% -99% while 731/s 348% 276% 147% 142% 112% 43% -- + -22% -22% -96% -98% rindex 939/s 476% 383% 217% 211% 172% 84% 28% + -- -0% -95% -97% index 939/s 476% 383% 217% 211% 172% 84% 28% + 0% -- -95% -97% split_by 18553/s 11271% 9442% 6162% 6048% 5272% 3532% 2436% +1876% 1875% -- -46% tr / y 34462/s 21022% 17623% 11531% 11319% 9878% 6645% 4611% +3570% 3568% 86% -- perl-5.20.1 ========== Rate split_grep substr chop =()= chomp (?{}) while +index rindex split_by tr / y split_grep 181/s -- -25% -38% -41% -41% -54% -73% + -74% -74% -99% -99% substr 241/s 33% -- -17% -21% -21% -38% -64% + -65% -66% -99% -99% chop 290/s 61% 20% -- -5% -5% -26% -57% + -58% -59% -98% -99% =()= 304/s 68% 26% 5% -- -1% -22% -55% + -56% -57% -98% -99% chomp 307/s 70% 27% 6% 1% -- -22% -54% + -56% -56% -98% -99% (?{}) 392/s 117% 62% 35% 29% 28% -- -42% + -44% -44% -98% -99% while 673/s 273% 179% 132% 121% 120% 72% -- + -3% -4% -96% -98% index 697/s 286% 189% 140% 129% 127% 78% 4% + -- -1% -96% -98% rindex 704/s 289% 192% 142% 131% 129% 80% 5% + 1% -- -96% -98% split_by 16747/s 9166% 6845% 5665% 5403% 5360% 4177% 2387% +2302% 2280% -- -51% tr / y 34296/s 18876% 14124% 11707% 11170% 11081% 8658% 4994% +4819% 4773% 105% --
    Transliteration (perlop#tr y / tr) is absolutely fastest.
    Splitting by search value is second fastest. Although it counts as inverse, i.e. the number of not matched chunks. Here is important to note edge cases: if the pattern matches right at the beginning and/or right on the end, therefore I add 'x' at both sides, which should not be a substring of the pattern.
    Other ways are way slower.
    If look across versions, I can spot that (?{})(perlre#(?{-code-})) became about 2x slower between 5.20 and 5.32. The results of perl 5.14 (not shown here) are similar to 5.20, except that I need to use 'our $m' variable with (?{}) variant.

    And here are variations (less than searching for single-char) with pattern of two characters:
    cmpthese(-1,{ '=()=' => sub { my $m = 0; $m = () = $target =~ m/ab/g; }, 'while' => sub { my $m = 0; $m ++ while $target =~ m/ab/g; }, '(?{})' => sub { my $m = 0; $target =~ m/ab(?{ $m ++ })(*F)/g; + }, 'split_by' => sub { my $m = 0; $m = -1 + split 'ab', 'x' . $target . 'x'; }, 'chomp' => sub { my $m = 0; my $target2 = $target; local $/ = 'ab'; 0 while chomp $target2 and ++ $m or chop $target2; }, 'index' => sub { my $m = 0; my $pat_len = length 'ab'; my $pos = -$pat_len; $m ++ while -1 < ( $pos = index $target, 'ab', $pos + $pat_len + ); }, 'rindex' => sub { my $m = 0; my $pat_len = length 'ab'; my $pos = -1 + length $target; $m ++ while -1 < ( $pos = rindex $target, 'ab', $pos - $pat_le +n ); }, });
    OUTPUT:
    perl-5.38.2 ========== Rate (?{}) chomp =()= while rindex index +split_by (?{}) 216/s -- -64% -70% -75% -78% -81% + -96% chomp 599/s 177% -- -17% -31% -40% -47% + -90% =()= 718/s 232% 20% -- -18% -28% -36% + -88% while 872/s 303% 46% 21% -- -13% -22% + -85% rindex 999/s 362% 67% 39% 15% -- -11% + -83% index 1120/s 418% 87% 56% 28% 12% -- + -81% split_by 5749/s 2559% 860% 701% 559% 475% 413% + -- perl-5.32.0 ========== Rate (?{}) chomp =()= while index rindex +split_by (?{}) 228/s -- -59% -64% -75% -77% -78% + -95% chomp 555/s 143% -- -12% -39% -44% -47% + -89% =()= 627/s 175% 13% -- -31% -37% -41% + -88% while 913/s 300% 65% 46% -- -9% -13% + -82% index 999/s 338% 80% 59% 9% -- -5% + -80% rindex 1056/s 362% 90% 68% 16% 6% -- + -79% split_by 5046/s 2110% 810% 705% 452% 405% 378% + -- perl-5.20.1 ========== Rate =()= (?{}) chomp rindex index while +split_by =()= 372/s -- -19% -22% -54% -54% -56% + -96% (?{}) 462/s 24% -- -3% -43% -43% -45% + -95% chomp 478/s 29% 4% -- -41% -41% -43% + -95% rindex 807/s 117% 75% 69% -- -1% -3% + -92% index 814/s 119% 76% 70% 1% -- -3% + -92% while 836/s 125% 81% 75% 4% 3% -- + -91% split_by 9752/s 2524% 2013% 1941% 1108% 1099% 1066% + --
    Split by search pattern is way faster than other variants. But to use it I need to think about edge cases and possible overlapping after appending or prepending additional symbols. Update. However splitting by search pattern seems to become almost 2x slower somewhere between perl-5.20 and 5.32.

    Update-2. Using while chop and while chomp it is important to note, that condition will terminate if pattern or chopped character was 0, therefore to overcome this limitation I should have added length, e.g. while length cho(m)p.

Add your Meditation
Title:
Meditation:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (4)
As of 2024-06-19 07:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.