Beefy Boxes and Bandwidth Generously Provided by pair Networks Joe
No such thing as a small change
 
PerlMonks  

Re^2: Help with speeding up regex

by eversuhoshin (Sexton)
on Aug 14, 2012 at 03:31 UTC ( #987257=note: print w/ replies, xml ) Need Help??


in reply to Re: Help with speeding up regex
in thread Help with speeding up regex

Hellp Davido, Thank you for your kind reply. Sorry but I am using active perl using Komodo, can you help me with utilizing Devel::NYTProf? What do I have to do to assess my script? I am still learning and I didn't fully understand the cspan explanation. Thank you again


Comment on Re^2: Help with speeding up regex
Re^3: Help with speeding up regex
by davido (Archbishop) on Aug 14, 2012 at 09:08 UTC

    ActiveState has Devel::NYTProf in its ppm4 repository here: http://code.activestate.com/ppm/Devel-NYTProf/. Once installed, you would cd into the target script's directory and execute a one-liner that invokes your script: perl -d:NYTProf some_perl.pl input_file.txt. And after it completes, you can review the results by executing the following statement: nytprofhtml --open (while still in the same directory). You should get a browser window with more useful information than you can shake a stick at.

    My optimized regex is going to help as an optimization of the exact regex you provided. But it's tricky to implement and maintain as your needs continue to evolve. A better solution would be to use threads, or to fork processes. BrowserUk already had some suggestions on how you might implement such a strategy. The beauty of that sort of approach is that you don't have to concern yourself quite as much with how efficient the regular expressions themselves are because you're processing several files in parallel.

    If you end up with a ton of data every day that has to get chewed through before tomorrow, you might look into a Map-Reduce strategy such as with hadoop.


    Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://987257]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2014-04-18 03:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (461 votes), past polls