Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re: grepping a large file and stopping on first match to a list

by perlfan (Vicar)
on Feb 23, 2016 at 16:56 UTC ( [id://1155935]=note: print w/replies, xml ) Need Help??


in reply to grepping a large file and stopping on first match to a list

In practical terms, it would behoove you to pre-process the nearly 1 million lines of text ahead of time so you're searching over a much smaller set of possible matches.

If this is a relatively stable corpus of some kind, then there are any number of things you could do - use the commandline grep, throw it in SQLite/MySQL - or even better, Sphinx.

Once you do this, any suggestion you use in this discussion for efficiency in the Perl script itself will work even better for you.

Replies are listed 'Best First'.
Re^2: grepping a large file and stopping on first match to a list
by msh210 (Monk) on Feb 23, 2016 at 17:03 UTC

    Thanks! I'll look into it.

    $_="msh210";$"=$\;@_=@{[split//,uc]}[2,0];$_="@_$\1";$\=$/;++$_[0]for$...1;print lc substr crypt($_,"@_"),1,6

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1155935]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (6)
As of 2024-04-23 18:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found