Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: processing large files

by Tuna (Friar)
on Jul 04, 2001 at 05:18 UTC ( #93746=note: print w/replies, xml ) Need Help??


in reply to processing large files

As a network analyst for a tier-1 ISP, I can happily report that Perl can, and on a daily basis, does, process files in the neighborhood of several hundred gig. Are you sure that your limitation is not hardware-related?

Replies are listed 'Best First'.
Re: Re: processing large files
by Anonymous Monk on Jul 07, 2001 at 01:12 UTC
    How could it be hardware related if these files are on
    disk at that size and C++ programs process them?
    how do you process your files with perl? Is it a specfic
    perl? or compile option? or algorithm?
    thanks

    -E
      Easily! However, you never said that you had other programs that were successfully processing your data. =) As far as my processing is concerned, I churn through Cisco NetFlow data, along with bgp tables, route summaries, snmp data, merge it all together and spit it back out again to create traffice matrices. The algorithms aren't that complicated at all; it's just a shite-load of data that I process continuously. If I tried to run on anything less than the E450 that it runs on, I would be in trouble.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://93746]
help
Chatterbox?
[marto]: I'm all for that
[choroba]: Trainspotting without OV is just a half of the movie!
LanX just single handedly stopped Brexit!

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2018-02-19 10:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    When it is dark outside I am happiest to see ...














    Results (262 votes). Check out past polls.

    Notices?