Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^2: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

by NeilF (Sexton)
on Jan 01, 2006 at 19:48 UTC ( #520280=note: print w/replies, xml ) Need Help??


in reply to Re: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!
in thread Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

Aristotle, yes it is for a forum...

I wrote it 4-5 years ago and its been chugging along very nicely for all those years. It's not heavily used, ie: typically only 0-12 people using it at any one time...

However, the io processing has been noticed by my ISP hence my attempts to reduce it.

I've made a number of mods already that have generally improved/reduced io processes, but one of the biggest hogs is when postting a message.

It seems at the least with writing:-
a) Just by adding "binmode" prior to the write that will more than half the io. And this can be applied to all such writes to improve across the board!
b) By consolidating an array into a variable that will further reduce the io. However, I don't like the idea of having a loop which goes through X thousand records, and having the data held twice (once in the array and once in a variable) at the same time!

So my two questions are currently:-
a) Is there a simple means of improving the reading, as has seemingly been found for the write. Reading in 4K chunks, blah!
b) Is there a better means of printing the array as a consolidated variable, so the data is not held twice in memory etc.
  • Comment on Re^2: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!

Replies are listed 'Best First'.
Re^3: Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO!
by merlyn (Sage) on Jan 01, 2006 at 22:02 UTC
    However, the io processing has been noticed by my ISP hence my attempts to reduce it.
    If your ISP is poking into it that much, it's either time to renegotiate with your ISP to make that a permitted activity, or get any one of the four million ISPs that won't care.

    Most ISPs charge for network bandwidth and disk storage, and would never care how many times you call read() or write().

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

      They have a prefered limit of 20,000,000 IO processes a day...

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://520280]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (2)
As of 2022-05-18 03:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (68 votes). Check out past polls.

    Notices?