Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Re: Re: Re: STOP Trading Memory for Speed

by Anonymous Monk
on Sep 25, 2002 at 22:53 UTC ( #200761=note: print w/replies, xml ) Need Help??

in reply to Re: Re: STOP Trading Memory for Speed
in thread STOP Trading Memory for Speed

You didn't read his problem.

The memory limit is not physical memory, it is the fact that you can only address 4 GB with a 32-bit pointer.

BerkeleyDB might solve this by taking less memory. It might also be able to use anonymous paging to work even though it cannot directly address most of that memory. The documentation says that data sets that large usually speed up by switching from hashing to a BTREE.

But letting it swap won't work.

  • Comment on Re: Re: Re: STOP Trading Memory for Speed

Replies are listed 'Best First'.
Re: Re: Re: Re: STOP Trading Memory for Speed
by perrin (Chancellor) on Sep 25, 2002 at 23:26 UTC
    I read the problem. You can only address 4GB of physical memory, but you can use more in swap if you run Linux with PAE (Page Address Extensions), as available in Red Hat's "Advanced Server" distribution. There may be something similar for other OSes.

    Note that I have not used this and can't speak about it from experience. My situation was with a process going over 2GB on a FreeBSD machine with 2GB of physical RAM.

      PAE lets the OS and specifically aware processes address more memory than what a simple pointer can address. If you turn it on then your OS can use more memory. That doesn't let any individual application push the 32-bit limit unless the OS supplies hooks for mapping and unmapping memory into what the program can directly address, and the program is aware of and uses those hooks.

      If you had a single process address over 4 GB of RAM on a 32-bit CPU, then the OS has to provide those hooks and the application has to be using it. It would make sense for BerkeleyDB to try this, but I don't know if does. If it doesn't and you want it to, you could pay Sleepycat to develop it for you.

        Good info. Thanks for stopping by, Anon.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://200761]
[hippo]: Indeed you are.
[thao4]: Hi all, I am new in perl. I need do somthing in perl: exemple: - search a patern "CADRE 1.1 CPU" in one file.txt - incremente +30 lines from the line with the patern found - replace the patern "Model Regle" by "Heath Rule A" in this new line (30 line
[marto]: good morning all
[hippo]: Welcome. Tell me, have you read perlintro? That's a great place to start.
[thao4]: I seach to do that since a week but I can't found the answer in google. Thank for your help
[Eily]: hello thao4 (and hello all :D), you should [id://http:// 479#post|open a question on Seekers of Perl Wisdom], be sure to include whatever you have tried that didn't work, or only does the job partially
[thao4]: yes, I do a little perl since some years but not fréquently, I can correct the triggers in Clearcase of IBM, but write the code : no
[Eily]: meh, such good link ...
[Eily]: here
[Eily]: still not xD there

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (10)
As of 2018-02-21 09:46 GMT
Find Nodes?
    Voting Booth?
    When it is dark outside I am happiest to see ...

    Results (276 votes). Check out past polls.