Re: To Fork or Not to Fork (bottle necks)

[tye]

Forking certainly can make processing a large number of large files go much faster. We have a system that does exactly that and forking allows things to run many times faster. But then, our processing of files is mostly CPU-bound as we are transcoding the files and this runs on a system with 32 cores exactly because of this.

We just finished benchmarks on a revamp of this and it is about 4x faster than it used to be (despite it previously forking more workers than there are CPU cores). The old process worked pretty much exactly like Parallel::ForkManager. The new strategy pre-forks the same number of workers and just continuously feeds filenames to them over a simple pipe.

There are several advantages to the new approach. The children are forked off of the process before it has built up the list of files to be processed, which will often be a huge list, so there are much fewer copy-on-write pages to eventually be copied. The children live a very long time now, so there is less overhead from fork()ing (once per worker instead of once per file). The above two features also mean that it makes sense for the children to be the ones to talk to the database, which is probably the biggest "win". It also significantly simplified the code.

If your processing of files is mostly I/O bound, then doing a bunch of them in parallel could actually be slower than doing them in serial. Though, I would expect that your processing of one file isn't perfectly I/O bound and having at least two running will provide some speed-up as often one can use the CPU while the other is waiting for I/O.

Once you have enough processes that you are maxing out the available throughput of either your CPU cores or your I/O subsystem, then adding more processes will just add overhead.

[oiskuu]

And a related question: Design advice: Classic boss/worker program memory consumption.

Did you also consider/bench a threaded version?

Might there be a practical way to manage multiple malloc areas in one perl process? A separate arena (mmap) per package? This could improve locality, and also segment the data so that CoW benefits are subdivided.

[tye]

No, we didn't consider using threads (nor what Perl 5 calls "threads"). If using threads offered any performance benefits for our use case, the benefits would be truly trivial.

The cost of using threads in terms of complexity would not be trivial. In some cases, you can use Perl threads with the Perl code mostly untouched and the added complexity is mostly just hiding inside of the perl executable. Though it will also add operational complexity when it comes to trouble shooting and similar tasks. But that complexity inside the perl executable very often doesn't stay completely hidden (go read the p5p thread on why Perl threads have been "depreciated", if you aren't familiar with the typical experience of trying to use Perl threads).

But I don't think this is even one of those cases. Since the workers spawn other executables to do parts of the transcoding and spawning jobs involves waiting for SIGHCLD and mixing signals and threads is often pointed out as a bad idea, I suspect that the Perl code would actually have to get significantly more complex.

So I didn't even consider adding significant complexity in at least 2 or 3 layers for a possible but at-most-trivial performance gain and maybe a (likely trivial) performance decrease.

[pimperator]

Thank you for the detailed reply. To clarify, my code sends a command to the system to convert a large file into a larger file using another program. Because there is no I/O, it would be advantageous to use Parallel::ForkManager.

But when I open each file and read them, it's better to do it in a serial fashion.

[tye]

Because there is no I/O

It is true that your Perl script isn't doing the I/O, but it is very much not true that "there is no I/O".

Whether Perl is doing the I/O or computing vs. some other program doing it has little bearing on the performance impact of having Perl fork() so you can have more than 1 instance running at once.