http://www.perlmonks.org?node_id=488163


in reply to distributed computing

Welcome to the world of super computing! Your computers are linked in a NOW (network of workstations) I presume. Cluster computing with Perl is very easy. Let me explain it...

The ideal configuration for a beowulf cluster is to have a head node (a node is a computer) that has access to the external network, and a private network to link up the rest of the nodes with the head node. The head node will have two network cards, one for the internal cluster, one for communicating with the rest of the world.
C--+--C | C--+--C | C--Hub--C(head) | | ---------+----------- Backbone LAN
This is the ideal world, I doublt that you want to configure your cluster in this mannar.

The easier way is to connect all the computers via the back bone LAN into a NOW (network of workstations), no special hardware is required.
C C C C(head) | | | | | | | | -+--+--+--+---- Backbone LAN
You will need to prepare your systems as follows:

1. have a shared storage area such as NFS mount.
2. have all the computers mount to the same NFS share.

The next step is to install a parallel virtual machine. Check out the project PVM at (Parallel Virtual Machine).
You may want to set up ssh on all the machines to make PVM run better and more secure.

Go to CPAN and fetch the module Parallel::PVM, which is the Perl interface to the parallel virtual machine.

PVM has pretty much everything you want for parallel computing, except for, distributed shared memory (which you really don't need anyway). Your code can be written in manager-worker mode. The manager will launch workers, and the launching process is transparent to your program. PVM will decide which machine the program will run on, depending on the work load.

By the way, the machines do not have to be of the same architecture or running the same OS. It's perfectly ok to run a cluster of linux/solaris/windows mixed nodes.

I am not going to say more here, I hope that you find this information useful, and take joy in doing research into PVM.