Mpirun problem


We have following configuration:
4 nodes with 2x4 cores processor on each which means that we are running on 4 x 8 processors and all machines are connected using 1gbit LAN.
Debian linux 4.0 installed

We are running weather forecast model compiled with pgi/mpich and we have performances problem when we run it on all 4 nodes.Usage per all cores is about 30-40 % and it should be over 90%.
It even works faster if we run it using mpirun on one machine than on 4.

what could be the problem ?
Do we try to install 10gb network for this model ?



It depends.

This either the connection is not fast enough or each is competing for memory access on the same machine when run at its full capacity.

You will need to check how much memory does the program use and if it has enough for each node.

If you compare:

  1. 4 processes on 4 different machines
  2. 4 processes on one machine

If (1) is slower than (2), then perhaps it is the connection.



We have 4 GB RAM memory of every node, so about 15% of memory is used when our model is run.The problem is with network then,so maybe gigabit network is too slow for our model.
Every node have two Xeon Quad Core 2.8Ghz CPU`s.