Charm++ parallel programming system

[Sam White <samt.white AT gmail.com>]

Yes, to get the full benefits of AMPI you will want to run with overdecomposition, meaning more virtual processors than physical processors (or OS processes). To do this, you will usually want to launch one process (+p) per core, with more virtual processors (+vp). Your example command line will launch 8 processes per node, with 8 virtual processors per process, ignoring some of your cores/node as you noted. To make full use of your 24-core nodes, you would want to do the following:

./charmrun +p192 ++mpiexec ++remote-shell $HOME/script.sh ./lulesh2.0 +vp512 -i 300

To increase the degree of overdecomposition, just increase the number of virtual processes:

./charmrun +p192 ++mpiexec ++remote-shell $HOME/script.sh ./lulesh2.0 +vp4096 -i 300

Let us know if you have any other questions,
Sam

On Thu, Nov 10, 2016 at 9:46 AM, Maksym Planeta <mplaneta AT os.inf.tu-dresden.de> wrote:

Dear Charm++ group,

as far as I understand, the main mode of operation of AMPI is running it in oversubscribed mode.

Does it mean that I should create less processes (+p parameter), than I have cores? And I should create at least as many virtual processes (+vp) as I have cores, or even more?

I tried to create 64 processes on 8 nodes and specified +vp 512. What I observed is that at each node there were 8 processes created, and the rest 16 cores of each 24-cores node stayed unoccupied. Did I do something wrong?

The way how I launched the program is following:

./charmrun +p64 ++mpiexec ++remote-shell $HOME/script.sh ./lulesh2.0 +vp512 -i 300

The script basically only does "exec srun -N 8 -n $*"

--
Regards,
Maksym Planeta