Charm++ parallel programming system

["Lee, Ian" <lee1001 AT llnl.gov>]

Abhinav,

 

Here is the grep for the BL_HOME in the charm directory, not sure if it is helpful for you but it is definitely something that is coming from Charm++

 

$ grep BL_HOME * -r

mpi-bluegenep/tmp/charmrun:6:#  Copies executable to $BL_HOME directory before running.

mpi-bluegenep/tmp/charmrun:11:if [ -z "$BL_HOME" ]

mpi-bluegenep/tmp/charmrun:13:  echo "You must set the BL_HOME environment variable"

mpi-bluegenep/tmp/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

mpi-bluegenep/tmp/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

mpi-bluegenep/tmp/charmrun:42:  cp $firstArg $BL_HOME

mpi-bluegenep/bin/charmrun:6:#  Copies executable to $BL_HOME directory before running.

mpi-bluegenep/bin/charmrun:11:if [ -z "$BL_HOME" ]

mpi-bluegenep/bin/charmrun:13:  echo "You must set the BL_HOME environment variable"

mpi-bluegenep/bin/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

mpi-bluegenep/bin/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

mpi-bluegenep/bin/charmrun:42:  cp $firstArg $BL_HOME

mpi-bluegenep/tests/charm++/simplearrayhello/charmrun:6:#  Copies executable to $BL_HOME directory before running.

mpi-bluegenep/tests/charm++/simplearrayhello/charmrun:11:if [ -z "$BL_HOME" ]

mpi-bluegenep/tests/charm++/simplearrayhello/charmrun:13:       echo "You must set the BL_HOME environment variable"

mpi-bluegenep/tests/charm++/simplearrayhello/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

mpi-bluegenep/tests/charm++/simplearrayhello/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

mpi-bluegenep/tests/charm++/simplearrayhello/charmrun:42:  cp $firstArg $BL_HOME

mpi-bluegenep-xlc/tmp/charmrun:6:#  Copies executable to $BL_HOME directory before running.

mpi-bluegenep-xlc/tmp/charmrun:11:if [ -z "$BL_HOME" ]

mpi-bluegenep-xlc/tmp/charmrun:13:      echo "You must set the BL_HOME environment variable"

mpi-bluegenep-xlc/tmp/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

mpi-bluegenep-xlc/tmp/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

mpi-bluegenep-xlc/tmp/charmrun:42:  cp $firstArg $BL_HOME

mpi-bluegenep-xlc/bin/charmrun:6:#  Copies executable to $BL_HOME directory before running.

mpi-bluegenep-xlc/bin/charmrun:11:if [ -z "$BL_HOME" ]

mpi-bluegenep-xlc/bin/charmrun:13:      echo "You must set the BL_HOME environment variable"

mpi-bluegenep-xlc/bin/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

mpi-bluegenep-xlc/bin/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

mpi-bluegenep-xlc/bin/charmrun:42:  cp $firstArg $BL_HOME

mpi-bluegenep-xlc/tests/charm++/simplearrayhello/charmrun:6:#  Copies executable to $BL_HOME directory before running.

mpi-bluegenep-xlc/tests/charm++/simplearrayhello/charmrun:11:if [ -z "$BL_HOME" ]

mpi-bluegenep-xlc/tests/charm++/simplearrayhello/charmrun:13:   echo "You must set the BL_HOME environment variable"

mpi-bluegenep-xlc/tests/charm++/simplearrayhello/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

mpi-bluegenep-xlc/tests/charm++/simplearrayhello/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

mpi-bluegenep-xlc/tests/charm++/simplearrayhello/charmrun:42:  cp $firstArg $BL_HOME

src/arch/mpi-bluegenel/charmrun:6:#  Copies executable to $BL_HOME directory before running.

src/arch/mpi-bluegenel/charmrun:11:if [ -z "$BL_HOME" ]

src/arch/mpi-bluegenel/charmrun:13:     echo "You must set the BL_HOME environment variable"

src/arch/mpi-bluegenel/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

src/arch/mpi-bluegenel/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

src/arch/mpi-bluegenel/charmrun:42:  cp $firstArg $BL_HOME

src/arch/mpi-bluegenep/charmrun:6:#  Copies executable to $BL_HOME directory before running.

src/arch/mpi-bluegenep/charmrun:11:if [ -z "$BL_HOME" ]

src/arch/mpi-bluegenep/charmrun:13:     echo "You must set the BL_HOME environment variable"

src/arch/mpi-bluegenep/charmrun:39:if [ $firstArg -nt $BL_HOME/$firstArg ]

src/arch/mpi-bluegenep/charmrun:41:  echo "Copying $firstArg to $BL_HOME..."

src/arch/mpi-bluegenep/charmrun:42:  cp $firstArg $BL_HOME

 

 

---------------------------------------------

Ian Lee

Engineering Technologies Division

Engineering Directorate

Lawrence Livermore National Laboratory

P.O. Box 808

Livermore, CA  94551-0808

Phone:  925-424-6414

 

From: bhatele AT gmail.com [mailto:bhatele AT gmail.com] On Behalf Of Abhinav Bhatele
Sent: Monday, August 29, 2011 1:58 PM
To: Lee, Ian
Cc: Jim Phillips; charm AT cs.uiuc.edu; Bennion, Brian
Subject: Re: [charm] [ppl] Appropriate build type for LLNL linux clusters

 

Hello Ian,

I am not sure where the BL_HOME error is coming from for the mpi-* builds, never seen it.

But I would anyways suggest using the bluegenep-xlc build which is significantly faster (since it runs on top of DCMF and not MPI). Instead of using charmrun, you can run the executable directly with mpirun. I think charmrun tries qsub because we mostly run on the Argonne BG/P which uses qsub.

- Abhinav

On Mon, Aug 29, 2011 at 1:39 PM, Lee, Ian <lee1001 AT llnl.gov> wrote:

Brian,

For what it's worth, I have also been looking at some of the LLNL clusters and was able to build and run the test cases on a single node using:

On Hera:
./build net-linux-x86_64 gcc

I was not able to build any of the MPI builds on Hera or Atlas due to they do not have mpicxx installed.  Though from your email just now it sounds like maybe you got this working somehow?

I was also able to successfully build on udawn using:
./build charm++ mpi-bluegenep xlc
./build charm++ mpi-bluegenep
./build charm++ bluegenep

However....

Charm++ mailing list:

I am receiving an error for the two mpi-* builds when I try to run that "You must set the BL_HOME environment variable" but I can't seem to find anything to point me to what that should be set as.

On the "./build charm++ bluegenep" case I receive the following message, which I'm actually not sure if it is coming from Charm++ or BG/P:

$ ./charmrun ./hello
Running on 1 processors:  ./hello
Submitting batch job for> 1  ./hello
 using the command> qsub -t 30 -n 1   ./hello
./charmrun: line 68: qsub: command not found


~ Ian

---------------------------------------------
Ian Lee
Engineering Technologies Division
Engineering Directorate
Lawrence Livermore National Laboratory
P.O. Box 808
Livermore, CA  94551-0808

-----Original Message-----
From: charm-bounces AT cs.uiuc.edu [mailto:charm-bounces AT cs.uiuc.edu] On Behalf Of Jim Phillips
Sent: Monday, August 29, 2011 1:00 PM
To: Abhinav Bhatele

Cc: charm AT cs.uiuc.edu; Bennion, Brian
Subject: Re: [charm] [ppl] Appropriate build type for LLNL linux clusters


Try it without the mpicxx option first, though.  -Jim

On Mon, 29 Aug 2011, Abhinav Bhatele wrote:

> It should work with most mpi installs as long as the include and lib files
> are in the usual places. If you have mpicc and mpicxx pointing to the intel
> compilers you can add that to the build like this:
>
> ./build charm++ mpi-linux-x86_64 mpicxx --with-production
>
> - Abhinav
>
>
> On Mon, Aug 29, 2011 at 12:17 PM, Bennion, Brian <Bennion1 AT llnl.gov> wrote:
>
>> Hello Abhinav,
>>
>> Thank you for the quick reply.  What flavor of mpi libraries do the builds
>> below assume is being used?
>> We have mpi wrappers for intel compilers that might need to be inserted
>> with the first two builds.
>>
>> Brian
>>
>>
>> -----Original Message-----
>> From: bhatele AT gmail.com [mailto:bhatele AT gmail.com] On Behalf Of Abhinav
>> Bhatele
>> Sent: Monday, August 29, 2011 12:01 PM
>> To: Bennion, Brian
>> Cc: charm AT cs.uiuc.edu
>> Subject: Re: [ppl] [charm] Appropriate build type for LLNL linux clusters
>>
>> Hi Brian,
>>
>> You can try a couple different charm builds and then benchmark NAMD to see
>> which one gives you the best performance:
>>
>> ./build charm++ mpi-linux-x86_64 --with-production
>> ./build charm++ mpi-linux-x86_64 smp --with-production
>> ./build charm++ net-linux-x86_64 ibverbs --with-production
>> ./build charm++ net-linux-x86_64 ibverbs smp --with-production
>>
>> The smp builds of Charm/NAMD require slightly different runtime options
>> than the regular builds. Please e-mail us if you have any problems.
>>
>> - Abhinav
>>
>>
>>
>> On Mon, Aug 29, 2011 at 11:10 AM, Bennion, Brian <Bennion1 AT llnl.gov>
>> wrote:
>>
>>
>>        Hello,
>>
>>        I would like to build the proper version of charm++ for our linux
>> clusters in order to use NAMD2.8
>>
>>        Based on the specs listed on this website, which build parameters
>> would give me the best charm++ performance
>>
>>        https://computing.llnl.gov/?set=resources&page=OCF_resources#sierra
>>
>>        In the past, i have built namd/charm++ but the nodes then had only a
>> handful of cpus, now they are a bit more complicated.
>>
>>        Any suggestions would be helpful.
>>
>>
>>        brian bennion
>>
>>        _______________________________________________
>>        charm mailing list
>>        charm AT cs.uiuc.edu
>>        http://lists.cs.uiuc.edu/mailman/listinfo/charm
>>        _______________________________________________
>>        ppl mailing list
>>        ppl AT cs.uiuc.edu
>>        http://lists.cs.uiuc.edu/mailman/listinfo/ppl
>>
>>
>>
>>
>>
>> --
>> Abhinav Bhatele
>> Postdoctoral Research Staff Member
>> Center for Applied Scientific Computing
>> Lawrence Livermore National Laboratory
>>
>>
>
>
> --
> Abhinav Bhatele
> Postdoctoral Research Staff Member
> Center for Applied Scientific Computing
> Lawrence Livermore National Laboratory
>
_______________________________________________
charm mailing list
charm AT cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/charm

--
Abhinav Bhatele
Postdoctoral Research Staff Member
Center for Applied Scientific Computing
Lawrence Livermore National Laboratory