Charm++ parallel programming system

[Ekaterina Tutlyaeva <xgl AT rsc-tech.ru>]

Dear Eric,

Thank you for your fix!

Now compilation phase passes successfully, but I've encountered another issue during linking stage. It requires some "module comlib":

Generating dependencies for driver.ci
Generating dependencies for lu.ci
Generating dependencies for luUtils.ci
Generating dependencies for benchmark.C
Generating dependencies for scheduler.C
Generating dependencies for lu.C
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1 -E luUtils.ci && touch luUtils.ci.stamp
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1 -E lu.ci && touch lu.ci.stamp
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1 -E driver.ci && touch driver.ci.stamp
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -language charm++ -O3 -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1  -c -o lu.o lu.C
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -language charm++ -O3 -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1  -c -o scheduler.o scheduler.C
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -language charm++ -O3 -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1  -c -o benchmark.o benchmark.C
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -language charm++ -O3 -module comlib -module CkMulticast  -o charmlu lu.o scheduler.o benchmark.o -L/opt/intel/mkl/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_intel_ilp64.a /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_sequential.a /opt/intel/mkl/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -Wl,--end-group -lpthread -lm -ldl
/usr/bin/ld: cannot find -lmodulecomlib
collect2: error: ld returned 1 exit status
Fatal Error by charmc in directory /storage/CHARM/charmlu/charmlu

My CHARM is builded with the next options:

./build LIBS mpi-linux-x86_64 mpicxx

and I can't found the module comlib; there also no tests with it. I have charm-6.7.1.

What am I doing wrong now?

I'm tried to compile EvelyLB target for charm:

./build everylb mpi-linux-x86_64 mpicxx

where mpicxx is Intel mpi.

But the dense LU benchmark again encountered compilation issue, now with the ckmulticast.h header file for this Charm build:

Generating dependencies for driver.ci
Generating dependencies for lu.ci
Generating dependencies for luUtils.ci
Generating dependencies for benchmark.C
Generating dependencies for scheduler.C
Generating dependencies for lu.C
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1 -E luUtils.ci && touch luUtils.ci.stamp
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1 -E lu.ci && touch lu.ci.stamp
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1 -E driver.ci && touch driver.ci.stamp
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -language charm++ -O3 -DSEND_LIM=2 -I/opt/intel/mkl/include -I/opt/intel/mkl/include -DLU_REVISION=a336a1b68dec67d067035a829eea046ddf118ac1  -c -o lu.o lu.C
In file included from manager.h:5:0,
                 from lu.h:1,
                 from lu.C:13:
messages.h:5:25: fatal error: ckmulticast.h: No such file or directory
 #include <ckmulticast.h>
                         ^
compilation terminated.

Could you, please, give me advise, how the CHARM should be configured to use the heterogeneous environment, Intel MPI (and what about avx2 features for the compiler)?
I want to try to run the dense LU benchmark in heterogeneous environment with Intel CPUs...

Thank you very much for your time!

best regards,

Ekaterina

2016-10-03 3:40 GMT+03:00 Mikida, Eric P <mikida2 AT illinois.edu>:

Hi Ekaterina,

This one was actually our fault. There was a syntax error in lu.ci that popped up after a small change to how the SDAG language is defined. I fixed that error and pushed my change back to the repository. If you do a pull and try to compile again, it should work this time but let me know if more problems crop up.

Thanks

Eric

On Oct 2, 2016, at 10:03 AM, Ekaterina Tutlyaeva <xgl AT rsc-tech.ru> wrote:

Dear Sir/Madam,

I'm trying to compile the dense LU benchmark: http://charmplusplus.org/miniApps/#denselu

and I've encountered problems in linking it with Intel MKL BLAS.

It's my first experience with CHARM, so sorry for the maybe obvious question.

I have next error:

Generating dependencies for driver.ci
Generating dependencies for lu.ci
STDIN:303:575-575: error: invalid SDAG member
  };
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            ^
Fatal Error by charmc in directory /storage/CHARM/charmlu/charmlu
   Command /storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmxi -M -orig-file lu.ci returned error code 1
charmc exiting...
Generating dependencies for luUtils.ci
Generating dependencies for benchmark.C
In file included from benchmark.C:2:0:
platformBlas.h:38:2: error: #error "No BLAS Header files included!"
 #error "No BLAS Header files included!"
  ^
Generating dependencies for scheduler.C
Generating dependencies for lu.C
In file included from lu.C:15:0:
platformBlas.h:38:2: error: #error "No BLAS Header files included!"
 #error "No BLAS Header files included!"
  ^
/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/charmc -language charm++ -O3 -DSEND_LIM=2 -I/opt/intel/mkl/include -DLU_REVISION=7cec5b0ba2a6752d4160976d00cc3881e4e64b51  -c -o lu.o lu.C
In file included from lu.h:1:0,
                 from lu.C:13:
manager.h:4:26: fatal error: luUtils.decl.h: No such file or directory
 #include "luUtils.decl.h"
                          ^
compilation terminated.
Fatal Error by charmc in directory /storage/CHARM/charmlu/charmlu
   Command mpicxx -m64 -fPIC -I/storage/CHARM/charm-6.7.1/mpi-linux-x86_64-mpicxx/bin/../include -D__CHARMC__=1 -DSEND_LIM=2 -I/opt/intel/mkl/include -DLU_REVISION=7cec5b0ba2a6752d4160976d00cc3881e4e64b51 -O3 -fno-stack-protector -std=c++0x -c lu.C -o lu.o returned error code 1
charmc exiting...
make: *** [lu.o] Error 1

My config.mk contains:

SEND_LIM  = 2
MKLROOT = /opt/intel/mkl
BLAS_INC =  -I${MKLROOT}/include
BLAS_LIBS = ${MKLROOT}/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_ilp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -Wl,--end-group -lpthread -lm -ldl

The include path do contains BLAS headers:

ls -la /opt/intel/mkl/include/mkl_blas.h
-rw-r--r--. 1 root root 76919 Jun 29 14:25 /opt/intel/mkl/include/mkl_blas.h

 

 

Please, could you be so kind to give me some hint, what am I doing wrong? Where should I start?

Should I define MKL usage somewhere?

 Thank you for your time!

 

__________

Best regards,

Ekaterina Tyutlyaeva

--

__________

С уважением,

Тютляева Екатерина