CUDA: undefined reference to CUDA_hlag2d
I'm trying to build Chameleon with StarPU and CUDA. I've installed StarPU (latest git revision) with support for CUDA. When building Chameleon, I get the following error
[ 90%] Linking C executable chameleon_stesting
cd /home/ftrahay/catherine/scripts/software/src/chameleon/build/testing && /usr/bin/cmake -E cmake_link_script CMakeFiles/chameleon_stesting.dir/link.txt --verbose=1
/usr/bin/cc -O2 -g -DNDEBUG -Wl,-rpath -Wl,/home/ftrahay/catherine/scripts/software/install/mpich/lib -Wl,--enable-new-dtags CMakeFiles/chameleon_stesting.dir/chameleon_stesting.c.o CMakeFiles/chameleon_stesting.dir/testing_scheck_aux.c.o CMakeFiles/chameleon_stesting.dir/testing_scheck_blas.c.o CMakeFiles/chameleon_stesting.dir/testing_scheck_facto.c.o CMakeFiles/chameleon_stesting.dir/testing_scheck_qr_lq.c.o CMakeFiles/chameleon_stesting.dir/testing_scheck_svd.c.o CMakeFiles/chameleon_stesting.dir/testing_scheck_polar_decomp.c.o CMakeFiles/chameleon_stesting.dir/testing_sprint.c.o CMakeFiles/chameleon_stesting.dir/testing_slange.c.o CMakeFiles/chameleon_stesting.dir/testing_slansy.c.o CMakeFiles/chameleon_stesting.dir/testing_slantr.c.o CMakeFiles/chameleon_stesting.dir/testing_sgemm.c.o CMakeFiles/chameleon_stesting.dir/testing_ssymm.c.o CMakeFiles/chameleon_stesting.dir/testing_ssyrk.c.o CMakeFiles/chameleon_stesting.dir/testing_ssyr2k.c.o CMakeFiles/chameleon_stesting.dir/testing_strmm.c.o CMakeFiles/chameleon_stesting.dir/testing_strsm.c.o CMakeFiles/chameleon_stesting.dir/testing_spotrf.c.o CMakeFiles/chameleon_stesting.dir/testing_spotrs.c.o CMakeFiles/chameleon_stesting.dir/testing_sposv.c.o CMakeFiles/chameleon_stesting.dir/testing_spotri.c.o CMakeFiles/chameleon_stesting.dir/testing_spoinv.c.o CMakeFiles/chameleon_stesting.dir/testing_slacpy.c.o CMakeFiles/chameleon_stesting.dir/testing_sgeadd.c.o CMakeFiles/chameleon_stesting.dir/testing_stradd.c.o CMakeFiles/chameleon_stesting.dir/testing_slascal.c.o CMakeFiles/chameleon_stesting.dir/testing_strtri.c.o CMakeFiles/chameleon_stesting.dir/testing_slauum.c.o CMakeFiles/chameleon_stesting.dir/testing_sgenm2.c.o CMakeFiles/chameleon_stesting.dir/testing_sgesv_nopiv.c.o CMakeFiles/chameleon_stesting.dir/testing_sgesvd.c.o CMakeFiles/chameleon_stesting.dir/testing_sgetrf.c.o CMakeFiles/chameleon_stesting.dir/testing_sgetrf_nopiv.c.o CMakeFiles/chameleon_stesting.dir/testing_sgetrs_nopiv.c.o CMakeFiles/chameleon_stesting.dir/testing_sgeqrf.c.o CMakeFiles/chameleon_stesting.dir/testing_sorgqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sormqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sgelqf.c.o CMakeFiles/chameleon_stesting.dir/testing_sorglq.c.o CMakeFiles/chameleon_stesting.dir/testing_sormlq.c.o CMakeFiles/chameleon_stesting.dir/testing_sgels.c.o CMakeFiles/chameleon_stesting.dir/testing_sgeqrf_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sorgqr_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sormqr_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sgelqf_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sorglq_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sormlq_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sgels_hqr.c.o CMakeFiles/chameleon_stesting.dir/testing_sgepdf_qdwh.c.o CMakeFiles/chameleon_stesting.dir/testing_sgepdf_qr.c.o CMakeFiles/chameleon_stesting.dir/testing_splrnk.c.o CMakeFiles/chameleon_stesting.dir/testing_scesca.c.o CMakeFiles/chameleon_stesting.dir/testing_sgram.c.o CMakeFiles/chameleon_stesting.dir/values.c.o CMakeFiles/chameleon_stesting.dir/run_list.c.o CMakeFiles/chameleon_stesting.dir/parameters.c.o CMakeFiles/chameleon_stesting.dir/testings.c.o -o chameleon_stesting -Wl,-rpath,/home/ftrahay/catherine/scripts/software/src/chameleon/build/lapack_api:/home/ftrahay/catherine/scripts/software/src/chameleon/build/compute:/home/ftrahay/catherine/scripts/software/src/chameleon/build/runtime/starpu:/home/ftrahay/catherine/scripts/software/src/chameleon/build/hqr/src:/home/ftrahay/catherine/scripts/software/install/mpich/lib:/home/ftrahay/catherine/scripts/software/src/chameleon/build/gpucublas/compute:/usr/local/cuda/lib64:/home/ftrahay/catherine/scripts/software/src/chameleon/build/coreblas/compute: ../lapack_api/libchameleon_lapack.so.1.3.0 ../compute/libchameleon.so.1.3.0 ../runtime/starpu/libchameleon_starpu.so.1.3.0 ../hqr/src/libhqr.so.0.1.0 /home/ftrahay/catherine/scripts/software/install/mpich/lib/libmpi.so ../gpucublas/compute/libgpucublas.so.1.3.0 /usr/local/cuda/lib64/libcublas.so /usr/local/cuda/lib64/libcudart_static.a -ldl /usr/lib/x86_64-linux-gnu/librt.a ../coreblas/compute/libcoreblas.so.1.3.0 -lm /usr/lib/x86_64-linux-gnu/libmkl_gf_lp64.so /usr/lib/x86_64-linux-gnu/libmkl_sequential.so /usr/lib/x86_64-linux-gnu/libmkl_core.so -lm -ldl
/usr/bin/ld : ../runtime/starpu/libchameleon_starpu.so.1.3.0 : référence indéfinie vers « CUDA_hlag2d »
/usr/bin/ld : ../runtime/starpu/libchameleon_starpu.so.1.3.0 : référence indéfinie vers « CUDA_dlag2h »
/usr/bin/ld : ../runtime/starpu/libchameleon_starpu.so.1.3.0 : référence indéfinie vers « CUDA_slag2h »
/usr/bin/ld : ../runtime/starpu/libchameleon_starpu.so.1.3.0 : référence indéfinie vers « CUDA_hlag2s »
/usr/bin/ld : ../runtime/starpu/libchameleon_starpu.so.1.3.0 : référence indéfinie vers « CUDA_dlag2s »
/usr/bin/ld : ../runtime/starpu/libchameleon_starpu.so.1.3.0 : référence indéfinie vers « CUDA_slag2d »
collect2: error: ld returned 1 exit status
make[2]: *** [testing/CMakeFiles/chameleon_stesting.dir/build.make:1378 : testing/chameleon_stesting] Erreur 1
make[2] : on quitte le répertoire « /home/ftrahay/catherine/scripts/software/src/chameleon/build »
make[1]: *** [CMakeFiles/Makefile2:1955 : testing/CMakeFiles/chameleon_stesting.dir/all] Erreur 2
make[1] : on quitte le répertoire « /home/ftrahay/catherine/scripts/software/src/chameleon/build »
make: *** [Makefile:166 : all] Erreur 2
Indeed, these symbols are not defined in libchameleon_starpu.so.1.3.0
:
$ nm ./runtime/starpu/libchameleon_starpu.so.1.3.0 |grep "CUDA_.lag"
U CUDA_dlag2h
U CUDA_dlag2s
U CUDA_hlag2d
U CUDA_hlag2s
U CUDA_slag2d
U CUDA_slag2h
CUDA_dlag2h
is defined in gpucublas/compute/cuda_dlag2h.cu
, but it looks like it is not compiled (adding #error plop
does not raise any error). I can see that this file is "converted":
[ 8%] Generating cuda_slag2h.cu
cd /home/ftrahay/catherine/scripts/software/src/chameleon/build/gpucublas/compute && /usr/bin/cmake -E remove -f cuda_slag2h.cu && /usr/bin/python3 /home/ftrahay/catherine/scripts/software/src/chameleon/cmake_modules/morse_cmake/modules/precision_generator/codegen.py -g -f /home/ftrahay/catherine/scripts/software/src/chameleon/gpucublas/compute/cuda_dlag2h.cu -b /home/ftrahay/catherine/scripts/software/src/chameleon/build/gpucublas/compute -p s -P ./ -D /home/ftrahay/catherine/scripts/software/src/chameleon/cmake_modules/local_subs.py && chmod a-w cuda_slag2h.cu
But the generated file is not used after that.
Do you know how to fix this problem ?