Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
Chameleon
Chameleon
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 11
    • Issues 11
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 8
    • Merge Requests 8
  • Operations
    • Operations
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • solverstack
  • ChameleonChameleon
  • Issues
  • #70

Closed
Open
Opened Dec 12, 2018 by Ghost User@ghost

Issues with LAPACK detection building on Cray XC40

I am trying to compile Chameleon on our Cray XC40. Unfortunately, I am running into issues with the LAPACK detection. From the logs:

$ module swap PrgEnv-cray PrgEnv-intel
$ module load numlib/intel/mkl/2018.1
$ cmake .. -DBLAS_DIR=$MKLROOT
-- The Fortran compiler identification is Intel 18.0.1.20171018
-- The C compiler identification is Intel 18.0.1.20171018
-- The CXX compiler identification is Intel 18.0.1.20171018
-- Cray Programming Environment 2.5.15 Fortran
-- Check for working Fortran compiler: /opt/cray/pe/craype/2.5.15/bin/ftn
-- Check for working Fortran compiler: /opt/cray/pe/craype/2.5.15/bin/ftn  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /opt/cray/pe/craype/2.5.15/bin/ftn supports Fortran 90
-- Checking whether /opt/cray/pe/craype/2.5.15/bin/ftn supports Fortran 90 -- yes
-- Cray Programming Environment 2.5.15 C
-- Check for working C compiler: /opt/cray/pe/craype/2.5.15/bin/cc
-- Check for working C compiler: /opt/cray/pe/craype/2.5.15/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Cray Programming Environment 2.5.15 CXX
-- Check for working CXX compiler: /opt/cray/pe/craype/2.5.15/bin/CC
-- Check for working CXX compiler: /opt/cray/pe/craype/2.5.15/bin/CC -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/C Compiler Compatibility
-- Verifying Fortran/C Compiler Compatibility - Success
-- Found PythonInterp: /usr/bin/python2 (found version "2.7.13") 
-- CHAMELEON_SCHED_STARPU is set to ON: CHAMELEON uses StarPU runtime
   To use CHAMELEON with Quark  runtime: set CHAMELEON_SCHED_QUARK  to ON
   To use CHAMELEON with PaRSEC runtime: set CHAMELEON_SCHED_PARSEC to ON
   (CHAMELEON_SCHED_STARPU will be disabled)
-- CHAMELEON_USE_MPI is set to OFF, turn it ON to use MPI (unsupported by Quark)
-- CHAMELEON_USE_CUDA is set to OFF, turn it ON to use CUDA (unsupported by Quark)
-- CHAMELEON_ENABLE_TRACING is set to OFF, turn it ON to use FxT (with StarPU)
-- CHAMELEON_ENABLE_EXAMPLE is set to ON, turn it OFF to avoid building examples
-- CHAMELEON_ENABLE_TESTING is set to ON, turn it OFF to avoid building testing
-- CHAMELEON_ENABLE_TIMING is set to ON, turn it OFF to avoid building timing
-- CHAMELEON_SIMULATION is set to OFF, turn it ON to use SIMULATION mode (only with StarPU compiled with SimGrid)
-- CHAMELEON_ENABLE_PRUNING_STATS is set to OFF, turn it ON to build pruning statistics
-- Performing Test HAVE_WALL
-- Performing Test HAVE_WALL - Success
-- Performing Test HAVE_WEXTRA
-- Performing Test HAVE_WEXTRA - Success
-- Performing Test HAVE_FALLTHROUGH
-- Performing Test HAVE_FALLTHROUGH - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- A cache variable, namely CBLAS_DIR, has been set to specify the install directory of CBLAS
-- A cache variable, namely BLAS_DIR, has been set to specify the install directory of BLAS
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.28") 
-- Try OpenMP C flag = [-qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP Fortran flag = [-qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -qopenmp  
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - not found
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Looking for MKL BLAS: found
-- A library with BLAS API found.
-- BLAS_LIBRARIES /sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_lp64.a;/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_sequential.a;/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a;/usr/lib64/libm.a
-- Looking for cblas_dscal
-- Looking for cblas_dscal - found
-- Looking for cblas_zgemm3m
-- Looking for cblas_zgemm3m - not found
-- Looking for cblas: test with blas succeeds
-- Looking for cblas_dscal
-- Looking for cblas_dscal - found
-- Found CBLAS: /sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_lp64.a;/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_sequential.a;/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a;/usr/lib64/libm.a  
-- A cache variable, namely LAPACKE_DIR, has been set to specify the install directory of LAPACKE
-- A cache variable, namely TMG_DIR, has been set to specify the install directory of TMG
-- A cache variable, namely LAPACK_DIR, has been set to specify the install directory of LAPACK
-- Looking for MKL BLAS: found
-- A library with BLAS API found.
-- BLAS_LIBRARIES /sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_lp64.a;/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_sequential.a;/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a;/usr/lib64/libm.a
-- Looking for Fortran CHEEV
-- Looking for Fortran CHEEV - not found
-- Looking for Fortran cheev
-- Looking for Fortran cheev - not found
-- Looking for Fortran cheev
-- Looking for Fortran cheev - not found
-- Looking for MKL LAPACK: not found
-- Looking for Goto LAPACK: not found
-- Looking for Open LAPACK: not found
-- Looking for IBM ESSL LAPACK: not found
-- Looking for IBM ESSL MT LAPACK: not found
-- Looking for Apple Accelerate LAPACK: not found
-- Looking for NAS LAPACK: not found
-- Looking for Generic LAPACK: not found
CMake Warning at cmake_modules/morse_cmake/modules/find/FindLAPACK.cmake:782 (message):
  BLA_VENDOR has been set to All but LAPACK libraries could not be found or
  check of symbols failed.

  Please indicate where to find LAPACK libraries.  You have three options:

  - Option 1: Provide the installation directory of LAPACK library with cmake
  option: -DLAPACK_DIR=your/path/to/lapack

  - Option 2: Provide the directory where to find BLAS libraries with cmake
  option: -DBLAS_LIBDIR=your/path/to/blas/libs

  - Option 3: Update your environment variable (Linux: LD_LIBRARY_PATH,
  Windows: LIB, Mac: DYLD_LIBRARY_PATH)

  

  To follow libraries detection more precisely you can activate a verbose
  mode with -DLAPACK_VERBOSE=ON at cmake configure.

  You could also specify a BLAS vendor to look for by setting
  -DBLA_VENDOR=blas_vendor_name.

  List of possible BLAS vendor: Goto, ATLAS PhiPACK, CXML, DXML, SunPerf,
  SCSL, SGIMATH, IBMESSL, Intel10_32 (intel mkl v10 32 bit),Intel10_64lp
  (intel mkl v10 64 bit, lp thread model, lp64 model), Intel10_64lp_seq
  (intel mkl v10 64 bit, sequential code, lp64 model),Intel( older versions
  of mkl 32 and 64 bit), ACML, ACML_MP, ACML_GPU, Apple, NAS, Generic
Call Stack (most recent call first):
  cmake_modules/morse_cmake/modules/find/FindTMG.cmake:77 (find_package)
  cmake_modules/morse_cmake/modules/find/FindLAPACKE.cmake:109 (find_package)
  CMakeLists.txt:345 (find_package)


CMake Error at cmake_modules/morse_cmake/modules/find/FindLAPACK.cmake:793 (message):
  A required library with LAPACK API not found.  Please specify library
  location.
Call Stack (most recent call first):
  cmake_modules/morse_cmake/modules/find/FindTMG.cmake:77 (find_package)
  cmake_modules/morse_cmake/modules/find/FindLAPACKE.cmake:109 (find_package)
  CMakeLists.txt:345 (find_package)


-- Configuring incomplete, errors occurred![CMakeError.log](/uploads/f1c4c87d42a8f940df6820261b88efe3/CMakeError.log)

I am attaching the CMake error log. I assume that there is an issue with the link order of the static libraries.

If I try building with support for dynamic libraries (export CRAYPE_LINK_TYPE=dynamic), LAPACK detection succeeds but linking of executables fails as follows:

[ 87%] Linking Fortran executable time_dgetrs_nopiv_tile
CMakeFiles/time_sgeqrf_hqr.dir/time_sgeqrf_hqr.c.o: In function `main':
time_sgeqrf_hqr.c:(.text+0x0): multiple definition of `main'
/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin/for_main.o:for_main.c:(.text+0x0): first defined here
/sw/hazelhen-cle6/hlrs/compiler/intel/Compiler/18.0.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin/for_main.o: In function `main':
for_main.c:(.text+0x2a): undefined reference to `MAIN__'
timing/CMakeFiles/time_sgeqrf_hqr.dir/build.make:114: recipe for target 'timing/time_sgeqrf_hqr' failed
make[2]: *** [timing/time_sgeqrf_hqr] Error 1

If I manually replace ftn with cc the linking succeeds. I tried running CMake with -DCMAKE_LINKER=/opt/cray/pe/craype/2.5.15/bin/cc but that seems to be ignored.

Tested on commit 55938b987201c89f007eae3cc321597536096c06.

Please let me know if I can provide any additional information on this.

To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
Chameleon 0.9.2
Milestone
Chameleon 0.9.2
Assign milestone
Time tracking
None
Due date
None
Reference: solverstack/chameleon#70