Logo Cineca Logo SCAI
MARCONI status
GALILEO status
DAVIDE status

You are here

Tips for Linking Libraries on the IBM SP6 (AIX)

In this page:

 
Introduction
 
In this document we describe how libraries are found when an executable is launched on the IBM SP6 at CINECA, together with some useful linking options. A description of how to create static and shared libraries is left to another document.
 
For reference, you may find this link useful:
 
 
In the following we use the example of an external library, FFTW, that a user has in his own work area (lib/ and include/). The program to compile is a FORTRAN source called f77_test.F
 
-bash-3.2$ ls
exe* f77_test.F include/ lib/
 
(From time-to-time we will rename the library lib to lib2 to illustrate some point: remember to rename it back to lib for later examples).
 
Standard Linking with the -L compiler option
 
The usual way of linking with an external library is as follows:
 
xlf f77_test.F -o exe -Iinclude -Llib -lfftw3
 
Even if the compiling+linking gave no errors you should anyway check the executable with the ldd command:
 
-bash-3.2$ ldd exe
exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lpp/xlf/lib/libxlf90.a(io_64.o)
 /gpfs/scratch/userinternal/aem0/aix-linking/lib/libfftw3.a(libfftw3.so.3)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)
 
This command will show you what libraries are needed by the executable. If ldd indicates a library hasn’t been found then the program wont run. Note that in the simple example above xlf will not create the executable if it cant find the FFTW library, but this is no guarantee that the resulting executable will be usable as we will see.
 
Another useful command is dump:
 
dump -Hv exe
 
exe:

 ***Loader Section***
 Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000010 0x00000022 0x00000064 
 
#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000004 0x000003d8 0x000000f0 0x0000043c 
  
 ***Import File Strings***

INDEX PATH BASE MEMBER 
0 lib:/usr/lpp/xlf/lib:/usr/lib:/lib 
1 libc.a shr_64.o 
2 libxlf90.a io_64.o 
3 libfftw3.a libfftw3.so.3 
 
Look at the Import File Strings section (from now on we won't show the Loader Section). The line containing 0 as INDEX indicates where the loader will look to find libraries before running the program.  There should always be the paths /usr/lib:/lib - other paths come from the -L option to the compiler. For example, if you compile like this
 
xlf f77_test.F -o exe -Iinclude -L. -Llib -lfftw3 
 
then the output of dump -Hv will contain
 
 ***Import File Strings***
INDEX PATH BASE MEMBER 
0 .:lib:/usr/lpp/xlf/lib:/usr/lib:/lib 
 
i.e. it will also contain the current directory . as specified in the first -L option.
 
Linking with the -L option and LIBPATH
 
If the library is moved between linking and runtime then the program will not execute. For example
 
mv lib lib2
ldd exe
exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lpp/xlf/lib/libxlf90.a(io_64.o)
Cannot find libfftw3.a(libfftw3.so.3)
 /unix
 /usr/lib/libcrypt.a(shr_64.o) 
 
In this case you can use the LIBPATH variable to tell the loader to look as well in the new location.
 
export LIBPATH=lib2
 
This has the effect of adding the LIBPATH location(s) to the list given by the INDEX 0 section of the dump output. Now we have
 
-bash-3.2$ ldd exe 

exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lpp/xlf/lib/libxlf90.a(io_64.o)
 lib2/libfftw3.a(libfftw3.so.3)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)
 
Linking without the -L option
 
For simple installations, the procedure above is the most convenient way of linking external libraries but in more complex cases it can be desirable 
to link a library directly into an executable without using the -L option of the compiler. For example,
 
xlf f77_test.F -o exe -Iinclude lib/libfftw3.a 
 
This executable will run as shown by ldd:
 
ldd exe
exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lpp/xlf/lib/libxlf90.a(io_64.o)
 lib/libfftw3.a(libfftw3.so.3)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)
 
Note however the output of dump:
 
INDEX PATH BASE MEMBER 
0 /usr/lpp/xlf/lib:/usr/lib:/lib 
1 libc.a shr_64.o 
2 libxlf90.a io_64.o 
3 lib libfftw3.a libfftw3.so.3 
 
Since the -L option wasn’t used the INDEX 0 line wasn’t changed and the entry for libfftw3.a now has a path, i.e. lib/, something which never happens with the -L$DIR -llibname combination. This means that the libfftw3 library must be found in this location and changing LIBPATH will have no effect. Try this
 
mv lib lib2
export LIBPATH=lib2
ldd exe
exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lpp/xlf/lib/libxlf90.a(io_64.o)
Cannot find lib/libfftw3.a(libfftw3.so.3)
 /unix
 /usr/lib/libcrypt.a(shr_64.o) 
 
To avoid this you can remove this path information from the executable with the -bnoipath linker option:
 
xlf f77_test.F -o exe -Iinclude -bnoipath lib/libfftw3.a
dump -Hv exe
 
 ***Import File Strings***

INDEX PATH BASE MEMBER 
0 /usr/lpp/xlf/lib:/usr/lib:/lib 
1 libc.a shr_64.o 
2 libxlf90.a io_64.o 
3 libfftw3.a libfftw3.so.3 
 
Now we can use LIBPATH if libfftw3 is moved:
 
-bash-3.2$ mv lib lib2; export LIBPATH=lib2
-bash-3.2$ ldd exe 
exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lpp/xlf/lib/libxlf90.a(io_64.o)
 lib2/libfftw3.a(libfftw3.so.3)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)
 
In situations where it is desirable not to use the -L option you should instead make sure to specify -bnoipath otherwise you will tie the executable to a particular directory structure.
 
 
The -blibpath option   
 
As you can see from the dump command, an executable can easily accumulate a long list of library paths, some of which may not be used in the final version of the executable. In addition these paths may not be relevant to other installations of AIX. For example, if you decide to link against the FFTW module present on the CINECA  SP6:
 
module load fftw/3.2.2--xl--10.1
xlf f77_test.F -o exe -I$FFTW_INC -L$FFTW_LIB -lfftw3
 
dump -Hv exe
0 /cineca/prod/libraries/fftw/3.2.2/xl--10.1/lib:/usr/lpp/xlf/lib:/usr/lib:/lib 
1 libc.a shr_64.o 
2 libxlf90.a io_64.o 
3 libfftw3.a libfftw3.so.3 
 
Clearly this probably wont work on a different site unless LIBPATH is used and even when it is the /cineca/prod.. path will be redundant.
When compiling programs which might be used on different machines, or where the libraries can be move, it is possible to explicitly set the search path with the -blibpath linker option (often using the default /usr/lib:/lib )
 
xlf f77_test.F -o exe -I$FFTW_INC -L$FFTW_LIB -lfftw3 -blibpath:/usr/lib:/lib
dump -Hv exe
0 /usr/lib:/lib 
1 libc.a shr_64.o 
2 libxlf90.a io_64.o 
3 libfftw3.a libfftw3.so.3 
 
Thus, the executable won’t run unless LIBPATH is also set (which is in fact what the CINECA module does). This is useful because you now have the option of changing the FFTW library at runtime, rather than recompiling the whole executable if you change your mind.
 
Be careful though! Use of -blibpath requires that you know all the libraries that need to be linked. For example, let’s assume that we have an MPI program compiled like this:
 
mpxlf f77_test.F -o exe -I$FFTW_INC -L$FFTW_LIB -lfftw3 -blibpath:/usr/lib:/lib 
 
If we look at the executable,
 
-bash-3.2$ ldd exe
exe needs:
 /usr/lib/libc.a(shr_64.o)
 /usr/lib/libpthreads.a(shr_xpg5_64.o)
 /usr/lib/libxlf90.a(io_64.o)
 /cineca/prod/libraries/fftw/3.2.2/xl--10.1/lib/libfftw3.a(libfftw3.so.3)
Cannot find libmpi_r.a(mpipoe64_r.o)
 /unix
 /usr/lib/libcrypt.a(shr_64.o)
 
we see that this wont work because we didn’t include the MPI library paths in -blibpath or LIBPATH. Obviously this can be done but it might be easier just not to use it in the first place.  
 
Summary
 
1.      The standard way used to link external libraries into an executable is with the -L and -l linker options. The path after each -L option is inserted into the executable.
2.      If the libraries are moved then you need to set the library search path with the LIBPATH environment variable, but otherwise it can be unset.
3.      You can link the libraries directly into the executable without the -L option but in this case you should add the -bnoipath option otherwise the path of the library will be hardcoded into the executable and this is unaffected by LIBPATH.
4.      You can use -blibpath to reset the default library search path but it is only really advisable if you are a code developer or if it is likely that the library search path will change often.