Welcome to LZN's Blog!
Wind extinguishes a candle but energizes fire.-
METCTM1
Final Configurations
PGI Compilers
Check version for the environment:
which pgcc /usr/local/pgi-20cos7/linux86-64/2020/bin/pgccMPICH3
No MPI available. Try to compile by myself. Official Site
./configure --prefix=/home/metctm1/soft/mpich3-pgi20 FC=pgfortran F77=pgfortran CC=pgcc CXX=pgc++ rsh=ssh # no error, 10 minutes make # works! more than 1 hour make installEnvironmental variables
export MPICH=/home/metctm1/soft/mpich3-pgi20 export PATH=$MPICH/bin:$PATH export LD_LIBRARY_PATH=$MPICH/lib:$LD_LIBRARY_PATH export INCLUDE=$MPICH/include:$INCLUDEZLIB
Zlib for HDF5 and NetCDF4.
Install zlib from source first, defualt configurations.
Environmental variables
export ZDIR=/home/metctm1/soft/zlib-1211-gcc export LD_LIBRARY_PATH=${ZDIR}/lib:${LD_LIBRARY_PATH} export INCLUDE=${ZDIR}/include:${INCLUDE}HDF5
Choose HDF5 1.10 to avoid compatibility issues.
Try the following command to compile the HDF5 with PGI compilers.
$CPP=cpp CFLAGS="-fPIC -m64 -tp=px" CXXFLAGS="-fPIC -m64 -tp=px" FCFLAGS="-fPIC -m64 -tp=px" CC=pgcc CXX=pgc++ FC=pgfortran ./configure --with-zlib=/home/metctm1/array/soft/zlib-1.2.11-gcc --prefix=/home/metctm1/array/soft/hdf5-1.10.7-pgi20-amd --enable-hl --enable-threadsafe --enable-cxx --enable-fortran --enable-unsupported $make # Successful! about 1 hour $make installEnvironmental variables
HDF5=/home/metctm1/soft/hdf5-1107-pgi export PATH=$HDF5/bin:$PATH export LD_LIBRARY_PATH=$HDF5/lib:$LD_LIBRARY_PATH export INCLUDE=$HDF5/include:$INCLUDENetCDF
Get the latest release for C and Fortran.
Note to assign the paths explicitly!
# C Lib CPPFLAGS='-I/home/metctm1/array/soft/hdf5-1.10.7-pgi20/include -I/home/metctm1/soft/zlib-1211-gcc/include' LDFLAGS='-L/home/metctm1/array/soft/hdf5-1.10.7-pgi20/lib -L/home/metctm1/soft/zlib-1211-gcc/lib' ./configure --prefix=/home/metctm1/array/soft/netcdf-472c453f-pgi20 --disable-dap CC=pgcc make make check make install # Fortran Lib CPPFLAGS='-I/home/metctm1/array/soft/hdf5-1.10.7-pgi20/include -I/home/metctm1/soft/zlib-1211-gcc/include -I/home/metctm1/array/soft/netcdf-474c453f-pgi20/include' LDFLAGS='-L/home/metctm1/array/soft/hdf5-1.10.7-pgi20/lib -L/home/metctm1/soft/zlib-1211-gcc/lib -L/home/metctm1/array/soft/netcdf-474c453f-pgi20/lib' ./configure --prefix=/home/metctm1/array/soft/netcdf-474c453f-pgi20 --disable-dap FC=pgfortranEnvironmental variables
export NETCDF=/home/metctm1/array/soft/netcdf-474c453f-pgi20/ export PATH=$NETCDF/bin:$PATH export LD_LIBRARY_PATH=$NETCDF/lib:$LD_LIBRARY_PATH export INCLUDE=$NETCDF/include/:$INCLUDEMCT
./configure --prefix=/.... FC=pgfortran CC=pgccModify the
Makefile.confto specify the MPI lib and include. Done.COAWST
Issues
HDF5
Using the PGI compiler to compile HDF5 is quite torturous, a good reference from Fluid Numerics:
HDF5 has long been a standard for sharing scientific data across High Performance Computing (HPC) platforms. From the HDF website "HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5." Other data models, like NetCDF from UCAR, are built on top of HDF5 and are heavily used in oceanography, meteorology, and other earth science domains. Given this, many applications rely on having a usable working version of HDF5 that they can link into their applications. The Portland Group Compilers are gaining popularity with the growing interest in accelerating HPC applications with Graphics Processing Units (GPUs). PGI compilers ship completely with the CUDA toolkit and a Fortran compiler (pgfortran) that support OpenACC and CUDA-Fortran, languages for writing code that operates on GPUs. This is a plus for Fortran programmers, who have a large presence in Department of Energy labs and in earth science departments around the world. Other options for GPU acceleration include FortranCL, CUDA (C/C++), OpenCL, and HIP. It is fair to say that the lack of activity on the main FortranCL repository for the last 6 years suggests the project has long been forgotten; this makes it unattractive for developers to latch on to this as a solution. The other languages are supported in C/C++ syntax and require Fortran programmers to develop C/C++ in addition to an ISO_C_BINDING layer to call their C-kernels from Fortran. Building HDF5 with parallel and Fortran support with PGI compilers is not as straight-forward as building with other compilers, like GCC. I came across this issue while setting up clusters for hackathons, and defining build instructions for SELF-Fluids ( personal software project ). Through this experience, I have discovered that there are hints at the solution on the HDF Forums; in fact, this is where I found that a GNU C-preprocessor needs to be used in place of PGI's. This did enable a build of HDF5 with serial support, but the OpenMPI build that ships with PGI compilers cannot be used for parallel support, and instead OpenMPI was built from scratch. This document provides details necessary for compiling HDF5 from source with the PGI compilers. HDF5 is built using an autotools build system. Template configure incantations are provided with the install notes along with expected output at the configure stage of the build. Ultimately, this was a roughly 16 hour exploration into this build issue that ultimately led to its resolution.NetCDF
It was quite anoying that the root reason of the failure in compiling NetCDF is that:
- The system manager has compiled a legacy HDF5 (1.8.12 or so) in the root paths (
/bin,/lib/usr/include) with unknown environment. - If we use the following command (default from the netCDF website) with pre-assigned variables
ZDIRandH5DIR, it seemsmakewill give higher priority to the root paths than the variable-assigned paths.
ZDIR=/home/metctm1/soft/zlib-1211-gcc H5DIR=/home/metctm1/soft/hdf5-1107-pgi CPPFLAGS='-I${H5DIR}/include -I${ZDIR}/include' LDFLAGS='-L${H5DIR}/lib -L${ZDIR}/lib' ./configure --prefix=${NCDIR} --disable-dap CC=pgccError Log
configure: error: HDF5 was not built with zlib, which is required. Rebuild HDF5 with zlib.Interesting. It seems the NETCDF does not recognize the PGI-built HDF5 embeded with GNU-built ZLIB. Go Trial II.
TRY:
CPP=cpp CPPFLAGS='-I${H5DIR}/include -I${ZDIR}/include' LDFLAGS='-L${H5DIR}/lib -L${ZDIR}/lib' ./configure --prefix=${NCDIR} --disable-dap CC=pgccgot error:
configure: error: C preprocessor "cpp" fails sanity checkThe NETCDF seems to ignore the $CPP variable shift to .
Here I went back to recompile HDF5 with gcc compiler, still not work. I noticed it could be a version issue in NetCDF. Change NetCDF-C-474 back to NetCDF-C-473, both HDF5 compiled by GCC or PGI works! The question is in the $make check, the NetCDF still fails all HDF5 test with PGI compiled HDF5…
Bad situation, we need to permutate all possible combinations to get the best practice…
need to refer the official guide.
COAWST
pgi-16cos7 Error in SWAN: PGF90-S-0285-Source line too long (mod_strings.f90: 228)
This is a legacy problem of PGI compiler:
Interestingly, the latest version of the PGI compilers (18.10, release last Friday) now support line lengths up to 1000 characters. That means that the problem in this bugreport is ‘solved’, or at least no longer a problem, compilation is successful out-of-the-box on the 18.10 build.
Changing compiler version does not work as the HDF5 and NetCDF depend on the PGI16. We modify the source code in
ROMS/Modules/mod_stringsto hard-code the multi-line char initialization.character (len=512) :: my_fflags =" -fastsse -Mipa=fast -Kieee -I/home/metctm1/array/soft/MCT4COAWST/include& & -I/home/metctm1/array/app/COAWST/COAWST201205//WRF/main -I/home/metctm1/array/app/COAWST/COAWST201205//WRF/external/esmf_time_f90& & -I/home/metctm1/array/app/COAWST/COAWST201205//WRF/frame -I/home/metctm1/array/app/COAWST/COAWST201205//WRF/share -Mfree -Mfree"Then, the final link problem:
IPA: no IPA optimizationsAfter comment out -Mipa=fast in ROMS, still not work (We finally found this is not needed).
We found another possible error:
WRF/main/libwrflib.a(module_ra_rrtmg_lwf.o): In function `memory_': module_ra_rrtmg_lwf.f90:(.text+0x0): multiple definition of `memory_' ./Build/libUTIL.a(memory.o):memory.f90:(.text+0x0): first defined hereWe rename the module
memorytomemory_wrfand it works!Refrence
https://www.fluidnumerics.com/resources/building-hdf5-with-pgi
Updated 2020-12-10**
- The system manager has compiled a legacy HDF5 (1.8.12 or so) in the root paths (
-
Scratch Notes for Stanford CS107: Programming Paradigm Ch05
L5: Generic Lsearch - Prototype
Notes
LSearch
First review the lsearch function:
void *lsearch(void *k, void *base, int n, int elemSize, int(*cmpfn)(void *, void *)) { for (int i=0; i<n; i++){ void *elemAddr= base+i*elemSize; //01 }01: This is a strictly pointer arithmetic against a typeless (void *) pointer.
Appendix
Glossary
ebb 退潮 mantissa 尾数Updated 2020-10-20
-
Scratch Notes for Stanford CS107: Programming Paradigm Ch04
L4: Creating a Generic Swap Function for Data Types of Arbitrary Size
Notes
Generic Swap
The generic swap function in the last lecture is useful for 4-byte figures. How to make it be a universial function? Look at the following realization:
void swap(void *vp1, void *vp2, int size) { char buffer[size]; memcpy(buffer, vp1, size); memcpy(vp1, vp2, size); memcpy(vp2, buffer, size); }; int x=17; y=37; swap(&x,&y,sizeof(int));void *means a generic pointer.char buffer[size]is to set aside enough space to hold byte info.memcpy(target, source, size)is a more generic function to copy byte info with respect to the byte length. Most energy is used to dynamically allocate and deallocate buffer.The perk of using this method compared to C++ template is that this way is lean and economical, while the C++ template will generate patches as much as the call times in the executables.
Wrong usage:
int i = 44; short j=5; swap(&i, &j, sizeof(short))If using generic code, you have to be very careful about the type. The above usage will cause unexpected result as the
shortandintare different types with different byte lengthes.Another example of the generic code (a very complicated case):
char * husband = strdup("Fred"); char * wife = strdup("Wilma"); // Right usage swap(&husband, &wife, sizeof(char *))Note here we are acturally swap the bit patterns (address stored in char *) of two char pointers. We still need the reference operator
&. See the blackboard:
Some terrible errors:
swap(husband, wife, sizeof(char *)) // will swap the first 4 chars in heap swap(husband, &wife, sizeof(char *)) // will make things chaos, the first 4 chars will be interpreted as an address!Generic lsearch
An example for linear search:
int lsearch(int key, int array[], int size){ for (i=0;i<size;i++){ if (array[i]==key){ return i; } } return -1; }Let’s make it generic! How about a undefinite type of array?
void *lsearch(void *key, void *base, int n, int elemSize){ for (i=0;i<n;i++){ void *elemAddr = (char *) base + i*elemSize; if (memcmp(key, elemAddr, elemSize)==0) return elemAddr; return NULL; } }It does not work for character pointers or C-strings.
Qustions
Appendix
C++ Template
Function templates are special functions that can operate with generic types. This allows us to create a function template whose functionality can be adapted to more than one type or class without repeating the entire code for each type.
template <class myType> myType GetMax (myType a, myType b) { return (a>b?a:b); } int x, y; GetMax <int> (x, y);Glossary
perk 额外好处 sedate v. 安慰 sentinel 哨兵Updated 2020-08-06
- A bit of experience and bitter reflection from my visit to CD
- 货币的价值
- 2018年2月5日“波动性末日”（Volmageddon）的回顾与思考
- 24年10月市场观察
- EGU23-10723 Supplementary Materials: Mangkhut (2018)
- Intel MPI specified compiler
- git submodules
- PYROMS Installation
- Coursera Machine Learning : W9 Anomaly Detection
- PGI21 WRF-CMAQ 2WAY