Welcome to LZN's Blog!

Wind extinguishes a candle but energizes fire.

PGI21 WRF-CMAQ 2WAY

2021-08-29

LZN

technology

linux

YQ need to compile CMAQ-WRF 2-way, now we archive the process.

Final Configurations

Libs

NVIDIA/hpc_sdk.2021.2
hdf5-1.12.2
netCDF-C v4.8.1
netCDF-F v4.5.3

HDF5

CPP=cpp CFLAGS="-fPIC -m64 -tp=px" CXXFLAGS="-fPIC -m64 -tp=px" FCFLAGS="-fPIC -m64 -tp=px" CC=pgcc CXX=pgc++ FC=pgfortran ./configure --with-zlib=/home/yhuangci/soft/zlib-1.2.11-gcc --prefix=/home/yhuangci/soft/hdf5-1.12.1-pgi21 --enable-hl --enable-threadsafe --enable-cxx --enable-fortran --enable-unsupported
make
make install

NetCDF-C

CPPFLAGS='-I/home/yhuangci/soft/hdf5-1.12.1-pgi21/include -I/home/yhuangci/soft/zlib-1.2.11-gcc/include' LDFLAGS='-L/home/yhuangci/soft/hdf5-1.12.1-pgi21/lib -L/home/yhuangci/soft/zlib-1.2.11-gcc/lib' ./configure --prefix=/home/yhuangci/soft/netcdf-481c453f-pgi21 --disable-dap CC=pgcc
make
make install

NetCDF-F

CPPFLAGS='-I/home/yhuangci/soft/hdf5-1.12.1-pgi21/include -I/home/yhuangci/soft/zlib-1.2.11-gcc/include -I/home/yhuangci/soft/netcdf-481c453f-pgi21/include' LDFLAGS='-L/home/yhuangci/hdf5-1.12.1-pgi21/lib -L/home/yhuangci/soft/zlib-1.2.11-gcc/lib -L/home/yhuangci/soft/netcdf-481c453f-pgi21/lib' ./configure --prefix=/home/yhuangci/soft/netcdf-481c453f-pgi21 --disable-dap FC=pgfortran
make
make install

Whole Porting Process

Libs

NVIDIA/hpc_sdk.2021.2
mpich-3.4.2
hdf5-1.12.2
netCDF-C v4.8.1
netCDF-F v4.5.3

MPICH

 ./configure --prefix=/home/yhuangci/soft/mpich342-pgi21 FC=pgfortran F77=pgfortran CC=pgcc CXX=pgc++ rsh=ssh

with error:

configure: error: no ch4 netmod selected

  The default ch4 device could not detect a preferred network
  library. Supported options are ofi (libfabric) and ucx:

    --with-device=ch4:ofi or --with-device=ch4:ucx

  Configure will use an embedded copy of libfabric or ucx if one is
  not found in the user environment. An installation can be specified
  by adding

    --with-libfabric=<path/to/install> or --with-ucx=<path/to/install>

  to the configuration.

  The previous MPICH default device (ch3) is also available and
  supported with option:

    --with-device=ch3

Try:

./configure --prefix=/home/yhuangci/soft/mpich342-pgi21 FC=pgfortran F77=pgfortran CC=pgcc CXX=pgc++ rsh=ssh --with-device=ch4:ofi

Pass!

make 2>&1 | tee make.log    # about 1 hour

Got error:

/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:338: undefined reference to `fi_dupinfo'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:313: undefined reference to `fi_getinfo'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:321: undefined reference to `fi_getparams'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:323: undefined reference to `fi_tostr'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:327: undefined reference to `fi_tostr'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:327: undefined reference to `fi_tostr'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:321: undefined reference to `fi_freeparams'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:329: undefined reference to `fi_freeinfo'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:431: undefined reference to `fi_freeinfo'
/home/yhuangci/soft/mpich-3.4.2/modules/libfabric/util/info.c:353: undefined reference to `fi_tostr'
make[3]: *** [util/fi_info] Error 2
make[3]: Leaving directory `/home/yhuangci/soft/mpich-3.4.2/modules/libfabric'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/yhuangci/soft/mpich-3.4.2/modules/libfabric'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/yhuangci/soft/mpich-3.4.2'
make: *** [all] Error 2

Retry config:

./configure --prefix=/home/yhuangci/soft/mpich342-pgi21 FC=pgfortran F77=pgfortran CC=pgcc CXX=pgc++ rsh=ssh --with-device=ch3
make 2>&1 | tee make.log

Pass! make with on error but make check with 4 items not passed. Let’s leave it alone first.

HDF5

CPP=cpp CFLAGS="-fPIC -m64 -tp=px" CXXFLAGS="-fPIC -m64 -tp=px" FCFLAGS="-fPIC -m64 -tp=px" CC=pgcc CXX=pgc++ FC=pgfortran ./configure --with-zlib=/home/yhuangci/soft/zlib-1.2.11-gcc --prefix=/home/yhuangci/soft/hdf5-1.12.1-pgi21 --enable-hl --enable-threadsafe --enable-cxx --enable-fortran --enable-unsupported
make
make install

NetCDF-C

CPPFLAGS='-I/home/yhuangci/soft/hdf5-1.12.1-pgi21/include -I/home/yhuangci/soft/zlib-1.2.11-gcc/include' LDFLAGS='-L/home/yhuangci/soft/hdf5-1.12.1-pgi21/lib -L/home/yhuangci/soft/zlib-1.2.11-gcc/lib' ./configure --prefix=/home/yhuangci/soft/netcdf-481c453f-pgi21 --disable-dap CC=pgcc
make
make install

NetCDF-F

CPPFLAGS='-I/home/yhuangci/soft/hdf5-1.12.1-pgi21/include -I/home/yhuangci/soft/zlib-1.2.11-gcc/include -I/home/yhuangci/soft/netcdf-481c453f-pgi21/include' LDFLAGS='-L/home/yhuangci/hdf5-1.12.1-pgi21/lib -L/home/yhuangci/soft/zlib-1.2.11-gcc/lib -L/home/yhuangci/soft/netcdf-481c453f-pgi21/lib' ./configure --prefix=/home/yhuangci/soft/netcdf-481c453f-pgi21 --disable-dap FC=pgfortran
make
make install

Read All

Coursera Machine Learning : W8 Unsupervised Learning

2021-08-21

LZN

technology

coursera cs229
W8 Unsupervised Learning

Notes

KMeans
- Process:
  - 1. Randomly initialize cluster centroids
  - 1. Assign all data points to their nearest cluster centroids
  - 1. Move cluster centroids to the mean value of all of the samples assigned to each previous centroid.
  - 1. Repeats these last two steps until this value is less than a threshold.
  - If no sample is assigned to the k-th centroid, just eliminate that centroid.
- Optimization Objectives
  - Distortion cost function
  - Multiple random initializations to avoid local optima. For large K, not work.
  - Elbow method: Cost function J as a function of numer of clusters K.
Dimentionality Reduction
- Aim: 1. Data compression 2. Visualization
- Princical Component Analysis (PCA)
- Feature scaling / mean normalization before performing PCA.
- PCA procedure:
  - 1. compute covariance matrix Sigma
  - 1. eigen vector: [U, S, V]=svd(Sigma) or eig(Sigma)
  - 1. U=[U1, U2, …, Uk (col)] the first k cols will represent k-dim features reduced from n-dim original features.
  - 1. z=transpose(Ureduce)*x
- Choose k with the smallest value that 99% of variance is retained.
- [U, S, V], S is used for check variance, with diagnal entries S11 S22 … added up to check the variance.
- Do not use PCA to avoid overfitting (no info from Y)
Read All
Coursera Machine Learning : W7 Large Margin Classification

2021-08-06

LZN

technology

coursera cs229
W7 Large Margin Classification

Notes

SVM - Support Vector Machine “Large Margin Classifiers”
- SVM gives computational advantages compared with Logistic Regression, easier to solve.
- Form of SVM cost function: J=A+lambda*B –> J=C*A+B
- For Linerly seperable data set, larger C, sensitive to outliers, but yields large margin.
Read All

4/115

Welcome to LZN's Blog!

PGI21 WRF-CMAQ 2WAY

Final Configurations

Libs

HDF5

NetCDF-C

NetCDF-F

Whole Porting Process

Libs

MPICH

HDF5

NetCDF-C

NetCDF-F

Coursera Machine Learning : W8 Unsupervised Learning

W8 Unsupervised Learning

Notes

KMeans

Dimentionality Reduction

Coursera Machine Learning : W7 Large Margin Classification

W7 Large Margin Classification

Notes

SVM - Support Vector Machine “Large Margin Classifiers”