Welcome to LZN's Blog!

Wind extinguishes a candle but energizes fire.

Coursera Machine Learning : W3

2021-07-08

LZN

technology

coursera cs229
W3

Notes

Classification and Logistic Regression
- Sigmoid function – Logistic function 1/(1+e^(-z))
- Logistic regression: thetax=1/(1+e^(thetaX)), estimates the probability that y=1 on input x
Decision Boundary
- The decision boundary is the line that separates the area where y = 0 and where y = 1. It is created by our hypothesis function.
- Directly use the cost function of logistic regression will lead to a non-convex function with many local optima.
- New cost function, cost(hthetax, y)=-log(htheta(x)), if y=1; and cost(hthetax,y)= -log(1-htheta(x)), if y=0. This will promise when hthetax gives a wrong answer, a very large penalty will be imposed.
- Single expression: cost(thetax, y)=-y*log(htheta(x))-(1-y)*log(1-htheta(x))
- Advantages: 1. Maximum likelihood estimation 2. Convex
- The form of gradient decent of logistic regression is the same as linear regression, just the hypothesis changes.
Advanced Optimization
- Gradient descent, Conjugate gradient, BFGS, L-BFGS
- Advanced algorithm advantages:
  - No need to manually pick alpha
  - faster
- Disadvantage: Much more complex
Multiclass Classification
- One-vs-all, One-vs-rest
- We are basically choosing one class and then lumping all the others into a single second class. We do this repeatedly, applying binary logistic regression to each case, and then use the hypothesis that returned the highest value as our prediction.
Overfitting
- underfitting – high bias; overfitting – high variance
Regularization
- Penalize the unuseful term
- Small values for parameters tend to a simpler hypothesis
- Add regularization term to cost function: lambda*sigma(thetaj^2), theta from 1 to n
- The λ, or lambda, is the regularization parameter. It determines how much the costs of our theta parameters are inflated.
- New gradient descent with regularized term: thetaj=thetaj(1-alphalambda/m)
- preconception 偏见
Read All
python package/module/class

2021-07-01

LZN

technology

linux python
Module

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. The __init__.py files are required to make Python treat directories containing the file as packages.

Naming

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. When an extension module written in C or C++ has an accompanying Python module that provides a higher level (e.g. more object oriented) interface, the C/C++ module has a leading underscore (e.g. _socket).

Package

A package is a collection of modules. Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name A.B designates a submodule named B in a package named A.

Naming

Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

Class

Naming

Class names should normally use the CapWords convention.

Instance

Naming
- Instance variable names should be all lower case
- Words in an instance variable name should be separated by an underscore
- Non-public instance variables should begin with a single underscore
- If an instance name needs to be mangled, two underscores may begin its name
Reference
- https://docs.python.org/3/tutorial/modules.html
- https://www.python.org/dev/peps/pep-0008/#package-and-module-names
- https://visualgit.readthedocs.io/en/latest/pages/naming_convention.html
Read All
Coursera Machine Learning Recap: W1-2

2021-06-28

LZN

technology

cs229 coursera
W1

Notes
- “Batch” Gradient Descent: Each step use all training examples.
W2

Notes

Multivar linear regression
- learning rate alpha choices: 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1… 3X test
- quadratic function / cubic function / third-order term
- polynomial regression
- Matrix inverse computational complexity O(n^3)
- Normal equation, use when feature numbers less than 10000 or so.
- Gradient descent complexity O(kn^2), where k is the number of iterations (epochs)
- Non-invertable matrix (singular/degenerate)
  - Cause 1: Redundent feature with linear relationship
  - Cause 2: Too many features (m«n)
- Clunkier 笨重的
Octave
- v= 1:0.1:2 arithmetic sequence
- ones(2,3) zeros(2,3) eye(3) rand(2,3)
- randn(2,3) % Guassian Distribution ~N(0,1)
- size(X), size(X,1), size(X,2)
- load(featureX.dat) % load a data file
- whos % command gives you variables in the current scope.
- clear X % delete X variable
- v = X(1:10) % slice
- save test.mat v % save v into test.mat file
- save test.txt v -ascii
- clear % clear all variables
- A=[A, [2; 4]] % matrix append a col vector to the rightmost position of A
- A(:) % squeeze nd data to 1d
- C = [A B]; C=[A;B] % right append and bottom append
- AB; A.B; % matrix multiplication and element-wise multiplication
- log(A); exp(A) % element-wise operation
- v + ones(length(v),1)
- [val, ind] = max(a)
- find(a<3) % return index
- [nrow, ncol]=find(A >= 7)
- sum(a); prod(a); % prod is the product of each element in a
- floor(a); ceil((a);
- max(A,[],1) % get max in each column
- max(A,[],2) % get max in each row
- sum(A, 1); sum(A,2); % column sum and row sum
- filpud(A) % FLIPUD Flip array in up/down direction.
- Plot
  - plot(t,y1); hold on; plot(t,y2); xlabel(‘time’); ylabel(‘val’); legend(‘sin’, ‘cos’)
  - print -dpng ‘myplot.png’
  - figure(1); plot(t,y1)
  - figure(2); plot(t,y2)
  - subplot(1,2,1); % divides canvas into 1x2 subdomain and access the first domain.
  - subplot(1,2,1); plot(t,y1); subplot(1,2,2); plot(t,y2)
  - axis([0.5 1 -1 1]) % set X axis from 0.5 to 1, and Y axis from -1 to 1
  - imagesc(A) % plot the matrix in heatmap
  - imagesc(A), colorbar, colormap gray; % in gray scale
  - comma chaining of commands would echo the result.
- Control
  - for i=1:10, v(i)=i^2; end;
  - ind = 1:10; for i=ind,
  - if elseif else
  - function y = f(x); y = x^2;
  - function y1, y2 = f(x) % multiple values return
- summation n. 求和
- reciprocal 倒数
- apostrophe 撇号
Programming

Carefully follow the matrix size after each vectorized operation, of particular caution is that matlab would perform add/subtract between 1-by-n and n-by-1 matrix by broadcasting the matrix to fit n-by-n size.
Read All

6/115

Welcome to LZN's Blog!

Coursera Machine Learning : W3

W3

Notes

Classification and Logistic Regression

Decision Boundary

Advanced Optimization

Multiclass Classification

Overfitting

Regularization

python package/module/class

Module

Package

Class

Instance

Reference

Coursera Machine Learning Recap: W1-2

W1

Notes

W2

Notes

Multivar linear regression

Octave

Programming