Welcome to LZN's Blog!

Wind extinguishes a candle but energizes fire.

Coursera Machine Learning : W5 Neural Network

2021-07-19

LZN

technology

coursera cs229
W5 Backward Propagation

Notes

Cost Function and Backpropagation
- total layer number: L, nodes # in layer l: s_l, k # of units in output layer
- cost function
- Backpropagation
- Random initialization: symmetry breaking
- Arch: Reasonable default: 1 hidden layer; multiple layers: same # of nodes in each layer; more layers, better performance
- backward propagation
- cyan /saian/ 青色
- magenta 品红
Read All
Coursera Machine Learning : W4 Neural Network

2021-07-12

LZN

technology

coursera cs229
W4 Neural Network

Notes

Neural Network
- Non-linear classification: Logistic regression may take in too many polynomial features.
- Neuron: Dendrite (input wires), Axon (output wires)
- Logistic unit: x0 –> bias unit
- An artificial neuron (single) with a sigmoid activation function
- Neural Network: Input layer, Hidden layer, and Output layer.
- a^(j)_i “activation” of unit i in layer j.
- theta^(j) = matrix of weights controlling function mapping from layer j to j+1.
- If network has s_j units in layer j and s_j+1 units in layer j+1, the theta j would be of dimension s<sub>j+1</sub>*s<sub>j</sub>+1
- Important Forward propagation representation
- Be careful with the MATLAB syntax: for i = 1:10 COLON not COMMA!
- negation 否定
- parenthese 括号
- auditory cortex 听觉皮层
Read All
Coursera Machine Learning : W3

2021-07-08

LZN

technology

coursera cs229
W3

Notes

Classification and Logistic Regression
- Sigmoid function – Logistic function 1/(1+e^(-z))
- Logistic regression: thetax=1/(1+e^(thetaX)), estimates the probability that y=1 on input x
Decision Boundary
- The decision boundary is the line that separates the area where y = 0 and where y = 1. It is created by our hypothesis function.
- Directly use the cost function of logistic regression will lead to a non-convex function with many local optima.
- New cost function, cost(hthetax, y)=-log(htheta(x)), if y=1; and cost(hthetax,y)= -log(1-htheta(x)), if y=0. This will promise when hthetax gives a wrong answer, a very large penalty will be imposed.
- Single expression: cost(thetax, y)=-y*log(htheta(x))-(1-y)*log(1-htheta(x))
- Advantages: 1. Maximum likelihood estimation 2. Convex
- The form of gradient decent of logistic regression is the same as linear regression, just the hypothesis changes.
Advanced Optimization
- Gradient descent, Conjugate gradient, BFGS, L-BFGS
- Advanced algorithm advantages:
  - No need to manually pick alpha
  - faster
- Disadvantage: Much more complex
Multiclass Classification
- One-vs-all, One-vs-rest
- We are basically choosing one class and then lumping all the others into a single second class. We do this repeatedly, applying binary logistic regression to each case, and then use the hypothesis that returned the highest value as our prediction.
Overfitting
- underfitting – high bias; overfitting – high variance
Regularization
- Penalize the unuseful term
- Small values for parameters tend to a simpler hypothesis
- Add regularization term to cost function: lambda*sigma(thetaj^2), theta from 1 to n
- The λ, or lambda, is the regularization parameter. It determines how much the costs of our theta parameters are inflated.
- New gradient descent with regularized term: thetaj=thetaj(1-alphalambda/m)
- preconception 偏见
Read All

14/124

Welcome to LZN's Blog!

Coursera Machine Learning : W5 Neural Network

W5 Backward Propagation

Notes

Cost Function and Backpropagation

Coursera Machine Learning : W4 Neural Network

W4 Neural Network

Notes

Neural Network

Coursera Machine Learning : W3

W3

Notes

Classification and Logistic Regression

Decision Boundary

Advanced Optimization

Multiclass Classification

Overfitting

Regularization