Welcome to LZN's Blog!

Wind extinguishes a candle but energizes fire.

Coursera Machine Learning : W4 Neural Network

2021-07-12

LZN

technology

coursera cs229
W4 Neural Network

Notes

Neural Network
- Non-linear classification: Logistic regression may take in too many polynomial features.
- Neuron: Dendrite (input wires), Axon (output wires)
- Logistic unit: x0 –> bias unit
- An artificial neuron (single) with a sigmoid activation function
- Neural Network: Input layer, Hidden layer, and Output layer.
- a^(j)_i “activation” of unit i in layer j.
- theta^(j) = matrix of weights controlling function mapping from layer j to j+1.
- If network has s_j units in layer j and s_j+1 units in layer j+1, the theta j would be of dimension s<sub>j+1</sub>*s<sub>j</sub>+1
- Important Forward propagation representation
- Be careful with the MATLAB syntax: for i = 1:10 COLON not COMMA!
- negation 否定
- parenthese 括号
- auditory cortex 听觉皮层
Read All
Coursera Machine Learning : W3

2021-07-08

LZN

technology

coursera cs229
W3

Notes

Classification and Logistic Regression
- Sigmoid function – Logistic function 1/(1+e^(-z))
- Logistic regression: thetax=1/(1+e^(thetaX)), estimates the probability that y=1 on input x
Decision Boundary
- The decision boundary is the line that separates the area where y = 0 and where y = 1. It is created by our hypothesis function.
- Directly use the cost function of logistic regression will lead to a non-convex function with many local optima.
- New cost function, cost(hthetax, y)=-log(htheta(x)), if y=1; and cost(hthetax,y)= -log(1-htheta(x)), if y=0. This will promise when hthetax gives a wrong answer, a very large penalty will be imposed.
- Single expression: cost(thetax, y)=-y*log(htheta(x))-(1-y)*log(1-htheta(x))
- Advantages: 1. Maximum likelihood estimation 2. Convex
- The form of gradient decent of logistic regression is the same as linear regression, just the hypothesis changes.
Advanced Optimization
- Gradient descent, Conjugate gradient, BFGS, L-BFGS
- Advanced algorithm advantages:
  - No need to manually pick alpha
  - faster
- Disadvantage: Much more complex
Multiclass Classification
- One-vs-all, One-vs-rest
- We are basically choosing one class and then lumping all the others into a single second class. We do this repeatedly, applying binary logistic regression to each case, and then use the hypothesis that returned the highest value as our prediction.
Overfitting
- underfitting – high bias; overfitting – high variance
Regularization
- Penalize the unuseful term
- Small values for parameters tend to a simpler hypothesis
- Add regularization term to cost function: lambda*sigma(thetaj^2), theta from 1 to n
- The λ, or lambda, is the regularization parameter. It determines how much the costs of our theta parameters are inflated.
- New gradient descent with regularized term: thetaj=thetaj(1-alphalambda/m)
- preconception 偏见
Read All
python package/module/class

2021-07-01

LZN

technology

linux python
Module

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. The __init__.py files are required to make Python treat directories containing the file as packages.

Naming

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. When an extension module written in C or C++ has an accompanying Python module that provides a higher level (e.g. more object oriented) interface, the C/C++ module has a leading underscore (e.g. _socket).

Package

A package is a collection of modules. Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name A.B designates a submodule named B in a package named A.

Naming

Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

Class

Naming

Class names should normally use the CapWords convention.

Instance

Naming
- Instance variable names should be all lower case
- Words in an instance variable name should be separated by an underscore
- Non-public instance variables should begin with a single underscore
- If an instance name needs to be mangled, two underscores may begin its name
Reference
- https://docs.python.org/3/tutorial/modules.html
- https://www.python.org/dev/peps/pep-0008/#package-and-module-names
- https://visualgit.readthedocs.io/en/latest/pages/naming_convention.html
Read All

11/121

Welcome to LZN's Blog!

Coursera Machine Learning : W4 Neural Network

W4 Neural Network

Notes

Neural Network

Coursera Machine Learning : W3

W3

Notes

Classification and Logistic Regression

Decision Boundary

Advanced Optimization

Multiclass Classification

Overfitting

Regularization

python package/module/class

Module

Package

Class

Instance

Reference