import numpy as np def sigmoid(x): """ define a sigmoid function x is input: An array This function is to compute the sigmoid function value """ s = 1.0 / (1 + (1/np.exp(x))) return s #return the sigmoid function value #main function m = np.array([1,2,3]) print(sigmoid(m))
1.2 – Sigmoid gradient
import numpy as np def sigmoid_derivative(x): s = 1.0 / (1 + 1 / np.exp(x)) ds = s * (1-s) return ds x = np.array([1, 2, 3]) print ("sigmoid_derivative(x) = " + str(sigmoid_derivative(x)))
1.3 – Reshaping arrays
v = v.reshape((v.shape*v.shape, v.shape)) # v.shape=a;v.shape=b;v.shape=c
Please don’t hardcode the dimensions of image as a constant. Instead look up the quantities you need with
#Reshaping arrays def image2vector(imag): """ takes an input of shape (length,height,3)and returns a vector of shape(length*height*3,1) """ v = img.reshape((img.shape*img.shape*img.shape,1 )) return v img = np.array([[[ 0.67826139, 0.29380381], [ 0.90714982, 0.52835647], [ 0.4215251 , 0.45017551]], [[ 0.92814219, 0.96677647], [ 0.85304703, 0.52351845], [ 0.19981397, 0.27417313]], [[ 0.60659855, 0.00533165], [ 0.10820313, 0.49978937], [ 0.34144279, 0.94630077]]]) m = image2vector(img) print ("image2vector(image) = " + str(m))
1.4 – Normalizing rows
Another common technique we use in Machine Learning and Deep Learning is to normalize our data. It often leads to a better performance because gradient descent converges faster after normalization. Here, by normalization we mean changing x to (dividing each row vector of x by its norm).
# GRADED FUNCTION: normalizeRows def normalizeRows(x): x_norm = np.linalg.norm(x,axis=1,keepdims=True)#x代表对x求解，ord后面的数字 #表示是几范数，无穷范数是np.inf，axis=1代表1维数据，keepdims表示如果将其设置 #为true，则将赋范的轴作为尺寸为1的尺寸保留在结果中。 x = x / x_norm return x x = np.array([[0, 3, 4],[1, 6, 4]]) print("normalizeRows(x) = " + str(normalizeRows(x)))
What you need to remember:
– np.exp(x) works for any np.array x and applies the exponential function to every coordinate
– the sigmoid function and its gradient
– image2vector is commonly used in deep learning
– np.reshape is widely used. In the future, you’ll see that keeping your matrix/vector dimensions straight will go toward eliminating a lot of bugs.
– numpy has efficient built-in functions
– broadcasting is extremely useful
np.dot() performs a matrix-matrix or matrix-vector multiplication. This is different from
np.multiply() and the
* operator (which is equivalent to
.* in Matlab/Octave), which performs an element-wise multiplication.
2.1 Implement the L1 and L2 loss functions
# GRADED FUNCTION: L1 import numpy as np def L1(yhat, y): loss = np.sum(np.abs( y - yhat)) return loss yhat = np.array([.9, 0.2, 0.1, .4, .9]) y = np.array([1, 0, 0, 1, 1]) print("L1 = " + str(L1(yhat,y)))
Exercise: Implement the numpy vectorized version of the L2 loss. There are several way of implementing the L2 loss but you may find the function np.dot() useful. As a reminder, if , then
np.dot(x,x) = .
#GRADED FUNCTION: L2 def L2(yhat,y): #loss=np.sum(np.power((y-yhat),2)) loss= np.sum(np.dot(y-yhat,y-yhat)) return loss yhat = np.array([.9, 0.2, 0.1, .4, .9]) y = np.array([1, 0, 0, 1, 1]) print("L2 = " + str(L2(yhat,y)))
What to remember:
– Vectorization is very important in deep learning. It provides computational efficiency and clarity.
– You have reviewed the L1 and L2 loss.
– You are familiar with many numpy functions such as np.sum, np.dot, np.multiply, np.maximum, etc…
Part 2： Logistic Regression with a Neural Network mindset
You will learn to:
– Build the general architecture of a learning algorithm, including:
– Initializing parameters
– Calculating the cost function and its gradient
– Using an optimization algorithm (gradient descent)
– Gather all three functions above into a main model function, in the right order.
First, let’s run the cell below to import all the packages that you will need during this assignment.
– numpy is the fundamental package for scientific computing with Python.
– h5py is a common package to interact with a dataset that is stored on an H5 file.
– matplotlib is a famous library to plot graphs in Python.
– PIL and scipy are used here to test your model with your own picture at the end.
2 – Overview of the Problem set
Problem Statement: You are given a dataset (“data.h5”) containing:
– a training set of m_train images labeled as cat (y=1) or non-cat (y=0)
– a test set of m_test images labeled as cat or non-cat
– each image is of shape (num_px, num_px, 3) where 3 is for the 3 channels (RGB). Thus, each image is square (height = num_px) and (width = num_px).
You will build a simple image-recognition algorithm that can correctly classify pictures as cat or non-cat.
Let’s get more familiar with the dataset. Load the data by running the following code.
train_set_x_orig, train_set_y, test_set_x_orig, test_set_y, classes = load_dataset()
We added “_orig” at the end of image datasets (train and test) because we are going to preprocess them. After preprocessing, we will end up with train_set_x and test_set_x (the labels train_set_y and test_set_y don’t need any preprocessing).
Each line of your train_set_x_orig and test_set_x_orig is an array representing an image. You can visualize an example by running the following code. Feel free also to change the
index value and re-run to see other images.
# Example of a picture
index = 19 plt.imshow(train_set_x_orig[index]) print("y="+str(train_set_y[:,index])+","+classes[np.squeeze(train_set_y[:,index])].decode("utf-8")+"'picture.")
Many software bugs in deep learning come from having matrix/vector dimensions that don’t fit. If you can keep your matrix/vector dimensions straight you will go a long way toward eliminating many bugs.
Exercise: Find the values for:
– m_train (number of training examples)
– m_test (number of test examples)
– num_px (= height = width of a training image)
train_set_x_orig is a numpy-array of shape (m_train, num_px, num_px, 3). For instance, you can access
m_train by writing
#access m_train,m_test,num_px m_train = train_set_x_orig.shape m_test = test_set_x_orig.shape num_px = train_set_x_orig.shape print ("Number of training examples: m_train = " + str(m_train)) print ("Number of testing examples: m_test = " + str(m_test)) print ("Height/Width of each image: num_px = " + str(num_px)) print ("Each image is of size: (" + str(num_px) + ", " + str(num_px) + ", 3)") print ("train_set_x shape: " + str(train_set_x_orig.shape)) print ("train_set_y shape: " + str(train_set_y.shape)) print ("test_set_x shape: " + str(test_set_x_orig.shape)) print ("test_set_y shape: " + str(test_set_y.shape))
For convenience, you should now reshape images of shape (num_px, num_px, 3) in a numpy-array of shape (num_px num_px 3, 1). After this, our training (and test) dataset is a numpy-array where each column represents a flattened image. There should be m_train (respectively m_test) columns.
Exercise: Reshape the training and test data sets so that images of size (num_px, num_px, 3) are flattened into single vectors of shape (num_px num_px 3, 1).
#reshape the training and test examples train_set_x_flatten = train_set_x_orig.reshape(m_train, -1).T test_set_x_flatten = test_set_x_orig.reshape(m_test, -1).T print("====================分割线======================") print ("train_set_x_flatten shape: " + str(train_set_x_flatten.shape)) print ("train_set_y shape: " + str(train_set_y.shape)) print ("test_set_x_flatten shape: " + str(test_set_x_flatten.shape)) print ("test_set_y shape: " + str(test_set_y.shape)) print ("sanity check after reshaping: " + str(train_set_x_flatten[0:5,0])) train_set_x = train_set_x_flatten/255 test_set_x = test_set_x_flatten/255
To represent color images, the red, green and blue channels (RGB) must be specified for each pixel, and so the pixel value is actually a vector of three numbers ranging from 0 to 255.
One common preprocessing step in machine learning is to center and standardize your dataset, meaning that you substract the mean of the whole numpy array from each example, and then divide each example by the standard deviation of the whole numpy array. But for picture datasets, it is simpler and more convenient and works almost as well to just divide every row of the dataset by 255 (the maximum value of a pixel channel).
Let’s standardize our dataset.
train_set_x = train_set_x_flatten/255. test_set_x = test_set_x_flatten/255.
What you need to remember:
Common steps for pre-processing a new dataset are:
– Figure out the dimensions and shapes of the problem (m_train, m_test, num_px, …)
– Reshape the datasets such that each example is now a vector of size (num_px * num_px * 3, 1)
– “Standardize” the data