深度学习系列1-逻辑回归分类

1. Logistic Regression

1.1. Theory

One sample: Loss function:

\[L(\hat{y},y)=-(ylog(\hat{y}+(1-y)log(1-\hat{y}))) \]

All training samples: Cost function

\[J(w,b)= -\frac 1m \sum_{i=1}^{m}[y^{(i)}log(\hat{y}^{(i)}) + (1-y^{i})log(1-\hat{y}^{(i)})] \]

1.2. Gradient Descent

One sample: Derivative of the LR:

\[dz^{(i)} = a^{(i)} - y^{(i)} \\ dw_{j}^{(i)} = \frac{\delta L}{\delta w_{j}^{(i)}} = x_{j}^{(i)} \cdot dz^{(i)}(j=1,2,...,n_{x}) \db^{(i)} = dz^{(i)} \]

All training samples: Derivative of the Cost function:

\[\delta w_{j}J(w,b) = \frac 1m \sum_{i=1}^{m} \frac{\delta}{\delta w_{j}}L(a^{(i)},y^{(i)}) \]

1.3. Implementation

搭建LR的分类器

import numpy as np

class LogisticRegression:

    def __init__(self, n_iters = 50, learning_rate = 0.001):
        self.epoch = n_iters
        self.lr = learning_rate
        self.weights = None
        self.bias = None


    def fit(self,X,y):
        # X: input data   (m,n)
        # y: output label (m,1)
        # weights: weight  (n,1)
        # bias:            na

        # init parameter
        n_samples, n_features = X.shape
        self.weights = np.random.rand(n_features)
        self.bias = 0

        # Gradient descent
        for _ in range(self.epoch):
            linear_value = np.dot(X, self.weights) + self.bias
            y_predicted = self._sigmoid(linear_value)

            # compute the derivative
            dw,db = self._derivative(X,y,y_predicted,n_samples)
            # update the value
            self.weights -= self.lr * dw
            self.bias -= self.lr * db

    def predict(self, X):
        # the weight and bia has already been updated
        linear_value = np.dot(X, self.weights) + self.bias
        y_predicted = self._sigmoid(linear_value)

        # compute the predicted class
        y_predicted_class = [1 if i>0.5 else 0 for i in y_predicted]

        return y_predicted_class

    def _sigmoid(self,x):
        return 1/(1+np.exp(-x))

    def _derivative(self, X, y, y_predicted, n_samples):
        dz = y_predicted - y
        dw = (1/n_samples)*np.dot(X.T, dz)  # (n,m) * (m,1) = (n,1)
        db = (1/n_samples)*np.sum(dz)

        return dw, db

Test

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn import datasets
import matplotlib.pyplot as plt

from Library import LogisticRegression

data = datasets.load_breast_cancer()
X,y = data.data, data.target

X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2, random_state=1234)

def accuracy(y_true,y_pred):
    return np.sum(y_true == y_pred) / len(y_true)

regressor = LogisticRegression(learning_rate=0.0001, n_iters=1000)
regressor.fit(X_train, y_train)
predictions = regressor.predict(X_test)

print("LR Classification accuract:", accuracy(y_test,predictions))

最终的结果达到了90%以上的准确度。

深度学习系列1-逻辑回归分类

上一篇:心存好奇,心怀敬意


下一篇:选择思维导图软件,你只需学会这几招!