线性回归模型
\(y=Ax+v,b是噪声\)
\(v=y-Ax\)
高斯分布
\(此时令b\sim 正态分布\)
\(P(x)=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}\)
\(对数似然函数MLE=L(x)=\prod_{i=1}^{m}lnP(v_i)\)
点击查看代码
# 高斯分布图像
from scipy.stats import norm
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.subplots(1, 1)
x = np.linspace(norm.ppf(0.01),
norm.ppf(0.99), 100)
ax.plot(x, norm.pdf(x),
'r-', lw=5, alpha=0.6, label='norm pdf')
\(=\sum_{i=1}^{m}(log\frac{1}{\sqrt{2\pi\sigma^2}}-\frac{1}{2\sigma^2}(y_i-a_i^Tx_i)^2)\)
\(=-\frac{m}{2}log\frac{1}{2\pi\sigma^2}-\frac{1}{2\sigma^2}\sum_{i=1}^{m}(y_i-a_i^Tx_i)^2\)
\(max_{x}L(x)\Leftrightarrow min_x \frac{1}{2}\sum_{i=1}^{m}(y_i-a_i^Tx_i)^2\)
\(即对误差的优化问题等价于最小二乘问题\)
拉普拉斯分布
\(拉普拉斯概率密度函数\)
\(P(x)=\frac{1}{2\lambda}e^{-\frac{|x-\mu|}{\lambda}}\)
点击查看代码
from scipy.stats import laplace
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 1)
x = np.linspace(laplace.ppf(0.01),
laplace.ppf(0.99), 100)
ax.plot(x, laplace.pdf(x),
'r-', lw=5, alpha=0.6, label='laplace pdf')
\(对数似然函数=L(v)=\sum_{i=1}^{m}log \frac{1}{2\lambda}e^{-\frac{|y_i-a_i^Tx|}{\lambda}}\)
\(=-mlog2\lambda \sum_{i=1}^{m}\frac{1}{\lambda}|y_i-a_i^Tx|\)
\(max\ L(x) \Leftrightarrow min\ \sum_{i=1}^{m}|y_i-a_i^Tx|\)
\(这是L1范数下的优化问题,最小二乘就是L2范数下的优化问题\)