主要是关于矩阵的求导。
-
∂ y ∂ x \frac{\partial y}{\partial \mathbf{x}} ∂x∂y [ y y y是标量， x \mathbf{x} x是列向量，导数是行向量]
x = [ x 1 x 2 ⋮ x n ] , ∂ y ∂ x = [ ∂ y ∂ x 1 , ∂ y ∂ x 2 , ⋯ , ∂ y ∂ x n ] \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \quad \frac{\partial y}{\partial \mathbf{x}} = [\frac{\partial y}{\partial x_1}, \frac{\partial y}{\partial x_2}, \cdots, \frac{\partial y}{\partial x_n}] x=⎣⎢⎢⎢⎡x1x2⋮xn⎦⎥⎥⎥⎤,∂x∂y=[∂x1∂y,∂x2∂y,⋯,∂xn∂y]
例如： ∂ ∂ x x 1 2 + 2 x 2 2 = [ 2 x 1 , 4 x 2 ] \frac{\partial}{\partial \mathbf{x}}x_1^2 + 2x_2^2 = [2x_1, 4x_2 ] ∂x∂x12+2x22=[2x1,4x2]一些样例
y y y a a a a u au au sum ( x ) \text{sum}(x) sum(x) ∥ x ∥ 2 \|\mathbf{x}\|^2 ∥x∥2 ∂ y ∂ x \frac{\partial y}{\partial \mathbf{x}} ∂x∂y 0 T \mathbf{0}^T 0T a ∂ u ∂ x a\frac{\partial u}{\partial \mathbf{x}} a∂x∂u 1 T \mathbf{1}^T 1T 2 x T 2\mathbf{x}^T 2xT y y y u + v u+v u+v u v uv uv < u , v > <\mathbf{u}, \mathbf{v}> <u,v> ∂ y ∂ x \frac{\partial y}{\partial \mathbf{x}} ∂x∂y ∂ u ∂ x + ∂ v ∂ x \frac{\partial u}{\partial \mathbf{x}} + \frac{\partial v}{\partial \mathbf{x}} ∂x∂u+∂x∂v ∂ u ∂ x v + ∂ v ∂ x u \frac{\partial u}{\partial \mathbf{x}} v+ \frac{\partial v}{\partial \mathbf{x}} u ∂x∂uv+∂x∂vu u T ∂ v ∂ x + v T ∂ u ∂ x \mathbf{u}^T \frac{\partial \mathbf{v}}{\partial \mathbf{x}} + \mathbf{v}^T \frac{\partial \mathbf{u}}{\partial \mathbf{x}} uT∂x∂v+vT∂x∂u -
∂ y ∂ x \frac{\partial \mathbf{y}}{\partial x} ∂x∂y [ y \mathbf{y} y是列向量， x x x是标量，导数是列向量]
y = [ y 1 y 2 ⋮ y m ] , ∂ y ∂ x = [ ∂ y 1 x ∂ y 2 x ⋮ ∂ y m x ] \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{bmatrix}, \quad \frac{\partial \mathbf{y}}{\partial x} = \begin{bmatrix} \frac{\partial y_1}{x} \\ \frac{\partial y_2}{x} \\ \vdots \\ \frac{\partial y_m}{x} \end{bmatrix} y=⎣⎢⎢⎢⎡y1y2⋮ym⎦⎥⎥⎥⎤,∂x∂y=⎣⎢⎢⎢⎡x∂y1x∂y2⋮x∂ym⎦⎥⎥⎥⎤ -
∂ y ∂ x \frac{\partial \mathbf{y}}{\partial \mathbf{x}} ∂x∂y [ y \mathbf{y} y是列向量， x \mathbf{x} x是列向量，导数是矩阵]
x = [ x 1 x 2 ⋮ x n ] , y = [ y 1 y 2 ⋮ y m ] , ∂ y ∂ x = [ ∂ y 1 x ∂ y 2 x ⋮ ∂ y m x ] = [ ∂ y 1 ∂ x 1 , ∂ y 1 ∂ x 2 , ⋯ , ∂ y 1 ∂ x n ∂ y 2 ∂ x 1 , ∂ y 2 ∂ x 2 , ⋯ , ∂ y 2 ∂ x n ⋮ ∂ y m ∂ x 1 , ∂ y m ∂ x 2 , ⋯ , ∂ y m ∂ x n ] \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{bmatrix}, \frac{\partial \mathbf{y}}{\partial \mathbf{x}} = \begin{bmatrix} \frac{\partial y_1}{\mathbf{x}} \\ \frac{\partial y_2}{\mathbf{x}} \\ \vdots \\ \frac{\partial y_m}{\mathbf{x}} \end{bmatrix} = \begin{bmatrix} \frac{\partial y_1}{\partial x_1}, & \frac{\partial y_1}{\partial x_2}, & \cdots, & \frac{\partial y_1}{\partial x_n} \\ \frac{\partial y_2}{\partial x_1}, & \frac{\partial y_2}{\partial x_2}, & \cdots, & \frac{\partial y_2}{\partial x_n} \\ \vdots \\ \frac{\partial y_m}{\partial x_1}, & \frac{\partial y_m}{\partial x_2}, & \cdots, &\frac{\partial y_m}{\partial x_n} \\ \end{bmatrix} x=⎣⎢⎢⎢⎡x1x2⋮xn⎦⎥⎥⎥⎤,y=⎣⎢⎢⎢⎡y1y2⋮ym⎦⎥⎥⎥⎤,∂x∂y=⎣⎢⎢⎢⎡x∂y1x∂y2⋮x∂ym⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎢⎡∂x1∂y1,∂x1∂y2,⋮∂x1∂ym,∂x2∂y1,∂x2∂y2,∂x2∂ym,⋯,⋯,⋯,∂xn∂y1∂xn∂y2∂xn∂ym⎦⎥⎥⎥⎥⎤
一些样例y \mathbf{y} y a \mathbf{a} a x \mathbf{x} x A x \mathbf{Ax} Ax x T A \mathbf{x^TA} xTA ∂ y ∂ x \frac{\partial \mathbf{y}}{\partial \mathbf{x}} ∂x∂y 0 \mathbf{0} 0 I \mathbf{I} I A \mathbf{A} A A T \mathbf{A}^T AT y \mathbf{y} y a u a\mathbf{u} au A u \mathbf{Au} Au u + v \mathbf{u}+ \mathbf{v} u+v ∂ y ∂ x \frac{\partial \mathbf{y}}{\partial \mathbf{x}} ∂x∂y a ∂ u ∂ x a\frac{\partial \mathbf{u}}{\partial \mathbf{x}} a∂x∂u A ∂ u ∂ x \mathbf{A}\frac{\partial \mathbf{u}}{\partial \mathbf{x}} A∂x∂u ∂ u ∂ x + ∂ v ∂ x \frac{\partial \mathbf{u}}{\partial \mathbf{x}} + \frac{\partial \mathbf{v}}{\partial \mathbf{x}} ∂x∂u+∂x∂v