幂迭代法求第k大的特征值和特征向量

数学表述

设矩阵A
\[ A = \begin{matrix} X_{11}&\ldots&X_{1n} \newline X_{21} & \ldots& X_{2n} \newline &\vdots& \newline X_{n1}&\ldots &X_{nn} \newline \end{matrix} \]
求其最大特征值对应的特征向量\(b_k = [v_1,v_2,\ldots,v_n]^T\):
\[b_{k+1} = \frac{Ab_k}{||Ab_k ||}..........(1)\]

证明：见英文wiki Power_iteration

代码：

import numpy as np
def power_iteration(A, num_simulations: int):
# Ideally choose a random vector
# To decrease the chance that our vector
# Is orthogonal to the eigenvector
b_k = np.random.rand(A.shape[1])

for _ in range(num_simulations):
# calculate the matrix-by-vector product Ab
b_k1 = np.dot(A, b_k)

# calculate the norm
b_k1_norm = np.linalg.norm(b_k1)

# re normalize the vector
b_k = b_k1 / b_k1_norm

return b_k

例子

\[ A = \left[ \begin{matrix} 2&1\newline 1&2\newline \end{matrix} \right] \]

求其特征值：
\[ |A-\lambda I| = \left| \begin{matrix} 2-\lambda &1\newline 1 &2-\lambda\newline \end{matrix} \right| = 3 - 4\lambda + \lambda^2 \]
求得最大特征值\(\lambda_2 = 3\)，另一特征值\(\lambda_1 = 1\)
\(\lambda_2 = 3\) 时：
\[ |(A-\lambda I)V| = 0 \]

\[ (A-3I)V = \left[ \begin{array} {cccc} -1 &1\newline 1 &-1\newline \end{array} \right] \left[ \begin{array} {cccc} v_1\newline v_2\newline \end{array} \right] = \left[ \begin{array} {cccc} 0\newline 0\newline \end{array} \right] \]

求得\(v_1 = v_2\),归一化后，\(V_{\lambda=3} = [0.707,0.707]^T\)

验证代码：

A = np.array([[2,1],[1,2]])
V = power_iteration(A, 100).reshape(1,-1)
V

array([[0.70710678, 0.70710678]])

求得特征向量后，特征值
\[ \lambda = VAV^T........(2) \]
代码

lbd = np.dot(V,np.dot(A,V.T))
lbd 

array([[3.]])

扩展

求次大特征向量与特征值:
\[ B = A - \frac{\lambda}{||V||^2}V^TV ........(3) \]

\[ b_{k+1}' = \frac{Bb_k'}{||Bb_k' ||} ...........(4) \]

代码

B = A - lbd / np.linalg.norm(V)**2 * np.dot(V.T,V)
V_2 = power_iteration(B, 100)
lbd1 = np.dot(V_2.T,np.dot(B,V_2))
lbd1

1.0000000000000002

以此类推，我们可以得到任意第k大特征值。这在很多机器学习方法中，取前k重要特征有着重要的作用。
欢迎访问个人博客。转载请注明出处。