参考资料

常用矩阵求导公式

对于一个矩阵A，向量 x mathrm{x} x，有如下求导公式： d x T d x = I , d x d x T = I (1) ag{1} frac{mathrm{dx}^{mathrm{T}}}{mathrm{dx}}=I ext{,} quad quad frac{mathrm{dx}}{mathrm{dx}^{mathrm{T}}}=I dxdxT=I,dxTdx=I(1)

d x T A d x = A , d A x d x T = A (2) ag{2} egin{gathered} frac{mathrm{dx}^{mathrm{T}} mathrm{A}}{mathrm{dx}}=mathrm{A} ext{,} quad quad quad frac{mathrm{dAx}}{mathrm{dx}^{mathrm{T}}}=mathrm{A} \ end{gathered} dxdxTA=A,dxTdAx=A(2)

d A x d x = A T , d x A d x = A T (3) ag{3} egin{gathered} frac{mathrm{dAx}}{mathrm{dx}}=mathrm{A}^{mathrm{T}} ext{,} quad quad quad frac{mathrm{dxA}}{mathrm{dx}}=mathrm{A}^{mathrm{T}} end{gathered} dxdAx=AT,dxdxA=AT(3)

d x T x d x = 2 x , d x T A x d x = ( A + A T ) x (4-1) ag{4-1} egin{gathered} frac{mathrm{dx}^{mathrm{T}} mathrm{x}}{mathrm{dx}}=2 mathrm{x} ext{,} quad quad frac{mathrm{dx}^{mathrm{T}} mathrm{Ax}}{mathrm{dx}}=left(mathrm{A}+mathrm{A}^{mathrm{T}} ight) mathrm{x} end{gathered} dxdxTx=2x,dxdxTAx=(A+AT)x(4-1) d x T A x d x x T = d d x ( d x T A x d x ) = A T + A (4-2) ag{4-2} frac{mathrm{dx}^{mathrm{T}}mathrm {A x}}{mathrm{d x x}^{mathrm{T}}}=frac{d}{ mathrm{d x}}left(frac{mathrm{ dx}^{mathrm{T}} mathrm{A x}}{ mathrm{d x}} ight)=mathrm{A}^{mathrm{T}}+mathrm{A} dxxTdxTAx=dxd(dxdxTAx)=AT+A(4-2)

∂ u ∂ x T = ( ∂ u T ∂ x ) T (5-1) ag{5-1} frac{partial mathrm{u}}{partial mathrm{x}^{mathrm{T}}}=left(frac{partial mathrm{u}^{mathrm{T}}}{partial mathrm{x}} ight)^{mathrm{T}} ∂xT∂u=(∂x∂uT)T(5-1) ∂ u T v ∂ x = ∂ u T ∂ x v + ∂ v T ∂ x u T , ∂ u v T ∂ x = ∂ u ∂ x v T + u ∂ v T ∂ x (5-2) ag{5-2} egin{aligned} frac{partial mathrm{u}^{mathrm{T}} mathrm{v}}{partial mathrm{x}}=frac{partial mathrm{u}^{mathrm{T}}}{partial mathrm{x}} mathrm{v}+frac{partial mathrm{v}^{mathrm{T}}}{partial mathrm{x}} mathrm{u}^{mathrm{T}} ext{,} quad quad frac{partial mathrm{uv}^{mathrm{T}}}{partial mathrm{x}}=frac{partial mathrm{u}}{partial mathrm{x}} mathrm{v}^{mathrm{T}}+mathrm{u} frac{partial mathrm{v}^{mathrm{T}}}{partial mathrm{x}} end{aligned} ∂x∂uTv=∂x∂uTv+∂x∂vTuT,∂x∂uvT=∂x∂uvT+u∂x∂vT(5-2)

∂ [ ( x u − v ) T ( x u − v ) ] ∂ x = 2 ( x u − v ) u T (6) ag{6} frac{partialleft[(mathrm{xu}-mathrm{v})^{mathrm{T}}(mathrm{x} u-mathrm{v}) ight]}{partial mathrm{x}}=2(mathrm{xu}-mathrm{v}) mathrm{u}^{mathrm{T}} ∂x∂[(xu−v)T(xu−v)]=2(xu−v)uT(6)

∂ u T x v ∂ x = u v T , ∂ u T x T x u ∂ x = 2 x u u T (7) ag{7} egin{gathered} frac{partial mathrm{u}^{mathrm{T}} mathrm{xv}}{partial mathrm{x}}=mathrm{uv}^{mathrm{T}} ext{,} quad quad frac{partial mathrm{u}^{mathrm{T}} mathrm{x}^{mathrm{T}} mathrm{xu}}{partial mathrm{x}}=2 mathrm{xuu}^{mathrm{T}} end{gathered} ∂x∂uTxv=uvT,∂x∂uTxTxu=2xuuT(7)

特别地，当 A = A T A=A^T A=AT时，公式(4-1)和公式(4-2)有 d x T A x d x = 2 A x (8) ag{8} egin{gathered} frac{mathrm{dx}^{mathrm{T}} mathrm{Ax}}{mathrm{dx}}=2mathrm{A}mathrm{x} end{gathered} dxdxTAx=2Ax(8) d x T A x d x x T = 2 A (9) ag{9} frac{ mathrm{dx}^{mathrm{T}}mathrm {A x}}{mathrm{dx x}^{mathrm{T}}}=2mathrm{A} dxxTdxTAx=2A(9)