利用者:BigNeuron/行列の微分

数学において、行列の微分^[1]とは、多変数微積分を行うための特殊な表記法のことで、特に行列で記したものである。ある関数の、多くの変数に関する偏微分、もしくは複数の関数の、ある変数に関する偏微分の集まりであり、これらをベクトルや行列で表してひとつの実体として扱う。これにより、関数の最大最小値を求めたり微分方程式系を解くことなどの演算が大幅に簡素化される。ここでの記法は統計と工学で一般的に使用されるものだが、物理学ではテンソルインデックス記法が好まれる。

行列の微分には2つの異なる記法があり、それによりこの分野は2つに区別できる。それは、ベクトルに関するスカラーの導関数を列ベクトルとして書くか、行ベクトルとして書くか、の違いである。ベクトルを行列と組み合わせる場合、（行ベクトルではなく）列ベクトルとして扱う必要があるという一般的な仮定が置かれているときでも、両方の記法が可能である。一般にこの記法は、行列の微積分を用いる分野ごとに、ある程度どちらかに定まっている（たとえば、計量経済学、統計、推定理論、機械学習）。ただし特定の分野内であっても、著者によって異なる記法を用いる場合がある。このとき著者はそれぞれ、その記法が標準であるかのように書くことが多い。異なる表記法が用いられていることを慎重に検証せずに、異なる著者の結果を組み合わせると、重大な間違いが生じる可能性がある。これらの2つの記法の定義とそれらの比較は、レイアウト規則セクションにまとめられている。

扱う範囲

行列の微分とは、各独立変数に関する各従属変数の微分をまとめるために、行列とベクトルを用いる表記法のことで、いくつか種類がある。一般に、独立変数はスカラ、ベクトル、または行列であり、従属変数もこれらのいずれかである。それぞれ異なる種類の記法では、広義の用語を用いながらも、異なる規則や個別の微積分がある。行列表記は、多くの導関数を体系的にまとめる便利な方法である。

最初の例として、ベクトル解析の分野から勾配を考える。三独立変数のスカラ関数 $f(x_{1},x_{2},x_{3})$ を考えると、その勾配はベクトル方程式で与えられる。

\nabla f={\frac {\partial f}{\partial x_{1}}}{\hat {x}}_{1}+{\frac {\partial f}{\partial x_{2}}}{\hat {x}}_{2}+{\frac {\partial f}{\partial x_{3}}}{\hat {x}}_{3}

、

ここで、 ${\hat {x}}_{i}$ は $x_{i}$ 方向の単位ベクトルを表す（ $1\leq i\leq 3$ ）。この種の一般的な微分は、ベクトル $\mathbf {x}$ に関するスカラ $f$ の微分として見ることができ、その結果はベクトル形式で容易にまとめられる。

\nabla f={\frac {\partial f}{\partial \mathbf {x} }}^{\textsf {T}}={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&{\frac {\partial f}{\partial x_{3}}}\\\end{bmatrix}}^{\textsf {T}}.

より複雑な例として、勾配行列として知られる、行列に関するスカラ関数の導関数がある。勾配行列とは、各行列要素に関するスカラ関数の微分を、対応する位置に記述したものである。その場合、スカラは行列内の各独立変数の関数でなければならない。別の例としては、 $m$ 次元の独立変数と $n$ 次元の従属変数を持つ関数があるとき、独立変数に関する従属変数の微分を考えることができる。結果は、考えうるすべての微分の組み合わせで構成される $m\times n$ 行列にまとめられる。当然、変数にスカラ、ベクトル、および行列のどれを用いるかで9つの可能性がある。ただし独立変数と従属変数のそれぞれで、3つより多い候補を考えれば、非常に多くの可能性が残ることに留意する。

行列形式でよく整理できる6種類の微分を次の表にまとめる。 ^[2] ^[2]

Types of matrix derivative
種類	スカラ	ベクトル	行列
スカラ	${\frac {\partial y}{\partial x}}$	${\frac {\partial \mathbf {y} }{\partial x}}$	${\frac {\partial \mathbf {Y} }{\partial x}}$
ベクトル	${\frac {\partial y}{\partial \mathbf {x} }}$	${\frac {\partial \mathbf {y} }{\partial \mathbf {x} }}$
行列	${\frac {\partial y}{\partial \mathbf {X} }}$

ここでは、最も一般的な意味で「行列」という用語を用いた。ベクトルとスカラは、それぞれ1列と1行1列の単純な行列である。さらに、ベクトルを表すために太字を用いて、行列には太字の大文字を用いた。この表記は全体を通して使用される。

また、行列に関するベクトルの微分や表の他の塗りつぶされていない項目についても議論できることに注意すること。ただし、これらの微分は、2を超えるランクのテンソルで最も自然に構成されるため、行列にうまく収まらない。以下の3節では、これらそれぞれの微分を定義し、数学の他の分野に関連付ける。より詳細な表については、レイアウト規則の節を参照すること。

恒等式

上記のように、一般に、分子表記と分母表記を切り替えると、演算の結果が入れ替わる。

以下のすべての恒等式を理解するために、最も重要な規則である連鎖律、積の微分則、および加法性に留意すること。加法性はいつでも当てはまる。積の微分則は、行列の積の順序が保たれている場合なら、以下のほとんどの場合に当てはまる。というのも行列の積は可換ではないためである。連鎖律はいくつかの場合に当てはまるが、残念ながら、行列ごとの微分またはスカラごとの微分には当てはまらない（後者の場合、主に行列に作用するトレース演算子が関係する）。後者の場合、積の微分則を直接当てはめることできないが、微分恒等式を用いてもう少し手を加えることで同等のことを行うことができる。

以下の恒等式は、次の規則に従う。

スカラ $a,b,c,d,e$ は、 $\mathrm {x} ,\mathbf {x} ,\mathbf {X}$ のいずれかに関する定数である。また、スカラ $u,v$ は、 $\mathrm {x} ,\mathbf {x} ,\mathbf {X}$ のいずれかに関する関数である。
ベクトル $\mathbf {a} ,\mathbf {b} ,\mathbf {c} ,\mathbf {d} ,\mathbf {e}$ は、 $\mathrm {x} ,\mathbf {x} ,\mathbf {X}$ のいずれかに関する定数である。また、ベクトル $\mathbf {u} ,\mathbf {v}$ は、 $\mathrm {x} ,\mathbf {x} ,\mathbf {X}$ のいずれかに関する関数である。
行列 $\mathbf {A} ,\mathbf {B} ,\mathbf {C} ,\mathbf {D} ,\mathbf {E}$ は、 $\mathrm {x} ,\mathbf {x} ,\mathbf {X}$ のいずれかに関する定数である。また、行列 $\mathbf {U} ,\mathbf {V}$ は、 $\mathrm {x} ,\mathbf {x} ,\mathbf {X}$ のいずれかに関する関数である。

ベクトルによるベクトルの微分の恒等式

ベクトルによるベクトルの微分の恒等式は、分子か分母の次元を削減してスカラにすることで、直接ベクトによるスカラの微分やスカラによるベクトルの微分の恒等式にも当てはまる。よってこれを始めに記す。

恒等式: ベクトルによるベクトルの微分 ${\frac {\partial \mathbf {y} }{\partial \mathbf {x} }}$
条件	式	分子表記, i.e. by y and x^T	分母表記, i.e. by y^T and x
$\mathbf {a}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \mathbf {a} }{\partial \mathbf {x} }}=$	$\mathbf {0}$
	${\frac {\partial \mathbf {x} }{\partial \mathbf {x} }}=$	$\mathbf {I}$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \mathbf {A} \mathbf {x} }{\partial \mathbf {x} }}=$	$\mathbf {A}$	$\mathbf {A} ^{\top }$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \mathbf {x} ^{\top }\mathbf {A} }{\partial \mathbf {x} }}=$	$\mathbf {A} ^{\top }$	$\mathbf {A}$
$\mathbf {a}$ は $\mathbf {x}$ の関数でない, $\mathbf {u} =\mathbf {u} (\mathbf {x} )$	${\frac {\partial a\mathbf {u} }{\partial \,\mathbf {x} }}=$	$a{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$
$\mathbf {u} =\mathbf {u} (\mathbf {x} ),v=v(\mathbf {x} )$	${\frac {\partial v\mathbf {u} }{\partial \mathbf {x} }}=$	$v{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}+\mathbf {u} {\frac {\partial v}{\partial \mathbf {x} }}$	$v{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}+{\frac {\partial v}{\partial \mathbf {x} }}\mathbf {u} ^{\top }$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない, $\mathbf {u} =\mathbf {u} (\mathbf {x} )$	${\frac {\partial \mathbf {A} \mathbf {u} }{\partial \mathbf {x} }}=$	$\mathbf {A} {\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}\mathbf {A} ^{\top }$
$\mathbf {u} =\mathbf {u} (\mathbf {x} ),\mathbf {v} =\mathbf {v} (\mathbf {x} )$	${\frac {\partial (\mathbf {u} +\mathbf {v} )}{\partial \mathbf {x} }}=$	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}+{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}$
$\mathbf {u} =\mathbf {u} (\mathbf {x} )$	${\frac {\partial \mathbf {g(u)} }{\partial \mathbf {x} }}=$	${\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}{\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}$
$\mathbf {u} =\mathbf {u} (\mathbf {x} )$	${\frac {\partial \mathbf {f(g(u))} }{\partial \mathbf {x} }}=$	${\frac {\partial \mathbf {f(g)} }{\partial \mathbf {g} }}{\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}{\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}{\frac {\partial \mathbf {f(g)} }{\partial \mathbf {g} }}$

ベクトルによるスカラの微分の恒等式

基本的な恒等式は太い黒線より上にあるものである。

恒等式: ベクトルによるスカラの微分 ${\frac {\partial y}{\partial \mathbf {x} }}=\nabla _{\mathbf {x} }y$
Condition	Expression	Numerator layout, i.e. by x^T; result is row vector	Denominator layout, i.e. by x; result is column vector
$a$ は $\mathbf {x}$ の関数でない	${\frac {\partial a}{\partial \mathbf {x} }}=$	$\mathbf {0} ^{\top }$ ^[3]	$\mathbf {0}$ ^[3]
$a$ は $\mathbf {x}$ の関数でない, $u=u(\mathbf {x} )$	${\frac {\partial au}{\partial \mathbf {x} }}=$	$a{\frac {\partial u}{\partial \mathbf {x} }}$
$u=u(\mathbf {x} ),v=v(\mathbf {x} )$	${\frac {\partial (u+v)}{\partial \mathbf {x} }}=$	${\frac {\partial u}{\partial \mathbf {x} }}+{\frac {\partial v}{\partial \mathbf {x} }}$
$u=u(\mathbf {x} ),v=v(\mathbf {x} )$	${\frac {\partial uv}{\partial \mathbf {x} }}=$	$u{\frac {\partial v}{\partial \mathbf {x} }}+v{\frac {\partial u}{\partial \mathbf {x} }}$
$u=u(\mathbf {x} )$	${\frac {\partial g(u)}{\partial \mathbf {x} }}=$	${\frac {\partial g(u)}{\partial u}}{\frac {\partial u}{\partial \mathbf {x} }}$
$u=u(\mathbf {x} )$	${\frac {\partial f(g(u))}{\partial \mathbf {x} }}=$	${\frac {\partial f(g)}{\partial g}}{\frac {\partial g(u)}{\partial u}}{\frac {\partial u}{\partial \mathbf {x} }}$
$\mathbf {u} =\mathbf {u} (\mathbf {x} ),\mathbf {v} =\mathbf {v} (\mathbf {x} )$	${\frac {\partial (\mathbf {u} \cdot \mathbf {v} )}{\partial \mathbf {x} }}={\frac {\partial \mathbf {u} ^{\top }\mathbf {v} }{\partial \mathbf {x} }}=$	$\mathbf {u} ^{\top }{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}+\mathbf {v} ^{\top }{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$ ${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }},{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}$ in numerator layout	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}\mathbf {v} +{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}\mathbf {u}$ ${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }},{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}$ in denominator layout
$\mathbf {u} =\mathbf {u} (\mathbf {x} ),\mathbf {v} =\mathbf {v} (\mathbf {x} )$ , $\mathbf {A}$ は $\mathbf {x}$ の関数でない	${\frac {\partial (\mathbf {u} \cdot \mathbf {A} \mathbf {v} )}{\partial \mathbf {x} }}={\frac {\partial \mathbf {u} ^{\top }\mathbf {A} \mathbf {v} }{\partial \mathbf {x} }}=$	$\mathbf {u} ^{\top }\mathbf {A} {\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}+\mathbf {v} ^{\top }\mathbf {A} ^{\top }{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$ ${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }},{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}$ in numerator layout	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}\mathbf {A} \mathbf {v} +{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}\mathbf {A} ^{\top }\mathbf {u}$ ${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }},{\frac {\partial \mathbf {v} }{\partial \mathbf {x} }}$ in denominator layout
	${\frac {\partial ^{2}f}{\partial \mathbf {x} \partial \mathbf {x} ^{\top }}}=$	$\mathbf {H} ^{\top }$	$\mathbf {H}$ , the Hessian matrix^[4]
$\mathbf {a}$ は $\mathbf {x}$ の関数でない	${\frac {\partial (\mathbf {a} \cdot \mathbf {x} )}{\partial \mathbf {x} }}={\frac {\partial (\mathbf {x} \cdot \mathbf {a} )}{\partial \mathbf {x} }}=$ ${\frac {\partial \mathbf {a} ^{\top }\mathbf {x} }{\partial \mathbf {x} }}={\frac {\partial \mathbf {x} ^{\top }\mathbf {a} }{\partial \mathbf {x} }}=$	$\mathbf {a} ^{\top }$	$\mathbf {a}$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない $\mathbf {b}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \mathbf {b} ^{\top }\mathbf {A} \mathbf {x} }{\partial \mathbf {x} }}=$	$\mathbf {b} ^{\top }\mathbf {A}$	$\mathbf {A} ^{\top }\mathbf {b}$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \mathbf {x} ^{\top }\mathbf {A} \mathbf {x} }{\partial \mathbf {x} }}=$	$\mathbf {x} ^{\top }\left(\mathbf {A} +\mathbf {A} ^{\top }\right)$	$\left(\mathbf {A} +\mathbf {A} ^{\top }\right)\mathbf {x}$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない $\mathbf {A}$ は対称行列	${\frac {\partial \mathbf {x} ^{\top }\mathbf {A} \mathbf {x} }{\partial \mathbf {x} }}=$	$2\mathbf {x} ^{\top }\mathbf {A}$	$2\mathbf {A} \mathbf {x}$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない	${\frac {\partial ^{2}\mathbf {x} ^{\top }\mathbf {A} \mathbf {x} }{\partial \mathbf {x} \partial \mathbf {x} ^{\top }}}=$	$\mathbf {A} +\mathbf {A} ^{\top }$
$\mathbf {A}$ は $\mathbf {x}$ の関数でない $\mathbf {A}$ は対称行列	${\frac {\partial ^{2}\mathbf {x} ^{\top }\mathbf {A} \mathbf {x} }{\partial \mathbf {x} \partial \mathbf {x} ^{\top }}}=$	$2\mathbf {A}$
	${\frac {\partial (\mathbf {x} \cdot \mathbf {x} )}{\partial \mathbf {x} }}={\frac {\partial \mathbf {x} ^{\top }\mathbf {x} }{\partial \mathbf {x} }}={\frac {\partial \left\Vert \mathbf {x} \right\Vert ^{2}}{\partial \mathbf {x} }}=$	$2\mathbf {x} ^{\top }$	$2\mathbf {x}$
$\mathbf {a}$ は $\mathbf {x}$ の関数でない, $\mathbf {u} =\mathbf {u} (\mathbf {x} )$	${\frac {\partial (\mathbf {a} \cdot \mathbf {u} )}{\partial \mathbf {x} }}={\frac {\partial \mathbf {a} ^{\top }\mathbf {u} }{\partial \mathbf {x} }}=$	$\mathbf {a} ^{\top }{\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$ ${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$ in numerator layout	${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}\mathbf {a}$ ${\frac {\partial \mathbf {u} }{\partial \mathbf {x} }}$ in denominator layout
$\mathbf {a} ,\mathbf {b}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \;{\textbf {a}}^{\top }{\textbf {x}}{\textbf {x}}^{\top }{\textbf {b}}}{\partial \;{\textbf {x}}}}=$	${\textbf {x}}^{\top }\left({\textbf {a}}{\textbf {b}}^{\top }+{\textbf {b}}{\textbf {a}}^{\top }\right)$	$\left({\textbf {a}}{\textbf {b}}^{\top }+{\textbf {b}}{\textbf {a}}^{\top }\right){\textbf {x}}$
$\mathbf {A} ,\mathbf {B} ,\mathbf {C} ,\mathbf {D} ,\mathbf {E} ,$ は $\mathbf {x}$ の関数でない	${\frac {\partial \;({\textbf {A}}{\textbf {x}}+{\textbf {b}})^{\top }{\textbf {C}}({\textbf {D}}{\textbf {x}}+{\textbf {e}})}{\partial \;{\textbf {x}}}}=$	$({\textbf {D}}{\textbf {x}}+{\textbf {e}})^{\top }{\textbf {C}}^{\top }{\textbf {A}}+({\textbf {A}}{\textbf {x}}+{\textbf {b}})^{\top }{\textbf {C}}{\textbf {D}}$	${\textbf {D}}^{\top }{\textbf {C}}^{\top }({\textbf {A}}{\textbf {x}}+{\textbf {b}})+{\textbf {A}}^{\top }{\textbf {C}}({\textbf {D}}{\textbf {x}}+{\textbf {e}})$
$\mathbf {a}$ は $\mathbf {x}$ の関数でない	${\frac {\partial \;\\|\mathbf {x} -\mathbf {a} \\|}{\partial \;\mathbf {x} }}=$	${\frac {(\mathbf {x} -\mathbf {a} )^{\top }}{\\|\mathbf {x} -\mathbf {a} \\|}}$	${\frac {\mathbf {x} -\mathbf {a} }{\\|\mathbf {x} -\mathbf {a} \\|}}$

スカラによるベクトルの微分の恒等式

恒等式: スカラによるベクトルの微分 ${\frac {\partial \mathbf {y} }{\partial x}}$
条件	式	分子表記, i.e. by y, result is column vector	分母表記, i.e. by y^T, result is row vector
$\mathbf {a}$ は $x$ の関数でない	${\frac {\partial \mathbf {a} }{\partial x}}=$	$\mathbf {0}$ ^[3]
$a$ は $x$ の関数でない, $\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial a\mathbf {u} }{\partial x}}=$	$a{\frac {\partial \mathbf {u} }{\partial x}}$
$\mathbf {A}$ は $x$ の関数でない, $\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial \mathbf {A} \mathbf {u} }{\partial x}}=$	$\mathbf {A} {\frac {\partial \mathbf {u} }{\partial x}}$	${\frac {\partial \mathbf {u} }{\partial x}}\mathbf {A} ^{\top }$
$\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial \mathbf {u} ^{\top }}{\partial x}}=$	$\left({\frac {\partial \mathbf {u} }{\partial x}}\right)^{\top }$
$\mathbf {u} =\mathbf {u} (x),\mathbf {v} =\mathbf {v} (x)$	${\frac {\partial (\mathbf {u} +\mathbf {v} )}{\partial x}}=$	${\frac {\partial \mathbf {u} }{\partial x}}+{\frac {\partial \mathbf {v} }{\partial x}}$
$\mathbf {u} =\mathbf {u} (x),\mathbf {v} =\mathbf {v} (x)$	${\frac {\partial (\mathbf {u} ^{\top }\times \mathbf {v} )}{\partial x}}=$	$\left({\frac {\partial \mathbf {u} }{\partial x}}\right)^{\top }\times \mathbf {v} +\mathbf {u} ^{\top }\times {\frac {\partial \mathbf {v} }{\partial x}}$	${\frac {\partial \mathbf {u} }{\partial x}}\times \mathbf {v} +\mathbf {u} ^{\top }\times \left({\frac {\partial \mathbf {v} }{\partial x}}\right)^{\top }$
$\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial \mathbf {g(u)} }{\partial x}}=$	${\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}{\frac {\partial \mathbf {u} }{\partial x}}$	${\frac {\partial \mathbf {u} }{\partial x}}{\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}$
$\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial \mathbf {g(u)} }{\partial x}}=$	Assumes consistent matrix layout; see below.
$\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial \mathbf {f(g(u))} }{\partial x}}=$	${\frac {\partial \mathbf {f(g)} }{\partial \mathbf {g} }}{\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}{\frac {\partial \mathbf {u} }{\partial x}}$	${\frac {\partial \mathbf {u} }{\partial x}}{\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}{\frac {\partial \mathbf {f(g)} }{\partial \mathbf {g} }}$
$\mathbf {u} =\mathbf {u} (x)$	${\frac {\partial \mathbf {f(g(u))} }{\partial x}}=$	Assumes consistent matrix layout; see below.
$\mathbf {U} =\mathbf {U} (x),\mathbf {v} =\mathbf {v} (x)$	${\frac {\partial (\mathbf {U} \times \mathbf {v} )}{\partial x}}=$	${\frac {\partial \mathbf {U} }{\partial x}}\times \mathbf {v} +\mathbf {U} \times {\frac {\partial \mathbf {v} }{\partial x}}$	$\mathbf {v} ^{\top }\times \left({\frac {\partial \mathbf {U} }{\partial x}}\right)+{\frac {\partial \mathbf {v} }{\partial x}}\times \mathbf {U} ^{\top }$

NOTE: The formulas involving the vector-by-vector derivatives ${\frac {\partial \mathbf {g(u)} }{\partial \mathbf {u} }}$ and ${\frac {\partial \mathbf {f(g)} }{\partial \mathbf {g} }}$ (whose outputs are matrices) assume the matrices are laid out consistent with the vector layout, i.e. numerator-layout matrix when numerator-layout vector and vice versa; otherwise, transpose the vector-by-vector derivatives.

行列によるスカラの微分の恒等式

Note that exact equivalents of the scalar product rule and chain rule do not exist when applied to matrix-valued functions of matrices. However, the product rule of this sort does apply to the differential form (see below), and this is the way to derive many of the identities below involving the trace function, combined with the fact that the trace function allows transposing and cyclic permutation, i.e.:

{\begin{aligned}\operatorname {tr} (\mathbf {A} )&=\operatorname {tr} \left(\mathbf {A^{\top }} \right)\\\operatorname {tr} (\mathbf {ABCD} )&=\operatorname {tr} (\mathbf {BCDA} )=\operatorname {tr} (\mathbf {CDAB} )=\operatorname {tr} (\mathbf {DABC} )\end{aligned}}

For example, to compute ${\frac {\partial \operatorname {tr} (\mathbf {AXBX^{\top }C} )}{\partial \mathbf {X} }}:$

{\begin{aligned}d\operatorname {tr} (\mathbf {AXBX^{\top }C} )&=d\operatorname {tr} \left(\mathbf {CAXBX^{\top }} \right)=\operatorname {tr} \left(d\left(\mathbf {CAXBX^{\top }} \right)\right)\\&=\operatorname {tr} \left(\mathbf {CAX} d(\mathbf {BX^{\top }} \right)+d\left(\mathbf {CAX} )\mathbf {BX^{\top }} \right)\\&=\operatorname {tr} \left(\mathbf {CAX} d\left(\mathbf {BX^{\top }} \right)\right)+\operatorname {tr} \left(d(\mathbf {CAX} )\mathbf {BX^{\top }} \right)\\&=\operatorname {tr} \left(\mathbf {CAXB} d\left(\mathbf {X^{\top }} \right)\right)+\operatorname {tr} \left(\mathbf {CA} (d\mathbf {X} )\mathbf {BX^{\top }} \right)\\&=\operatorname {tr} \left(\mathbf {CAXB} (d\mathbf {X} )^{\top }\right)+\operatorname {tr} (\mathbf {CA} \left(d\mathbf {X} )\mathbf {BX^{\top }} \right)\\&=\operatorname {tr} \left(\left(\mathbf {CAXB} (d\mathbf {X} )^{\top }\right)^{\top }\right)+\operatorname {tr} \left(\mathbf {CA} (d\mathbf {X} )\mathbf {BX^{\top }} \right)\\&=\operatorname {tr} \left((d\mathbf {X} )\mathbf {B^{\top }X^{\top }A^{\top }C^{\top }} \right)+\operatorname {tr} \left(\mathbf {CA} (d\mathbf {X} )\mathbf {BX^{\top }} \right)\\&=\operatorname {tr} \left(\mathbf {B^{\top }X^{\top }A^{\top }C^{\top }} (d\mathbf {X} )\right)+\operatorname {tr} \left(\mathbf {BX^{\top }} \mathbf {CA} (d\mathbf {X} )\right)\\&=\operatorname {tr} \left(\left(\mathbf {B^{\top }X^{\top }A^{\top }C^{\top }} +\mathbf {BX^{\top }} \mathbf {CA} \right)d\mathbf {X} \right)\end{aligned}}

Therefore,

{\frac {\partial \operatorname {tr} \left(\mathbf {AXBX^{\top }C} \right)}{\partial \mathbf {X} }}=\mathbf {CAXB} +\mathbf {A^{\top }C^{\top }} \mathbf {XB^{\top }} .

(For the last step, see the 'Conversion from differential to derivative form' section.)

Identities: scalar-by-matrix ${\frac {\partial y}{\partial \mathbf {X} }}$
Condition	Expression	Numerator layout, i.e. by X^T	Denominator layout, i.e. by X
a is not a function of X	${\frac {\partial a}{\partial \mathbf {X} }}=$	$\mathbf {0} ^{\top }$ ^[5]	$\mathbf {0}$ ^[5]
a is not a function of X, u = u(X)	${\frac {\partial au}{\partial \mathbf {X} }}=$	$a{\frac {\partial u}{\partial \mathbf {X} }}$
u = u(X), v = v(X)	${\frac {\partial (u+v)}{\partial \mathbf {X} }}=$	${\frac {\partial u}{\partial \mathbf {X} }}+{\frac {\partial v}{\partial \mathbf {X} }}$
u = u(X), v = v(X)	${\frac {\partial uv}{\partial \mathbf {X} }}=$	$u{\frac {\partial v}{\partial \mathbf {X} }}+v{\frac {\partial u}{\partial \mathbf {X} }}$
u = u(X)	${\frac {\partial g(u)}{\partial \mathbf {X} }}=$	${\frac {\partial g(u)}{\partial u}}{\frac {\partial u}{\partial \mathbf {X} }}$
u = u(X)	${\frac {\partial f(g(u))}{\partial \mathbf {X} }}=$	${\frac {\partial f(g)}{\partial g}}{\frac {\partial g(u)}{\partial u}}{\frac {\partial u}{\partial \mathbf {X} }}$
U = U(X)	^[4] ${\frac {\partial g(\mathbf {U} )}{\partial X_{ij}}}=$	$\operatorname {tr} \left({\frac {\partial g(\mathbf {U} )}{\partial \mathbf {U} }}{\frac {\partial \mathbf {U} }{\partial X_{ij}}}\right)$	$\operatorname {tr} \left(\left({\frac {\partial g(\mathbf {U} )}{\partial \mathbf {U} }}\right)^{\top }{\frac {\partial \mathbf {U} }{\partial X_{ij}}}\right)$
U = U(X)	^[4] ${\frac {\partial g(\mathbf {U} )}{\partial X_{ij}}}=$	Both forms assume numerator layout for ${\frac {\partial \mathbf {U} }{\partial X_{ij}}},$ i.e. mixed layout if denominator layout for X is being used.
a and b are not functions of X	${\frac {\partial a^{\top }\mathbf {X} b}{\partial \mathbf {X} }}=$	$ba^{\top }$	$ab^{\top }$
a and b are not functions of X	${\frac {\partial a^{\top }\mathbf {X} ^{\top }b}{\partial \mathbf {X} }}=$	$ab^{\top }$	$ba^{\top }$
a, b and C are not functions of X	${\frac {\partial (\mathbf {X} a+b)^{\top }\mathbf {C} (\mathbf {X} a+b)}{\partial \mathbf {X} }}=$	$\left(\left(\mathbf {C} +\mathbf {C} ^{\top }\right)(\mathbf {X} a+b)a^{\top }\right)^{\top }$	$\left(\mathbf {C} +\mathbf {C} ^{\top }\right)(\mathbf {X} a+b)a^{\top }$
a, b and C are not functions of X	${\frac {\partial (\mathbf {X} a)^{\top }\mathbf {C} (\mathbf {X} b)}{\partial \mathbf {X} }}=$	$\left(\mathbf {C} \mathbf {X} ba^{\top }+\mathbf {C} ^{\top }\mathbf {X} ab^{\top }\right)^{\top }$	$\mathbf {C} \mathbf {X} ba^{\top }+\mathbf {C} ^{\top }\mathbf {X} ab^{\top }$
	${\frac {\partial \operatorname {tr} (\mathbf {X} )}{\partial \mathbf {X} }}=$	$\mathbf {I}$
U = U(X), V = V(X)	${\frac {\partial \operatorname {tr} (\mathbf {U} +\mathbf {V} )}{\partial \mathbf {X} }}=$	${\frac {\partial \operatorname {tr} (\mathbf {U} )}{\partial \mathbf {X} }}+{\frac {\partial \operatorname {tr} (\mathbf {V} )}{\partial \mathbf {X} }}$
a is not a function of X, U = U(X)	${\frac {\partial \operatorname {tr} (a\mathbf {U} )}{\partial \mathbf {X} }}=$	$a{\frac {\partial \operatorname {tr} (\mathbf {U} )}{\partial \mathbf {X} }}$
g(X) is any polynomial with scalar coefficients, or any matrix function defined by an infinite polynomial series (e.g. e^X, sin(X), cos(X), ln(X), etc. using a Taylor series); g(x) is the equivalent scalar function, g′(x) is its derivative, and g′(X) is the corresponding matrix function	${\frac {\partial \operatorname {tr} (\mathbf {g(X)} )}{\partial \mathbf {X} }}=$	$\mathbf {g} '(\mathbf {X} )$	$\left(\mathbf {g} '(\mathbf {X} )\right)^{\top }$
A is not a function of X	^[6] ${\frac {\partial \operatorname {tr} (\mathbf {AX} )}{\partial \mathbf {X} }}={\frac {\partial \operatorname {tr} (\mathbf {XA} )}{\partial \mathbf {X} }}=$	$\mathbf {A}$	$\mathbf {A} ^{\top }$
A is not a function of X	^[4] ${\frac {\partial \operatorname {tr} \left(\mathbf {AX^{\top }} \right)}{\partial \mathbf {X} }}={\frac {\partial \operatorname {tr} \left(\mathbf {X^{\top }A} \right)}{\partial \mathbf {X} }}=$	$\mathbf {A} ^{\top }$	$\mathbf {A}$
A is not a function of X	^[4] ${\frac {\partial \operatorname {tr} \left(\mathbf {X^{\top }AX} \right)}{\partial \mathbf {X} }}=$	$\mathbf {X} ^{\top }\left(\mathbf {A} +\mathbf {A} ^{\top }\right)$	$\left(\mathbf {A} +\mathbf {A} ^{\top }\right)\mathbf {X}$
A is not a function of X	^[4] ${\frac {\partial \operatorname {tr} (\mathbf {X^{-1}A} )}{\partial \mathbf {X} }}=$	$-\mathbf {X} ^{-1}\mathbf {A} \mathbf {X} ^{-1}$	$-\left(\mathbf {X} ^{-1}\right)^{\top }\mathbf {A} ^{\top }\left(\mathbf {X} ^{-1}\right)^{\top }$
A, B are not functions of X	${\frac {\partial \operatorname {tr} (\mathbf {AXB} )}{\partial \mathbf {X} }}={\frac {\partial \operatorname {tr} (\mathbf {BAX} )}{\partial \mathbf {X} }}=$	$\mathbf {BA}$	$\mathbf {A^{\top }B^{\top }}$
A, B, C are not functions of X	${\frac {\partial \operatorname {tr} \left(\mathbf {AXBX^{\top }C} \right)}{\partial \mathbf {X} }}=$	$\mathbf {BX^{\top }CA} +\mathbf {B^{\top }X^{\top }A^{\top }C^{\top }}$	$\mathbf {A^{\top }C^{\top }XB^{\top }} +\mathbf {CAXB}$
n is a positive integer	^[4] ${\frac {\partial \operatorname {tr} \left(\mathbf {X} ^{n}\right)}{\partial \mathbf {X} }}=$	$n\mathbf {X} ^{n-1}$	$n\left(\mathbf {X} ^{n-1}\right)^{\top }$
A is not a function of X, n is a positive integer	^[4] ${\frac {\partial \operatorname {tr} \left(\mathbf {A} \mathbf {X} ^{n}\right)}{\partial \mathbf {X} }}=$	$\sum _{i=0}^{n-1}\mathbf {X} ^{i}\mathbf {A} \mathbf {X} ^{n-i-1}$	$\sum _{i=0}^{n-1}\left(\mathbf {X} ^{i}\mathbf {A} \mathbf {X} ^{n-i-1}\right)^{\top }$
	^[4] ${\frac {\partial \operatorname {tr} \left(e^{\mathbf {X} }\right)}{\partial \mathbf {X} }}=$	$e^{\mathbf {X} }$	$\left(e^{\mathbf {X} }\right)^{\top }$
	^[4] ${\frac {\partial \operatorname {tr} (\sin(\mathbf {X} ))}{\partial \mathbf {X} }}=$	$\cos(\mathbf {X} )$	$(\cos(\mathbf {X} ))^{\top }$
	^[7] ${\frac {\partial \|\mathbf {X} \|}{\partial \mathbf {X} }}=$	$\operatorname {cofactor} (X)^{\top }=\|\mathbf {X} \|\mathbf {X} ^{-1}$	$\operatorname {cofactor} (X)=\|\mathbf {X} \|\left(\mathbf {X} ^{-1}\right)^{\top }$
a is not a function of X	^[4] ${\frac {\partial \ln \|a\mathbf {X} \|}{\partial \mathbf {X} }}=$ ^[8]	$\mathbf {X} ^{-1}$	$\left(\mathbf {X} ^{-1}\right)^{\top }$
A, B are not functions of X	^[4] ${\frac {\partial \|\mathbf {AXB} \|}{\partial \mathbf {X} }}=$	$\|\mathbf {AXB} \|\mathbf {X} ^{-1}$	$\|\mathbf {AXB} \|\left(\mathbf {X} ^{-1}\right)^{\top }$
n is a positive integer	^[4] ${\frac {\partial \left\|\mathbf {X} ^{n}\right\|}{\partial \mathbf {X} }}=$	$n\left\|\mathbf {X} ^{n}\right\|\mathbf {X} ^{-1}$	$n\left\|\mathbf {X} ^{n}\right\|\left(\mathbf {X} ^{-1}\right)^{\top }$
(see pseudo-inverse)	^[4] ${\frac {\partial \ln \left\|\mathbf {X} ^{\top }\mathbf {X} \right\|}{\partial \mathbf {X} }}=$	$2\mathbf {X} ^{+}$	$2\left(\mathbf {X} ^{+}\right)^{\top }$
(see pseudo-inverse)	^[4] ${\frac {\partial \ln \left\|\mathbf {X} ^{\top }\mathbf {X} \right\|}{\partial \mathbf {X} ^{+}}}=$	$-2\mathbf {X}$	$-2\mathbf {X} ^{\top }$
A is not a function of X, X is square and invertible	${\frac {\partial \left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|}{\partial \mathbf {X} }}=$	$2\left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|\mathbf {X} ^{-1}=2\left\|\mathbf {X^{\top }} \right\|\|\mathbf {A} \|\|\mathbf {X} \|\mathbf {X} ^{-1}$	$2\left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|\left(\mathbf {X} ^{-1}\right)^{\top }$
A is not a function of X, X is non-square, A is symmetric	${\frac {\partial \left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|}{\partial \mathbf {X} }}=$	$2\left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|\left(\mathbf {X^{\top }A^{\top }X} \right)^{-1}\mathbf {X^{\top }A^{\top }}$	$2\left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|\mathbf {AX} \left(\mathbf {X^{\top }AX} \right)^{-1}$
A is not a function of X, X is non-square, A is non-symmetric	${\frac {\partial \|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \|}{\partial \mathbf {X} }}=$	${\begin{aligned}\left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|{\Big (}&\left(\mathbf {X^{\top }AX} \right)^{-1}\mathbf {X^{\top }A} +{}\\&\left(\mathbf {X^{\top }A^{\top }X} \right)^{-1}\mathbf {X^{\top }A^{\top }} {\Big )}\end{aligned}}$	${\begin{aligned}\left\|\mathbf {X^{\top }} \mathbf {A} \mathbf {X} \right\|{\Big (}&\mathbf {AX} \left(\mathbf {X^{\top }AX} \right)^{-1}+{}\\&\mathbf {A^{\top }X} \left(\mathbf {X^{\top }A^{\top }X} \right)^{-1}{\Big )}\end{aligned}}$

Matrix-by-scalar identities

Identities: matrix-by-scalar ${\frac {\partial \mathbf {Y} }{\partial x}}$
Condition	Expression	Numerator layout, i.e. by Y
U = U(x)	${\frac {\partial a\mathbf {U} }{\partial x}}=$	$a{\frac {\partial \mathbf {U} }{\partial x}}$
A, B are not functions of x, U = U(x)	${\frac {\partial \mathbf {AUB} }{\partial x}}=$	$\mathbf {A} {\frac {\partial \mathbf {U} }{\partial x}}\mathbf {B}$
U = U(x), V = V(x)	${\frac {\partial (\mathbf {U} +\mathbf {V} )}{\partial x}}=$	${\frac {\partial \mathbf {U} }{\partial x}}+{\frac {\partial \mathbf {V} }{\partial x}}$
U = U(x), V = V(x)	${\frac {\partial (\mathbf {U} \mathbf {V} )}{\partial x}}=$	$\mathbf {U} {\frac {\partial \mathbf {V} }{\partial x}}+{\frac {\partial \mathbf {U} }{\partial x}}\mathbf {V}$
U = U(x), V = V(x)	${\frac {\partial (\mathbf {U} \otimes \mathbf {V} )}{\partial x}}=$	$\mathbf {U} \otimes {\frac {\partial \mathbf {V} }{\partial x}}+{\frac {\partial \mathbf {U} }{\partial x}}\otimes \mathbf {V}$
U = U(x), V = V(x)	${\frac {\partial (\mathbf {U} \circ \mathbf {V} )}{\partial x}}=$	$\mathbf {U} \circ {\frac {\partial \mathbf {V} }{\partial x}}+{\frac {\partial \mathbf {U} }{\partial x}}\circ \mathbf {V}$
U = U(x)	${\frac {\partial \mathbf {U} ^{-1}}{\partial x}}=$	$-\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial x}}\mathbf {U} ^{-1}$
U = U(x,y)	${\frac {\partial ^{2}\mathbf {U} ^{-1}}{\partial x\partial y}}=$	$\mathbf {U} ^{-1}\left({\frac {\partial \mathbf {U} }{\partial x}}\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial y}}-{\frac {\partial ^{2}\mathbf {U} }{\partial x\partial y}}+{\frac {\partial \mathbf {U} }{\partial y}}\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial x}}\right)\mathbf {U} ^{-1}$
A is not a function of x, g(X) is any polynomial with scalar coefficients, or any matrix function defined by an infinite polynomial series (e.g. e^X, sin(X), cos(X), ln(X), etc.); g(x) is the equivalent scalar function, g′(x) is its derivative, and g′(X) is the corresponding matrix function	${\frac {\partial \,\mathbf {g} (x\mathbf {A} )}{\partial x}}=$	$\mathbf {A} \mathbf {g} '(x\mathbf {A} )=\mathbf {g} '(x\mathbf {A} )\mathbf {A}$
A is not a function of x	${\frac {\partial e^{x\mathbf {A} }}{\partial x}}=$	$\mathbf {A} e^{x\mathbf {A} }=e^{x\mathbf {A} }\mathbf {A}$

Further see Derivative of the exponential map.

Scalar-by-scalar identities

With vectors involved

Identities: scalar-by-scalar, with vectors involved
Condition	Expression	Any layout (assumes dot product ignores row vs. column layout)
u = u(x)	${\frac {\partial g(\mathbf {u} )}{\partial x}}=$	${\frac {\partial g(\mathbf {u} )}{\partial \mathbf {u} }}\cdot {\frac {\partial \mathbf {u} }{\partial x}}$
u = u(x), v = v(x)	${\frac {\partial (\mathbf {u} \cdot \mathbf {v} )}{\partial x}}=$	$\mathbf {u} \cdot {\frac {\partial \mathbf {v} }{\partial x}}+{\frac {\partial \mathbf {u} }{\partial x}}\cdot \mathbf {v}$

With matrices involved

Identities: scalar-by-scalar, with matrices involved^[4]
Condition	Expression	Consistent numerator layout, i.e. by Y and X^T	Mixed layout, i.e. by Y and X
U = U(x)	${\frac {\partial \|\mathbf {U} \|}{\partial x}}=$	$\|\mathbf {U} \|\operatorname {tr} \left(\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial x}}\right)$
U = U(x)	${\frac {\partial \ln \|\mathbf {U} \|}{\partial x}}=$	$\operatorname {tr} \left(\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial x}}\right)$
U = U(x)	${\frac {\partial ^{2}\|\mathbf {U} \|}{\partial x^{2}}}=$	$\|\mathbf {U} \|\left[\operatorname {tr} \left(\mathbf {U} ^{-1}{\frac {\partial ^{2}\mathbf {U} }{\partial x^{2}}}\right)+\operatorname {tr} ^{2}\left(\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial x}}\right)-\operatorname {tr} \left(\left(\mathbf {U} ^{-1}{\frac {\partial \mathbf {U} }{\partial x}}\right)^{2}\right)\right]$
U = U(x)	${\frac {\partial g(\mathbf {U} )}{\partial x}}=$	$\operatorname {tr} \left({\frac {\partial g(\mathbf {U} )}{\partial \mathbf {U} }}{\frac {\partial \mathbf {U} }{\partial x}}\right)$	$\operatorname {tr} \left(\left({\frac {\partial g(\mathbf {U} )}{\partial \mathbf {U} }}\right)^{\top }{\frac {\partial \mathbf {U} }{\partial x}}\right)$
A is not a function of x, g(X) is any polynomial with scalar coefficients, or any matrix function defined by an infinite polynomial series (e.g. e^X, sin(X), cos(X), ln(X), etc.); g(x) is the equivalent scalar function, g′(x) is its derivative, and g′(X) is the corresponding matrix function.	${\frac {\partial \operatorname {tr} (\mathbf {g} (x\mathbf {A} ))}{\partial x}}=$	$\operatorname {tr} \left(\mathbf {A} \mathbf {g} '(x\mathbf {A} )\right)$
A is not a function of x	${\frac {\partial \operatorname {tr} \left(e^{x\mathbf {A} }\right)}{\partial x}}=$	$\operatorname {tr} \left(\mathbf {A} e^{x\mathbf {A} }\right)$

Identities in differential form

It is often easier to work in differential form and then convert back to normal derivatives. This only works well using the numerator layout. In these rules, "a" is a scalar.

Differential identities: scalar involving matrix^[2]^[4]
Condition	Expression	Result (numerator layout)
	$d(\operatorname {tr} (\mathbf {X} ))=$	$\operatorname {tr} (d\mathbf {X} )$
	$d(\|\mathbf {X} \|)=$	$\|\mathbf {X} \|\operatorname {tr} \left(\mathbf {X} ^{-1}d\mathbf {X} \right)=\operatorname {tr} (\operatorname {adj} (\mathbf {X} )d\mathbf {X} )$
	$d(\ln \|\mathbf {X} \|)=$	$\operatorname {tr} \left(\mathbf {X} ^{-1}d\mathbf {X} \right)$

Differential identities: matrix^[2]^[4]
Condition	Expression	Result (numerator layout)
A is not a function of X	$d(\mathbf {A} )=$	$0$
a is not a function of X	$d(a\mathbf {X} )=$	$a\,d\mathbf {X}$
	$d(\mathbf {X} +\mathbf {Y} )=$	$d\mathbf {X} +d\mathbf {Y}$
	$d(\mathbf {X} \mathbf {Y} )=$	$(d\mathbf {X} )\mathbf {Y} +\mathbf {X} (d\mathbf {Y} )$
(Kronecker product)	$d(\mathbf {X} \otimes \mathbf {Y} )=$	$(d\mathbf {X} )\otimes \mathbf {Y} +\mathbf {X} \otimes (d\mathbf {Y} )$
(Hadamard product)	$d(\mathbf {X} \circ \mathbf {Y} )=$	$(d\mathbf {X} )\circ \mathbf {Y} +\mathbf {X} \circ (d\mathbf {Y} )$
	$d\left(\mathbf {X} ^{\top }\right)=$	$(d\mathbf {X} )^{\top }$
	$d\left(\mathbf {X} ^{-1}\right)=$	$-\mathbf {X} ^{-1}\left(d\mathbf {X} \right)\mathbf {X} ^{-1}$
(conjugate transpose)	$d\left(\mathbf {X} ^{\rm {H}}\right)=$	$(d\mathbf {X} )^{\rm {H}}$

To convert to normal derivative form, first convert it to one of the following canonical forms, and then use these identities:

Conversion from differential to derivative form^[2]
Canonical differential form	Equivalent derivative form
$dy=a\,dx$	${\frac {dy}{dx}}=a$
$dy=\mathbf {a} \,d\mathbf {x}$	${\frac {dy}{d\mathbf {x} }}=\mathbf {a}$
$dy=\operatorname {tr} (\mathbf {A} \,d\mathbf {X} )$	${\frac {dy}{d\mathbf {X} }}=\mathbf {A}$
$d\mathbf {y} =\mathbf {a} \,dx$	${\frac {d\mathbf {y} }{dx}}=\mathbf {a}$
$d\mathbf {y} =\mathbf {A} \,d\mathbf {x}$	${\frac {d\mathbf {y} }{d\mathbf {x} }}=\mathbf {A}$
$d\mathbf {Y} =\mathbf {A} \,dx$	${\frac {d\mathbf {Y} }{dx}}=\mathbf {A}$

[[Category:多変数微分積分学]] [[Category:線型代数学]] [[Category:行列論]]

^ Matrix Calculusには定着した日本語訳が見当たらない。Vector Calculus（ベクトル解析）の類推をすれば行列解析と訳されるべきであるが、ここでは行列解析のうち特に微分を扱っているため、行列の微分と訳した。
^ ^a ^b ^c ^d ^e Thomas P. (December 28, 2000). “Old and New Matrix Algebra Useful for Statistics”. 5 February 2016閲覧。
^ ^a ^b ^c Here, $\mathbf {0}$ refers to a column vector of all 0's, of size n, where n is the length of x.
^ ^a ^b ^c ^d ^e ^f ^g ^h ⁱ ^j ^k ^l ^m ⁿ ^o ^p ^q Petersen, Kaare Brandt; Pedersen, Michael Syskind. The Matrix Cookbook. オリジナルの2 March 2010時点におけるアーカイブ。 5 February 2016閲覧。 This book uses a mixed layout, i.e. by Y in ${\frac {\partial \mathbf {Y} }{\partial x}},$ by X in ${\frac {\partial y}{\partial \mathbf {X} }}.$
^ ^a ^b Here, $\mathbf {0}$ refers to a matrix of all 0's, of the same shape as X.
^ Duchi, John C. “Properties of the Trace and Matrix Derivatives”. Stanford University. 5 February 2016閲覧。
^ See Determinant#Derivative for the derivation.
^ The constant a disappears in the result. This is intentional. In general,
${\frac {d\ln au}{dx}}={\frac {1}{au}}{\frac {d(au)}{dx}}={\frac {1}{au}}a{\frac {du}{dx}}={\frac {1}{u}}{\frac {du}{dx}}={\frac {d\ln u}{dx}}.$
or, also
${\frac {d\ln au}{dx}}={\frac {d(\ln a+\ln u)}{dx}}={\frac {d\ln a}{dx}}+{\frac {d\ln u}{dx}}={\frac {d\ln u}{dx}}.$

[1] Matrix Calculusには定着した日本語訳が見当たらない。Vector Calculus（ベクトル解析）の類推をすれば行列解析と訳されるべきであるが、ここでは行列解析のうち特に微分を扱っているため、行列の微分と訳した。

[minka-2] Thomas P. (December 28, 2000). “Old and New Matrix Algebra Useful for Statistics”. 5 February 2016閲覧。

[zerovec-3] Here, $\mathbf {0}$ refers to a column vector of all 0's, of size n, where n is the length of x.

[matrix-cookbook-4] ^ ^a ^b ^c ^d ^e ^f ^g ^h ⁱ ^j ^k ^l ^m ⁿ ^o ^p ^q Petersen, Kaare Brandt; Pedersen, Michael Syskind. The Matrix Cookbook. オリジナルの2 March 2010時点におけるアーカイブ。 5 February 2016閲覧。 This book uses a mixed layout, i.e. by Y in ${\frac {\partial \mathbf {Y} }{\partial x}},$ by X in ${\frac {\partial y}{\partial \mathbf {X} }}.$

[zeromatrix-5] Here, $\mathbf {0}$ refers to a matrix of all 0's, of the same shape as X.

[6] Duchi, John C. “Properties of the Trace and Matrix Derivatives”. Stanford University. 5 February 2016閲覧。

[7] See Determinant#Derivative for the derivation.

[8] The constant a disappears in the result. This is intentional. In general,
${\frac {d\ln au}{dx}}={\frac {1}{au}}{\frac {d(au)}{dx}}={\frac {1}{au}}a{\frac {du}{dx}}={\frac {1}{u}}{\frac {du}{dx}}={\frac {d\ln u}{dx}}.$
or, also
${\frac {d\ln au}{dx}}={\frac {d(\ln a+\ln u)}{dx}}={\frac {d\ln a}{dx}}+{\frac {d\ln u}{dx}}={\frac {d\ln u}{dx}}.$

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]