Matrix derivative of square root
$begingroup$
Let A be an $ntimes n$ matrix (real valued), and set $S := sqrt{A^t A}$. What is the matrix derivative of $S$ w.r.t. $A$? I.e., what is $frac{partial S}{partial A}$?
(In case of invertibility issues, I might for the time being assume that $A^t A$ is positive definite, although for the applications I have in mind I might have only positive semi-definiteness).
Thanks for help!
linear-algebra matrices matrix-calculus
asked Jan 2 at 19:48
user2019user2019
61
$endgroup$
add a comment |
$begingroup$
Let A be an $ntimes n$ matrix (real valued), and set $S := sqrt{A^t A}$. What is the matrix derivative of $S$ w.r.t. $A$? I.e., what is $frac{partial S}{partial A}$?
(In case of invertibility issues, I might for the time being assume that $A^t A$ is positive definite, although for the applications I have in mind I might have only positive semi-definiteness).
Thanks for help!
linear-algebra matrices matrix-calculus
asked Jan 2 at 19:48
user2019user2019
61
$endgroup$
$begingroup$
What is the motivation you have in mind? What was your attempt at solving the problem?
$endgroup$
– A. Pongrácz
Jan 2 at 19:59
$begingroup$
see math.stackexchange.com/questions/1884536/… and references therein
$endgroup$
– rych
Jan 20 at 11:23
add a comment |
$begingroup$
Let A be an $ntimes n$ matrix (real valued), and set $S := sqrt{A^t A}$. What is the matrix derivative of $S$ w.r.t. $A$? I.e., what is $frac{partial S}{partial A}$?
(In case of invertibility issues, I might for the time being assume that $A^t A$ is positive definite, although for the applications I have in mind I might have only positive semi-definiteness).
Thanks for help!
linear-algebra matrices matrix-calculus
asked Jan 2 at 19:48
user2019user2019
61
$endgroup$
Let A be an $ntimes n$ matrix (real valued), and set $S := sqrt{A^t A}$. What is the matrix derivative of $S$ w.r.t. $A$? I.e., what is $frac{partial S}{partial A}$?
(In case of invertibility issues, I might for the time being assume that $A^t A$ is positive definite, although for the applications I have in mind I might have only positive semi-definiteness).
Thanks for help!
linear-algebra matrices matrix-calculus
linear-algebra matrices matrix-calculus
asked Jan 2 at 19:48
user2019user2019
61
asked Jan 2 at 19:48
user2019user2019
61
asked Jan 2 at 19:48
user2019user2019
61
asked Jan 2 at 19:48
user2019user2019
61
asked Jan 2 at 19:48
user2019user2019
61
61
$begingroup$
What is the motivation you have in mind? What was your attempt at solving the problem?
$endgroup$
– A. Pongrácz
Jan 2 at 19:59
$begingroup$
see math.stackexchange.com/questions/1884536/… and references therein
$endgroup$
– rych
Jan 20 at 11:23
add a comment |
$begingroup$
What is the motivation you have in mind? What was your attempt at solving the problem?
$endgroup$
– A. Pongrácz
Jan 2 at 19:59
$begingroup$
see math.stackexchange.com/questions/1884536/… and references therein
$endgroup$
– rych
Jan 20 at 11:23
$begingroup$
What is the motivation you have in mind? What was your attempt at solving the problem?
$endgroup$
– A. Pongrácz
Jan 2 at 19:59
$begingroup$
What is the motivation you have in mind? What was your attempt at solving the problem?
$endgroup$
– A. Pongrácz
Jan 2 at 19:59
$begingroup$
see math.stackexchange.com/questions/1884536/… and references therein
$endgroup$
– rych
Jan 20 at 11:23
$begingroup$
see math.stackexchange.com/questions/1884536/… and references therein
$endgroup$
– rych
Jan 20 at 11:23
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
Recall that you can perform a polar decomposition on $A$ such that:
$$A = QS$$
where $Q$ is an orthogonal matrix and $S$ is defined uniquely as precisely what you describe above when it exists. Hence, what you really want to calculate is:
$$frac{partial S}{partial A} = frac{partial [Q^{-1}A]}{partial A} = Q^{-1}otimesmathbf{I}$$
Note the result is a fourth-order linear transformation, since we are the describing the rate of change of a second-order object $S$ with respect to another second-order linear transformation $A$.
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
add a comment |
$begingroup$
If you are familiar with the vec operation for matrices, then you could proceed as follows.
$$eqalign{
&S,S
&= A^TA cr
&S,dS,(I)+(I),dS,S
&= A^T,dA,(I)+(I),dA^T,A cr
&(I^Totimes S+S^Totimes I),{rm vec}(dS)
&= (I^Totimes A^T),{rm vec}(dA)+(A^Totimes I),{rm vec}(dA^T) cr
&Big(Iotimes S+Sotimes IBig),{rm vec}(dS)
&= Big((Iotimes A^T)+(A^Totimes I)KBig),{rm vec}(dA) cr
&frac{partial{,rm vec}(S)}{partial{,rm vec}(A)}
&= Big(Iotimes S+Sotimes IBig)^+
Big((Iotimes A^T)+(A^Totimes I)KBig) crcr
}$$
where $M^+$ denotes the pseudoinverse of $M$, $I$ is the identity matrix, and $K$ is the commutation matrix associated with the Kronecker product. The solution also takes advantage of the fact that $I$ and $S$ are symmetric.
answered Jan 2 at 21:15
greggreg
8,6851823
$endgroup$
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3059888%2fmatrix-derivative-of-square-root%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Recall that you can perform a polar decomposition on $A$ such that:
$$A = QS$$
where $Q$ is an orthogonal matrix and $S$ is defined uniquely as precisely what you describe above when it exists. Hence, what you really want to calculate is:
$$frac{partial S}{partial A} = frac{partial [Q^{-1}A]}{partial A} = Q^{-1}otimesmathbf{I}$$
Note the result is a fourth-order linear transformation, since we are the describing the rate of change of a second-order object $S$ with respect to another second-order linear transformation $A$.
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
add a comment |
$begingroup$
Recall that you can perform a polar decomposition on $A$ such that:
$$A = QS$$
where $Q$ is an orthogonal matrix and $S$ is defined uniquely as precisely what you describe above when it exists. Hence, what you really want to calculate is:
$$frac{partial S}{partial A} = frac{partial [Q^{-1}A]}{partial A} = Q^{-1}otimesmathbf{I}$$
Note the result is a fourth-order linear transformation, since we are the describing the rate of change of a second-order object $S$ with respect to another second-order linear transformation $A$.
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
add a comment |
$begingroup$
Recall that you can perform a polar decomposition on $A$ such that:
$$A = QS$$
where $Q$ is an orthogonal matrix and $S$ is defined uniquely as precisely what you describe above when it exists. Hence, what you really want to calculate is:
$$frac{partial S}{partial A} = frac{partial [Q^{-1}A]}{partial A} = Q^{-1}otimesmathbf{I}$$
Note the result is a fourth-order linear transformation, since we are the describing the rate of change of a second-order object $S$ with respect to another second-order linear transformation $A$.
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
$endgroup$
Recall that you can perform a polar decomposition on $A$ such that:
$$A = QS$$
where $Q$ is an orthogonal matrix and $S$ is defined uniquely as precisely what you describe above when it exists. Hence, what you really want to calculate is:
$$frac{partial S}{partial A} = frac{partial [Q^{-1}A]}{partial A} = Q^{-1}otimesmathbf{I}$$
Note the result is a fourth-order linear transformation, since we are the describing the rate of change of a second-order object $S$ with respect to another second-order linear transformation $A$.
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
answered Jan 2 at 20:12
aghostinthefiguresaghostinthefigures
1,2641217
1,2641217
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
add a comment |
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
$begingroup$
No problem @user2019! If you find either my answer or greg’s to be correct/sufficient, remember to check it off/upvote it.
$endgroup$
– aghostinthefigures
Jan 4 at 15:12
add a comment |
$begingroup$
If you are familiar with the vec operation for matrices, then you could proceed as follows.
$$eqalign{
&S,S
&= A^TA cr
&S,dS,(I)+(I),dS,S
&= A^T,dA,(I)+(I),dA^T,A cr
&(I^Totimes S+S^Totimes I),{rm vec}(dS)
&= (I^Totimes A^T),{rm vec}(dA)+(A^Totimes I),{rm vec}(dA^T) cr
&Big(Iotimes S+Sotimes IBig),{rm vec}(dS)
&= Big((Iotimes A^T)+(A^Totimes I)KBig),{rm vec}(dA) cr
&frac{partial{,rm vec}(S)}{partial{,rm vec}(A)}
&= Big(Iotimes S+Sotimes IBig)^+
Big((Iotimes A^T)+(A^Totimes I)KBig) crcr
}$$
where $M^+$ denotes the pseudoinverse of $M$, $I$ is the identity matrix, and $K$ is the commutation matrix associated with the Kronecker product. The solution also takes advantage of the fact that $I$ and $S$ are symmetric.
answered Jan 2 at 21:15
greggreg
8,6851823
$endgroup$
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
add a comment |
$begingroup$
If you are familiar with the vec operation for matrices, then you could proceed as follows.
$$eqalign{
&S,S
&= A^TA cr
&S,dS,(I)+(I),dS,S
&= A^T,dA,(I)+(I),dA^T,A cr
&(I^Totimes S+S^Totimes I),{rm vec}(dS)
&= (I^Totimes A^T),{rm vec}(dA)+(A^Totimes I),{rm vec}(dA^T) cr
&Big(Iotimes S+Sotimes IBig),{rm vec}(dS)
&= Big((Iotimes A^T)+(A^Totimes I)KBig),{rm vec}(dA) cr
&frac{partial{,rm vec}(S)}{partial{,rm vec}(A)}
&= Big(Iotimes S+Sotimes IBig)^+
Big((Iotimes A^T)+(A^Totimes I)KBig) crcr
}$$
where $M^+$ denotes the pseudoinverse of $M$, $I$ is the identity matrix, and $K$ is the commutation matrix associated with the Kronecker product. The solution also takes advantage of the fact that $I$ and $S$ are symmetric.
answered Jan 2 at 21:15
greggreg
8,6851823
$endgroup$
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
add a comment |
$begingroup$
If you are familiar with the vec operation for matrices, then you could proceed as follows.
$$eqalign{
&S,S
&= A^TA cr
&S,dS,(I)+(I),dS,S
&= A^T,dA,(I)+(I),dA^T,A cr
&(I^Totimes S+S^Totimes I),{rm vec}(dS)
&= (I^Totimes A^T),{rm vec}(dA)+(A^Totimes I),{rm vec}(dA^T) cr
&Big(Iotimes S+Sotimes IBig),{rm vec}(dS)
&= Big((Iotimes A^T)+(A^Totimes I)KBig),{rm vec}(dA) cr
&frac{partial{,rm vec}(S)}{partial{,rm vec}(A)}
&= Big(Iotimes S+Sotimes IBig)^+
Big((Iotimes A^T)+(A^Totimes I)KBig) crcr
}$$
where $M^+$ denotes the pseudoinverse of $M$, $I$ is the identity matrix, and $K$ is the commutation matrix associated with the Kronecker product. The solution also takes advantage of the fact that $I$ and $S$ are symmetric.
answered Jan 2 at 21:15
greggreg
8,6851823
$endgroup$
If you are familiar with the vec operation for matrices, then you could proceed as follows.
$$eqalign{
&S,S
&= A^TA cr
&S,dS,(I)+(I),dS,S
&= A^T,dA,(I)+(I),dA^T,A cr
&(I^Totimes S+S^Totimes I),{rm vec}(dS)
&= (I^Totimes A^T),{rm vec}(dA)+(A^Totimes I),{rm vec}(dA^T) cr
&Big(Iotimes S+Sotimes IBig),{rm vec}(dS)
&= Big((Iotimes A^T)+(A^Totimes I)KBig),{rm vec}(dA) cr
&frac{partial{,rm vec}(S)}{partial{,rm vec}(A)}
&= Big(Iotimes S+Sotimes IBig)^+
Big((Iotimes A^T)+(A^Totimes I)KBig) crcr
}$$
where $M^+$ denotes the pseudoinverse of $M$, $I$ is the identity matrix, and $K$ is the commutation matrix associated with the Kronecker product. The solution also takes advantage of the fact that $I$ and $S$ are symmetric.
answered Jan 2 at 21:15
greggreg
8,6851823
edited Jan 2 at 21:21
answered Jan 2 at 21:15
greggreg
8,6851823
answered Jan 2 at 21:15
greggreg
8,6851823
answered Jan 2 at 21:15
greggreg
8,6851823
8,6851823
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
add a comment |
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
$begingroup$
thanks for your help!
$endgroup$
– user2019
Jan 4 at 15:09
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3059888%2fmatrix-derivative-of-square-root%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
What is the motivation you have in mind? What was your attempt at solving the problem?
$endgroup$
– A. Pongrácz
Jan 2 at 19:59
$begingroup$
see math.stackexchange.com/questions/1884536/… and references therein
$endgroup$
– rych
Jan 20 at 11:23