Derivative of the trace of a Kronecker product
up vote
0
down vote
favorite
I am trying to compute the derivative
$frac{partial}{partial W} text{Tr}(W^top A (Iotimes W)B),$
where $Winmathbb{R}^{Dtimes d}, Iinmathbb{R}^{Ttimes T}$ is an identity matrix, $Ainmathbb{R}^{Dtimes DT}$, and $Binmathbb{R}^{dTtimes d}$.
I have found a similar post:
Derivative involving the trace of a Kronecker product
but it seems that the method is not applicable to my problem.
Thank you!
derivatives trace kronecker-product
asked Feb 10 at 3:07
user3138073
1678
add a comment |
up vote
0
down vote
favorite
I am trying to compute the derivative
$frac{partial}{partial W} text{Tr}(W^top A (Iotimes W)B),$
where $Winmathbb{R}^{Dtimes d}, Iinmathbb{R}^{Ttimes T}$ is an identity matrix, $Ainmathbb{R}^{Dtimes DT}$, and $Binmathbb{R}^{dTtimes d}$.
I have found a similar post:
Derivative involving the trace of a Kronecker product
but it seems that the method is not applicable to my problem.
Thank you!
derivatives trace kronecker-product
asked Feb 10 at 3:07
user3138073
1678
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am trying to compute the derivative
$frac{partial}{partial W} text{Tr}(W^top A (Iotimes W)B),$
where $Winmathbb{R}^{Dtimes d}, Iinmathbb{R}^{Ttimes T}$ is an identity matrix, $Ainmathbb{R}^{Dtimes DT}$, and $Binmathbb{R}^{dTtimes d}$.
I have found a similar post:
Derivative involving the trace of a Kronecker product
but it seems that the method is not applicable to my problem.
Thank you!
derivatives trace kronecker-product
asked Feb 10 at 3:07
user3138073
1678
I am trying to compute the derivative
$frac{partial}{partial W} text{Tr}(W^top A (Iotimes W)B),$
where $Winmathbb{R}^{Dtimes d}, Iinmathbb{R}^{Ttimes T}$ is an identity matrix, $Ainmathbb{R}^{Dtimes DT}$, and $Binmathbb{R}^{dTtimes d}$.
I have found a similar post:
Derivative involving the trace of a Kronecker product
but it seems that the method is not applicable to my problem.
Thank you!
derivatives trace kronecker-product
derivatives trace kronecker-product
asked Feb 10 at 3:07
user3138073
1678
asked Feb 10 at 3:07
user3138073
1678
asked Feb 10 at 3:07
user3138073
1678
asked Feb 10 at 3:07
user3138073
1678
asked Feb 10 at 3:07
user3138073
1678
1678
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
The technique from the linked post can be applied to the current problem.
Write the function in terms of the trace/Frobenius product, and find its differential
$$eqalign{
phi &= W:A(Iotimes W)B = A^TWB^T:(Iotimes W) cr
dphi &= A(Iotimes W)B:dW + A^TWB^T:(Iotimes dW)
}$$
At this point, we need use the Pitsianis decomposition on that last term.
$$eqalign{
A^TWB^T &= sum_k Y_kotimes Z_k cr
}$$
The matrices $(Y_k,Z_k)$ are shaped like $(I,W)$ respectively.
Finish calculating the differential, then on to the gradient.
$$eqalign{
dphi
&= A(Iotimes W)B:dW + sum_kY_kotimes Z_k:(Iotimes dW) cr
&= Big(A(Iotimes W)B + sum_k(I:Y_k)Z_kBig):dW cr
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k {rm tr}(Y_k),Z_k crcr
}$$
Another technique uses the SVD of
$$B=sum_ksigma_ku_kv_k^T$$
to handle the second term of $dphi$ as follows.
$$eqalign{
A^TW:(Iotimes dW)B
&= sum_k,A^TW:(Iotimes dW)sigma_ku_kv_k^T cr
&= sum_k,(A^TWsigma_kv_k):(Iotimes dW)u_k cr
&= sum_k,q_k:{rm vec}(dW,U_k) cr
&= sum_k,Q_k:dW,U_k cr
&= sum_k,Q_kU_k^T:dW cr
}$$
where
$$eqalign{
{rm vec}(Q_k) &= q_k = A^TWsigma_kv_k cr
{rm vec}(U_k) &= u_k cr
}$$
Yielding the gradient as
$$eqalign{
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k Q_kU_k^T cr
}$$
answered Feb 11 at 16:26
greg
7,4201721
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2644106%2fderivative-of-the-trace-of-a-kronecker-product%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
The technique from the linked post can be applied to the current problem.
Write the function in terms of the trace/Frobenius product, and find its differential
$$eqalign{
phi &= W:A(Iotimes W)B = A^TWB^T:(Iotimes W) cr
dphi &= A(Iotimes W)B:dW + A^TWB^T:(Iotimes dW)
}$$
At this point, we need use the Pitsianis decomposition on that last term.
$$eqalign{
A^TWB^T &= sum_k Y_kotimes Z_k cr
}$$
The matrices $(Y_k,Z_k)$ are shaped like $(I,W)$ respectively.
Finish calculating the differential, then on to the gradient.
$$eqalign{
dphi
&= A(Iotimes W)B:dW + sum_kY_kotimes Z_k:(Iotimes dW) cr
&= Big(A(Iotimes W)B + sum_k(I:Y_k)Z_kBig):dW cr
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k {rm tr}(Y_k),Z_k crcr
}$$
Another technique uses the SVD of
$$B=sum_ksigma_ku_kv_k^T$$
to handle the second term of $dphi$ as follows.
$$eqalign{
A^TW:(Iotimes dW)B
&= sum_k,A^TW:(Iotimes dW)sigma_ku_kv_k^T cr
&= sum_k,(A^TWsigma_kv_k):(Iotimes dW)u_k cr
&= sum_k,q_k:{rm vec}(dW,U_k) cr
&= sum_k,Q_k:dW,U_k cr
&= sum_k,Q_kU_k^T:dW cr
}$$
where
$$eqalign{
{rm vec}(Q_k) &= q_k = A^TWsigma_kv_k cr
{rm vec}(U_k) &= u_k cr
}$$
Yielding the gradient as
$$eqalign{
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k Q_kU_k^T cr
}$$
answered Feb 11 at 16:26
greg
7,4201721
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
add a comment |
up vote
0
down vote
The technique from the linked post can be applied to the current problem.
Write the function in terms of the trace/Frobenius product, and find its differential
$$eqalign{
phi &= W:A(Iotimes W)B = A^TWB^T:(Iotimes W) cr
dphi &= A(Iotimes W)B:dW + A^TWB^T:(Iotimes dW)
}$$
At this point, we need use the Pitsianis decomposition on that last term.
$$eqalign{
A^TWB^T &= sum_k Y_kotimes Z_k cr
}$$
The matrices $(Y_k,Z_k)$ are shaped like $(I,W)$ respectively.
Finish calculating the differential, then on to the gradient.
$$eqalign{
dphi
&= A(Iotimes W)B:dW + sum_kY_kotimes Z_k:(Iotimes dW) cr
&= Big(A(Iotimes W)B + sum_k(I:Y_k)Z_kBig):dW cr
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k {rm tr}(Y_k),Z_k crcr
}$$
Another technique uses the SVD of
$$B=sum_ksigma_ku_kv_k^T$$
to handle the second term of $dphi$ as follows.
$$eqalign{
A^TW:(Iotimes dW)B
&= sum_k,A^TW:(Iotimes dW)sigma_ku_kv_k^T cr
&= sum_k,(A^TWsigma_kv_k):(Iotimes dW)u_k cr
&= sum_k,q_k:{rm vec}(dW,U_k) cr
&= sum_k,Q_k:dW,U_k cr
&= sum_k,Q_kU_k^T:dW cr
}$$
where
$$eqalign{
{rm vec}(Q_k) &= q_k = A^TWsigma_kv_k cr
{rm vec}(U_k) &= u_k cr
}$$
Yielding the gradient as
$$eqalign{
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k Q_kU_k^T cr
}$$
answered Feb 11 at 16:26
greg
7,4201721
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
add a comment |
up vote
0
down vote
up vote
0
down vote
The technique from the linked post can be applied to the current problem.
Write the function in terms of the trace/Frobenius product, and find its differential
$$eqalign{
phi &= W:A(Iotimes W)B = A^TWB^T:(Iotimes W) cr
dphi &= A(Iotimes W)B:dW + A^TWB^T:(Iotimes dW)
}$$
At this point, we need use the Pitsianis decomposition on that last term.
$$eqalign{
A^TWB^T &= sum_k Y_kotimes Z_k cr
}$$
The matrices $(Y_k,Z_k)$ are shaped like $(I,W)$ respectively.
Finish calculating the differential, then on to the gradient.
$$eqalign{
dphi
&= A(Iotimes W)B:dW + sum_kY_kotimes Z_k:(Iotimes dW) cr
&= Big(A(Iotimes W)B + sum_k(I:Y_k)Z_kBig):dW cr
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k {rm tr}(Y_k),Z_k crcr
}$$
Another technique uses the SVD of
$$B=sum_ksigma_ku_kv_k^T$$
to handle the second term of $dphi$ as follows.
$$eqalign{
A^TW:(Iotimes dW)B
&= sum_k,A^TW:(Iotimes dW)sigma_ku_kv_k^T cr
&= sum_k,(A^TWsigma_kv_k):(Iotimes dW)u_k cr
&= sum_k,q_k:{rm vec}(dW,U_k) cr
&= sum_k,Q_k:dW,U_k cr
&= sum_k,Q_kU_k^T:dW cr
}$$
where
$$eqalign{
{rm vec}(Q_k) &= q_k = A^TWsigma_kv_k cr
{rm vec}(U_k) &= u_k cr
}$$
Yielding the gradient as
$$eqalign{
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k Q_kU_k^T cr
}$$
answered Feb 11 at 16:26
greg
7,4201721
The technique from the linked post can be applied to the current problem.
Write the function in terms of the trace/Frobenius product, and find its differential
$$eqalign{
phi &= W:A(Iotimes W)B = A^TWB^T:(Iotimes W) cr
dphi &= A(Iotimes W)B:dW + A^TWB^T:(Iotimes dW)
}$$
At this point, we need use the Pitsianis decomposition on that last term.
$$eqalign{
A^TWB^T &= sum_k Y_kotimes Z_k cr
}$$
The matrices $(Y_k,Z_k)$ are shaped like $(I,W)$ respectively.
Finish calculating the differential, then on to the gradient.
$$eqalign{
dphi
&= A(Iotimes W)B:dW + sum_kY_kotimes Z_k:(Iotimes dW) cr
&= Big(A(Iotimes W)B + sum_k(I:Y_k)Z_kBig):dW cr
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k {rm tr}(Y_k),Z_k crcr
}$$
Another technique uses the SVD of
$$B=sum_ksigma_ku_kv_k^T$$
to handle the second term of $dphi$ as follows.
$$eqalign{
A^TW:(Iotimes dW)B
&= sum_k,A^TW:(Iotimes dW)sigma_ku_kv_k^T cr
&= sum_k,(A^TWsigma_kv_k):(Iotimes dW)u_k cr
&= sum_k,q_k:{rm vec}(dW,U_k) cr
&= sum_k,Q_k:dW,U_k cr
&= sum_k,Q_kU_k^T:dW cr
}$$
where
$$eqalign{
{rm vec}(Q_k) &= q_k = A^TWsigma_kv_k cr
{rm vec}(U_k) &= u_k cr
}$$
Yielding the gradient as
$$eqalign{
frac{partialphi}{partial W} &= A(Iotimes W)B + sum_k Q_kU_k^T cr
}$$
answered Feb 11 at 16:26
greg
7,4201721
edited Dec 4 at 0:20
answered Feb 11 at 16:26
greg
7,4201721
answered Feb 11 at 16:26
greg
7,4201721
answered Feb 11 at 16:26
greg
7,4201721
7,4201721
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
add a comment |
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
Thanks a lot for the reply! But one question is: $A^top W B^top$ is also a function of $W$, so we cannot do the decomposition, am I right?
– user3138073
Feb 12 at 21:30
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
The dependence of the function on the $A^TWB^T$ term was accounted for by the first term in the differential expression. Perform the Pitsianis decomposition using the current value of $W$.
– greg
Feb 13 at 1:15
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2644106%2fderivative-of-the-trace-of-a-kronecker-product%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
