The Derivative of a Vector
It is stated in this video that some books attempt to define the derivative of a vector, with the added caveat that tensor notation is a much better approach. Presumably, this is a different concept than the derivative of a vector-valued function, as the latter is uncontroversial and ubiquitous. What is this unpalatable process used by such books to take the derivative of a vector, and a vector transposed?
calculus linear-algebra derivatives vectors transpose
add a comment |
It is stated in this video that some books attempt to define the derivative of a vector, with the added caveat that tensor notation is a much better approach. Presumably, this is a different concept than the derivative of a vector-valued function, as the latter is uncontroversial and ubiquitous. What is this unpalatable process used by such books to take the derivative of a vector, and a vector transposed?
calculus linear-algebra derivatives vectors transpose
I didn't click on the youtube link, but as a general comment derivatives are always based on changes in functions, which are then taken to the limit as some parameter approaches zero. Notation doesn't change this, it's more a matter of convenience.
– Cuspy Code
Dec 6 at 17:44
The best framework for this is geometric algebra or Clifford Algebra.
– John L Winters
Dec 6 at 19:02
add a comment |
It is stated in this video that some books attempt to define the derivative of a vector, with the added caveat that tensor notation is a much better approach. Presumably, this is a different concept than the derivative of a vector-valued function, as the latter is uncontroversial and ubiquitous. What is this unpalatable process used by such books to take the derivative of a vector, and a vector transposed?
calculus linear-algebra derivatives vectors transpose
It is stated in this video that some books attempt to define the derivative of a vector, with the added caveat that tensor notation is a much better approach. Presumably, this is a different concept than the derivative of a vector-valued function, as the latter is uncontroversial and ubiquitous. What is this unpalatable process used by such books to take the derivative of a vector, and a vector transposed?
calculus linear-algebra derivatives vectors transpose
calculus linear-algebra derivatives vectors transpose
asked Dec 6 at 17:29
user10478
388111
388111
I didn't click on the youtube link, but as a general comment derivatives are always based on changes in functions, which are then taken to the limit as some parameter approaches zero. Notation doesn't change this, it's more a matter of convenience.
– Cuspy Code
Dec 6 at 17:44
The best framework for this is geometric algebra or Clifford Algebra.
– John L Winters
Dec 6 at 19:02
add a comment |
I didn't click on the youtube link, but as a general comment derivatives are always based on changes in functions, which are then taken to the limit as some parameter approaches zero. Notation doesn't change this, it's more a matter of convenience.
– Cuspy Code
Dec 6 at 17:44
The best framework for this is geometric algebra or Clifford Algebra.
– John L Winters
Dec 6 at 19:02
I didn't click on the youtube link, but as a general comment derivatives are always based on changes in functions, which are then taken to the limit as some parameter approaches zero. Notation doesn't change this, it's more a matter of convenience.
– Cuspy Code
Dec 6 at 17:44
I didn't click on the youtube link, but as a general comment derivatives are always based on changes in functions, which are then taken to the limit as some parameter approaches zero. Notation doesn't change this, it's more a matter of convenience.
– Cuspy Code
Dec 6 at 17:44
The best framework for this is geometric algebra or Clifford Algebra.
– John L Winters
Dec 6 at 19:02
The best framework for this is geometric algebra or Clifford Algebra.
– John L Winters
Dec 6 at 19:02
add a comment |
1 Answer
1
active
oldest
votes
The linked video appears to be a discussion about minimizing
$$eqalign{ f(x) &= tfrac{1}{2}x^TAx - x^Tb}$$
Here's how one might solve it using matrix notation.
First find the differential and gradient of the function.
$$eqalign{
f &= tfrac{1}{2}x^TAx - b^Tx cr
df &= tfrac{1}{2}(dx^TAx+x^TA,dx) - b^Tdx cr
&= (x^TA -b^T),dx cr
&= (Ax-b)^T,dx cr
frac{partial f}{partial x} &= Ax - b cr
}$$
Then set the gradient to zero and solve
$$(Ax-b)=0 implies x=A^{-1}b$$
So it's not really as bad as the lecturer is making it out to be.
The same calculation in index/tensor notation looks like
$$eqalign{
f &= tfrac{1}{2}x_iA_{ij}x_j - b_ix_i cr
frac{partial f}{partial x_k}
&= tfrac{1}{2}(delta_{ik}A_{ij}x_j+x_iA_{ij}delta_{jk}) - b_idelta_{ik} cr
&= tfrac{1}{2}(A_{kj}x_j+x_iA_{ik}) - b_k cr
&= tfrac{1}{2}(A_{kj}x_j+A_{ki}^Tx_i) - b_k cr
&= A_{kj}x_j - b_k cr
}$$
For dealing with scalars, vectors, and matrices (as in this problem) the two notations are roughly equivalent. The real power of index/tensor notation is in dealing with higher-order tensors.
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3028794%2fthe-derivative-of-a-vector%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The linked video appears to be a discussion about minimizing
$$eqalign{ f(x) &= tfrac{1}{2}x^TAx - x^Tb}$$
Here's how one might solve it using matrix notation.
First find the differential and gradient of the function.
$$eqalign{
f &= tfrac{1}{2}x^TAx - b^Tx cr
df &= tfrac{1}{2}(dx^TAx+x^TA,dx) - b^Tdx cr
&= (x^TA -b^T),dx cr
&= (Ax-b)^T,dx cr
frac{partial f}{partial x} &= Ax - b cr
}$$
Then set the gradient to zero and solve
$$(Ax-b)=0 implies x=A^{-1}b$$
So it's not really as bad as the lecturer is making it out to be.
The same calculation in index/tensor notation looks like
$$eqalign{
f &= tfrac{1}{2}x_iA_{ij}x_j - b_ix_i cr
frac{partial f}{partial x_k}
&= tfrac{1}{2}(delta_{ik}A_{ij}x_j+x_iA_{ij}delta_{jk}) - b_idelta_{ik} cr
&= tfrac{1}{2}(A_{kj}x_j+x_iA_{ik}) - b_k cr
&= tfrac{1}{2}(A_{kj}x_j+A_{ki}^Tx_i) - b_k cr
&= A_{kj}x_j - b_k cr
}$$
For dealing with scalars, vectors, and matrices (as in this problem) the two notations are roughly equivalent. The real power of index/tensor notation is in dealing with higher-order tensors.
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
add a comment |
The linked video appears to be a discussion about minimizing
$$eqalign{ f(x) &= tfrac{1}{2}x^TAx - x^Tb}$$
Here's how one might solve it using matrix notation.
First find the differential and gradient of the function.
$$eqalign{
f &= tfrac{1}{2}x^TAx - b^Tx cr
df &= tfrac{1}{2}(dx^TAx+x^TA,dx) - b^Tdx cr
&= (x^TA -b^T),dx cr
&= (Ax-b)^T,dx cr
frac{partial f}{partial x} &= Ax - b cr
}$$
Then set the gradient to zero and solve
$$(Ax-b)=0 implies x=A^{-1}b$$
So it's not really as bad as the lecturer is making it out to be.
The same calculation in index/tensor notation looks like
$$eqalign{
f &= tfrac{1}{2}x_iA_{ij}x_j - b_ix_i cr
frac{partial f}{partial x_k}
&= tfrac{1}{2}(delta_{ik}A_{ij}x_j+x_iA_{ij}delta_{jk}) - b_idelta_{ik} cr
&= tfrac{1}{2}(A_{kj}x_j+x_iA_{ik}) - b_k cr
&= tfrac{1}{2}(A_{kj}x_j+A_{ki}^Tx_i) - b_k cr
&= A_{kj}x_j - b_k cr
}$$
For dealing with scalars, vectors, and matrices (as in this problem) the two notations are roughly equivalent. The real power of index/tensor notation is in dealing with higher-order tensors.
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
add a comment |
The linked video appears to be a discussion about minimizing
$$eqalign{ f(x) &= tfrac{1}{2}x^TAx - x^Tb}$$
Here's how one might solve it using matrix notation.
First find the differential and gradient of the function.
$$eqalign{
f &= tfrac{1}{2}x^TAx - b^Tx cr
df &= tfrac{1}{2}(dx^TAx+x^TA,dx) - b^Tdx cr
&= (x^TA -b^T),dx cr
&= (Ax-b)^T,dx cr
frac{partial f}{partial x} &= Ax - b cr
}$$
Then set the gradient to zero and solve
$$(Ax-b)=0 implies x=A^{-1}b$$
So it's not really as bad as the lecturer is making it out to be.
The same calculation in index/tensor notation looks like
$$eqalign{
f &= tfrac{1}{2}x_iA_{ij}x_j - b_ix_i cr
frac{partial f}{partial x_k}
&= tfrac{1}{2}(delta_{ik}A_{ij}x_j+x_iA_{ij}delta_{jk}) - b_idelta_{ik} cr
&= tfrac{1}{2}(A_{kj}x_j+x_iA_{ik}) - b_k cr
&= tfrac{1}{2}(A_{kj}x_j+A_{ki}^Tx_i) - b_k cr
&= A_{kj}x_j - b_k cr
}$$
For dealing with scalars, vectors, and matrices (as in this problem) the two notations are roughly equivalent. The real power of index/tensor notation is in dealing with higher-order tensors.
The linked video appears to be a discussion about minimizing
$$eqalign{ f(x) &= tfrac{1}{2}x^TAx - x^Tb}$$
Here's how one might solve it using matrix notation.
First find the differential and gradient of the function.
$$eqalign{
f &= tfrac{1}{2}x^TAx - b^Tx cr
df &= tfrac{1}{2}(dx^TAx+x^TA,dx) - b^Tdx cr
&= (x^TA -b^T),dx cr
&= (Ax-b)^T,dx cr
frac{partial f}{partial x} &= Ax - b cr
}$$
Then set the gradient to zero and solve
$$(Ax-b)=0 implies x=A^{-1}b$$
So it's not really as bad as the lecturer is making it out to be.
The same calculation in index/tensor notation looks like
$$eqalign{
f &= tfrac{1}{2}x_iA_{ij}x_j - b_ix_i cr
frac{partial f}{partial x_k}
&= tfrac{1}{2}(delta_{ik}A_{ij}x_j+x_iA_{ij}delta_{jk}) - b_idelta_{ik} cr
&= tfrac{1}{2}(A_{kj}x_j+x_iA_{ik}) - b_k cr
&= tfrac{1}{2}(A_{kj}x_j+A_{ki}^Tx_i) - b_k cr
&= A_{kj}x_j - b_k cr
}$$
For dealing with scalars, vectors, and matrices (as in this problem) the two notations are roughly equivalent. The real power of index/tensor notation is in dealing with higher-order tensors.
answered Dec 7 at 6:46
greg
7,5001721
7,5001721
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
add a comment |
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Why does $dx^TAx = x^TAdx$?
– user10478
Dec 8 at 17:42
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
Because of two things: 1) $A=A^T$ and 2) for real vectors $x^Ty=y^Tx$
– greg
Dec 8 at 18:48
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3028794%2fthe-derivative-of-a-vector%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I didn't click on the youtube link, but as a general comment derivatives are always based on changes in functions, which are then taken to the limit as some parameter approaches zero. Notation doesn't change this, it's more a matter of convenience.
– Cuspy Code
Dec 6 at 17:44
The best framework for this is geometric algebra or Clifford Algebra.
– John L Winters
Dec 6 at 19:02