Why the average of a set of value has the least square error?












3












$begingroup$


Now we have the equation
$$sum_{i}(x_i-hat x_i)^2,$$
where $x_i$ is the observed value of a data sample $S$. Here is the question:




Why does this expression get its minimum value when $hat x_i$ is the average of the data sample $S$ ?




I tried to take the derivatives of that equation and make it to zero, but it seems there's something wrong, because $hat x_i$ is kind of multi-variable.
Can anyone help me out? Thanks a lot!






statistics least-squares







share|cite|improve this question











$endgroup$








  • 4




    $begingroup$
    Your notation is confusing. You can set each $hat{x}_{i}$ equal to $x_{i}$ to make the sum zero. Do you want to minimizie $sum_{i=1}^{n} (x_{i}-hat{x})^{2}$, where $hat{x}$ is a single variable?
    $endgroup$
    – Brian Borchers
    Dec 24 '18 at 17:36






  • 4




    $begingroup$
    Your expression is confusing. If $hat{x_i}$ is an average, why the subscript?
    $endgroup$
    – herb steinberg
    Dec 24 '18 at 17:38








  • 1




    $begingroup$
    you are right ! thanks..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:17
















3












$begingroup$


Now we have the equation
$$sum_{i}(x_i-hat x_i)^2,$$
where $x_i$ is the observed value of a data sample $S$. Here is the question:




Why does this expression get its minimum value when $hat x_i$ is the average of the data sample $S$ ?




I tried to take the derivatives of that equation and make it to zero, but it seems there's something wrong, because $hat x_i$ is kind of multi-variable.
Can anyone help me out? Thanks a lot!






statistics least-squares







share|cite|improve this question











$endgroup$








  • 4




    $begingroup$
    Your notation is confusing. You can set each $hat{x}_{i}$ equal to $x_{i}$ to make the sum zero. Do you want to minimizie $sum_{i=1}^{n} (x_{i}-hat{x})^{2}$, where $hat{x}$ is a single variable?
    $endgroup$
    – Brian Borchers
    Dec 24 '18 at 17:36






  • 4




    $begingroup$
    Your expression is confusing. If $hat{x_i}$ is an average, why the subscript?
    $endgroup$
    – herb steinberg
    Dec 24 '18 at 17:38








  • 1




    $begingroup$
    you are right ! thanks..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:17














3












3








3


0



$begingroup$


Now we have the equation
$$sum_{i}(x_i-hat x_i)^2,$$
where $x_i$ is the observed value of a data sample $S$. Here is the question:




Why does this expression get its minimum value when $hat x_i$ is the average of the data sample $S$ ?




I tried to take the derivatives of that equation and make it to zero, but it seems there's something wrong, because $hat x_i$ is kind of multi-variable.
Can anyone help me out? Thanks a lot!






statistics least-squares







share|cite|improve this question











$endgroup$




Now we have the equation
$$sum_{i}(x_i-hat x_i)^2,$$
where $x_i$ is the observed value of a data sample $S$. Here is the question:




Why does this expression get its minimum value when $hat x_i$ is the average of the data sample $S$ ?




I tried to take the derivatives of that equation and make it to zero, but it seems there's something wrong, because $hat x_i$ is kind of multi-variable.
Can anyone help me out? Thanks a lot!






statistics least-squares




statistics least-squares






share|cite|improve this question















share|cite|improve this question













share|cite|improve this question




share|cite|improve this question








edited Dec 24 '18 at 18:43









amWhy

1




1










asked Dec 24 '18 at 17:32









Li ChenLi Chen

161




161








  • 4




    $begingroup$
    Your notation is confusing. You can set each $hat{x}_{i}$ equal to $x_{i}$ to make the sum zero. Do you want to minimizie $sum_{i=1}^{n} (x_{i}-hat{x})^{2}$, where $hat{x}$ is a single variable?
    $endgroup$
    – Brian Borchers
    Dec 24 '18 at 17:36






  • 4




    $begingroup$
    Your expression is confusing. If $hat{x_i}$ is an average, why the subscript?
    $endgroup$
    – herb steinberg
    Dec 24 '18 at 17:38








  • 1




    $begingroup$
    you are right ! thanks..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:17














  • 4




    $begingroup$
    Your notation is confusing. You can set each $hat{x}_{i}$ equal to $x_{i}$ to make the sum zero. Do you want to minimizie $sum_{i=1}^{n} (x_{i}-hat{x})^{2}$, where $hat{x}$ is a single variable?
    $endgroup$
    – Brian Borchers
    Dec 24 '18 at 17:36






  • 4




    $begingroup$
    Your expression is confusing. If $hat{x_i}$ is an average, why the subscript?
    $endgroup$
    – herb steinberg
    Dec 24 '18 at 17:38








  • 1




    $begingroup$
    you are right ! thanks..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:17








4




4




$begingroup$
Your notation is confusing. You can set each $hat{x}_{i}$ equal to $x_{i}$ to make the sum zero. Do you want to minimizie $sum_{i=1}^{n} (x_{i}-hat{x})^{2}$, where $hat{x}$ is a single variable?
$endgroup$
– Brian Borchers
Dec 24 '18 at 17:36




$begingroup$
Your notation is confusing. You can set each $hat{x}_{i}$ equal to $x_{i}$ to make the sum zero. Do you want to minimizie $sum_{i=1}^{n} (x_{i}-hat{x})^{2}$, where $hat{x}$ is a single variable?
$endgroup$
– Brian Borchers
Dec 24 '18 at 17:36




4




4




$begingroup$
Your expression is confusing. If $hat{x_i}$ is an average, why the subscript?
$endgroup$
– herb steinberg
Dec 24 '18 at 17:38






$begingroup$
Your expression is confusing. If $hat{x_i}$ is an average, why the subscript?
$endgroup$
– herb steinberg
Dec 24 '18 at 17:38






1




1




$begingroup$
you are right ! thanks..
$endgroup$
– Li Chen
Dec 24 '18 at 18:17




$begingroup$
you are right ! thanks..
$endgroup$
– Li Chen
Dec 24 '18 at 18:17










2 Answers
2






active

oldest

votes


















3












$begingroup$

Let's take the function:



$$f(hat x)=sum_{i=1}^n (x_i-hat x)^2$$



Here, we want to find the value of $hat x$ which minimizes $f(hat x)$. Now, even though there are multiple variables of this function because of $x_i$, we can just treat these variables as constants since they are independent from the $hat x$, which essentially changes this to a single-variable calculus problem. Now, let's take the derivative of $f$ with respect to $hat x$.



$$f'(hat x)=sum_{i=1}^n 2(hat x-x_i)$$



From here, can you find the value of $hat x$ satisfying $f(hat x)=0$? Once you solve that equation, use second-derivative test to show that it is indeed an absolute minimum.






share|cite|improve this answer









$endgroup$









  • 1




    $begingroup$
    thanks , turns out my notation confuses me..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:21



















1












$begingroup$

This can be solved
without calculus.



Let
$f(z)
=sum_{i}(x_i-z)^2
$.



Then,
since
$sum_{i}x_i
=nhat x$,



$begin{array}\
f(z)-f(hat x)
&=sum_{i}(x_i-z)^2-sum_{i}(x_i-hat x)^2\
&=sum_{i}((x_i-z)^2-(x_i-hat x)^2)\
&=sum_{i}(x_i^2-2x_iz+z^2-(x_i^2-2x_ihat x+hat x^2))\
&=sum_{i}(2x_i(hat x-z)+z^2-hat x^2)\
&=2nhat x(hat x-z)+n(z^2-hat x^2)\
&=2nhat x(hat x-z)+n(z-hat x)(z+hat x)\
&=n(hat x-z)(2hat x-(z+hat x))\
&=n(hat x-z)(hat x-z)\
&=n(hat x-z)^2\
&ge 0
quad text{with equality iff } z=hat x\
end{array}
$






share|cite|improve this answer









$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3051480%2fwhy-the-average-of-a-set-of-value-has-the-least-square-error%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    3












    $begingroup$

    Let's take the function:



    $$f(hat x)=sum_{i=1}^n (x_i-hat x)^2$$



    Here, we want to find the value of $hat x$ which minimizes $f(hat x)$. Now, even though there are multiple variables of this function because of $x_i$, we can just treat these variables as constants since they are independent from the $hat x$, which essentially changes this to a single-variable calculus problem. Now, let's take the derivative of $f$ with respect to $hat x$.



    $$f'(hat x)=sum_{i=1}^n 2(hat x-x_i)$$



    From here, can you find the value of $hat x$ satisfying $f(hat x)=0$? Once you solve that equation, use second-derivative test to show that it is indeed an absolute minimum.






    share|cite|improve this answer









    $endgroup$









    • 1




      $begingroup$
      thanks , turns out my notation confuses me..
      $endgroup$
      – Li Chen
      Dec 24 '18 at 18:21
















    3












    $begingroup$

    Let's take the function:



    $$f(hat x)=sum_{i=1}^n (x_i-hat x)^2$$



    Here, we want to find the value of $hat x$ which minimizes $f(hat x)$. Now, even though there are multiple variables of this function because of $x_i$, we can just treat these variables as constants since they are independent from the $hat x$, which essentially changes this to a single-variable calculus problem. Now, let's take the derivative of $f$ with respect to $hat x$.



    $$f'(hat x)=sum_{i=1}^n 2(hat x-x_i)$$



    From here, can you find the value of $hat x$ satisfying $f(hat x)=0$? Once you solve that equation, use second-derivative test to show that it is indeed an absolute minimum.






    share|cite|improve this answer









    $endgroup$









    • 1




      $begingroup$
      thanks , turns out my notation confuses me..
      $endgroup$
      – Li Chen
      Dec 24 '18 at 18:21














    3












    3








    3





    $begingroup$

    Let's take the function:



    $$f(hat x)=sum_{i=1}^n (x_i-hat x)^2$$



    Here, we want to find the value of $hat x$ which minimizes $f(hat x)$. Now, even though there are multiple variables of this function because of $x_i$, we can just treat these variables as constants since they are independent from the $hat x$, which essentially changes this to a single-variable calculus problem. Now, let's take the derivative of $f$ with respect to $hat x$.



    $$f'(hat x)=sum_{i=1}^n 2(hat x-x_i)$$



    From here, can you find the value of $hat x$ satisfying $f(hat x)=0$? Once you solve that equation, use second-derivative test to show that it is indeed an absolute minimum.






    share|cite|improve this answer









    $endgroup$



    Let's take the function:



    $$f(hat x)=sum_{i=1}^n (x_i-hat x)^2$$



    Here, we want to find the value of $hat x$ which minimizes $f(hat x)$. Now, even though there are multiple variables of this function because of $x_i$, we can just treat these variables as constants since they are independent from the $hat x$, which essentially changes this to a single-variable calculus problem. Now, let's take the derivative of $f$ with respect to $hat x$.



    $$f'(hat x)=sum_{i=1}^n 2(hat x-x_i)$$



    From here, can you find the value of $hat x$ satisfying $f(hat x)=0$? Once you solve that equation, use second-derivative test to show that it is indeed an absolute minimum.







    share|cite|improve this answer












    share|cite|improve this answer



    share|cite|improve this answer










    answered Dec 24 '18 at 18:09









    Noble MushtakNoble Mushtak

    15.3k1835




    15.3k1835








    • 1




      $begingroup$
      thanks , turns out my notation confuses me..
      $endgroup$
      – Li Chen
      Dec 24 '18 at 18:21














    • 1




      $begingroup$
      thanks , turns out my notation confuses me..
      $endgroup$
      – Li Chen
      Dec 24 '18 at 18:21








    1




    1




    $begingroup$
    thanks , turns out my notation confuses me..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:21




    $begingroup$
    thanks , turns out my notation confuses me..
    $endgroup$
    – Li Chen
    Dec 24 '18 at 18:21











    1












    $begingroup$

    This can be solved
    without calculus.



    Let
    $f(z)
    =sum_{i}(x_i-z)^2
    $    .



    Then,
    since
    $sum_{i}x_i
    =nhat x$    ,



    $begin{array}\
    f(z)-f(hat x)
    &=sum_{i}(x_i-z)^2-sum_{i}(x_i-hat x)^2\
    &=sum_{i}((x_i-z)^2-(x_i-hat x)^2)\
    &=sum_{i}(x_i^2-2x_iz+z^2-(x_i^2-2x_ihat x+hat x^2))\
    &=sum_{i}(2x_i(hat x-z)+z^2-hat x^2)\
    &=2nhat x(hat x-z)+n(z^2-hat x^2)\
    &=2nhat x(hat x-z)+n(z-hat x)(z+hat x)\
    &=n(hat x-z)(2hat x-(z+hat x))\
    &=n(hat x-z)(hat x-z)\
    &=n(hat x-z)^2\
    &ge 0
    quad text{with equality iff } z=hat x\
    end{array}
    $






    share|cite|improve this answer









    $endgroup$


















      1












      $begingroup$

      This can be solved
      without calculus.



      Let
      $f(z)
      =sum_{i}(x_i-z)^2
      $      .



      Then,
      since
      $sum_{i}x_i
      =nhat x$      ,



      $begin{array}\
      f(z)-f(hat x)
      &=sum_{i}(x_i-z)^2-sum_{i}(x_i-hat x)^2\
      &=sum_{i}((x_i-z)^2-(x_i-hat x)^2)\
      &=sum_{i}(x_i^2-2x_iz+z^2-(x_i^2-2x_ihat x+hat x^2))\
      &=sum_{i}(2x_i(hat x-z)+z^2-hat x^2)\
      &=2nhat x(hat x-z)+n(z^2-hat x^2)\
      &=2nhat x(hat x-z)+n(z-hat x)(z+hat x)\
      &=n(hat x-z)(2hat x-(z+hat x))\
      &=n(hat x-z)(hat x-z)\
      &=n(hat x-z)^2\
      &ge 0
      quad text{with equality iff } z=hat x\
      end{array}
      $






      share|cite|improve this answer









      $endgroup$
















        1












        1








        1





        $begingroup$

        This can be solved
        without calculus.



        Let
        $f(z)
        =sum_{i}(x_i-z)^2
        $        .



        Then,
        since
        $sum_{i}x_i
        =nhat x$        ,



        $begin{array}\
        f(z)-f(hat x)
        &=sum_{i}(x_i-z)^2-sum_{i}(x_i-hat x)^2\
        &=sum_{i}((x_i-z)^2-(x_i-hat x)^2)\
        &=sum_{i}(x_i^2-2x_iz+z^2-(x_i^2-2x_ihat x+hat x^2))\
        &=sum_{i}(2x_i(hat x-z)+z^2-hat x^2)\
        &=2nhat x(hat x-z)+n(z^2-hat x^2)\
        &=2nhat x(hat x-z)+n(z-hat x)(z+hat x)\
        &=n(hat x-z)(2hat x-(z+hat x))\
        &=n(hat x-z)(hat x-z)\
        &=n(hat x-z)^2\
        &ge 0
        quad text{with equality iff } z=hat x\
        end{array}
        $






        share|cite|improve this answer









        $endgroup$



        This can be solved
        without calculus.



        Let
        $f(z)
        =sum_{i}(x_i-z)^2
        $        .



        Then,
        since
        $sum_{i}x_i
        =nhat x$        ,



        $begin{array}\
        f(z)-f(hat x)
        &=sum_{i}(x_i-z)^2-sum_{i}(x_i-hat x)^2\
        &=sum_{i}((x_i-z)^2-(x_i-hat x)^2)\
        &=sum_{i}(x_i^2-2x_iz+z^2-(x_i^2-2x_ihat x+hat x^2))\
        &=sum_{i}(2x_i(hat x-z)+z^2-hat x^2)\
        &=2nhat x(hat x-z)+n(z^2-hat x^2)\
        &=2nhat x(hat x-z)+n(z-hat x)(z+hat x)\
        &=n(hat x-z)(2hat x-(z+hat x))\
        &=n(hat x-z)(hat x-z)\
        &=n(hat x-z)^2\
        &ge 0
        quad text{with equality iff } z=hat x\
        end{array}
        $







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered Dec 24 '18 at 19:45









        marty cohenmarty cohen

        73.6k549128




        73.6k549128






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3051480%2fwhy-the-average-of-a-set-of-value-has-the-least-square-error%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Måne

            Image
            För jordens måne, se Månen. Några månar i solsystemet i skala till vår måne En måne , naturlig satellit eller drabant är en himlakropp som kretsar kring en planet eller en asteroid i ett solsystem. Ordet i bestämd form – månen – avser planeten jordens måne. Det finns inte i något fall belagt att någon måne har egna månar. De flesta stora månar i solsystemet är bundna i synkron rotation med en och samma sida vänd mot planeten. Ett exempel på en måne som inte beter sig på detta vis, är Hyperion som kaotiskt roterar kring Saturnus. En asteroidmåne är en måne som kretsar kring en asteroid. En måne kan ha bildats samtidigt som moderplaneten eller senare infångats av planetens gravitationskraft. I vårt eget solsystem känner vi i nuläget till 173 månar runt sex av de åtta planeterna [ 1 ] : Jorden – månen Mars – Phobos och Deimos Jupiter – Ganymedes (som är solsystemets största måne), Io, Europa, Callisto och 59 andra kända månar Saturnus – Titan (den enda måne...
            Read more

            Storängen

            Image
            För stadsdelen i Mariehamn, se Storängen, Åland. För ett naturreservat i Norrtälje kommun, se Storängens naturreservat. Storängen från luften i september 1967, vy mot väst. Till vänster syns Järlasjön och Saltsjöbanans spår. Till höger löper Värmdövägen och längst ut till höger syns anläggningsarbetena för Värmdöleden. Storängen är ett villaområde i centrala Nacka i Stockholms län. Området är beläget strax sydost om Nacka Forum vid Saltsjöbanan mellan Lillängen och Saltsjö-Duvnäs. Storängens villasamhälle grundades 1904 av "Tjänstemännens Egnahemsförening vid Storängen" och de första villorna började byggas samma år. Redan 1909 var 107 tomter bebyggda av totalt runt 160. [ 1 ] År 2013 fanns drygt 150 villor med lika många familjer i området. Egnahemsföreningen existerar fortfarande och fungerar idag som intresseförening för de boende i Storängen. Storängen anses vara ett av Sveriges bäst bevarade villasamhällen från 1900-talets början, tack vare sina tidstypis...
            Read more

            VLT Carioca

            Image
            VLT Carioca Informações Proprietário Prefeitura do Rio de Janeiro Local [[Rio de Janeiro, RJ]] País {{country data Brasil| country flag2| name = Brasil| variant = | size = }} Tipo de transporte Bonde (Veículo leve sobre trilhos) Número de linhas 2 (+1 em construção) Número de estações 26 (+3 em construção) Tráfego 80 mil passageiros/dia [ 1 ] (dezembro/2018) Website www.vltrio.com.br Funcionamento Início de funcionamento 5 de junho de 2016 (2 anos) Operadora(s) Concessionária do VLT Carioca S.A. Número de veículos 32 Alstom Citadis Comprimento dos veículos 44 m (144 ft) Dados técnicos Extensão do sistema 28 km (17,4 mi) [ 2 ] Frequência 5–10 minutos Bitola Bitola padrão/internacional 1 435 mm (4,71 ft) Eletrificação APS, 750V DC Velocidade média 15 km/h (9,32 mph) [ 2 ] Velocidade máxima 50 km/h (31,1 mph) ...
            Read more