# 程序代写代做代考 ECON 61001: Review of OLS in simple linear regression

ECON 61001: Review of OLS in simple linear regression

Alastair R. Hall

The University of Manchester

Alastair R. Hall

ECON61001: OLS Review 1 / 12

Outline of this revision session

Origins of OLS

Simple linear regression model

Intuition behind OLS

Formal definition of OLS

Other approaches and advantages of OLS

Resources: any introductory econometrics text such as Wooldridge (2019) Introductory Econometrics.

Alastair R. Hall

ECON61001: OLS Review 2 / 12

Origins of OLS

The origin of Ordinary Least Squares (OLS) is shrouded in controversy.

The method is published by

Adrien Legendre, a French mathematician, in 1805. Robert Adrain, an American, in 1808.

Carl Friedrich Gauss, a German, in 1809

but Gauss claimed he had been using the method since 1795.

Stigler (1981): “The method of least squares is the automobile of modern statistical analysis” but who was the “Henry Ford of statistics”?

Alastair R. Hall

ECON61001: OLS Review 3 / 12

Origins of OLS

Gauss was a very eminent mathematician whose key contributions include:

the Normal (or “Gaussian”) distribution. development of Least Squares theory Gaussian elimination

Alastair R. Hall

ECON61001: OLS Review 4 / 12

Origins of OLS

Stigler (1981) evaluates Gauss’s claim using historical records and concludes

Just as the automobile was not the product of one man of genius, so too the method of least squares is due to […] at least two independent discoverers. Gauss may well have been the first of these, but he was no Henry Ford of statistics. If there was any single scientist who first put the method within the reach of the common man, it was Legendre.(Stigler, 1981,p.472)

Alastair R. Hall

ECON61001: OLS Review 5 / 12

Simple linear regression

Suppose we are interesting in modeling the relationship between y, the annual salary of the CEO of a firm i, and xi the average return on equity for the CEO’s firm for the previous 3 years.

Assume simple linear regression model

yi = β0,1 +β0,2xi +ui = xi′β0 +ui

where

ui is the unobserved error term;

xi′ =(1,xi);

β0,1

β0 = β are unknown (regression) parameters. 0,2

Alastair R. Hall

ECON61001: OLS Review 6 / 12

Simple linear regression

Under assumptions discussed in the lecture, we have: E[yi |xi] = β0,1 + β0,2xi

So E[yi |xi] is linear function of xi but weights (the regression parameters) are unknown.

So collect sample on {yi , xi } and use this to estimate β0.

Alastair R. Hall

ECON61001: OLS Review 7 / 12

Simple linear regression

Suppose our data consists of the following five observations on (yi,xi):

{(1095, 14.1), (1001, 10.9), (1122, 23.5), (578, 5.9), (1145, 20.0)}.

Alastair R. Hall

ECON61001: OLS Review 8 / 12

Simple linear regression

No single line passes through all the points.

Choose line that comes “closest” to fitting the scatter plot → issue of how to measure distance of actual y from value predicted by line, β1 + β2x.

Alastair R. Hall

ECON61001: OLS Review 9 / 12

Simple linear regression and OLS

Define ui (β) = yi − β1 − β2xi

Measure of distance from line must be non-negative: in OLS we measure this distance by {ui (β)}2

OLS estimator of (β1, β2) is value that minimizes 5i=1{ui (β)}2.

Alastair R. Hall

ECON61001: OLS Review 10 / 12

OLS in the simple linear regression model

In this example the OLS line is:

yˆi = 573.63 + 27.86xi

Alastair R. Hall

ECON61001: OLS Review 11 / 12

OLS in the simple linear regression model

Notice that {ui(β)}2 is just one possible measure of distance.

For example, could also use ui (β):

If choose (β1, β2) to minimize 5 ui (β) → Least Absolute i=1

Deviation (LAD) regression

LAD actually proposed in 1757 by Roger Boscovich but OLS became the “automobile of modern statistical analysis” because:

calculus of OLS is far easier

OLS can be shown to have some desirable statistical properties (see lectures).

Alastair R. Hall

ECON61001: OLS Review 12 / 12