# CS代考 MAST 90138: MULTIVARIATE STATISTICAL TECHNIQUES – cscodehelp代写

MAST 90138: MULTIVARIATE STATISTICAL TECHNIQUES
See Ha ̈rdle and Simar, chapter 16.
7 7.1
CANONICAL CORRELATION ANALYSIS (CCA) MOST INTERESTING LINEAR COMBINATIONS
 A tool developed by Hotelling for discovering and quantifying as- sociation between two sets of variables
 Setup: two random vectors X ∈ Rq and Y ∈ Rp, X∼(μ,ΣXX), Y∼(ν,ΣYY).
 Also,
a q × p matrix.
Cov(X,Y)=E(X−μ)(Y −ν)T =ΣXY =ΣTYX,
1

 Goal: Find two coefficient vectors a ∈ Rq and b ∈ Rp such that the correlation between
is maximized .
 For any a and b
aT X and bT Y
ρ(a, b) := Corr (aT X, bT Y ) = aT ΣXY b
(aT ΣXXa)1/2(bT ΣY Y b)1/2
 Also, for any constant c, d ∈ R+, we have ρ(a, b) = ρ(ca, db)
2

 Refined goal: Find a and b such that aT ΣXY b is maximized subject to the constraints
aTΣXXa = 1 and bTΣXXb = 1  How to solve this?
3

 For this problem, define
K = Σ−1/2ΣXY Σ−1/2
XX YY
 Also, look at its singular value decomposition (SVD): K = ΓΛ∆T
 Recall: Γ = (γ1|…|γk), ∆ = (δ1|…|δk), Λ = diag(λ1/2,…,λ1/2) 1k
 Also, the number k is
– rank(K)
– rank(ΣXY ) – rank(ΣY X)
– the number of non-zero eigenvalues of KKT or KT K. (Precisely, λ1, . . . , λk are the non-zero eigenvalues of these two matrices)
4

 Now, define
for i = 1, . . . , k. They are called canonical correlation vectors.
 Using them, define the canonical correlation variables ηi =aTi Xandψi =bTi Y
ai = Σ−1/2γi and bi = Σ−1/2δi XX YY
for i = 1,…,k.
 The quantities
fori=1,…,karecalled canonicalcorrelationcoefficients
ρi = λ1/2 i
5

 Solution to our problem:
η1 =a1X, ψ1 =b1Y
 Proof: demonstrated in class (can be in exam!)  Precisely, Cov(η1, ψ1) = ρ1 = √λ1
6

 In fact, similar to PCA and PLS, one can continue to ask: Givenη1 =a1Xandψ1 =b1Y,finda∈Rq andb∈Rp suchthat
Cov(aTX,bTY)
is maximized subject to
– Corr(aT X, η1) = Corr(bT Y, η1) = 0, – Corr(aT X, ψ1) = Corr(bT Y, ψ1) = 0 – aTΣXXa = 1 and bTΣXXb = 1
 Turns out, the solution is: η2 = a2X and ψ2 = b2Y .  Proof: shown in class. (maybe on exam again!)  Precisely, Corr(η2, ψ2) = ρ2 = √λ2
7

 Continuing, we can further ask:
Given η1,…,etal and ψ1,…,ψl, find a ∈ Rq and b ∈ Rp such that
Cov(aTX,bTY)
is maximized subject to
– Corr(aTX,ηi) = Corr(bTY,ηi) = 0, for all i = 1,…,l – Corr(aTX,ψi) = Corr(bTY,ψi) = 0, for all i = 1,…,l – aTΣXXa = 1 and bTΣXXb = 1
 As expected the solution is: ηl+1 = al+1X and ψl+1 = bl+1Y .  This process can continue until l = k.
 Theorem (Ha ̈rdle and Simar p.447):
Let η = (η1,…,ηk)T and ψ = (ψ1,…,ψk)T, we have
􏰂 T T T􏰃 􏰈Ik Λ􏰉 Cov(η,ψ) = ΛIk
8

Theorem(invariancetoinvertibletransformation,Ha ̈rdleandSimar p.448):
LetX∗ =UTX+uandY∗ =VTY+v,whereUandVareinvertible matrices, and u and v are constant vectors. Then
– the canonical correlations between X∗ and Y ∗ are the same as those between X and Y ;
– the canonical correlation vectors of X∗ and Y ∗ are given by a∗i = U−1ai, b∗i = V−1bi, i = 1,…,k.
9