# TU Wien:Multivariate Statistik VO (Filzmoser)/Multivariate Statistics Possible Exam Questions

## Why do we need multivariate statistics?[Bearbeiten | Quelltext bearbeiten]

## What is the Spectral Decomposition Theorem?[Bearbeiten | Quelltext bearbeiten]

## What is the expectation of the Principal Components?[Bearbeiten | Quelltext bearbeiten]

## What is the density of the multivariate normal distribution?[Bearbeiten | Quelltext bearbeiten]

## Name distances for clustering, methods, and their respective objective functions/criteria.[Bearbeiten | Quelltext bearbeiten]

## Explain model-based clustering and difficulties that could occur.[Bearbeiten | Quelltext bearbeiten]

Note: We have to estimate k multivariate normal distribution covariance structures. That can potentially be a lot of parameters to estimate and lead to instability in the estimates. There are some assumptions that make the model simpler.

## Explain fuzzy clustering, what is the objective function?[Bearbeiten | Quelltext bearbeiten]

## How can we evaluate clustering solutions -- Hetero/Homogeneity, Calinski-Harabasz, Hartigan, silhouette width, Gap statistic (principles)?[Bearbeiten | Quelltext bearbeiten]

## What is the least squares estimator?[Bearbeiten | Quelltext bearbeiten]

## How does multivariate linear regression work, what is the objective function, solution and appropriate inference (estimation of covariance of errors). Basic model selection?[Bearbeiten | Quelltext bearbeiten]

## What are problems with non-robustness? How is it connected to the (empirical) influence function, maxbias curve, breakdown point and efficiency?[Bearbeiten | Quelltext bearbeiten]

## What are M-estimators? What are the M-estimating equations? Why is it a weighted least squares estimator?[Bearbeiten | Quelltext bearbeiten]

## What are S-estimators? What is the MM-estimator and the properties inherited from S- and M-estimators?[Bearbeiten | Quelltext bearbeiten]

## Define affine equivariance.[Bearbeiten | Quelltext bearbeiten]

## [Bearbeiten | Quelltext bearbeiten]

## What are problems in classic regression diagnostics (hat matrix)? What are robust regression diagnostics?[Bearbeiten | Quelltext bearbeiten]

## How does robust multivariate regression work? (estimate covariance matrix with M-estimator of scale)[Bearbeiten | Quelltext bearbeiten]

## Principal Component Analysis - how to select the vectors for the transformation, Lagrange problem definition.[Bearbeiten | Quelltext bearbeiten]

## Why is PCA sensitive to scale? What happens if we center-scale the data?[Bearbeiten | Quelltext bearbeiten]

## What are some rules for the number of principal components to select? (for hypothesis tests: only concept, not formulas)[Bearbeiten | Quelltext bearbeiten]

## [Bearbeiten | Quelltext bearbeiten]

Let be a mean-centered matrix (columns have mean 0).

Then there exists an orthogonal matrix and an orthogonal matrix such that

where is an "diagonal" matrix i.e. the only non-zero values are . The "diagonal" elements of are called *singular values* of .

We can show that

,

which means the columns of are the eigenvectors of with eigenvalues . Furthermore, it holds that the covariance matrix , because is mean-centered. We know that in PCA,

Therefore, and . Hence, for the scores we obtain .

SVD is preferable when , which means we have more features than observations. In that case, the covariance matrix would be non-singular and the spectral decomposition theorem would not be applicable.

## How can we define the PCA problem in terms of reconstruction error (Frobenius norm)?[Bearbeiten | Quelltext bearbeiten]

Note: I got this question in my exam. Only the definition was enough with a natural language explanation.

## What are Biplots? What is the rank-2 approximation? Define the G/H matrix. What are the properties of the biplot? (inner row product of G and H approximates elements of the X-matrix, etc.)[Bearbeiten | Quelltext bearbeiten]

## Which diagnostics do we have for PCA (formal definition of orthogonal, score distance)?[Bearbeiten | Quelltext bearbeiten]

## What is the factor analysis model (formal definition, assumptions)? What is the difference to PCA?[Bearbeiten | Quelltext bearbeiten]

## Explain the decomposition of the correlation matrix in factor analysis.[Bearbeiten | Quelltext bearbeiten]

## [Bearbeiten | Quelltext bearbeiten]

## What is the maximum number of factors we can include in the factor model, and why?[Bearbeiten | Quelltext bearbeiten]

## How can we estimate the communalities and loadings (PFA)?[Bearbeiten | Quelltext bearbeiten]

## How can we interpret factors? Give an overview of factor rotation criteria.[Bearbeiten | Quelltext bearbeiten]

## How are factor scores estimated (Bartlett and Regression method). Name the models, formulas and solutions for the factor scores estimates.[Bearbeiten | Quelltext bearbeiten]

## What is the problem setting in multiple correlation analysis? What is the objective function to minimize?[Bearbeiten | Quelltext bearbeiten]

## What is the linear prediction function in multiple correlation analysis? Describe the structure of the proof.[Bearbeiten | Quelltext bearbeiten]

## Name a hypothesis test for the multiple correlation coefficient.[Bearbeiten | Quelltext bearbeiten]

## What is the problem setting in canonical correlation, what is the maximization problem?[Bearbeiten | Quelltext bearbeiten]

## How do we get the linear combinations for canonical correlation? Why is a matrix product and Eigenvector/Eigenvalue problem involved?[Bearbeiten | Quelltext bearbeiten]

## What happens if there is the same variable in X and Y in canonical correlation?[Bearbeiten | Quelltext bearbeiten]

The first canonical correlation coefficient is 1.