Computing partial correlations from a multiple regression

Author

Enrico Schumann

Keywords

partial correlation, regression, R

Review Status

A partial correlation between two variables $y$ and $x$ with respect to a set of other variables $Z$ is defined as the Bravais-Pearson correlation between the residuals of a regression of $y$ on $Z$ and a regression of $x$ on $Z$. There exists in fact a simple way to compute the partial correlations between $y$ and all other variables at once from one multiple regression of $y$ on $X = [x\ Z]$.

Greene [1, chapter 3] shows that in a multiple regression the partial correlations between $y$ and the regressors $X$ are related to the $t$-statistics of the model $y=X\beta+\epsilon$ by

(1)
\begin{align} \rho^2 = \frac{t^2}{t^2 + \mathrm{df}}\,. \end{align}

Here $\rho$ are the partial correlations, $t$ are the t-statistics, and $\mathrm{df}$ are the degrees of freedom of the model. Squaring $t$ means to square each element in $t$. Hence, taking the square root of this expression gives the partial correlations. Their sign is equal to the coefficients' sign in the multiple regression.

An R implementation

A short R program that implements the correlation-of-residuals method (method1) and the multiple-regression-method (method2).

# number of variables and observations
nV <- 3
nO <- 100

# create artificial data
X <- array(rnorm(nV * nO, mean = 0, sd = 0.02), dim = c(nO,nV))
y <- rnorm(nO, mean = 0, sd = 0.02)

# estimate model
model <- lm(y ~ X)
summary(model)

#########partial correlation###########

#method1: regression (vvv = x)
vvv <- 1
y.orth <- (lm(y ~ X[,-vvv]))$residuals
X.orth <- (lm(X[,vvv] ~ X[,-vvv]))$residuals
cor(y.orth,X.orth)

#method2: multiple regression
t.values <- model$coeff / sqrt(diag(vcov(model)))
partcorr <- sqrt((t.values^2) / ((t.values^2) + model$df.residual))
partcorr

Internal Links

Concepts
Tutorials
Tips
Related Articles

External links

References
1. Greene, W.H. (2005). Econometric Analysis. 5th ed. Pearson Education.
2. R Development Core Team (2008). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. http://www.R-project.org.
Weblinks

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License