Regression Example (Download Code)


In this example, a sparse regression example is built, noise is added, then the truth is hidden. This model has 200 variables ot choose from - 190 are simply noise (drawn from N(0,1)) and the other 10 drive the response vector, y. We use SparseLab tools to undercover the true sparse model.

 

The following plots show the true model, in red, and the estimated model in blue. The Forward Stepwise Algorithm was used to recover the model coefficients in the first plot; in the second plot Forward Stepwise was use, but with a cutoff dictated by False Discovery Rates; the third plot used Matching Pursuit, and the final plot used Orthogonal Matching Pursuit to recover the underlying model. It's easy to visually compare the algorithms in this setting.

 

All these tools are included in SparseLab, along with the code that generated this example (it is also linked to above). If one were so inclined, it is very easy to change the parameters (such as number of variable, observations, noise level, algorithm used) but simply modifying the script.