Structured Sparsity Reading Group (SSRG)
Day/time: 1-2pm, Tuesdays
Location: CS 3310
Mailing list: sign up
Organizer: Junming Sui
In this reading group, we will discuss recent works on high-dimensional sparse learning with prior structural information. Sparse models, which can conduct classification and feature selection simultaneously, are preferred in learning problems with high-dimensional data. In many applications, some intrinsic structure information is available on either input or output sides. For example, several genes may belong to the same functional group; in multi-task learning, several estimators are expected to share common types of covariates. Incorporating such structure information into sparse learning can achieve better prediction performance and produce interpretable results. We will cover several topics on this direction, including some structured sparsity regularizer and optimization techniques, multi-task learning, varying coefficient models, graphical models, dictionary learning, relation with submodular functions.
An up-to-date schedule will be maintained as a Google
Calendar (see below), but the order of topics will follow roughly what is mapped out
below. Schedule is adjustable.
Multiple papers are listed as 'suggested/optional' for one week. Whoever presents can decide to pick only one to focus on, or to present an overview of multiple papers.
- Week 1 (02/15): Group Lasso and Group Dantzig Selector (Presenter: Sang Lee)
- Week 2 (02/22): General Structured Sparse Learning (Presenter: Jie Liu)
An Efficient Proximal Gradient Method for General Structured Sparse Learning
Chen, X. and Lin, Q. and Kim, S. and Carbonell, J.G. and Xing, E.P.
Group Lasso with overlap and graph Lasso
Jacob, L. and Obozinski, G. and Vert, J.P.
Statistical estimation of correlated genome associations to a quantitative trait network
Kim, S. and Xing, E.P.
PLoS Genetics 2009
- Week 3 - 4 (03/01, 03/08): Multi-task Learning (Presenter: Junming Xu)
- Week 5 (03/22): Varying Coefficient Models (Presenter: Kwang-Sung Jun)
- Week 6 - 8 (03/29, 04/05, 04/12): Sparsity-Inducing Norms (Presenter: Sang Lee, Nikhil Rao)
- Week 9 - 10 (04/19, 04/26): Sparse Dictionary Learning (Presenter: Nikhil Rao)
- Week 11 (05/03): Graphical Models (Presenter: Ji Liu)
The nonparanormal: Semiparametric estimation of high dimensional undirected graphs
Liu, H. and Lafferty, J. and Wasserman, L.
Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models
Liu, H. and Roeder, K. and Wasserman, L.
Forest Density Estimation
Liu, H. and Xu, M. and Gu, H. and Gupta, A. and Lafferty, J. and Wasserman, L.
Other useful references:
Sparse methods for machine learning: Theory and algorithms, ECML/PKDD 2010 Tutorial
NIPS-2010 Workshop on Practical Applications of Sparse Modeling: Open Issues and New Directions