Structured Sparsity Reading Group (SSRG) 
Spring 2011 
 

Day/time: 1-2pm, Tuesdays
Location: CS 3310
Mailing list: sign up here
Organizer: Junming Sui

Description:

In this reading group, we will discuss recent works on high-dimensional sparse learning with prior structural information. Sparse models, which can conduct classification and feature selection simultaneously, are preferred in learning problems with high-dimensional data. In many applications, some intrinsic structure information is available on either input or output sides. For example, several genes may belong to the same functional group; in multi-task learning, several estimators are expected to share common types of covariates. Incorporating such structure information into sparse learning can achieve better prediction performance and produce interpretable results. We will cover several topics on this direction, including some structured sparsity regularizer and optimization techniques, multi-task learning, varying coefficient models, graphical models, dictionary learning, relation with submodular functions.

Schedule

An up-to-date schedule will be maintained as a Google Calendar (see below), but the order of topics will follow roughly what is mapped out below. Schedule is adjustable.

Tentative Plan

Multiple papers are listed as 'suggested/optional' for one week. Whoever presents can decide to pick only one to focus on, or to present an overview of multiple papers.

• Week 1 (02/15): Group Lasso and Group Dantzig Selector (Presenter: Sang Lee)
  
  • Model selection and estimation in regression with grouped variables
    Yuan, M. and Lin, Y.
    Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1):49-67, 2006
    
  • The group Dantzig selector
    Liu, H. and Zhang, J. and Jiang, X. and Liu, J.
    AISTATS 2010
    
• Week 2 (02/22): General Structured Sparse Learning (Presenter: Jie Liu)
  
  • An Efficient Proximal Gradient Method for General Structured Sparse Learning
    Chen, X. and Lin, Q. and Kim, S. and Carbonell, J.G. and Xing, E.P.
    JMLR 2010
    
  • Group Lasso with overlap and graph Lasso
    Jacob, L. and Obozinski, G. and Vert, J.P.
    ICML 2009
    
  • Statistical estimation of correlated genome associations to a quantitative trait network
    Kim, S. and Xing, E.P.
    PLoS Genetics 2009
    
• Week 3 - 4 (03/01, 03/08): Multi-task Learning (Presenter: Junming Xu)
  
  • Nonparametric regression and classification with joint sparsity constraints
    Liu, H. and Lafferty, J. and Wasserman, L.
    NIPS 2008
    
  • Ultra-high dimensional multiple output learning with simultaneous orthogonal matching pursuit: Screening approach
    Kolar, M. and Xing, E.P.
    AISTATS 2010
    
  • Tree-guided group lasso for multi-task regression with structured sparsity
    Kim, S. and Xing, E.P.
    ICML 2010
    
• Week 5 (03/22): Varying Coefficient Models (Presenter: Kwang-Sung Jun)
  
  • Sparsistent Learning of Varying-coefficient Models with Structural Changes
    Kolar, M. and Song, L. and Xing, E.P.
    NIPS 2009
    
• Week 6 - 8 (03/29, 04/05, 04/12): Sparsity-Inducing Norms (Presenter: Sang Lee, Nikhil Rao)
  
  • Convex Analysis and Optimization with Submodular Functions: a Tutorial
    Bach, F.
    Technical report, HAL 00527714, 2010
    
  • Structured Sparsity-Inducing Norms through Submodular Functions
    Bach, F.
    NIPS 2010
    
  • Structured Variable Selection with Sparsity-Inducing Norms
    Jenatton, R. and Audibert J. and Bach, F.
    Technical report, 2009
    
• Week 9 - 10 (04/19, 04/26): Sparse Dictionary Learning (Presenter: Nikhil Rao)
  
  • Structured Sparse Principal Component Analysis
    Jenatton, B.R. and Obozinski, G. and Bach, F.
    AISTATS 2010
    
  • Proximal Methods for Sparse Hierarchical Dictionary Learning
    Jenatton, B.R. and Mairal, J. and Obozinski, G. and Bach, F.
    ICML 2010
    
  • Online Group-Structured Dictionary Learning [supplementary material]
    Szabo, Z. and Poczos, B. and Lorincz A.
    CVPR 2011
    
• Week 11 (05/03): Graphical Models (Presenter: Ji Liu)
  
  • The nonparanormal: Semiparametric estimation of high dimensional undirected graphs
    Liu, H. and Lafferty, J. and Wasserman, L.
    JMLR 2009
    
  • Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models
    Liu, H. and Roeder, K. and Wasserman, L.
    NIPS 2010
    
  • Forest Density Estimation [Short Version]
    Liu, H. and Xu, M. and Gu, H. and Gupta, A. and Lafferty, J. and Wasserman, L.
    COLT 2010
    

Other useful references:

Sparse methods for machine learning: Theory and algorithms, ECML/PKDD 2010 Tutorial
NIPS-2010 Workshop on Practical Applications of Sparse Modeling: Open Issues and New Directions