ORIE 6700 Statistical Principles (Fall 2017)

1. Basic Info

Lectures: Mon/Wed 2:55–4:10, Hollister Hall 320
Sessions: Thu 2:55--4:25, Phillips Hall 407

Instructor:
Yudong Chen (yudong.chen at cornell edu, Rhodes 223)
Office hours: Monday 5:10-6:10pm

TA:
Wei Qian (wq34 at cornell edu)
Office hours: Tue 5:15--6:15, Rhodes 294

2. Syllabus

The course is about how to think rationally about the scientific extraction of information from data. This is a “theory” course, but mathematical formulations will be motivated by applications.

• • Basic concepts and models: exponential families, location/scale families, mixture models, order statistics, sufficient statistics, completeness.
• • Statistical decision theory: risk, optimality, admissibility, Bayesian framework.
• • Point estimation: uniform minimum variance unbiased estimators, Fisher Information, Cramer-Rao bounds, Rao-Blackwell and Lehmann-Scheffe, maximum likelihood estimators, asymptotic efficiency, linear regression.
• • Hypothesis testing: most powerful tests, Neyman-Pearson and Karlin-Rubin, z/t-tests, Bayesian hypothesis testing.
• • Interval estimation: duality with testing, Bayesian credible intervals, p-values.
• • Computational statistics: Expectation-Maximization (EM), gradient descent, Markov chain Monte Carlo, Gibbs sampler, model checking.
• • Modern high dimensional statistics: sparse regression, compressed sensing and Lasso, concentration inequalities, non-asymptotic analysis, and selected topics.

3. Prerequisites

• • Multivariate calculus including epsilon-delta proofs as taught in advanced calculus or mathematical analysis courses, such as Math 4130 at Cornell.
• Linear algebra and matrices (including eigenvalues/vectors, SVD)
• • One semester of undergraduate probability, including:
• - Probabilities, random variables and vectors, probability mass functions and probability density functions. Cumulative distribution functions. Joint probability mass/density functions, independence.
• - Expected values, moments, moment generating functions, (co)variance.
• - Modes of convergence (in distribution, in probability, and almost surely).
• - The law of large numbers and the central limit theorem.
• - Basic distributions (normal, uniform, exponential, gamma, beta, chi-square, t, F, binomial, Poisson, geometric, hypergeometric, negative binomial).
• Probability prerequisites can mostly be found in Casella & Berger: Sections 1.1-6, 2.1-3, 3.1-3 & 3.6.1, 4.1-3, 4.5-4.6, 5.1-5. You should read these sections carefully in the first week of class. They may give a more formal treatment of these topics than you have seen before. For a more basic reference, you may consult the textbook, “A First Course in Probability,” by Sheldon Ross, although you are required to understand the material at the level of Casella & Berger.

4. Textbooks

• Required: Casella, G. and Berger, R. (2002, 2nd ed.). Statistical Inference.
• • Helpful: Shao, J. (2003, 2nd edition). Mathematical Statistics. The proofs of some of the technical results in the course can be found in this book.
• • Helpful: Keener, R. (2010). Theoretical Statistics: Topics for a Core Course.
• • Helpful: Bickel, P.J. & Doksum, K.A. (2007, 2nd ed.). Mathematical Statistics.
• • Helpful: Wasserman, L. (2003). All of Statistics: A Concise Course in Statistical Inference.

5. Homework

Homework assignments will be posted weekly on Wednesday, and are due the following Wednesday by 2:55pm to the course dropbox on the 1st floor of Rhodes Hall. The first homework will be assigned by Aug 30. No HWs accepted late or to any other location.

You may discuss problems if you find this educational but solutions must be written up individually. Copying is a violation of the honor code.

6. Exams

• Prelim: Oct 3 7:30--9:30PM, Phillips 403. You are allowed to bring one double-sided, letter-sized sheet of notes.
• Final: Dec 9, 2-4pm, Phillips 403. You can bring two double-sided, letter-sized sheets of notes.

7. Website

Log in to Blackboard to access course materials. You should be automatically given access to the course Blackboard site when you enroll in the course. All course communication is via Blackboard.