• Jeremy Wu () motivated Meng's questions about multi-source inference
• How could we use a non-random sample?
• “Is an 80% non-random sample ‘better’ than a 5% random sample in measurable terms? 90%? 95%? 99%?”
• Meng (2014) defines multi-source inference as that arising in "situations where we need to draw inference by using data coming from different sources and some (but not all) of which were not collected for inference purposes."

• Two sources
• Administrative record (\(f_a\) fraction of population)
• Simple random sample (SRS) (\(f_s\) fraction of population)
• \(f_a >>f_s\)
• How to combine maximal information from both sources?
• Wu () asks how relative information varies with differences in ratio of \(\frac{f_a}{f_s}\)

• \(\lbrace x_1, ..., x_N\rbrace\) is our population of size \(N\)
  • \(R_i\) indicator for presence of \(x_i\) in administrative record
  • \(I_i\) indicator for presence of \(x_i\) in SRS
  • \(n_a = \sum_{i=1}^NR_i >>n_s = \sum_{i=1}^N I_i\)
• \(\bar{x}_a = \frac{\sum_{i=1}^Nx_iR_i}{n_a}\)
• \(\bar{x}_s = \frac{\sum_{i=1}^Nx_iI_i}{n_s}\)

• \(R_i = 1_{Z_i \le \alpha + \beta x_i}\)
• \(Z_i \sim N(0,1)\text{ iid}\) (latent refusal tendency of \(i^{th}\) individual)
• \(\beta\): strength of self-selecting mechanism (\(\beta \neq 0\) means non-ignorable missing data)

• \(x_s\) unbiased
• \(Var(\bar{x}_s) = \frac{1-f_s}{n_s}S_N^2(x)\)
• \(S_N^2(x) = \frac{1}{N-1}\sum_{i=1}^N(x_i - \bar{x}_N)^2\)

• biased (due to nature of sampling)
• \(R_i\) depends on \(x_i\)
• Assume: 1. \(N\) very large, 2. \(f_a = \frac{n_a}{N}\) not near 0
• Then \(\text{bias}^2\) dominates \(MSE(\bar{x}_a) \approx \text{bias}^2(\bar{x}_a)\)
• Large \(N\) and large \(n_a\) allow simplification:

\[MSE(\bar{x}_a) \approx \lbrace \frac{\sum_{i=1}^N(x_i - \bar{x}_N)p(x_i)}{\sum_{i=1}^Np(x_i)}\rbrace^2\]

• \(p(x_i) = \mathbb{E}(R_i|x_i) = \Phi(\alpha + x_i\beta)\)

• By LLN,

\[ \text{bias}^2(\bar{x}_a) = \frac{\text{cov}(X, p(X))}{\mathbb{E}(p(X))} = \frac{\sigma \mathbb{E}(Z\Phi(\tilde{\alpha}+ \tilde{\beta}Z))}{\mathbb{E}(\Phi(\tilde{\alpha}+ \tilde{\beta}Z)))} = \frac{\sigma \tilde{\beta}}{\sqrt{1+\tilde{\beta}^2}}\text{MR}(\frac{\tilde{\alpha}}{\sqrt{1+\tilde{\beta}^2}})\]

• \(MR(a) = \frac{\phi(a)}{\Phi(a)}\)

\[\frac{\text{MSE}(\bar{x}_a)}{\sigma^2} \approx \frac{\text{bias}^2(\bar{x}_a)}{\sigma^2} = \frac{\phi^2(z_{f_a})}{f_a^2} \frac{\tilde{\beta}^2}{1+\tilde{\beta}^2} = \frac{\tilde{\beta}^2}{1+\tilde{\beta}^2}\frac{e^{-z^2_{f_z}}}{2\pi f_a^2}\]

\[\frac{\text{MSE}(\bar{x}_s)}{\sigma^2} = \frac{1}{n_s} - \frac{1}{N}\approx \frac{1}{n_s}\]

• Non-sampling is arbitrarily small only when \(f_a \to 1\) (\(n_a\) alone is insufficient)

• Sampling error can be arbitrarily small by merely making \(n_s\) arbitrarily large, even if \(f_s \to 0\) as \(n\to \infty\)

• Big data may be "big" due to their size relative to the (finite) population size, rather than due to their absolute size

\[f_a > \frac{n_s\rho^2_N(x,p)}{1+n_s\rho^2_N(x,p)}\]

• \(\rho_N(x,p)\): correlation between x & p
• estimate \(\rho_N(x,p)\) from the SRS

• Given partial knowledge of the recording/response mechanism for a (large) biased sample, what is the optimal way to create an intentionally biased sub-sampling scheme to counter-balance the original bias so the resulting sub-sample is guaranteed to be less biased than the original biased sample in terms of the sample mean, or other estimators, or predictive power?

What should be the key considerations when combining small random samples with large non-random samples, and what are the sensible “corner- cutting” guidelines when facing resource constraints? How can the combined data help to estimate \(\rho_N (x, p)\)? In what ways can such estimators aid multi-source inference?

• What are theoretically sound and practically useful defect indices for pre- diction, hypothesis testing, model checking, clustering, classification, etc., as counterparts to the defect index for estimation, ρN(x,p)? What are their roles in determining information bounds for multi-source inference? What are the relevant information measures for multi-source inference?

• How can we develop the study of multi-source inference in the personalized (genomic) medicine context?
• Administrative record: EMR ?
• SRS: ?
• How close to SRS is the sample in, say, a GWAS? or TCGA study?
• Personalized medicine's goals are about prediction