কেন সম্ভাবনা-অনুপাতের পরীক্ষা চি-স্কোয়ার বিতরণ করা হয়?

Question

সম্ভাবনা অনুপাত পরীক্ষার পরীক্ষার পরিসংখ্যান কেন চি-স্কোয়ার বিতরণ করা হয়?

$2(\ln \text{ L}_{\rm alt\ model} - \ln \text{ L}_{\rm null\ model} ) \sim \chi^{2}_{df_{\rm alt}-df_{\rm null}}$

Comments

এই সাহায্য করে ?

— নিক সাব্বে

রেফারেন্সের জন্য ধন্যবাদ। আমার থেকে একটি এখানে: stats.stackexchange.com/faq#etiquette

— ডাঃ

সেখানে "আপনার হাস্যরসের অনুভূতি আনুন" লক্ষ্য করুন। আমি অভদ্র হওয়ার ইচ্ছা করি নি, তবে এই প্রশ্নের উত্তর তুলনামূলকভাবে ক্লান্তিকর এবং মূলত সেই নিবন্ধটির (বা আরও ভাল পরিসংখ্যানের পাঠ্যপুস্তকের কিছু) বিষয়বস্তুর সমন্বয়ে গঠিত হবে। আপনি যদি এর মধ্যে একটিতে ব্যাখ্যা দিয়ে আপনার নির্দিষ্ট সমস্যাটি বর্ণনা করেন তবে আমি আপনাকে সাহায্য করে খুশি হব।

— নিক সাব্বে

পে-ওয়াল ছাড়াই উইল্কসের মূল কাগজে সরাসরি লিঙ্ক ।

— আয়োরগো

Answer 1

As mentioned by @Nick this is a consequence of Wilks' theorem. But note that the test statistic is asymptotically $\chi^2$-distributed, not $\chi^2$-distributed.

আমি এই উপপাদ্যটি দ্বারা অত্যন্ত অভিভূত কারণ এটি একটি বিস্তৃত প্রসঙ্গে। সম্ভাবনা সঙ্গে একটি পরিসংখ্যান মডেল বিবেচনা ঠ ( θ | Y ) যেখানে Y ভেক্টর পর্যবেক্ষণ হল এন পরামিতি সঙ্গে একটি ডিস্ট্রিবিউশন থেকে স্বাধীন প্রতিলিপি পর্যবেক্ষণ θ একটি submanifold একাত্মতার বি 1 এর আর ঘ সঙ্গে মাত্রা অস্পষ্ট ( বি 1 ) = গুলি । যাক বি 0 ⊂ বি 1 একটি submanifold সঙ্গে মাত্রা হতে অস্পষ্ট ( বি $l(\theta \mid y)$$y$$n$$\theta$$B_1$$\mathbb{R}^d$$\dim(B_1)=s$$B_0 \subset B_1$$\dim(B_0)=m$. Imagine you are interested in testing $H_0\colon\{\theta \in B_0\}$.

The likelihood ratio is

$$lr(y) = \frac{\sup_{\theta \in B_1}l(\theta \mid y)}{\sup_{\theta \in B_0}l(\theta \mid y)}.$$ Define the deviance $d(y)=2 \log \big(lr(y)\big)$. Then Wilks' theorem says that, under usual regularity assumptions, $d(y)$ is asymptotically $\chi^2$-distributed with $s-m$ degrees of freedom when $H_0$ holds true.

It is proven in Wilk's original paper mentioned by @Nick. I think this paper is not easy to read. Wilks published a book later, perhaps with an easiest presentation of his theorem. A short heuristic proof is given in Williams' excellent book.

Comments

Sad that this theorem is not mentioned in the wikipedia page devoted to Samuel S. Wilks

— Stéphane Laurent

Oh come on Stephane. This is Wikipedia, you can edit it and improve it!

— StasK

@StasK I know that but I have never tried. And I already spend too much time of my life with statistics & mathematics ;)

— Stéphane Laurent

Is there an intuition for why the 2 is in front of the log in the definition of the deviance?

— user56834

@Programmer2134 It is derived from a second order taylor expansion.

— Frank Vel

Answer 2

I second Nick Sabbe's harsh comment, and my short answer is, It is not. I mean, it only is in the normal linear model. For absolutely any other sort of circumstances, the exact distribution is not a $\chi^2$. In many situations, you can hope that Wilks' theorem conditions are satisfied, and then asymptotically the log-likelihood ratio test statistics converges in distribution to $\chi^2$. Limitations and violations of the conditions of Wilks' theorem are too numerous to disregard.

1. The theorem assumes i.i.d. data $\Rightarrow$ expect issues with dependent data, such as time series or unequal probability survey samples (for which the likelihoods are poorly defined, anyway; the "regular" $\chi^2$ tests, such as independence tests in contingency tables, start behaving as a sum $\sum_k a_k v_k, v_k \sim \mbox{i.i.d.} \chi^2_1$ (Rao & Scott). For i.i.d. data, $a_k=1$, and the sum becomes the $\chi^2$. But for non-independent data, this is no longer the case.
2. The theorem assumes the true parameter to be in the interior of the parameter space. If you have a Euclidean space to work with, that's not an issue. However, in some problems, the natural restrictions may arise, such as variance $\ge$ 0 or correlation between -1 and 1. If the true parameter is one the boundary, then the asymptotic distribution is a mixture of $\chi^2$ with different degrees of freedom, in the sense that the cdf of the test is the sum of such cdfs (Andrews 2001, plus two or three more of his papers from the same period, with history going back to Chernoff 1954).
3. The theorem assumes that all the relevant derivatives are non-zero. This can be challenged with some nonlinear problems and/or parameterizations, and/or situations when a parameter is not identified under the null. Suppose you have a Gaussian mixture model, and your null is one component $N(\mu_0,\sigma^2_0)$ vs. the alternative of two distinct components $f N(\mu_1,\sigma_1^2) + (1-f) N(\mu_2,\sigma_2^2)$ with a mixing fraction $f$. The null is apparently nested in the alternative, but this can be expressed in a variety of ways: as $f=0$ (in which case the parameters $\mu_1,\sigma_1^2$ are not identified), $f=1$ (in which case $\mu_2, \sigma_2^2$ are not identified), or $\mu_1=\mu_2, \sigma_1=\sigma_2$ (in which case $f$ is not identified). Here, you can't even say how many degrees of freedom your test should have, as you have different number of restrictions depending on how you parameterize the nesting. See the work of Jiahua Chen on this, e.g. CJS 2001.
4. The $\chi^2$ may work OK if the distribution has been correctly specified. But if it was not, the test will break down again. In the (largely neglected by statisticians) subarea of multivariate analysis known as structural equation covariance modeling, a multivariate normal distribution is often assumed, but even if the structure is correct, the test will misbehave if the distribution is different. Satorra and Bentler 1995 show that the distribution will become $\sum_k a_k v_k, v_k \sim \mbox{i.i.d.} \chi^2_1$, the same story as with non-independent data in my point 1, but they've also demonstrated how the $a_k$s depend on the structure of the model and the fourth moments of the distribution.
5. For finite samples, in a large class of situations likelihood ratio is Bartlett-correctible: while ${\rm Prob}[d(y) \le x]=F(x;\chi^2_d)[1+O(n^{-1})]$ for a sample of size $n$, and $F(x;\chi^2_d)$ being the distribution function of the $\chi^2_d$ distribution, for the regular likelihood problems you can find a constant $b$ such that ${\rm Prob}[d(y)/(1+b/n) \le x]=F(x;\chi^2_d)[1+O(n^{-2})]$, i.e., to a higher order of accuracy. So the $\chi^2$ approximation for finite samples can be improved (and arguably should be improved if you know how). The constant $b$ depends on the structure of the model, and sometimes on the auxiliary parameters, but if it can be consistently estimated, that works, too, in improving the order of coverage.

For a review of these and similar esoteric issues in likelihood inference, see Smith 1989.

Comments

Thanks! Very instructive. What do you mean by "it only is in the normal linear model" ? For a Fisher test when $B_0$ and $B_1$ are linear subspaces, then the deviance is a monotone function of the Fisher statistic, and it is only asymptotically $\chi^2$.

— Stéphane Laurent

With known variance, I should add.

— StasK