Logistic Regression without Intercept
Asian Journal of Probability and Statistics,
Logistic regression is a popular statistic modelling algorithm in predicting a binary outcome. Although logistic regression almost always has an intercept, logistic regression without intercept is sometimes appropriate or even necessary. However, logistic regression without intercept has rarely been discussed other than being used explicitly or implicitly. In this paper, we aim at filling this gap by systematically studying logistic regression without intercept. Specifically, we study the 4 most important aspects of logistic regression: (1) Maximum Likelihood Estimate, (2) data configuration (complete separation, quasi-complete separation and overlap) to categorize the existence and uniqueness of maximum likelihood estimate, (3) multicollinearity, and (4) monotonic transformations of independent variables. We adopt an extensional method in that we first present results for logistic regression with intercept and then extend the results to the case of without intercept. Our numerical examples further compare logistic regression with intercept and without intercept.
- Logistic regression
- variance inflation factor
How to Cite
Eisenhauer JG. Regression through the Origin. Teaching Statistics. 2003;25(3):76-80.
Cincotta K. Multicollinearity in zero intercept regression: They are not who we thought they were. Presented at the Society of Cost Estimating and Analysis (SCEA) Conference. June, 2011, Albuquerque, NM; 2011.
Othman SA. Comparison between Models With and Without Intercept. Gen. Math. Notes. 2014;21(1):118-127.
Lee AH, Silvapulle MJ. Ridge estimation in logistic regression, Communications in Statistics - Simulation and Computation. 1988;17(4):1231-1257.
Duffy DE, Santner TJ. On the small sample properties of norm-restricted maximum likelihood estimators for logistic regression models. Communications in Statistics - Theory and Methods. 1989;18(3):959-980.
Le Cessie S, van Houwelingen JC. Ridge estimators in logistic regression. Journal of the Royal Statistical Society Series C (Applied Statistics). 1992;41(1):191-201.
Tian GL, Fang HB, Liu ZQ, Ming T Tan. Regularized (bridge) logistic regression for variable selection based on ROC criterion. Statistics and Its Inference. 2009;2:493-502.
SAS Institute Inc. SAS/STAT 13.1 User’s Guide. Cary, NC: SAS Institute Inc; 2013.
Zeng G. On the existence of an analytical solution in multiple logistic regression. International journal of applied mathematics & statistics. 2021;60(1):53-67.
Zeng G. On the analytical properties of category encodings in logistic regression. Communications in Statistics - Theory and Methods; 2021.
Farrar D, Glauber R. Multicollmearity in Regression analysts the problem revisited, Review of Economics and Statistics. 1967;92-107.
Zeng G, Zeng E. On the relationship between multicollinearity and separation in logistic regression. Communications in Statistics - Simulation and Computation; 2019b.
Voss DS. (n.d.) Multicollinearity. In The Encyclopedia of Social Measurement, ed. Kempf-Leonard, Kimberly. San Diego, CA: Academic Press.
Chatelain JB, Ralf K. Spurious regressions and near- multicollinearity, with an application to aid, Policies and growth. Journal of Macroeconomics. 2014;39(PA):85-96.
Vatcheva KP, Lee M, McCormick JB, Rahbar MH. Multicollinearity in regression analyses conducted in epidemiologic studies. Epidemiology (Sunnyvale). 2016;6(2):227.
Dohoo IR, Ducrot C, Fourichon C, Donald A, Hurnik D. An overview of techniques for dealing with large numbers of independent variables in epidemiologic studies. Preventive Veterinary Medicine. 1997;29(3):221–239.
Bewick V, Cheek L, Ball J. Statistics review 7: Correlation and regression: Presenting and summarizing data. Critical Care. 2003;7:451-459.
Shen J, Gao S. A solution to separation and multicollinearity in multiple logistic regression. Journal of Data Science. 2008;6:515-531.
Albert A, Anderson, JA. On the existence of maximum likelihood estimates in logistic regression models. Biometrika. 1984;71(1):1–10.
Zeng G. Invariant properties of logistic regression model in credit scoring under monotonic transformations. Communications in Statistics - Theory and Methods. 2017b;46(17):8791-8807.
Zeng G. On the existence of maximum likelihood estimates for weighted logistic regression. Communications in Statistics - Theory and Methods. 2017c;46(22):11194-11203.
Breslow NE, Day NE. Statistical methods in cancer research. The Analysis of Case-Control Studies. Lyon, France: IARC. 1980;1:249–251.
Zeng G. Metric divergence measures and information value in credit scoring. Journal of Mathematics. 2013;Article ID 848271.
Zeng G. A unified definition of mutual information with applications in machine learning. Mathematical Problems in Engineering. 2015;Article ID 201874.
Zeng G. A comparison study of computational methods of kolmogorov–smirnov statistic in credit scoring. Communications in Statistics - Theory and Methods. 2017a;46(10):7744-7760.
Lichman M. UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science. 2013.
Abstract View: 226 times
PDF Download: 72 times