# LOGISTIC REGRESSION

In Chapter 7, I gave the example of logistic regression: predicting whether something (Y) was 1 or 0 based on a series of observations (X). Thus, logistic regression can be thought of a classification method, and regularized multinomial regression is a respectable modern machine learning approach to multi-class classification. The estimation of the logistic regression parameters, b, is the “training” stage for this classification method. We assume that we are in a multivariate setting, so that we have a vector *b* with one component for each dimension, and we can proceed with estimation using one of the approaches we saw in the chapters on regression. To predict *Y _{n}+_{1}* (or do the classification) based on the features,

*X*for a new observation we simply follow the assumption of logistic regression, and calculate

_{n}+_{1},

Since there are only two possibilities (1 or 0), if this is greater than У2, we can assign observation *n* + 1 to the positive class by the MAP rule. The classification boundary is therefore

which works out to *bX* = 0 (see Exercises). More generally, we can choose a threshold, t, and set the classification boundary to be *bX* = t, above which we will assign observation *n* + 1 to the positive class. This formula is just the equation for a line if there are two dimensions, i.e., *X _{t}* = (X

_{;1}, X

_{i2}), (see Exercises), and a plane (or “hyperplane”), if there are more than two dimensions.