Logistic regression and decision boundaries

The credit team had a rule written in 2019 and never revised: reject any applicant trading for less than eighteen months. When the first classifier went in beside it, the argument that followed was not about accuracy. It was that a shop trading for sixteen months with a perfect payment record was being refused, and the model wanted to approve it, and nobody could say which of the two was making the better trade.

Both were drawing a line through the same space. The rule drew a vertical one. The model drew a slanted one, and the slant is the entire content of what it learned.

The model, in feature space

Two inputs, chosen because a credit officer can check them: months trading, and the share of past instalments paid on time. Fit a logistic regression to repayment and it returns coefficients.

log-odds of repaying = 0.122 * months + 7.3 * on-time rate - 7.3

Take an applicant trading for 30 months with an on-time rate of 0.80. The sum is 3.66 plus 5.84 minus 7.30, which is 2.20. That number is in log-odds. Exponentiate it to get odds of 9.03 to one, and turn odds into a probability by dividing by one plus itself: 9.03 divided by 10.03, which is 0.90.

A second applicant, 12 months and an on-time rate of 0.62, gives 1.46 plus 4.53 minus 7.30, which is minus 1.31. Odds of 0.27 to one, a probability of 0.21.

The linear part is doing everything. The logistic function only squashes the result into the interval between zero and one, which the scikit-learn documentation writes as the expit of the linear combination. Nothing curves, nothing interacts, and no input is ever consulted in the light of another.

The boundary is where the sum hits zero

Set the log-odds to zero and the probability is exactly one half. That equation is a straight line in the two features:

0.122 * months + 7.3 * on-time rate = 7.3

Rearrange it into something a credit officer can read: the required on-time rate is 1 minus 0.0167 times the months trading. An applicant with 30 months needs a rate of 0.50. An applicant with 12 months needs 0.80. Every extra ten months of trading buys about 0.17 off the payment record required, and that exchange rate is fixed everywhere on the line, because a line has one slope.

The old rule, reject below eighteen months, is also a line, drawn vertically at 18. It refuses the sixteen month shop with a spotless record and accepts the twenty month shop that has missed a third of its instalments. Both lines are wrong somewhere. The fitted one is at least wrong in a direction someone chose by looking at outcomes.

With three inputs the boundary is a flat plane, with twenty it is a flat sheet through twenty dimensional space, and the geometry keeps working while the picture stops. What survives in any number of dimensions is the useful part: the boundary is flat, so the trade between any two inputs is the same at every point on it.

Coefficients multiply odds, they do not add percentage points

The coefficient on months is 0.122 in log-odds. Exponentiate it and you get 1.13, so one additional month multiplies the odds of repayment by about 1.13. Ten months multiplies them by e to the 1.22, which is 3.39.

The thing that trips people up is what that does to a probability, because it depends where you start.

Begin at an applicant sitting exactly on the boundary, at a probability of 0.50 and odds of 1.0. Add a month. The odds become 1.13, and the probability becomes 1.13 divided by 2.13, which is 0.530. The month was worth three probability points.

Now begin at the first applicant above, at 0.90, whose odds are 9.03. Add a month. The odds become 10.20, and the probability becomes 10.20 divided by 11.20, which is 0.911. The same month is worth one point.

One coefficient, two different answers, and neither is a mistake. The coefficient is constant on the odds scale and varies on the probability scale. Any sentence of the form "each month adds one percent to the chance of repayment" is wrong on its face, and it is the single most common thing said about a logistic model in a meeting.

Two other properties of the fitted numbers are worth knowing before quoting them. Regularisation is applied by default in scikit-learn, which the documentation notes is common in machine learning though not in statistics, so the coefficients you print have been shrunk towards zero by an amount the C parameter controls. And the inputs were on different scales here, months running to sixty and a rate running to one, which is why one coefficient is sixty times the other. Rescale the inputs and the coefficients change without the model changing at all.

The threshold moves the line without turning it

The fit chose the slope. The threshold chooses where along its own perpendicular the line sits, and it is a separate decision made by a different person.

A threshold of 0.75 means acting only when the log-odds exceed the log of 3, which is 1.0986. The boundary equation becomes 0.122 * months + 7.3 * on-time rate = 8.40, and the required on-time rate is 1.15 minus 0.0167 times the months. Identical slope. The whole line has slid up by 0.15 on the payment axis. A threshold of 0.25 slides it down by the same 0.15.

Read the counts under the drawing and the trade is arithmetic rather than argument. At 0.75 the model catches six of the ten defaults and raises no false alarms. At 0.50 it catches seven and raises three. At 0.25 it catches all ten and raises four. Which of those three is correct depends entirely on what a missed default costs against what a wrongly refused applicant costs, which is worked through in The cost of a wrong answer and is not a modelling question at all.

The scikit-learn documentation carries a worked demonstration of the same point on the German credit dataset, where a cost matrix says a bad credit approved is five times worse than a good credit refused. The business metric sits at minus 209 at the default threshold of 0.50, and tuning the threshold against that metric moves it to 0.03 and the metric to minus 143. The model was untouched.

What the flat boundary cannot do

A straight boundary is an assumption, and it is wrong in named ways.

It cannot express a rule that reverses. If very short trading histories and very long ones are both risky, for different reasons, no single line separates the safe middle from both ends. A squared term added to the feature set will do it, because the boundary is linear in the features you supply, not in the quantities they were derived from.

It cannot express a condition. If a weak payment record only matters for shops under two years old, that is a product of two inputs, and it has to be built by hand or found by a model that searches for combinations. Trees do that without being asked.

It cannot rescue a feature set that does not contain the answer. A perfectly fitted line through inputs that miss the mechanism is still blind, and the honest fix is upstream, in How a dataset becomes features.

What it can do, better than most alternatives, is submit to inspection. Every prediction decomposes into a sum of named contributions, so a refusal can be explained to the applicant in the terms that produced it. Rudin argues that for high stakes decisions this is not a nicety: trying to explain an opaque model after the fact is likely to perpetuate bad practices and can cause serious harm, and the better path is designing models that are inherently interpretable. Credit is exactly the setting she has in mind.

The other thing logistic regression does well is produce probabilities that mean something, when it is fitted on enough data and not badly misspecified. That is a claim to verify rather than assume, and the way to verify it is in What probability buys you. A model whose stated 0.9 happens six times in ten will produce a threshold argument that cannot be settled, because both sides are computing expected cost from a number that is not a frequency.

What to do with this in practice

• Fit it first, for the same reason Linear regression is still the baseline gets fitted first. It gives every later classifier a number to beat.
• Report the boundary as an exchange rate between two inputs in their own units. It is the form a subject expert can dispute.
• Quote coefficients as odds multipliers, never as percentage points, and say what the inputs were scaled by.
• Keep the threshold in a configuration file with a named owner and a date, not buried in the training script.
• Plot the residual pattern of misclassified cases against each input. A crescent of errors on one side of the boundary is the shape of a missing squared term or a missing interaction.

The credit team kept both lines for a quarter. The rule refused the sixteen month shop, the model approved it, the shop repaid, and the case was reviewed by hand because someone had written down which line disagreed with which. That is the practical value of a flat boundary: the disagreements are countable.

The next question was what to do about the interactions the line could not hold. The answer starts by asking one question at a time and measuring how much each question is worth.

References

1. Linear Models, Logistic Regression. scikit-learn documentation, version 1.9.0, 2026.
2. Post-tuning the decision threshold for cost-sensitive learning. scikit-learn documentation, version 1.9.0, 2026.
3. Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead. Cynthia Rudin, arXiv, later published in Nature Machine Intelligence, 2019.