In [1]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scipy import stats
In [56]:
def sigma(t):
    return 1 / (1 + np.e**(-t))

def plot_logistic():
    t = np.linspace(-5, 5, 100)
    plt.plot(t, sigma(t), lw=2, color='darkblue') 
    plt.plot([-5, 5], [0, 0], lw=1, color='grey')
    plt.xticks(range(-5, 6))
    plt.yticks(np.arange(0, 1.1, 0.25))
    # axes and labels
    plt.xlabel('$t$', fontsize=15)
    plt.ylabel('$\sigma(t)$', fontsize=15, rotation=0)

    
def plot_logistic_symmetry():
    plot_logistic()
    plt.plot([-5, 5], [1, 1], lw=1, color='grey')
    plt.plot([2, 2], [sigma(2), 1], lw=2, color='red', linestyle='dotted')
    plt.plot([-2, -2], [0, sigma(-2)], lw=2, color='green', linestyle='dotted')

Properties of the Logistic Function

A. Adhikari for Data 100 Spring 2020

These are elementary derivations of:

  • the properties listed on Slide 24 of Lecture 23
  • the multiplicative effect stated in Slide 30

Definition

$$ \sigma(t) ~ = ~ \frac{1}{1 + e^{-t}} ~~~~~~ - \infty < t < \infty $$

Alternative expression: $$ \sigma(t) ~ = ~ \frac{e^t}{e^t} \cdot\frac{1}{1 + e^{-t}} ~ = ~ \frac{e^t}{e^t + 1} ~ = ~ \frac{e^t}{1 + e^t} $$

Values:

  • $0 < \sigma(t) < 1$
  • $\sigma(0) = 0.5$
  • $\sigma(t) \to 0$ as $t \to -\infty$
  • $\sigma(t) \to 1$ as $t \to \infty$
In [37]:
plot_logistic()

Inverse

On Slide 22, $\sigma(t)$ was defined by inverting $t = \log\big{(}\frac{p}{1-p}\big{)}$ for $0 < p < 1$. So

$$ \sigma^{-1}(p) ~ = ~ t ~ = ~ \log\big{(}\frac{p}{1-p}\big{)} ~~~~~ 0 < p < 1 $$

Reflection and Symmetry

$$ 1 - \sigma(t) ~ = ~ 1 - \frac{1}{1+e^{-t}} ~ = ~ \frac{1 + e^{-t} - 1}{1+e^{-t}} ~ = ~ \frac{e^{-t}}{1+e^{-t}} ~ = ~ \sigma(-t) $$

by the alterative expression for $\sigma(-t)$.

In the figure below, the length of the red segment is $1 - \sigma(2)$ and the length of the green segment is $\sigma(-2)$. The two lengths are equal.

In [57]:
plot_logistic_symmetry()

Derivative

By the Chain Rule, $$ \frac{d}{dt} \sigma(t) ~ = ~ -\big{(}\frac{1}{1 + e^{-t}}\big{)}^2 \cdot (-e^{-t}) ~ = ~ \frac{1}{1 + e^{-t}} \cdot \frac{e^{-t}}{1 + e^{-t}} ~ = ~ \sigma(t)\sigma(-t) $$

Multiplicative Effect (Slide 30)

Suppose the odds ratio is $$ OR ~ = ~ \frac{p}{1-p} ~ = ~ e^{\theta_0 + \theta_1 x} $$ Increase $x$ by 1. Then the new odds ratio is $$ OR_{new} ~ = ~ ~ e^{\theta_0 + \theta_1 (x+1)} $$ Compare relative sizes: $$ \frac{OR_{new}}{OR} ~ = ~ \frac{e^{\theta_0 + \theta_1 (x+1)}}{e^{\theta_0 + \theta_1 x}} ~ = ~ \frac{e^{\theta_0 + \theta_1x + \theta_1}}{e^{\theta_0 + \theta_1 x}} ~ = ~ e^{\theta_1} $$ So $$ OR_{new} ~ = ~ e^{\theta_1}OR $$