Confidence intervals in statistics.

A confidence interval (CI) defines the measure of uncertainty in any particular statistic with a certain margin of error.
Or you can say that it is a range of values that’s likely to include a population value with a certain degree of confidence. As a practical example it tells how confident you can be that the results from a poll or survey reflect what you would expect to find if it were possible to survey the entire population.

Confidence intervals by formula.

Confidence levels are expressed as a percentage ,for example a 95% confidence level.

CI = x̄ ± z * ơ / √n


x̄: Sample Mean
z: Confidence Coefficient
ơ: Population Standard Deviation
n: Sample Size

The factors affecting the width of the CI include the confidence level, the sample size, and the variability in the sample. Larger samples produce narrower confidence intervals. Greater variability in the sample will produce wider confidence intervals. A higher confidence level will produce a wider confidence interval.

Generating initial data for CI calculation:

import matplotlib.pyplot as plt
import numpy as np
import scipy.stats as stats
from matplotlib.patches import Polygon

## simulate data

popN = int(1e7)  # lots and LOTS of data!!

# the data (note: non-normal!)
population = (4*np.random.randn(popN))**2

# we can calculate the exact population mean
popMean = np.mean(population)

# let's see it
fig,ax = plt.subplots(2,1,figsize=(6,4))

# only plot every 1000th sample
ax[0].set_xlabel('Data index')
ax[0].set_ylabel('Data value')

ax[1].set_xlabel('Data value')

Generating initial data for CI calculation

Drawing a random sample with confidence intervals:

# parameters
samplesize = 40
confidence = 95 # in percent

# compute sample mean
randSamples = np.random.randint(0,popN,samplesize)
samplemean  = np.mean(population[randSamples])
samplestd   = np.std(population[randSamples],ddof=1)

# compute confidence intervals
citmp = (1-confidence/100)/2
confint = samplemean + stats.t.ppf([citmp, 1-citmp],samplesize-1) * samplestd/np.sqrt(samplesize)

# graph everything
fig,ax = plt.subplots(1,1)

y = np.array([ [confint[0],0],[confint[1],0],[confint[1],1],[confint[0],1] ])
p = Polygon(y,facecolor='g',alpha=.3)

# now add the lines
ax.plot([popMean,popMean],[0, 1.5],'k:',linewidth=2)
ax.plot([samplemean,samplemean],[0, 1],'r--',linewidth=3)
ax.set_xlim([popMean-30, popMean+30])
ax.set_xlabel('Data values')
ax.legend(('Population mean','Sample mean','%g%% CI region'%confidence))

Drawing a random sample with confidence intervals

Distribution of null hypothesis values:

## repeat for large number of samples

# parameters
samplesize = 50
confidence = 95  # in percent
numExperiments = 5000

withinCI = np.zeros(numExperiments)

# part of the CI computation can be done outside the loop
citmp = (1-confidence/100)/2
CI_T  = stats.t.ppf([citmp, 1-citmp],samplesize-1)
sqrtN = np.sqrt(samplesize)

for expi in range(numExperiments):
    # compute sample mean and CI as above
    randSamples = np.random.randint(0,popN,samplesize)
    samplemean  = np.mean(population[randSamples])
    samplestd   = np.std(population[randSamples],ddof=1)
    confint     = samplemean + CI_T * samplestd/sqrtN
    # determine whether the True mean is inside this CI
    if popMean>confint[0] and popMean<=confint[1]:
        withinCI[expi] = 1

print('%g%% of sample C.I.''s contained the true population mean.'%(100*np.mean(withinCI)))

OUT: 91.98% of sample C.I.s contained the true population mean.

See also related topics: