11  Parametric Regression

A parametric model is a function that has a fixed form, but parameters which can be tuned. Polynomial functions are parametric when we tune their coefficients. Take, for example, the quadratic: \[ y = a x^2 + bx + c\] Changing the values of \(a\), \(b\), and \(c\) will change the shape of the quadratic. If we have data we assume to be quadratic, we can turn the “dial” on these parameters until the model has a good fit to our data.

This section utilizes NumPy, Pyplot, and the curve_fit function from scipy:

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

11.1 Parametric Models

If we can write a parametric model as a Python function, then we can attempt to fit to data by optimizing the parameters using scipy.optimize’s curve_fit function.

To demonstrate this, we can generate synthetic data from a noisy sine wave.

# Sample data
x = np.linspace(0, 10, 50)
y = 3 * np.sin(2 * x + 0.5) + 1 + np.random.normal(0, 0.5, len(x))

plt.scatter(x,y)
plt.show()

That’s a lot of noise, but we can still make out the structure of the wave form. Assuming the underlying structure is sinusoidal, we might define our model as the following function.

# A sinusoidal model, y = A*sin(Bx+C)+D
def sine_model(x, A, B, C, D):
    return A * np.sin(B * x + C) + D

This model is a function that takes input values as x, along with four parameters, A, B, C, D. \[\hat{Y} = A \sin(B X +C) +D\]

We initialize the parameters with a guess, then the curve_fit function will iteratively improve the values of the parameters to fit the data. We can put any values in for the initial guess, so I will just use \(1\) for all four parameters.

initial_guess = [1, 1, 1, 1]
params, _ = curve_fit(sine_model, x, y, p0=initial_guess)
print(params)

plt.plot(x, sine_model(x, params[0], params[1], params[2], params[3]),'k')
plt.scatter(x,y)
[0.82396469 1.1089773  1.97790422 1.23861617]

That’s a terrible fit. What likely happened is that curve_fit got stuck in a local minimum of error and could not improve. This is why the initial guess of parameters is crtically important.

Examining our data, we can estimate parameter values for a much better initial guess.

# Initial guesses make a difference!
initial_guess = [3, 2, 1, 1]
params, _ = curve_fit(sine_model, x, y, p0=initial_guess)
print(params)

plt.plot(x, sine_model(x, params[0], params[1], params[2], params[3]),'k')
plt.scatter(x,y)
[2.98103245 1.99997555 0.50621157 0.97539252]

Exercises

  1. Here is NOAA Tide Data from Nantucket Harbor, Massachusetts.
    1. Extract and plot the data.
    2. Decide on a model for the data.
    3. Use curve_fit to fit the model to the data, and plot the model with data.
  2. Atmospheric CO\(_2\) data from Maunakea Observatories is reported at https://gml.noaa.gov/webdata/ccgg/trends/co2/co2_mm_mlo.txt.
    1. Extract and plot the data.
    2. Decide on a model for the data.
    3. Use curve_fit to fit the model to the data, and plot the model with data.