Using the missing_inds function to label interpolated values

  run this example as a notebook — or grab the .ipynb (also linked at the bottom of the page)

If you have data with missing values, Hypertools will try to interpolate them using PPCA. To visualize how well its doing, you can use the missing_inds function and then highlight the values that were interpolated. Here, we generated some synthetic data, removed some values, and then plotted the original data, data with missing values and highlighted the missing datapoints with stars.

/home/docs/checkouts/readthedocs.org/user_builds/hypertools/checkouts/latest/hypertools/tools/format_data.py:492: UserWarning: Missing data: filling missing values with PPCA (observed values are preserved exactly; only the NaN entries are reconstructed). Pass impute= to choose a different imputation model -- see hypertools.impute.
  warnings.warn('Missing data: filling missing values '
/home/docs/checkouts/readthedocs.org/user_builds/hypertools/checkouts/latest/hypertools/plot/backend.py:1353: UserWarning: Failed to switch to any interactive backend (TkAgg, QtAgg, Qt5Agg, Qt4Agg, GTK4Agg, GTK3Agg, WXAgg). Falling back to 'Agg'.
  warnings.warn(BACKEND_WARNING)

# Code source: Andrew Heusser
# License: MIT

# import
from scipy.linalg import toeplitz
import numpy as np
from copy import copy
import hypertools as hyp

# simulate data (seeded so the figure is reproducible)
np.random.seed(123)
K = 10 - toeplitz(np.arange(10))
data1 = np.cumsum(np.random.multivariate_normal(np.zeros(10), K, 250), axis=0)
data2 = copy(data1)

# randomly remove 5% of the data
missing = .05
inds = [(i,j) for i in range(data1.shape[0]) for j in range(data1.shape[1])]
missing_data = [inds[i] for i in np.random.choice(len(inds), int(len(inds)*missing), replace=False)]
for i,j in missing_data:
    data2[i,j]=np.nan

# reduce the data
data1_r,data2_r = hyp.reduce([data1, data2], ndims=3)

# pull out missing inds
missing_inds = hyp.tools.missing_inds(data2)
missing_data = data2_r[missing_inds, :]

# plot
hyp.plot([data1_r, data2_r, missing_data], ['-', '--', '*'],
         legend=['Full', 'Missing', 'Missing Points'])