hypertools.analyze

hypertools.analyze(data, manip=None, normalize=None, reduce=None, ndims=None, align=None, cluster=None, pipeline=None, return_model=False, internal=False, impute=None, random_state=None)

Wrapper function for manip -> normalize -> reduce -> align -> cluster transformations (the canonical 1.0 pipeline order, GH #153): each requested stage is applied to the previous stage’s output, in that order (e.g. normalize= output feeds reduce=, whose output feeds align=).

Parameters:

datanumpy array, pandas df, or list/tuple of arrays/dfs

The data to analyze. Each dataset must be 2-D (observations x features); 1-D vectors are treated as single-feature columns. A tuple of datasets is treated exactly like a list. None raises a TypeError and an empty list raises a ValueError (matching every other dispatcher), even when no stage kwargs are given.

manipmodel spec, False, or None

Cross-module stage kwarg (GH #138): a hypertools.manip spec (a registry name, dict spec, class/instance, or a list chaining several – see hypertools.manip.manip.manip), applied FIRST (the manip stage runs before normalize/reduce/align/cluster in the canonical order). False or None (default) skips this stage.

normalizestr or False or None

If set to ‘across’, the columns of the input data will be z-scored across lists. That is, the z-scores will be computed with respect to column n across all arrays passed in the list. If set to ‘within’, the columns will be z-scored within each list that is passed. If set to ‘row’, each row of the input data will be z-scored. If set to False or None (default), the input data will be returned with no z-scoring.

reducestr, dict, class, instance, fitted Reducer, False, or None

Decomposition/manifold learning model to use, or False/None (default) to SKIP dimensionality reduction entirely (in which case ndims= has no effect – see ndims below). Models supported: PCA, IncrementalPCA, SparsePCA, MiniBatchSparsePCA, KernelPCA, FastICA, FactorAnalysis, TruncatedSVD, DictionaryLearning, MiniBatchDictionaryLearning, TSNE, Isomap, SpectralEmbedding, LocallyLinearEmbedding, MDS, and UMAP; the mixture models GaussianMixture, BayesianGaussianMixture, LatentDirichletAllocation and NMF (GH #174); and the torch autoencoders Autoencoder, DeepAutoencoder, SparseAutoencoder, ConvolutionalAutoencoder, SequenceAutoencoder and VariationalAutoencoder (GH #162, pip install “hypertools[torch]”). Can be passed as a string, or for finer control as a dictionary, e.g. reduce={‘model’: ‘PCA’, ‘kwargs’: {‘whiten’: True}}. See scikit-learn model docs for details on parameters supported for each model.

ndimsint

Number of dimensions to reduce to. Only takes effect when reduce= is also given: if reduce is left at its default of None (or is False), no reduction runs and ndims= is ignored (a UserWarning is emitted so the request does not silently no-op).

alignstr, dict, False, or None

Alignment model to bring a list of datasets into a shared space. If str, ‘hyper’ (hyperalignment) or ‘SRM’ (shared response model). You can also pass a dictionary for finer control, where ‘model’ specifies the model and ‘kwargs’ holds its parameters, e.g. align={‘model’: ‘HyperAlign’, ‘kwargs’: {‘n_iter’: 10}}. If False or None, no alignment is applied (default: None).

clustermodel spec, False, or None

Cross-module stage kwarg (GH #138): a hypertools.cluster spec (a registry name, dict spec, or class/instance – see hypertools.cluster.cluster.cluster), applied LAST (after align, the canonical order). analyze still returns the TRANSFORMED DATA (not cluster labels) when cluster= is given; the cluster labels themselves are retrievable from the fitted hypertools.Pipeline’s ‘cluster’ step (model.named_steps[‘cluster’]) when return_model=True is also passed – pass the RETURNED transformed data back through that step’s .transform to recover the labels (this works for every clusterer, including hard clusterers such as DBSCAN / AgglomerativeClustering that have no out-of-sample predict). For a list of datasets, .transform returns one flat label sequence over the row-concatenated data; to get labels split PER dataset, split that flat sequence by each dataset’s row count, e.g. np.split(labels, np.cumsum([len(d) for d in data])[:-1]) (hypertools.cluster likewise returns one flat label sequence for a list input, because the datasets are row-stacked before clustering). False or None (default) skips this stage.

pipelinehypertools.Pipeline or None

A previously-FITTED Pipeline (e.g. from an earlier analyze(…, return_model=True) call) to apply to data via .transform – reusing its learned parameters rather than re-fitting them (GH #227). Mutually exclusive with manip=/normalize=/reduce=/align=/cluster= (all must be left at their default of None) – passing both raises ValueError naming the conflicting kwarg(s). internal=/impute= are still honored (internal=True still guarantees a list is returned, even for a single-dataset data; impute= overrides the PPCA missing-data fill exactly as on the fitting paths). ndims= is IGNORED on this path (with a UserWarning): a fitted Pipeline applies its reduce stage exactly as fitted – re-fit with analyze(…, reduce=…, ndims=…, return_model=True) to change the dimensionality. Reusing a pipeline whose last step is ‘cluster’ returns the TRANSFORMED DATA (matching the cluster= contract above, not the labels); recover the labels via pipeline.named_steps[‘cluster’].transform(returned_data) (default: None).

return_modelbool

If True, also return the fitted model: a fitted hypertools.Pipeline covering whichever stages ran (default: False). Using ONLY the legacy normalize=/reduce=/align= kwargs (no manip=/ cluster=/pipeline=) with return_model=False (the default) runs the exact same code path analyze has always used, so every existing caller (hyp.plot, hyp.load, pre-1.0 scripts) is byte-identical. return_model=True, or passing manip=/cluster=, routes through hypertools.core.pipeline.build_pipeline instead (needed to hand back a genuinely fit-once-reusable Pipeline); impute= is honored either way, but internal= is only otherwise meaningful for the legacy path – see pipeline= above for how it is handled on that path.

internalbool

(Internal use, e.g. by hyp.plot) if True, always return a list even when the input was a single dataset (default: False).

imputestr, dict, class, class instance or None

Overrides the default PPCA missing-data fill (applied at the format_data stage, before any pipeline stage runs) with a different hypertools.impute model, e.g. ‘Kalman’, ‘KNNImputer’. Honored on every path – with or without normalize=, and with pipeline= (default: None, i.e. PPCA – byte-compatible with pre-1.0 behavior).

random_stateint, numpy.random RandomState/Generator, or None

Seed (or seeded generator) threaded through to the reduce= and cluster= stages so stochastic models (e.g. TSNE, MDS, KMeans) give reproducible results across calls (default: None).

Returns:

analyzed_datalist of numpy arrays (or a single array)

The processed data: for a LIST of datasets, a list with one entry per dataset. For a SINGLE dataset, normalize=/reduce=-only calls return a single array, while combinations that include align= return a list of length 1 (alignment always operates on – and returns – a list); manip=-only calls return the manipulator’s own output type (typically pandas DataFrames). Pass internal=True to guarantee a list regardless of input shape. If return_model=True, an (analyzed_data, model) tuple is returned instead.

Examples

>>> import numpy as np
>>> import hypertools as hyp
>>> x = np.cumsum(np.random.default_rng(0).standard_normal((40, 5)),
...               axis=0)
>>> analyzed = hyp.analyze(x, normalize='within', reduce='PCA', ndims=3)
>>> analyzed.shape
(40, 3)