Decomposition Pca

pca

CLASS	DESCRIPTION
PrincipalComponentAnalysis	Principal component analysis with optional sample weighting.

FUNCTION	DESCRIPTION
principal_component_analysis	Perform principal component analysis with optional sample weighting.

Classes

PrincipalComponentAnalysis(*, standardize: bool = True, out_features: int = 0)

Bases: Decomposition

Principal component analysis with optional sample weighting.

Performs dimensionality reduction by finding the principal components that explain the most variance in the data. Supports weighted samples for cases where observations have different importance.

The algorithm follows these steps:

1. Center the data by subtracting the (weighted) mean.
2. Optionally standardize features to unit variance.
3. Compute the (weighted) covariance matrix.
4. Perform eigendecomposition of the covariance matrix.
5. Select the top out_features eigenvalues and corresponding eigenvectors.

PARAMETER	DESCRIPTION
standardize	Whether to standardize features before PCA. When True, features are centered and scaled to unit variance. TYPE: bool, optional, by default True DEFAULT: True
out_features	Number of principal components to compute. If 0, computes all components (of shape n_samples). Must be non-negative. TYPE: int, optional, by default 0 DEFAULT: 0

Properties

eigenvalues : torch.Tensor Eigenvalues (explained variance) for each component of shape (out_features, ).

eigenvectors : torch.Tensor Eigenvectors (principal component directions) of shape (in_features, out_features).

covariance : torch.Tensor Covariance matrix computed during fitting of shape (in_features, in_features).

mean : torch.Tensor Feature-wise mean computed during fitting of shape (in_features, ).

std : torch.Tensor | None Feature-wise standard deviation if standardize=True, otherwise None. Shape (in_features, ) when available.

explained_variance : torch.Tensor Explained variance for each component.

cumulative_explained_variance : torch.Tensor Cumulative explained variance for components.

Examples:

Basic PCA usage:

>>> import torch
>>> from spectre.decomposition import PCA
>>> X = torch.randn(100, 10)  # 100 samples, 10 features
>>> pca = PCA(out_features=3)
>>> pca.fit(X)
>>> X_reduced = pca.predict(X)  # Shape: (100, 3)

Weighted PCA:

>>> weights = torch.ones(100)  # Equal weights
>>> pca_weighted = PCA(out_features=5, standardize=True)
>>> pca_weighted.fit(X, weights=weights)
>>> explained_var = pca_weighted.explained_variance

References

[1] Stack Exchange: https://stats.stackexchange.com/a/113488

METHOD	DESCRIPTION
score	Compute reconstruction score (negative reconstruction error).
fit	Fit PCA to the provided data.
predict	Transform data to principal component space.
inverse_transform	Transform data back to original space.

ATTRIBUTE	DESCRIPTION
explained_variance	Proportion of total variance explained by each component. TYPE: Tensor
cumulative_explained_variance	Cumulative proportion of variance explained. TYPE: Tensor
eigenvectors	Eigenvectors of the covariance matrix. TYPE: Tensor
eigenvalues	Eigenvalues of the covariance matrix. TYPE: Tensor
covariance	Covariance matrix of the input data. TYPE: Tensor
mean	Feature-wise mean of the training data. TYPE: Tensor
std	Feature-wise standard deviation (if standardize=True). TYPE: Tensor | None

Source code in spectre/decomposition/pca.py

def __init__(
    self,
    *,
    standardize: bool = True,
    out_features: int = 0,
) -> None:
    super().__init__()

    if not isinstance(standardize, bool):
        raise TypeError(
            f"Expected `standardize` to be bool, got {type(standardize)}."
        )
    self.standardize = standardize

    if not isinstance(out_features, int):
        raise TypeError(
            f"Expected `out_features` to be int, got {type(out_features)}."
        )
    check_in_interval(out_features, "[0, inf)")
    self.out_features = out_features

    # Internal state.
    self._eigenvalues: torch.Tensor | None = None
    self._eigenvectors: torch.Tensor | None = None
    self._covariance: torch.Tensor | None = None
    self._mean: torch.Tensor | None = None
    self._std: torch.Tensor | None = None

Attributes

explained_variance: torch.Tensor property

Proportion of total variance explained by each component.

cumulative_explained_variance: torch.Tensor property

Cumulative proportion of variance explained.

eigenvectors: torch.Tensor property

Eigenvectors of the covariance matrix.

eigenvalues: torch.Tensor property

Eigenvalues of the covariance matrix.

covariance: torch.Tensor property

Covariance matrix of the input data.

mean: torch.Tensor property

Feature-wise mean of the training data.

std: torch.Tensor | None property

Feature-wise standard deviation (if standardize=True).

Functions

score(X: torch.Tensor, weights: torch.Tensor | None = None, target: torch.Tensor | None = None) -> float

Compute reconstruction score (negative reconstruction error).

Returns the negative mean squared reconstruction error, so higher values indicate better reconstruction quality.

PARAMETER	DESCRIPTION
X	Input data of shape (n_samples, in_features). TYPE: Tensor
weights	Sample weights of shape (n_samples, ). TYPE: torch.Tensor | None, optional, by default None DEFAULT: None
target	Not used in PCA (unsupervised method). TYPE: torch.Tensor | None, optional, by default None DEFAULT: None

RETURNS	DESCRIPTION
float	Negative reconstruction error (higher is better).

Source code in spectre/decomposition/pca.py

def score(
    self,
    X: torch.Tensor,
    weights: torch.Tensor | None = None,
    target: torch.Tensor | None = None,
) -> float:
    """
    Compute reconstruction score (negative reconstruction error).

    Returns the negative mean squared reconstruction error, so higher values
    indicate better reconstruction quality.

    Parameters
    ----------
    X : torch.Tensor
        Input data of shape (n_samples, in_features).

    weights : torch.Tensor | None, optional, by default None
        Sample weights of shape (n_samples, ).

    target : torch.Tensor | None, optional, by default None
        Not used in PCA (unsupervised method).

    Returns
    -------
    float
        Negative reconstruction error (higher is better).
    """
    self.check_fitted()

    X_projected = self.predict(X)
    X_reconstructed = self.inverse_transform(X_projected)

    return -reconstruction_error(X, X_reconstructed, reduce="mse")

fit(X: torch.Tensor, weights: torch.Tensor | None = None, target: torch.Tensor | None = None) -> PrincipalComponentAnalysis

Fit PCA to the provided data.

Computes the principal components of the input data, with optional sample weighting. After fitting, the estimator can transform new data.

PARAMETER	DESCRIPTION
X	Training data of shape (n_samples, in_features). TYPE: Tensor
weights	Sample weights of shape (n_samples, ). TYPE: torch.Tensor | None, optional, by default None DEFAULT: None
target	Not used in PCA (unsupervised method). TYPE: torch.Tensor | None, optional, by default None DEFAULT: None

RETURNS	DESCRIPTION
PrincipalComponentAnalysis	Returns self for method chaining.

Source code in spectre/decomposition/pca.py

def fit(
    self,
    X: torch.Tensor,
    weights: torch.Tensor | None = None,
    target: torch.Tensor | None = None,
) -> "PrincipalComponentAnalysis":
    """
    Fit PCA to the provided data.

    Computes the principal components of the input data, with optional sample
    weighting. After fitting, the estimator can transform new data.

    Parameters
    ----------
    X : torch.Tensor
        Training data of shape (n_samples, in_features).

    weights : torch.Tensor | None, optional, by default None
        Sample weights of shape (n_samples, ).

    target : torch.Tensor | None, optional, by default None
        Not used in PCA (unsupervised method).

    Returns
    -------
    PrincipalComponentAnalysis
        Returns self for method chaining.
    """
    self.in_features = X.shape[1]

    if self.out_features == 0:
        self.out_features = min(X.shape[0], X.shape[1])

    if weights is None:
        weights = torch.ones(size=(len(X),), dtype=X.dtype, device=X.device)

    result = _principal_component_analysis_impl(
        X,
        weights=weights,
        standardize=self.standardize,
        out_features=0,  # Compute all, slice later
    )

    self.is_fitted = True
    self._eigenvalues = result.eigenvalues
    self._eigenvectors = result.eigenvectors
    self._covariance = result.kernel

    # Store mean and std for transform/inverse_transform
    sum_weights = weights.sum()
    self._mean = (weights.unsqueeze(1) * X).sum(dim=0) / sum_weights

    if self.standardize:
        X_centered = X - self._mean
        weighted_var = (weights.unsqueeze(1) * X_centered**2).sum(
            dim=0
        ) / sum_weights
        eps = torch.finfo(X.dtype).eps
        self._std = torch.sqrt(weighted_var + eps)
    else:
        self._std = None

    return self

predict(X: torch.Tensor) -> torch.Tensor

Transform data to principal component space.

Projects the input data onto the fitted principal components, effectively performing dimensionality reduction.

PARAMETER	DESCRIPTION
X	Input data of shape (n_samples, in_features). TYPE: Tensor

RETURNS	DESCRIPTION
Tensor	Transformed data of shape (n_samples, out_features).

Source code in spectre/decomposition/pca.py

def predict(self, X: torch.Tensor) -> torch.Tensor:
    """
    Transform data to principal component space.

    Projects the input data onto the fitted principal components, effectively
    performing dimensionality reduction.

    Parameters
    ----------
    X : torch.Tensor
        Input data of shape (n_samples, in_features).

    Returns
    -------
    torch.Tensor
        Transformed data of shape (n_samples, out_features).
    """
    self.check_fitted()

    # Center the data
    X_centered = X - self._mean

    # Standardize if needed
    if self._std is not None:
        X_centered = X_centered / self._std

    # Project onto principal components
    return torch.matmul(X_centered, self._eigenvectors[:, : self.out_features])

inverse_transform(X: torch.Tensor) -> torch.Tensor

Transform data back to original space.

Reconstructs the original features from PCA components. Note that this is an approximation if not all components were retained (out_features < in_features).

PARAMETER	DESCRIPTION
X	Transformed data of shape (n_samples, out_features). TYPE: Tensor

RETURNS	DESCRIPTION
Tensor	Reconstructed data of shape (n_samples, in_features).

Source code in spectre/decomposition/pca.py

def inverse_transform(self, X: torch.Tensor) -> torch.Tensor:
    """
    Transform data back to original space.

    Reconstructs the original features from PCA components.
    Note that this is an approximation if not all components
    were retained (out_features < in_features).

    Parameters
    ----------
    X : torch.Tensor
        Transformed data of shape (n_samples, out_features).

    Returns
    -------
    torch.Tensor
        Reconstructed data of shape (n_samples, in_features).
    """
    self.check_fitted()

    # Project back to original space
    X_reconstructed = X @ self._eigenvectors[:, : self.out_features].T

    # Unstandardize if needed
    if self._std is not None:
        X_reconstructed = X_reconstructed * self._std

    # Add back the mean
    X_reconstructed = X_reconstructed + self._mean

    return X_reconstructed

Functions

principal_component_analysis(X: torch.Tensor, weights: torch.Tensor | None = None, standardize: bool = True, out_features: int = 0, eps: float = torch.finfo(torch.float32).eps) -> DecompositionResult

Perform principal component analysis with optional sample weighting.

Computes the principal components of the input data using eigendecomposition of the covariance matrix. Supports weighted samples and optional standardization.

PARAMETER	DESCRIPTION
X	Input data of shape (n_samples, in_features). Must be a 2D tensor. TYPE: Tensor
weights	Sample weights of shape (n_samples, ). If None, all samples have equal weight. When provided, must have same length as X. TYPE: torch.Tensor | None, optional, by default None DEFAULT: None
standardize	Whether to standardize features before PCA. When True, features are centered and scaled to unit variance. TYPE: bool, optional, by default True DEFAULT: True
out_features	Number of principal components to compute. If 0, computes all components. Must be non-negative and not exceed min(n_samples, in_features). TYPE: int, optional, by default 0 DEFAULT: 0
eps	Small value added to std for numerical stability. TYPE: float, optional, by default torch.finfo(torch.float32).eps DEFAULT: eps

RETURNS	DESCRIPTION
DecompositionResult	A NamedTuple containing: eigenvalues: Explained variance for each component of shape (out_features, ) eigenvectors: Principal components of shape (in_features, out_features) kernel: Covariance matrix (weighted) of shape (in_features, in_features)

RAISES	DESCRIPTION
TypeError	If input types are incorrect.
ValueError	If tensor shapes are incompatible or parameters are out of valid ranges.

Examples:

Basic PCA on random data:

>>> import torch
>>> X = torch.randn(50, 10)
>>> eigenvalues, eigenvectors, cov_matrix = pca(X, out_features=3)

Weighted PCA:

>>> weights = torch.rand(50)  # Random weights
>>> result = principal_component_analysis(X, weights=weights, standardize=True)
>>> result.eigenvalues.shape
torch.Size([10])
>>> result.eigenvectors.shape
torch.Size([10, 10])
>>> result.covariance.shape
torch.Size([10, 10])

References

[1] Stack Exchange: https://stats.stackexchange.com/a/113488

Source code in spectre/decomposition/pca.py

def principal_component_analysis(
    X: torch.Tensor,
    weights: torch.Tensor | None = None,
    standardize: bool = True,
    out_features: int = 0,
    eps: float = torch.finfo(torch.float32).eps,
) -> DecompositionResult:
    """
    Perform principal component analysis with optional sample weighting.

    Computes the principal components of the input data using eigendecomposition
    of the covariance matrix. Supports weighted samples and optional standardization.

    Parameters
    ----------
    X : torch.Tensor
        Input data of shape (n_samples, in_features). Must be a 2D tensor.

    weights : torch.Tensor | None, optional, by default None
        Sample weights of shape (n_samples, ). If None, all samples have equal weight.
        When provided, must have same length as X.

    standardize : bool, optional, by default True
        Whether to standardize features before PCA. When True, features are
        centered and scaled to unit variance.

    out_features : int, optional, by default 0
        Number of principal components to compute. If 0, computes all components.
        Must be non-negative and not exceed min(n_samples, in_features).

    eps : float, optional, by default torch.finfo(torch.float32).eps
        Small value added to std for numerical stability.

    Returns
    -------
    DecompositionResult
        A NamedTuple containing:

        - eigenvalues: Explained variance for each component of shape
            (out_features, )
        - eigenvectors: Principal components of shape (in_features, out_features)
        - kernel: Covariance matrix (weighted) of shape (in_features,
            in_features)

    Raises
    ------
    TypeError
        If input types are incorrect.
    ValueError
        If tensor shapes are incompatible or parameters are out of valid ranges.

    Examples
    --------
    Basic PCA on random data:

    >>> import torch
    >>> X = torch.randn(50, 10)
    >>> eigenvalues, eigenvectors, cov_matrix = pca(X, out_features=3)

    Weighted PCA:

    >>> weights = torch.rand(50)  # Random weights
    >>> result = principal_component_analysis(X, weights=weights, standardize=True)
    >>> result.eigenvalues.shape
    torch.Size([10])
    >>> result.eigenvectors.shape
    torch.Size([10, 10])
    >>> result.covariance.shape
    torch.Size([10, 10])

    References
    ----------
    [1] Stack Exchange: https://stats.stackexchange.com/a/113488
    """
    check_2d(X)

    if not isinstance(standardize, bool):
        raise TypeError(f"Expected `standardize` to be bool, got {type(standardize)}.")

    if not isinstance(out_features, int):
        raise TypeError(f"Expected `out_features` to be int, got {type(out_features)}.")

    n_eigen = min(X.shape[0], X.shape[1])
    check_in_interval(out_features, f"[0, {n_eigen}]")

    if out_features == 0:
        out_features = n_eigen

    if weights is None:
        weights = torch.ones(size=(len(X),), dtype=X.dtype, device=X.device)
    else:
        if not isinstance(weights, torch.Tensor):
            raise TypeError(
                f"Expected `weights` to be torch.Tensor, got {type(weights)}."
            )

    check_same_len(X, weights)

    return _principal_component_analysis_impl(
        X,
        weights=weights,
        standardize=standardize,
        out_features=out_features,
        eps=eps,
    )