Kernel T

t

CLASS	DESCRIPTION
TKernelResult	Result container for t-kernel.

FUNCTION	DESCRIPTION
t_kernel	Compute T-distribution kernel matrix from distance matrix.

Classes

TKernelResult

Bases: NamedTuple

Result container for t-kernel.

ATTRIBUTE	DESCRIPTION
kernel	Computed T-distribution kernel matrix. TYPE: Tensor
alpha	Alpha parameter value used in computation. TYPE: Tensor
beta	Beta parameter value used in computation. TYPE: Tensor

TKernel(alpha: torch.Tensor = torch.tensor(1.0), beta: torch.Tensor = torch.tensor(1.0), learnable: bool = False, learnable_log: bool = True, normalization: KernelNormalizer | None = None, dtype: torch.dtype = torch.float32)

Bases: Kernel

T-distribution kernel for computing pairwise similarity matrices.

Implements the t-distribution kernel function widely used in manifold learning and dimensionality reduction [1]. The kernel provides robust handling of outliers compared to Gaussian kernels due to its heavier tails, making it particularly suitable for datasets with non-uniform density or noise.

The kernel transforms pairwise distances into similarity values using the formula: \(K(d_{kl}) = (1 + \alpha d_{kl}^2)^{-\beta}\) where \(d_{kl}\) is the distance between samples \(k\) and \(l\), \(\alpha > 0\) controls the kernel bandwidth (scale), and \(\beta > 0\) controls the tail heaviness (shape).

PARAMETER	DESCRIPTION
alpha	Scale parameter \(\alpha > 0\) controlling kernel bandwidth. Higher values produce narrower kernels with faster decay. TYPE: torch.Tensor, optional, by default torch.tensor(1.0) DEFAULT: tensor(1.0)
beta	Shape parameter \(\beta > 0\) controlling tail behavior. Higher values produce Gaussian-like behavior with exponential decay, while lower values emphasize heavy tails with polynomial decay. TYPE: torch.Tensor, optional, by default torch.tensor(1.0) DEFAULT: tensor(1.0)
learnable	Whether alpha and beta should be learnable parameters during training. When True, parameters become torch.nn.Parameter objects with gradients. TYPE: bool, optional, by default False DEFAULT: False
learnable_log	Whether to parameterize learnable parameters in log space for numerical stability and to ensure positivity constraints. TYPE: bool, optional, by default True DEFAULT: True
normalization	Kernel normalization to apply after computing the kernel matrix. Use to create transition matrices for spectral methods (e.g., diffusion maps). If None, returns unnormalized kernel values. TYPE: KernelNormalizer | None, optional, by default None DEFAULT: None
dtype	Data type for internal tensor computations. TYPE: torch.dtype, optional, by default torch.float32 DEFAULT: float32

ATTRIBUTE	DESCRIPTION
alpha_constant	Constant alpha value when learnable=False. TYPE: Tensor
beta_constant	Constant beta value when learnable=False. TYPE: Tensor
alpha_learnable	Learnable alpha parameter when learnable=True. TYPE: Parameter
beta_learnable	Learnable beta parameter when learnable=True. TYPE: Parameter

Examples:

Basic usage with fixed parameters:

>>> import torch
>>> from spectre.kernel import TKernel
>>> # Compute distance matrix
>>> X = torch.randn(100, 10)
>>> D = torch.cdist(X, X)
>>>
>>> # Create kernel and compute similarity matrix
>>> kernel = TKernel(alpha=1.0, beta=2.0)
>>> K = kernel(D)
>>> K.shape
torch.Size([100, 100])

Using normalization for spectral methods:

>>> from spectre.kernel import TKernel, KernelNormalizer
>>> normalization = KernelNormalizer(norm_type="markov_symmetric")
>>> kernel = TKernel(alpha=1.0, beta=1.5, normalization=normalization)
>>> K_normalized = kernel(D)
>>> # Row sums are approximately 1
>>> K_normalized.sum(dim=1)
tensor([1.0000, 1.0000, ..., 1.0000])

Learnable parameters for neural networks:

>>> kernel = TKernel(alpha=0.5, beta=1.5, learnable=True, learnable_log=True)
>>> K = kernel(D)
>>> loss = K.sum()
>>> loss.backward()
>>> # Gradients are computed for alpha and beta
>>> kernel.alpha_learnable.grad is not None
True

Tensor-valued parameters for adaptive kernels:

>>> # Different alpha for each sample
>>> alpha_tensor = torch.ones(100, 1) * 0.5
>>> kernel = TKernel(alpha=alpha_tensor, beta=1.0, learnable=True)
>>> K = kernel(D)
>>> kernel.alpha_learnable.shape  # (100, 1) - per-sample alpha

Comparing different parameter settings:

>>> # Heavy-tailed kernel (robust to outliers)
>>> kernel_heavy = TKernel(alpha=1.0, beta=0.5)
>>> # Gaussian-like kernel (sensitive to outliers)
>>> kernel_light = TKernel(alpha=1.0, beta=5.0)

Notes

• The t-distribution kernel is particularly effective for noisy datasets with outliers due to its heavy-tailed behavior
• Higher \(eta\) values produce Gaussian-like exponential decay: :math:K \approx \exp(-\alpha \beta d^2) for large :math:\beta
• Lower \(eta\) values emphasize polynomial decay with heavier tails
• The Cauchy kernel is a special case: \(lpha=1, eta=1\)
• Unlike Gaussian kernels, t-distribution kernels maintain non-zero similarity even for large distances, providing better connectivity in sparse data

METHOD	DESCRIPTION
forward_step	Compute unnormalized T-distribution kernel matrix.
get_kernel_params	Return current alpha and beta parameters.

Source code in spectre/kernel/t.py

def __init__(
    self,
    alpha: torch.Tensor = torch.tensor(1.0),
    beta: torch.Tensor = torch.tensor(1.0),
    learnable: bool = False,
    learnable_log: bool = True,
    normalization: KernelNormalizer | None = None,
    dtype: torch.dtype = torch.float32,
) -> None:
    super().__init__(
        learnable=learnable,
        learnable_log=learnable_log,
        normalization=normalization,
        dtype=dtype,
    )

    # Validate and convert parameters to tensors
    if not isinstance(alpha, torch.Tensor):
        raise TypeError(
            f"Parameter `alpha` must be a torch.Tensor, got {type(alpha)}."
        )
    if not isinstance(beta, torch.Tensor):
        raise TypeError(
            f"Parameter `beta` must be a torch.Tensor, got {type(beta)}."
        )

    alpha = alpha.to(dtype=self.dtype)
    beta = beta.to(dtype=self.dtype)

    if learnable:
        # Create learnable parameters with same shape as provided tensors
        if self.learnable_log:
            self.alpha_learnable = torch.nn.Parameter(
                torch.log(alpha), requires_grad=True
            )
            self.beta_learnable = torch.nn.Parameter(
                torch.log(beta), requires_grad=True
            )
        else:
            self.alpha_learnable = torch.nn.Parameter(
                alpha.clone(), requires_grad=True
            )
            self.beta_learnable = torch.nn.Parameter(
                beta.clone(), requires_grad=True
            )
    else:
        self.alpha_constant = alpha
        self.beta_constant = beta

Functions

forward_step(D: torch.Tensor, weights: torch.Tensor | None = None, **kwargs) -> torch.Tensor

Compute unnormalized T-distribution kernel matrix.

Delegates to t_kernel() function for computation.

PARAMETER	DESCRIPTION
D	Pairwise distance matrix of shape (n_samples, n_samples) or (n_samples, m_samples). TYPE: Tensor
weights	Sample weights of shape (n_samples,). TYPE: torch.Tensor | None, optional, by default None DEFAULT: None
**kwargs	Additional keyword arguments passed to the kernel. TYPE: dict DEFAULT: {}

RETURNS	DESCRIPTION
Tensor	T-distribution kernel (similarity) matrix with same shape as input.

Source code in spectre/kernel/t.py

def forward_step(
    self, D: torch.Tensor, weights: torch.Tensor | None = None, **kwargs
) -> torch.Tensor:
    """
    Compute unnormalized T-distribution kernel matrix.

    Delegates to `t_kernel()` function for computation.

    Parameters
    ----------
    D : torch.Tensor
        Pairwise distance matrix of shape (n_samples, n_samples) or
        (n_samples, m_samples).

    weights : torch.Tensor | None, optional, by default None
        Sample weights of shape (n_samples,).

    **kwargs : dict
        Additional keyword arguments passed to the kernel.

    Returns
    -------
    torch.Tensor
        T-distribution kernel (similarity) matrix with same shape as input.

    """
    if self.learnable:
        # Pass learnable parameters directly
        alpha_arg = self._get_param(self.alpha_learnable)
        beta_arg = self._get_param(self.beta_learnable)
    else:
        alpha_arg = self.alpha_constant
        beta_arg = self.beta_constant

    # Delegate to t_kernel function (single source of truth)
    result = t_kernel(D, alpha=alpha_arg, beta=beta_arg, dtype=self.dtype)

    return result.kernel

get_kernel_params() -> dict[str, torch.Tensor]

Return current alpha and beta parameters.

For compact scalar statistics, use get_param_summary() instead.

RETURNS	DESCRIPTION
dict[str, Tensor]	Dictionary with alpha and beta keys containing current parameter values (tensors).

Examples:

Basic usage with constant parameters:

>>> kernel = TKernel(alpha=torch.tensor(1.0), beta=torch.tensor(2.0))
>>> params = kernel.get_kernel_params()
>>> params
{'alpha': tensor(1.), 'beta': tensor(2.)}

Learnable parameters:

>>> kernel = TKernel(
...     alpha=torch.tensor(1.0),
...     beta=torch.tensor(2.0),
...     learnable=True,
... )
>>> params = kernel.get_kernel_params()
>>> "alpha" in params and "beta" in params
True

Logging kernel parameters during training with PyTorch Lightning:

>>> import torch
>>> from spectre.parametric import SpectralMap
>>> from spectre.kernel import TKernel
>>> from spectre.utils.callbacks import MetricsCallback, kernel_params_extractor
>>> from pytorch_lightning import Trainer
>>>
>>> # Create spectral map with learnable T-kernel
>>> kernel = TKernel(
...     alpha=torch.tensor(0.5),
...     beta=torch.tensor(1.5),
...     learnable=True,
... )
>>> spectral_map = SpectralMap(model=my_encoder, kernel_fn=kernel, n_states=3)
>>>
>>> # Create callback to log kernel parameters
>>> callback = MetricsCallback(
...     extractors=[kernel_params_extractor],
...     log_every_n_epochs=1,
... )
>>>
>>> # Train with callback
>>> trainer = Trainer(callbacks=[callback], max_epochs=100)
>>> trainer.fit(spectral_map, train_dataloader)
>>>
>>> # For scalar parameters, logged metrics will include:
>>> # - alpha: scalar alpha value
>>> # - beta: scalar beta value
>>> # - kernel_type: "TKernel"
>>> # - kernel_learnable: True
>>>
>>> # For vector/matrix parameters, logged metrics will include:
>>> # - alpha_mean, alpha_std, alpha_min, alpha_max
>>> # - beta_mean, beta_std, beta_min, beta_max
>>> # - kernel_type: "TKernel"
>>> # - kernel_learnable: True

Source code in spectre/kernel/t.py

def get_kernel_params(self) -> dict[str, torch.Tensor]:
    """
    Return current alpha and beta parameters.

    For compact scalar statistics, use `get_param_summary()` instead.

    Returns
    -------
    dict[str, torch.Tensor]
        Dictionary with `alpha` and `beta` keys containing current parameter
        values (tensors).

    Examples
    --------
    Basic usage with constant parameters:

    >>> kernel = TKernel(alpha=torch.tensor(1.0), beta=torch.tensor(2.0))
    >>> params = kernel.get_kernel_params()
    >>> params
    {'alpha': tensor(1.), 'beta': tensor(2.)}

    Learnable parameters:

    >>> kernel = TKernel(
    ...     alpha=torch.tensor(1.0),
    ...     beta=torch.tensor(2.0),
    ...     learnable=True,
    ... )
    >>> params = kernel.get_kernel_params()
    >>> "alpha" in params and "beta" in params
    True

    Logging kernel parameters during training with PyTorch Lightning:

    >>> import torch
    >>> from spectre.parametric import SpectralMap
    >>> from spectre.kernel import TKernel
    >>> from spectre.utils.callbacks import MetricsCallback, kernel_params_extractor
    >>> from pytorch_lightning import Trainer
    >>>
    >>> # Create spectral map with learnable T-kernel
    >>> kernel = TKernel(
    ...     alpha=torch.tensor(0.5),
    ...     beta=torch.tensor(1.5),
    ...     learnable=True,
    ... )
    >>> spectral_map = SpectralMap(model=my_encoder, kernel_fn=kernel, n_states=3)
    >>>
    >>> # Create callback to log kernel parameters
    >>> callback = MetricsCallback(
    ...     extractors=[kernel_params_extractor],
    ...     log_every_n_epochs=1,
    ... )
    >>>
    >>> # Train with callback
    >>> trainer = Trainer(callbacks=[callback], max_epochs=100)
    >>> trainer.fit(spectral_map, train_dataloader)
    >>>
    >>> # For scalar parameters, logged metrics will include:
    >>> # - alpha: scalar alpha value
    >>> # - beta: scalar beta value
    >>> # - kernel_type: "TKernel"
    >>> # - kernel_learnable: True
    >>>
    >>> # For vector/matrix parameters, logged metrics will include:
    >>> # - alpha_mean, alpha_std, alpha_min, alpha_max
    >>> # - beta_mean, beta_std, beta_min, beta_max
    >>> # - kernel_type: "TKernel"
    >>> # - kernel_learnable: True
    """
    params = {}

    if self.learnable:
        params["alpha"] = self._get_param(self.alpha_learnable)
        params["beta"] = self._get_param(self.beta_learnable)
    else:
        params["alpha"] = self.alpha_constant
        params["beta"] = self.beta_constant

    return params

Functions

t_kernel(D: torch.Tensor, alpha: torch.Tensor = torch.tensor(1.0), beta: torch.Tensor = torch.tensor(1.0), dtype: torch.dtype = torch.float32) -> TKernelResult

Compute T-distribution kernel matrix from distance matrix.

Core T-distribution kernel computation - single source of truth for parameter logic. Returns the kernel matrix along with the parameter values used in computation.

This is a pure function that computes the unnormalized T-distribution kernel. For normalized kernels, use the TKernel class with a normalization.

Applies the t-distribution kernel formula: \(K(d) = (1+\alpha d^2)^{-\beta}\).

PARAMETER	DESCRIPTION
D	Pairwise distance matrix of shape (n_samples, n_samples) or (n_samples, m_samples). TYPE: Tensor
alpha	Scale parameter :math:\alpha > 0 controlling kernel bandwidth. Must be a torch.Tensor (scalar or broadcastable shape). TYPE: torch.Tensor, optional, by default torch.tensor(1.0) DEFAULT: tensor(1.0)
beta	Shape parameter :math:\beta > 0 controlling tail behavior. Must be a torch.Tensor (scalar or broadcastable shape). TYPE: torch.Tensor, optional, by default torch.tensor(1.0) DEFAULT: tensor(1.0)
dtype	Data type for computations. TYPE: torch.dtype, optional, by default torch.float32 DEFAULT: float32

RETURNS	DESCRIPTION
TKernelResult	NamedTuple containing: kernel: Unnormalized T-distribution kernel matrix with same shape as D alpha: Alpha parameter value used in computation beta: Beta parameter value used in computation

Examples:

Basic computation:

>>> import torch
>>> from spectre.kernel import t_kernel
>>> X = torch.randn(50, 5)
>>> D = torch.cdist(X, X)
>>> result = t_kernel(D, alpha=torch.tensor(1.0), beta=torch.tensor(2.0))
>>> result.kernel.shape
torch.Size([50, 50])
>>> result.alpha
tensor(1.)
>>> result.beta
tensor(2.)

Comparing different parameter settings:

>>> # Heavy-tailed kernel (robust to outliers)
>>> result_heavy = t_kernel(D, alpha=torch.tensor(1.0), beta=torch.tensor(0.5))
>>> # Gaussian-like kernel (sensitive to outliers)
>>> result_light = t_kernel(D, alpha=torch.tensor(1.0), beta=torch.tensor(5.0))

Source code in spectre/kernel/t.py

def t_kernel(
    D: torch.Tensor,
    alpha: torch.Tensor = torch.tensor(1.0),
    beta: torch.Tensor = torch.tensor(1.0),
    dtype: torch.dtype = torch.float32,
) -> TKernelResult:
    """
    Compute T-distribution kernel matrix from distance matrix.

    Core T-distribution kernel computation - single source of truth for parameter logic.
    Returns the kernel matrix along with the parameter values used in computation.

    This is a pure function that computes the unnormalized T-distribution kernel.
    For normalized kernels, use the `TKernel` class with a normalization.

    Applies the t-distribution kernel formula: $K(d) = (1+\\alpha d^2)^{-\\beta}$.

    Parameters
    ----------
    D : torch.Tensor
        Pairwise distance matrix of shape (n_samples, n_samples) or
        (n_samples, m_samples).

    alpha : torch.Tensor, optional, by default torch.tensor(1.0)
        Scale parameter :math:`\\alpha > 0` controlling kernel bandwidth. Must be a
        torch.Tensor (scalar or broadcastable shape).

    beta : torch.Tensor, optional, by default torch.tensor(1.0)
        Shape parameter :math:`\\beta > 0` controlling tail behavior. Must be a
        torch.Tensor (scalar or broadcastable shape).

    dtype : torch.dtype, optional, by default torch.float32
        Data type for computations.

    Returns
    -------
    TKernelResult
        NamedTuple containing:

        - kernel: Unnormalized T-distribution kernel matrix with same shape as `D`
        - alpha: Alpha parameter value used in computation
        - beta: Beta parameter value used in computation

    Examples
    --------
    Basic computation:

    >>> import torch
    >>> from spectre.kernel import t_kernel
    >>> X = torch.randn(50, 5)
    >>> D = torch.cdist(X, X)
    >>> result = t_kernel(D, alpha=torch.tensor(1.0), beta=torch.tensor(2.0))
    >>> result.kernel.shape
    torch.Size([50, 50])
    >>> result.alpha
    tensor(1.)
    >>> result.beta
    tensor(2.)

    Comparing different parameter settings:

    >>> # Heavy-tailed kernel (robust to outliers)
    >>> result_heavy = t_kernel(D, alpha=torch.tensor(1.0), beta=torch.tensor(0.5))
    >>> # Gaussian-like kernel (sensitive to outliers)
    >>> result_light = t_kernel(D, alpha=torch.tensor(1.0), beta=torch.tensor(5.0))
    """
    check_2d(D)

    # Validate parameters are tensors
    if not isinstance(alpha, torch.Tensor):
        raise TypeError(f"`alpha` must be a torch.Tensor, got {type(alpha)}.")
    if not isinstance(beta, torch.Tensor):
        raise TypeError(f"`beta` must be a torch.Tensor, got {type(beta)}.")

    # For learnable tensors, preserve requires_grad by not converting dtype
    if alpha.requires_grad:
        alpha_val = alpha
    else:
        alpha_val = alpha.to(dtype=dtype)

    if beta.requires_grad:
        beta_val = beta
    else:
        beta_val = beta.to(dtype=dtype)

    # Broadcast alpha and beta to matrix shape (n_samples, m_samples)
    alpha_broadcast = safe_broadcast(alpha_val, D, param_name="alpha")
    beta_broadcast = safe_broadcast(beta_val, D, param_name="beta")

    # Compute kernel
    kernel = torch.pow(1 + alpha_broadcast * torch.pow(D, 2), -beta_broadcast)

    return TKernelResult(kernel=kernel, alpha=alpha_val, beta=beta_val)