Loss Cosine

cosine

CLASS	DESCRIPTION
CosineEmbeddingLoss	Cosine embedding loss for similarity learning.

FUNCTION	DESCRIPTION
cosine_embedding_loss	Cosine embedding loss for similarity learning.

Classes

CosineEmbeddingLoss(margin: float = 0.0, reduce: Literal['mean', 'sum'] = 'mean', sign: float = 1.0, context_prefix: str | None = None)

Bases: Loss

Cosine embedding loss for similarity learning.

Measures the angle between embeddings, encouraging similar pairs to have small angles and dissimilar pairs to have large angles.

The loss is defined as:

\(L = \begin{cases} 1 - \cos(X, Z) & \text{if } \mathrm{label} = 1 \\ \max(0, \cos(X, Z) - m) & \text{if } \mathrm{label} = -1 \end{cases}\),

where \(\cos(X, Z)\) is the cosine similarity and \(m\) is the margin.

PARAMETER	DESCRIPTION
margin	Margin for dissimilar pairs. Must be in [-1, 1]. TYPE: float, by default 0.0 DEFAULT: 0.0
reduce	Reduction method. TYPE: Literal["mean", "sum"], by default "mean" DEFAULT: 'mean'
sign	Loss sign multiplier. TYPE: float, by default 1.0 DEFAULT: 1.0

Examples:

>>> loss_fn = CosineEmbeddingLoss(margin=0.2)
>>> X = torch.randn(32, 128)
>>> Z = torch.randn(32, 128)
>>> labels = torch.randint(0, 2, (32,)).float() * 2 - 1  # Convert to {-1, 1}
>>> context = {"X": X, "Z": Z, "labels": labels}
>>> loss = loss_fn(context)

METHOD	DESCRIPTION
forward	Compute cosine embedding loss.

Source code in spectre/loss/cosine.py

def __init__(
    self,
    margin: float = 0.0,
    reduce: Literal["mean", "sum"] = "mean",
    sign: float = 1.0,
    context_prefix: str | None = None,
):
    super().__init__(reduce=reduce, sign=sign, context_prefix=context_prefix)

    if not isinstance(margin, (int, float)):
        raise TypeError(f"margin must be numeric, got {type(margin).__name__}")
    check_in_interval(float(margin), "[-1, 1]")
    self.margin = float(margin)

Functions

forward(context: dict, **kwargs) -> torch.Tensor

Compute cosine embedding loss.

PARAMETER	DESCRIPTION
context	Dictionary containing the input tensors and labels. TYPE: dict
**kwargs	Additional keyword arguments. TYPE: dict DEFAULT: {}

RETURNS	DESCRIPTION
Tensor	Cosine embedding loss.

Source code in spectre/loss/cosine.py

def forward(self, context: dict, **kwargs) -> torch.Tensor:
    """
    Compute cosine embedding loss.

    Parameters
    ----------
    context : dict
        Dictionary containing the input tensors and labels.

    **kwargs : dict
        Additional keyword arguments.

    Returns
    -------
    torch.Tensor
        Cosine embedding loss.
    """
    X = self._get_context(context, "X", None)
    Z = self._get_context(context, "Z", None)
    labels = self._get_context(context, "labels", None)
    weights = self._get_context(context, "weights", None)

    if X is None or Z is None or labels is None:
        raise ValueError(
            "`CosineEmbeddingLoss` requires `X`, `Z`, and `labels` in context."
        )

    return cosine_embedding_loss(
        X,
        Z,
        labels,
        margin=self.margin,
        weights=weights,
        reduce=self.reduce,
        sign=self.sign,
    )

Functions

cosine_embedding_loss(X: torch.Tensor, Z: torch.Tensor, labels: torch.Tensor, margin: float = 0.0, weights: torch.Tensor | None = None, reduce: Literal['mean', 'sum'] = 'mean', sign: float = 1.0) -> torch.Tensor

Cosine embedding loss for similarity learning.

Measures the angle between embeddings, encouraging similar pairs to have small angles and dissimilar pairs to have large angles.

The loss is defined as:

\(L = \begin{cases} 1 - \cos(X, Z) & \text{if } \mathrm{label} = 1 \\ \max(0, \cos(X, Z) - m) & \text{if } \mathrm{label} = -1 \end{cases}\),

where \(\cos(X, Z)\) is the cosine similarity and \(m\) is the margin.

PARAMETER	DESCRIPTION
X	First set of embeddings of shape (batch_size, out_features). TYPE: Tensor
Z	Second set of embeddings of shape (batch_size, out_features). TYPE: Tensor
labels	Similarity labels of shape (batch_size,). 1 for similar pairs, -1 for dissimilar pairs. TYPE: Tensor
margin	Margin for dissimilar pairs. TYPE: float, by default 0.0 DEFAULT: 0.0
weights	Sample weights. TYPE: torch.Tensor | None, by default None DEFAULT: None
reduce	Reduction method. TYPE: Literal["mean", "sum"], by default "mean" DEFAULT: 'mean'
sign	Loss sign multiplier. TYPE: float, by default 1.0 DEFAULT: 1.0

RETURNS	DESCRIPTION
Tensor	Cosine embedding loss value.

Examples:

>>> X = torch.randn(10, 64)
>>> Z = torch.randn(10, 64)
>>> labels = torch.randint(0, 2, (10,)).float() * 2 - 1
>>> loss = cosine_embedding_loss(X, Z, labels, margin=0.2)

Source code in spectre/loss/cosine.py

def cosine_embedding_loss(
    X: torch.Tensor,
    Z: torch.Tensor,
    labels: torch.Tensor,
    margin: float = 0.0,
    weights: torch.Tensor | None = None,
    reduce: Literal["mean", "sum"] = "mean",
    sign: float = 1.0,
) -> torch.Tensor:
    """
    Cosine embedding loss for similarity learning.

    Measures the angle between embeddings, encouraging similar pairs to have
    small angles and dissimilar pairs to have large angles.

    The loss is defined as:

    $L = \\begin{cases}
    1 - \\cos(X, Z) & \\text{if } \\mathrm{label} = 1 \\\\
    \\max(0, \\cos(X, Z) - m) & \\text{if } \\mathrm{label} = -1
    \\end{cases}$,

    where $\\cos(X, Z)$ is the cosine similarity and $m$ is the margin.

    Parameters
    ----------
    X : torch.Tensor
        First set of embeddings of shape (batch_size, out_features).

    Z : torch.Tensor
        Second set of embeddings of shape (batch_size, out_features).

    labels : torch.Tensor
        Similarity labels of shape (batch_size,). 1 for similar pairs,
        -1 for dissimilar pairs.

    margin : float, by default 0.0
        Margin for dissimilar pairs.

    weights : torch.Tensor | None, by default None
        Sample weights.

    reduce : Literal["mean", "sum"], by default "mean"
        Reduction method.

    sign : float, by default 1.0
        Loss sign multiplier.

    Returns
    -------
    torch.Tensor
        Cosine embedding loss value.

    Examples
    --------
    >>> X = torch.randn(10, 64)
    >>> Z = torch.randn(10, 64)
    >>> labels = torch.randint(0, 2, (10,)).float() * 2 - 1
    >>> loss = cosine_embedding_loss(X, Z, labels, margin=0.2)
    """
    check_same_shape(X, Z)
    check_same_shape(X, labels, axis=0)
    check_2d(X)
    check_2d(Z)
    check_1d(labels)

    loss = F.cosine_embedding_loss(X, Z, labels, margin=margin, reduction="none")

    if weights is not None:
        check_same_shape(weights, X, axis=0)
        loss = loss * weights.squeeze()

    if reduce == "sum":
        loss = loss.sum()
    elif reduce == "mean":
        loss = loss.mean()
    else:
        raise ValueError(
            f"Unknown reduce operation '{reduce}'. Options: 'sum', 'mean'."
        )

    return sign * loss