SymmetricMeanAbsolutePercentageError

yohou.metrics.point.SymmetricMeanAbsolutePercentageError

Bases: BasePointScorer

Symmetric Mean Absolute Percentage Error metric for point forecasts.

Computes the symmetric average percentage error between predictions and actual values. This provides a scale-independent metric that treats over and under-predictions equally, unlike MAPE which is asymmetric.

The sMAPE is defined as:

\[\\text{sMAPE} = \\frac{100}{n}\\sum_{i=1}^{n}\\frac{|y_i - \\hat{y}_i|}{(|y_i| + |\\hat{y}_i|)/2}\]

where \(y_i\) is the actual value, \(\\hat{y}_i\) is the predicted value, and \(n\) is the number of observations. Values are bounded between 0 and 200.

Parameters

Name	Type	Description	Default
epsilon	float	Small constant added to denominator to prevent division by zero when both actual and predicted values are zero or near-zero.	1e-8
aggregation_method	list of str or str	Dimensions to aggregate over. Options: - "stepwise": Aggregate across forecasting steps. - "vintagewise": Aggregate across vintages (observed times). - "componentwise": Aggregate across components, return per-timestep DataFrame - "groupwise": Aggregate across panel groups (panel data only) - "all": Aggregate across all dimensions (returns scalar). Same as ["stepwise", "vintagewise", "componentwise", "groupwise"]. Example outputs: - ["stepwise", "vintagewise"]: Per-component (and per-group) DataFrame. - "componentwise" or ["componentwise"]: Per-timestep (and per-group) DataFrame. - "groupwise" or ["groupwise"]: Per-component per-timestep DataFrame (panel aggregated). - ["stepwise", "vintagewise", "componentwise"]: Scalar (global) or per-group DataFrame (panel). - "all": Scalar float (hierarchically aggregated for panel data).	"all"
groups	list of str, dict of str to float, or None	Panel group filter (list) or filter with weights (dict).	None
components	list of str, dict of str to float, or None	Component filter (list) or filter with weights (dict).	None

Attributes

Name	Type	Description
lower_is_better	bool	Always True for sMAPE.

Examples

>>> import polars as pl
>>> from datetime import datetime
>>> from yohou.metrics import SymmetricMeanAbsolutePercentageError
>>> y_true = pl.DataFrame({
...     "time": [datetime(2020, 1, 1), datetime(2020, 1, 2), datetime(2020, 1, 3)],
...     "value": [10.0, 20.0, 30.0],
... })
>>> y_pred = pl.DataFrame({
...     "vintage_time": [datetime(2019, 12, 31)] * 3,
...     "time": [datetime(2020, 1, 1), datetime(2020, 1, 2), datetime(2020, 1, 3)],
...     "value": [12.0, 19.0, 28.0],
... })
>>> smape = SymmetricMeanAbsolutePercentageError()
>>> _ = smape.fit(y_true)
>>> smape.score(y_true, y_pred)
10.06...

Notes

• sMAPE is symmetric: treats over-predictions and under-predictions equally
• Scale-independent and useful for comparing forecasts across different series
• Bounded between 0 and 200 (unlike MAPE which is unbounded)
• Less sensitive to very small actual values compared to MAPE
• Undefined when both actual and predicted values are zero; epsilon prevents division by zero

See Also

• MeanAbsolutePercentageError : Asymmetric version of percentage error
• MeanAbsoluteError : Absolute error in original units
• MeanAbsoluteScaledError : Scaled by naive forecast error

Source Code

View on GitHub

Show/Hide source

class SymmetricMeanAbsolutePercentageError(BasePointScorer):
    r"""Symmetric Mean Absolute Percentage Error metric for point forecasts.

    Computes the symmetric average percentage error between predictions and actual values.
    This provides a scale-independent metric that treats over and under-predictions equally,
    unlike MAPE which is asymmetric.

    The sMAPE is defined as:

    $$\\text{sMAPE} = \\frac{100}{n}\\sum_{i=1}^{n}\\frac{|y_i - \\hat{y}_i|}{(|y_i| + |\\hat{y}_i|)/2}$$

    where $y_i$ is the actual value, $\\hat{y}_i$ is the predicted value, and
    $n$ is the number of observations. Values are bounded between 0 and 200.

    Parameters
    ----------
    epsilon : float, default=1e-8
        Small constant added to denominator to prevent division by zero when both
        actual and predicted values are zero or near-zero.

    aggregation_method : list of str or str, default="all"
        Dimensions to aggregate over. Options:
        - "stepwise": Aggregate across forecasting steps.
        - "vintagewise": Aggregate across vintages (observed times).
        - "componentwise": Aggregate across components, return per-timestep DataFrame
        - "groupwise": Aggregate across panel groups (panel data only)
        - "all": Aggregate across all dimensions (returns scalar). Same as
          ["stepwise", "vintagewise", "componentwise", "groupwise"].
        Example outputs:
        - ["stepwise", "vintagewise"]: Per-component (and per-group) DataFrame.
        - "componentwise" or ["componentwise"]: Per-timestep (and per-group) DataFrame.
        - "groupwise" or ["groupwise"]: Per-component per-timestep DataFrame (panel aggregated).
        - ["stepwise", "vintagewise", "componentwise"]: Scalar (global) or per-group DataFrame (panel).
        - "all": Scalar float (hierarchically aggregated for panel data).
    groups : list of str, dict of str to float, or None, default=None
        Panel group filter (list) or filter with weights (dict).
    components : list of str, dict of str to float, or None, default=None
        Component filter (list) or filter with weights (dict).

    Attributes
    ----------
    lower_is_better : bool
        Always True for sMAPE.

    Examples
    --------
    >>> import polars as pl
    >>> from datetime import datetime
    >>> from yohou.metrics import SymmetricMeanAbsolutePercentageError
    >>> y_true = pl.DataFrame({
    ...     "time": [datetime(2020, 1, 1), datetime(2020, 1, 2), datetime(2020, 1, 3)],
    ...     "value": [10.0, 20.0, 30.0],
    ... })
    >>> y_pred = pl.DataFrame({
    ...     "vintage_time": [datetime(2019, 12, 31)] * 3,
    ...     "time": [datetime(2020, 1, 1), datetime(2020, 1, 2), datetime(2020, 1, 3)],
    ...     "value": [12.0, 19.0, 28.0],
    ... })
    >>> smape = SymmetricMeanAbsolutePercentageError()
    >>> _ = smape.fit(y_true)
    >>> smape.score(y_true, y_pred)  # doctest: +ELLIPSIS
    10.06...

    Notes
    -----
    - sMAPE is symmetric: treats over-predictions and under-predictions equally
    - Scale-independent and useful for comparing forecasts across different series
    - Bounded between 0 and 200 (unlike MAPE which is unbounded)
    - Less sensitive to very small actual values compared to MAPE
    - Undefined when both actual and predicted values are zero; epsilon prevents division by zero

    See Also
    --------
    - [`MeanAbsolutePercentageError`][yohou.metrics.point.MeanAbsolutePercentageError] : Asymmetric version of percentage error
    - [`MeanAbsoluteError`][yohou.metrics.point.MeanAbsoluteError] : Absolute error in original units
    - [`MeanAbsoluteScaledError`][yohou.metrics.point.MeanAbsoluteScaledError] : Scaled by naive forecast error

    """

    _parameter_constraints: dict = {
        **BasePointScorer._parameter_constraints,
        "epsilon": [Interval(numbers.Real, 0, None, closed="neither")],
    }

    _metric_name = "smape"

    def __init__(
        self,
        epsilon: float = 1e-8,
        aggregation_method: list[str] | str = "all",
        groups: list[str] | dict[str, float] | None = None,
        components: list[str] | dict[str, float] | None = None,
    ) -> None:
        super().__init__(
            aggregation_method=aggregation_method,
            groups=groups,
            components=components,
        )
        self.epsilon = epsilon

    def _compute_raw_errors(self, y_truth: pl.DataFrame, y_pred: pl.DataFrame) -> pl.DataFrame:
        """Compute per-row symmetric absolute percentage errors."""
        smape_errors_data = {}
        for col in y_truth.columns:
            abs_errors = (y_truth[col] - y_pred[col]).abs()
            denominator = (y_truth[col].abs() + y_pred[col].abs()) / 2.0 + self.epsilon
            smape_errors_data[col] = (abs_errors / denominator) * 100.0
        return pl.DataFrame(smape_errors_data)

Tutorials

The following example notebooks use this component:

• How to Use Point Forecast Metrics

  ---

  Evaluation-Search

  Compare MAE, MAPE, MASE, RMSE, and other point metrics across multiple forecasters with componentwise and groupwise aggregation.

  View · Open in marimo