MeanAbsoluteScaledError

yohou.metrics.point.MeanAbsoluteScaledError

Bases: BasePointScorer

Mean Absolute Scaled Error metric for point forecasts.

Computes MAE scaled by the in-sample naive seasonal forecast error. This provides a scale-independent metric that enables comparison across time series with different magnitudes. Requires training data to compute scaling factors.

The MASE is defined as:

\[\\text{MASE} = \\frac{1}{h}\\sum_{t=1}^{h}\\left|\\frac{y_t - \\hat{y}_t}{\\text{scale}}\\right|\]

where the scale is computed from training data as:

\[\\text{scale}_j = \\frac{1}{T-m}\\sum_{t=m+1}^{T}|y_{t,j} - y_{t-m,j}|\]

with \(m\) = seasonality, \(T\) = training length, \(h\) = forecast horizon, and \(j\) = column index. Per-column MASE values are averaged to produce the final score.

Parameters

Name	Type	Description	Default
seasonality	int	Seasonal period for computing scaling factors. Must be at least 1. Common values: 1 (non-seasonal), 7 (weekly), 12 (monthly), 24 (hourly daily pattern).	1
aggregation_method	list of str or str	Dimensions to aggregate over. Options: - "stepwise": Aggregate across forecasting steps. - "vintagewise": Aggregate across vintages (observed times). - "componentwise": Aggregate across components, return per-timestep DataFrame - "groupwise": Aggregate across panel groups (panel data only) - "all": Aggregate across all dimensions (returns scalar). Same as ["stepwise", "vintagewise", "componentwise", "groupwise"]. Example outputs: - ["stepwise", "vintagewise"]: Per-component (and per-group) DataFrame. - "componentwise" or ["componentwise"]: Per-timestep (and per-group) DataFrame. - "groupwise" or ["groupwise"]: Per-component per-timestep DataFrame (panel aggregated). - ["stepwise", "vintagewise", "componentwise"]: Scalar (global) or per-group DataFrame (panel). - "all": Scalar float (hierarchically aggregated for panel data).	"all"
groups	list of str, dict of str to float, or None	Panel group filter (list) or filter with weights (dict).	None
components	list of str, dict of str to float, or None	Component filter (list) or filter with weights (dict).	None

Attributes

Name	Type	Description
lower_is_better	bool	Always True for MASE.
naive_errors_	dict[str, float]	Fitted per-column scaling factors based on naive seasonal forecast MAE. Computed during fit() from training data.

Examples

>>> import polars as pl
>>> from datetime import datetime, timedelta
>>> from yohou.metrics import MeanAbsoluteScaledError
>>> # Training data
>>> y_train = pl.DataFrame({
...     "time": [datetime(2020, 1, 1) + timedelta(days=i) for i in range(10)],
...     "value": [10.0, 12.0, 11.0, 13.0, 12.0, 14.0, 13.0, 15.0, 14.0, 16.0],
... })
>>> # Test predictions
>>> y_true = pl.DataFrame({
...     "time": [datetime(2020, 1, 11), datetime(2020, 1, 12)],
...     "value": [15.0, 17.0],
... })
>>> y_pred = pl.DataFrame({
...     "vintage_time": [datetime(2020, 1, 10)] * 2,
...     "time": [datetime(2020, 1, 11), datetime(2020, 1, 12)],
...     "value": [15.5, 16.5],
... })
>>> mase = MeanAbsoluteScaledError(seasonality=2)
>>> mase.fit(y_train)
MeanAbsoluteScaledError(seasonality=2)
>>> mase.score(y_true, y_pred)
0.5...

Notes

• MASE values are scale-independent, enabling comparison across different time series
• Values < 1 indicate better performance than naive seasonal forecast on training data
• Values > 1 indicate worse performance than naive seasonal baseline
• Requires training data with length > seasonality
• Per-column scaling factors are stored and applied independently
• More interpretable than RMSSE as it uses absolute errors rather than squared errors

See Also

• RootMeanSquaredScaledError : Squared error version of scaled metric
• MeanAbsoluteError : Non-scaled MAE
• MeanAbsolutePercentageError : Percentage-based scale-independent metric

Source Code

View on GitHub

Show/Hide source

class MeanAbsoluteScaledError(BasePointScorer):
    r"""Mean Absolute Scaled Error metric for point forecasts.

    Computes MAE scaled by the in-sample naive seasonal forecast error. This provides
    a scale-independent metric that enables comparison across time series with different
    magnitudes. Requires training data to compute scaling factors.

    The MASE is defined as:

    $$\\text{MASE} = \\frac{1}{h}\\sum_{t=1}^{h}\\left|\\frac{y_t - \\hat{y}_t}{\\text{scale}}\\right|$$

    where the scale is computed from training data as:

    $$\\text{scale}_j = \\frac{1}{T-m}\\sum_{t=m+1}^{T}|y_{t,j} - y_{t-m,j}|$$

    with $m$ = seasonality, $T$ = training length, $h$ = forecast horizon, and $j$ = column index.
    Per-column MASE values are averaged to produce the final score.

    Parameters
    ----------
    seasonality : int, default=1
        Seasonal period for computing scaling factors. Must be at least 1.
        Common values: 1 (non-seasonal), 7 (weekly), 12 (monthly), 24 (hourly daily pattern).

    aggregation_method : list of str or str, default="all"
        Dimensions to aggregate over. Options:
        - "stepwise": Aggregate across forecasting steps.
        - "vintagewise": Aggregate across vintages (observed times).
        - "componentwise": Aggregate across components, return per-timestep DataFrame
        - "groupwise": Aggregate across panel groups (panel data only)
        - "all": Aggregate across all dimensions (returns scalar). Same as
          ["stepwise", "vintagewise", "componentwise", "groupwise"].
        Example outputs:
        - ["stepwise", "vintagewise"]: Per-component (and per-group) DataFrame.
        - "componentwise" or ["componentwise"]: Per-timestep (and per-group) DataFrame.
        - "groupwise" or ["groupwise"]: Per-component per-timestep DataFrame (panel aggregated).
        - ["stepwise", "vintagewise", "componentwise"]: Scalar (global) or per-group DataFrame (panel).
        - "all": Scalar float (hierarchically aggregated for panel data).
    groups : list of str, dict of str to float, or None, default=None
        Panel group filter (list) or filter with weights (dict).
    components : list of str, dict of str to float, or None, default=None
        Component filter (list) or filter with weights (dict).

    Attributes
    ----------
    lower_is_better : bool
        Always True for MASE.
    naive_errors_ : dict[str, float]
        Fitted per-column scaling factors based on naive seasonal forecast MAE.
        Computed during fit() from training data.

    Examples
    --------
    >>> import polars as pl
    >>> from datetime import datetime, timedelta
    >>> from yohou.metrics import MeanAbsoluteScaledError
    >>> # Training data
    >>> y_train = pl.DataFrame({
    ...     "time": [datetime(2020, 1, 1) + timedelta(days=i) for i in range(10)],
    ...     "value": [10.0, 12.0, 11.0, 13.0, 12.0, 14.0, 13.0, 15.0, 14.0, 16.0],
    ... })
    >>> # Test predictions
    >>> y_true = pl.DataFrame({
    ...     "time": [datetime(2020, 1, 11), datetime(2020, 1, 12)],
    ...     "value": [15.0, 17.0],
    ... })
    >>> y_pred = pl.DataFrame({
    ...     "vintage_time": [datetime(2020, 1, 10)] * 2,
    ...     "time": [datetime(2020, 1, 11), datetime(2020, 1, 12)],
    ...     "value": [15.5, 16.5],
    ... })
    >>> mase = MeanAbsoluteScaledError(seasonality=2)
    >>> mase.fit(y_train)
    MeanAbsoluteScaledError(seasonality=2)
    >>> mase.score(y_true, y_pred)  # doctest: +ELLIPSIS
    0.5...

    Notes
    -----
    - MASE values are scale-independent, enabling comparison across different time series
    - Values < 1 indicate better performance than naive seasonal forecast on training data
    - Values > 1 indicate worse performance than naive seasonal baseline
    - Requires training data with length > seasonality
    - Per-column scaling factors are stored and applied independently
    - More interpretable than RMSSE as it uses absolute errors rather than squared errors

    See Also
    --------
    - [`RootMeanSquaredScaledError`][yohou.metrics.point.RootMeanSquaredScaledError] : Squared error version of scaled metric
    - [`MeanAbsoluteError`][yohou.metrics.point.MeanAbsoluteError] : Non-scaled MAE
    - [`MeanAbsolutePercentageError`][yohou.metrics.point.MeanAbsolutePercentageError] : Percentage-based scale-independent metric

    """

    _parameter_constraints: dict = {
        **BasePointScorer._parameter_constraints,
        "seasonality": [Interval(numbers.Integral, 1, None, closed="left")],
    }

    _metric_name = "mase"

    def __init__(
        self,
        seasonality: int = 1,
        aggregation_method: list[str] | str = "all",
        groups: list[str] | dict[str, float] | None = None,
        components: list[str] | dict[str, float] | None = None,
    ) -> None:
        super().__init__(
            aggregation_method=aggregation_method,
            groups=groups,
            components=components,
        )
        self.seasonality = seasonality

    def __sklearn_tags__(self):
        """Get estimator tags.

        Returns
        -------
        Tags
            Estimator tags with requires_calibration=True.

        """
        tags = super().__sklearn_tags__()
        if tags.scorer_tags is not None:
            tags.scorer_tags.requires_calibration = True
        return tags

    @_fit_context(prefer_skip_nested_validation=True)
    def fit(self, y_train: pl.DataFrame, *, forecaster=None, **params) -> MeanAbsoluteScaledError:
        """Fit the scorer by computing per-column scaling factors.

        Parameters
        ----------
        y_train : pl.DataFrame
            Training set target values with "time" column.
        forecaster : BaseForecaster or None, default=None
            If provided, metadata is extracted directly from the fitted
            forecaster instead of being re-inferred from ``y_train``.
        **params : dict
            Metadata to route to nested estimators.

        Returns
        -------
        self

        Raises
        ------
        ValueError
            If y_train is None or seasonality > len(y_train) - 1.

        """
        # Call parent fit() to validate parameters (aggregation_method, groups, etc.)
        super().fit(y_train, forecaster=forecaster, **params)

        # Validate training data and remove time column
        y_train_values, _, _ = validate_scorer_data(scorer=self, y_true=y_train, y_pred=None, reset=True)

        # Compute per-column scaling factors using seasonal naive forecast MAE
        self.naive_errors_ = {}
        for col in y_train_values.columns:
            # Compute seasonal differences: |y_t - y_{t-seasonality}|
            col_data = y_train_values[col].to_numpy()
            naive_errors = np.abs(col_data[self.seasonality :] - col_data[: -self.seasonality])

            # Scale is mean absolute error of seasonal naive forecast
            scale = float(np.mean(naive_errors))

            if scale == 0:
                warnings.warn(
                    f"Training data for column '{col}' has zero scale "
                    f"(constant values with seasonality={self.seasonality}). "
                    "MASE scores for this column will use a scale floor of 1e-10.",
                    UserWarning,
                    stacklevel=2,
                )

            # Store non-zero scale (avoid division by zero in score())
            self.naive_errors_[col] = max(scale, 1e-10)

        return self

    def _compute_raw_errors(self, y_truth: pl.DataFrame, y_pred: pl.DataFrame) -> pl.DataFrame:
        """Compute per-row scaled absolute errors."""
        scaled_errors_data = {}
        for col in y_truth.columns:
            errors = (y_truth[col] - y_pred[col]).abs().to_numpy()
            scale = self.naive_errors_[col]
            scaled_errors_data[col] = errors / scale
        return pl.DataFrame(scaled_errors_data)

Methods

__sklearn_tags__()

Get estimator tags.

Returns

Type	Description
Tags	Estimator tags with requires_calibration=True.

Source Code

View on GitHub

Show/Hide source

def __sklearn_tags__(self):
    """Get estimator tags.

    Returns
    -------
    Tags
        Estimator tags with requires_calibration=True.

    """
    tags = super().__sklearn_tags__()
    if tags.scorer_tags is not None:
        tags.scorer_tags.requires_calibration = True
    return tags

fit(y_train, *, forecaster=None, **params)

Fit the scorer by computing per-column scaling factors.

Parameters

Name	Type	Description	Default
y_train	DataFrame	Training set target values with "time" column.	required
forecaster	BaseForecaster or None	If provided, metadata is extracted directly from the fitted forecaster instead of being re-inferred from y_train.	None
**params	dict	Metadata to route to nested estimators.	{}

Returns

Type	Description
self

Raises

Type	Description
ValueError	If y_train is None or seasonality > len(y_train) - 1.

Source Code

View on GitHub

Show/Hide source

@_fit_context(prefer_skip_nested_validation=True)
def fit(self, y_train: pl.DataFrame, *, forecaster=None, **params) -> MeanAbsoluteScaledError:
    """Fit the scorer by computing per-column scaling factors.

    Parameters
    ----------
    y_train : pl.DataFrame
        Training set target values with "time" column.
    forecaster : BaseForecaster or None, default=None
        If provided, metadata is extracted directly from the fitted
        forecaster instead of being re-inferred from ``y_train``.
    **params : dict
        Metadata to route to nested estimators.

    Returns
    -------
    self

    Raises
    ------
    ValueError
        If y_train is None or seasonality > len(y_train) - 1.

    """
    # Call parent fit() to validate parameters (aggregation_method, groups, etc.)
    super().fit(y_train, forecaster=forecaster, **params)

    # Validate training data and remove time column
    y_train_values, _, _ = validate_scorer_data(scorer=self, y_true=y_train, y_pred=None, reset=True)

    # Compute per-column scaling factors using seasonal naive forecast MAE
    self.naive_errors_ = {}
    for col in y_train_values.columns:
        # Compute seasonal differences: |y_t - y_{t-seasonality}|
        col_data = y_train_values[col].to_numpy()
        naive_errors = np.abs(col_data[self.seasonality :] - col_data[: -self.seasonality])

        # Scale is mean absolute error of seasonal naive forecast
        scale = float(np.mean(naive_errors))

        if scale == 0:
            warnings.warn(
                f"Training data for column '{col}' has zero scale "
                f"(constant values with seasonality={self.seasonality}). "
                "MASE scores for this column will use a scale floor of 1e-10.",
                UserWarning,
                stacklevel=2,
            )

        # Store non-zero scale (avoid division by zero in score())
        self.naive_errors_[col] = max(scale, 1e-10)

    return self

Tutorials

The following example notebooks use this component:

• How to Use Point Forecast Metrics

  ---

  Evaluation-Search

  Compare MAE, MAPE, MASE, RMSE, and other point metrics across multiple forecasters with componentwise and groupwise aggregation.

  View · Open in marimo

• Forecasting Workflow

  ---

  Getting-Started

  Evaluate forecasters with cross-validation, search hyperparameters with GridSearchCV, and inspect residuals to diagnose model weaknesses.

  View · Open in marimo

• Observe-Predict Workflow

  ---

  Getting-Started

  Walk through a test set in batches, updating forecasts as new data arrives with observe_predict.

  View · Open in marimo