MeanIntervalWidth

yohou.metrics.interval.MeanIntervalWidth

Bases: BaseIntervalScorer

Mean width of prediction intervals.

Measures the average width of prediction intervals. Narrower intervals are preferred (more informative), provided coverage is maintained.

The mean interval width for coverage rate α is:

\[\\text{Width}(\\alpha) = \\frac{1}{n}\\sum_{i=1}^{n} |U_i(\\alpha) - L_i(\\alpha)|\]

Parameters

Name	Type	Description	Default
aggregation_method	list of str or str	Dimensions to collapse when aggregating scores. Orthogonal modes: "stepwise": Collapse the forecasting-step dimension. "vintagewise": Collapse the vintage/observed-time dimension. "componentwise": Collapse components, return per-timestep scores. "groupwise": Collapse panel groups (panel data only). "coveragewise": Collapse coverage rates (return average coverage). "all": Collapse all dimensions (returns scalar).	"all"
coverage_rates	list of float, dict of float to float, or None	Coverage rate filter (list) or filter with weights (dict).	None
groups	list of str, dict of str to float, or None	Panel group filter (list) or filter with weights (dict).	None
components	list of str, dict of str to float, or None	Component filter (list) or filter with weights (dict).	None

Attributes

Name	Type	Description
lower_is_better	bool	True for width (narrower intervals are better).

Examples

>>> import polars as pl
>>> from datetime import datetime
>>> from yohou.metrics import MeanIntervalWidth
>>> y_true = pl.DataFrame({"time": [datetime(2020, 1, 1), datetime(2020, 1, 2)], "value": [10.0, 20.0]})
>>> y_pred = pl.DataFrame({
...     "vintage_time": [datetime(2019, 12, 31)] * 2,
...     "time": [datetime(2020, 1, 1), datetime(2020, 1, 2)],
...     "value_lower_0.9": [8.0, 18.0],
...     "value_upper_0.9": [12.0, 22.0],
... })
>>> width = MeanIntervalWidth()
>>> _ = width.fit(y_true)
>>> width.score(y_true, y_pred)
4.0

Notes

• Lower is better (sharper, more precise predictions)
• Should only be compared when coverage is approximately equal
• Scale-dependent (units match target variable)
• Missing values are excluded from computation
• Uses absolute width to handle bound inversions silently

See Also

• EmpiricalCoverage : Evaluates interval calibration
• IntervalScore : Combined coverage and sharpness metric

Source Code

View on GitHub

Show/Hide source

class MeanIntervalWidth(BaseIntervalScorer):
    r"""Mean width of prediction intervals.

    Measures the average width of prediction intervals. Narrower intervals are
    preferred (more informative), provided coverage is maintained.

    The mean interval width for coverage rate α is:

    $$\\text{Width}(\\alpha) = \\frac{1}{n}\\sum_{i=1}^{n} |U_i(\\alpha) - L_i(\\alpha)|$$

    Parameters
    ----------
    aggregation_method : list of str or str, default="all"
        Dimensions to collapse when aggregating scores. Orthogonal modes:

        - "stepwise": Collapse the forecasting-step dimension.
        - "vintagewise": Collapse the vintage/observed-time dimension.
        - "componentwise": Collapse components, return per-timestep scores.
        - "groupwise": Collapse panel groups (panel data only).
        - "coveragewise": Collapse coverage rates (return average coverage).

        - "all": Collapse all dimensions (returns scalar).
    coverage_rates : list of float, dict of float to float, or None, default=None
        Coverage rate filter (list) or filter with weights (dict).
    groups : list of str, dict of str to float, or None, default=None
        Panel group filter (list) or filter with weights (dict).
    components : list of str, dict of str to float, or None, default=None
        Component filter (list) or filter with weights (dict).

    Attributes
    ----------
    lower_is_better : bool
        True for width (narrower intervals are better).

    Examples
    --------
    >>> import polars as pl
    >>> from datetime import datetime
    >>> from yohou.metrics import MeanIntervalWidth
    >>> y_true = pl.DataFrame({"time": [datetime(2020, 1, 1), datetime(2020, 1, 2)], "value": [10.0, 20.0]})
    >>> y_pred = pl.DataFrame({
    ...     "vintage_time": [datetime(2019, 12, 31)] * 2,
    ...     "time": [datetime(2020, 1, 1), datetime(2020, 1, 2)],
    ...     "value_lower_0.9": [8.0, 18.0],
    ...     "value_upper_0.9": [12.0, 22.0],
    ... })
    >>> width = MeanIntervalWidth()
    >>> _ = width.fit(y_true)
    >>> width.score(y_true, y_pred)
    4.0

    Notes
    -----
    - Lower is better (sharper, more precise predictions)
    - Should only be compared when coverage is approximately equal
    - Scale-dependent (units match target variable)
    - Missing values are excluded from computation
    - Uses absolute width to handle bound inversions silently

    See Also
    --------
    - [`EmpiricalCoverage`][yohou.metrics.interval.EmpiricalCoverage] : Evaluates interval calibration
    - [`IntervalScore`][yohou.metrics.interval.IntervalScore] : Combined coverage and sharpness metric

    """

    _parameter_constraints: dict = {
        **BaseIntervalScorer._parameter_constraints,
    }

    _metric_name = "width"

    def __init__(
        self,
        aggregation_method: list[str] | str = "all",
        coverage_rates: list[float] | dict[float, float] | None = None,
        groups: list[str] | dict[str, float] | None = None,
        components: list[str] | dict[str, float] | None = None,
    ) -> None:
        agg_list = aggregation_method
        if aggregation_method == "all":
            agg_list = ["stepwise", "vintagewise", "componentwise", "groupwise", "coveragewise"]

        super().__init__(
            aggregation_method=agg_list,
            coverage_rates=coverage_rates,
            groups=groups,
            components=components,
        )

    def _compute_raw_scores(self, y_truth, y_pred, coverage_rates, target_columns):
        """Compute per-row interval width values."""
        frames = []
        for rate in coverage_rates:
            rate_data = {}
            for col in target_columns:
                lower_col = f"{col}_lower_{rate}"
                upper_col = f"{col}_upper_{rate}"
                if lower_col in y_pred.columns and upper_col in y_pred.columns:
                    rate_data[col] = (y_pred[upper_col] - y_pred[lower_col]).abs()
            frames.append(pl.DataFrame(rate_data).with_columns(pl.lit(rate).alias("coverage_rate")))
        return pl.concat(frames)

Tutorials

The following example notebooks use this component:

• How to Use Conformity Scorers

  ---

  Evaluation-Search

  Compare Residual, AbsoluteResidual, GammaResidual, and AbsoluteGammaResidual conformity scorers with coverage/width analysis and DistanceSimilarity interaction.

  View · Open in marimo

• How to Evaluate Interval Forecasts

  ---

  Evaluation-Search

  Evaluate prediction intervals with EmpiricalCoverage, IntervalScore, MeanIntervalWidth, PinballLoss, and CalibrationError across coverage levels.

  View · Open in marimo

• How to Search Interval Forecaster Hyperparameters

  ---

  Evaluation-Search

  Tune interval forecaster parameters directly with interval metrics in GridSearchCV, including mixed point+interval multimetric search.

  View · Open in marimo

• How to Forecast Intervals with CatBoost Multiquantile

  ---

  Forecasting-Models

  Use IntervalReductionForecaster with CatBoost's native multiquantile objective for simultaneous lower and upper bound estimation.

  View · Open in marimo

• How to Use Distance-Based Similarity for Intervals

  ---

  Forecasting-Models

  Adaptive prediction intervals via similarity-weighted conformal prediction using DistanceSimilarity with configurable distance metrics and bandwidths.

  View · Open in marimo

• How to Build Interval Forecasts with Reduction

  ---

  Forecasting-Models

  Wrap any quantile-capable sklearn estimator with IntervalReductionForecaster to produce calibrated prediction intervals across multiple horizons.

  View · Open in marimo

• Conformal Prediction Intervals

  ---

  Getting-Started

  Build distribution-free prediction intervals with SplitConformalForecaster using calibration holdouts and configurable conformity scoring functions.

  View · Open in marimo

• How to Create a Custom Interval Forecaster

  ---

  Getting-Started

  Implement a NaiveIntervalForecaster from scratch, validate it with the check generator, and compare it against SplitConformalForecaster.

  View · Open in marimo