ROCAuC

yohou.metrics.classification.ROCAuC

Bases: BaseRankingScorer

ROC AUC from class-probability forecasts.

Computes the Area Under the Receiver Operating Characteristic curve using a one-vs-rest (OvR) strategy for multiclass problems.

Parameters

Name	Type	Description	Default
average	str	Class averaging strategy: "macro" (unweighted mean across classes) or "weighted" (support-weighted mean).	"macro"
aggregation_method	list of str or str	Dimensions to aggregate over.	"all"
groups	list of str, dict of str to float, or None	Panel group filter or filter with weights.	None
components	list of str, dict of str to float, or None	Component filter or filter with weights.	None

Attributes

Name	Type	Description
lower_is_better	bool	Always False (higher AUC is better).

Examples

>>> import polars as pl
>>> from datetime import datetime
>>> from yohou.metrics.classification import ROCAuC
>>> y_true = pl.DataFrame({
...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
...     "weather": ["sunny", "rainy", "cloudy", "sunny", "rainy"],
... })
>>> y_pred = pl.DataFrame({
...     "vintage_time": [datetime(2019, 12, 31)] * 5,
...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
...     "weather_proba_sunny": [0.7, 0.1, 0.2, 0.6, 0.1],
...     "weather_proba_rainy": [0.2, 0.8, 0.1, 0.3, 0.8],
...     "weather_proba_cloudy": [0.1, 0.1, 0.7, 0.1, 0.1],
... })
>>> scorer = ROCAuC()
>>> _ = scorer.fit(y_true)
>>> scorer.score(y_true, y_pred)
1.0

See Also

• PRAuC : Precision-Recall AUC.

Source Code

View on GitHub

Show/Hide source

class ROCAuC(BaseRankingScorer):
    r"""ROC AUC from class-probability forecasts.

    Computes the Area Under the Receiver Operating Characteristic curve
    using a one-vs-rest (OvR) strategy for multiclass problems.

    Parameters
    ----------
    average : str, default="macro"
        Class averaging strategy: ``"macro"`` (unweighted mean across
        classes) or ``"weighted"`` (support-weighted mean).
    aggregation_method : list of str or str, default="all"
        Dimensions to aggregate over.
    groups : list of str, dict of str to float, or None, default=None
        Panel group filter or filter with weights.
    components : list of str, dict of str to float, or None, default=None
        Component filter or filter with weights.

    Attributes
    ----------
    lower_is_better : bool
        Always False (higher AUC is better).

    Examples
    --------
    >>> import polars as pl
    >>> from datetime import datetime
    >>> from yohou.metrics.classification import ROCAuC
    >>> y_true = pl.DataFrame({
    ...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
    ...     "weather": ["sunny", "rainy", "cloudy", "sunny", "rainy"],
    ... })
    >>> y_pred = pl.DataFrame({
    ...     "vintage_time": [datetime(2019, 12, 31)] * 5,
    ...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
    ...     "weather_proba_sunny": [0.7, 0.1, 0.2, 0.6, 0.1],
    ...     "weather_proba_rainy": [0.2, 0.8, 0.1, 0.3, 0.8],
    ...     "weather_proba_cloudy": [0.1, 0.1, 0.7, 0.1, 0.1],
    ... })
    >>> scorer = ROCAuC()
    >>> _ = scorer.fit(y_true)
    >>> scorer.score(y_true, y_pred)
    1.0

    See Also
    --------
    - [`PRAuC`][yohou.metrics.classification.PRAuC] : Precision-Recall AUC.

    """

    _metric_name = "roc_auc"
    _lower_is_better = False

    def _compute_ranking_metric(
        self,
        y_true_binary: np.ndarray,
        y_proba: np.ndarray,
        sample_weight: np.ndarray | None = None,
    ) -> float:
        """Compute ROC AUC via sklearn."""
        return float(roc_auc_score(y_true_binary, y_proba, sample_weight=sample_weight))