PRAuC

yohou.metrics.classification.PRAuC

Bases: BaseRankingScorer

Precision-Recall AUC from class-probability forecasts.

Computes the Area Under the Precision-Recall curve using a one-vs-rest (OvR) strategy for multiclass problems. Uses average_precision_score from scikit-learn.

Parameters

Name	Type	Description	Default
average	str	Class averaging strategy: "macro" (unweighted mean across classes) or "weighted" (support-weighted mean).	"macro"
aggregation_method	list of str or str	Dimensions to aggregate over.	"all"
groups	list of str, dict of str to float, or None	Panel group filter or filter with weights.	None
components	list of str, dict of str to float, or None	Component filter or filter with weights.	None

Attributes

Name	Type	Description
lower_is_better	bool	Always False (higher AUC is better).

Examples

>>> import polars as pl
>>> from datetime import datetime
>>> from yohou.metrics.classification import PRAuC
>>> y_true = pl.DataFrame({
...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
...     "weather": ["sunny", "rainy", "cloudy", "sunny", "rainy"],
... })
>>> y_pred = pl.DataFrame({
...     "vintage_time": [datetime(2019, 12, 31)] * 5,
...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
...     "weather_proba_sunny": [0.7, 0.1, 0.2, 0.6, 0.1],
...     "weather_proba_rainy": [0.2, 0.8, 0.1, 0.3, 0.8],
...     "weather_proba_cloudy": [0.1, 0.1, 0.7, 0.1, 0.1],
... })
>>> scorer = PRAuC()
>>> _ = scorer.fit(y_true)
>>> scorer.score(y_true, y_pred)
1.0

See Also

• ROCAuC : ROC AUC.

Source Code

View on GitHub

Show/Hide source

class PRAuC(BaseRankingScorer):
    r"""Precision-Recall AUC from class-probability forecasts.

    Computes the Area Under the Precision-Recall curve using a one-vs-rest
    (OvR) strategy for multiclass problems. Uses ``average_precision_score``
    from scikit-learn.

    Parameters
    ----------
    average : str, default="macro"
        Class averaging strategy: ``"macro"`` (unweighted mean across
        classes) or ``"weighted"`` (support-weighted mean).
    aggregation_method : list of str or str, default="all"
        Dimensions to aggregate over.
    groups : list of str, dict of str to float, or None, default=None
        Panel group filter or filter with weights.
    components : list of str, dict of str to float, or None, default=None
        Component filter or filter with weights.

    Attributes
    ----------
    lower_is_better : bool
        Always False (higher AUC is better).

    Examples
    --------
    >>> import polars as pl
    >>> from datetime import datetime
    >>> from yohou.metrics.classification import PRAuC
    >>> y_true = pl.DataFrame({
    ...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
    ...     "weather": ["sunny", "rainy", "cloudy", "sunny", "rainy"],
    ... })
    >>> y_pred = pl.DataFrame({
    ...     "vintage_time": [datetime(2019, 12, 31)] * 5,
    ...     "time": [datetime(2020, 1, i) for i in range(1, 6)],
    ...     "weather_proba_sunny": [0.7, 0.1, 0.2, 0.6, 0.1],
    ...     "weather_proba_rainy": [0.2, 0.8, 0.1, 0.3, 0.8],
    ...     "weather_proba_cloudy": [0.1, 0.1, 0.7, 0.1, 0.1],
    ... })
    >>> scorer = PRAuC()
    >>> _ = scorer.fit(y_true)
    >>> scorer.score(y_true, y_pred)
    1.0

    See Also
    --------
    - [`ROCAuC`][yohou.metrics.classification.ROCAuC] : ROC AUC.

    """

    _metric_name = "pr_auc"
    _lower_is_better = False

    def _compute_ranking_metric(
        self,
        y_true_binary: np.ndarray,
        y_proba: np.ndarray,
        sample_weight: np.ndarray | None = None,
    ) -> float:
        """Compute PR AUC via sklearn."""
        return float(average_precision_score(y_true_binary, y_proba, sample_weight=sample_weight))