RandomizedSearchCV¶
yohou.model_selection.search.RandomizedSearchCV
¶
Bases: BaseSearchCV
Randomized search on hyperparameters.
Important members are fit, predict, predict_interval, observe, and score.
RandomizedSearchCV implements a "fit" method that samples n_iter
parameter settings from the specified distributions using time series
cross-validation. In contrast to GridSearchCV, not all parameter values
are tried out, but rather a fixed number of parameter settings is sampled
from the specified distributions. The number of parameter settings that
are tried is given by n_iter.
It also implements "predict", "predict_interval", "observe", "rewind", "observe_predict", and "observe_predict_interval" if the underlying forecaster supports these methods and refit=True.
Parameters¶
| Name | Type | Description | Default |
|---|---|---|---|
forecaster
|
BaseForecaster
|
A forecaster object implementing the yohou forecaster interface with fit and predict methods. |
required |
param_distributions
|
dict or list of dict
|
Dictionary with parameter names ( Distributions are assumed to implement the Examples:: |
required |
n_iter
|
int
|
Number of parameter settings that are sampled. Higher |
10
|
scoring
|
BaseScorer or dict of {str: BaseScorer}
|
Strategy to evaluate the performance of the cross-validated model on the test set. If a single BaseScorer instance, the same scorer is used for all folds and stored in cv_results_ with key 'score'. If a dict, keys are scorer names and values are BaseScorer instances.
This enables multi-metric evaluation. The Unlike sklearn, string scorer names are not supported. You must use yohou.metrics BaseScorer instances (e.g., MeanAbsoluteError(), RootMeanSquaredError()). Examples:: Note: For multi-metric evaluation with dict, cv_results_ will contain keys like 'mean_test_mae', 'rank_test_mae', 'mean_test_rmse', etc. |
None
|
n_jobs
|
int
|
Number of jobs to run in parallel.
|
None
|
refit
|
bool, str, or callable
|
Refit a forecaster using the best found parameters on the whole dataset. For multiple metric evaluation, this needs to be a Where there are considerations other than maximum score in
choosing a best forecaster, The refitted forecaster is made available at the Also for multiple metric evaluation, the attributes See Examples:: |
True
|
cv
|
int, BaseSplitter, or None
|
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For time series data, typical splitters are:
|
None
|
verbose
|
int
|
Controls the verbosity: the higher, the more messages.
|
0
|
pre_dispatch
|
int or str
|
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
|
'2*n_jobs'
|
random_state
|
int, RandomState instance or None
|
Pseudo random number generator state used for random uniform sampling from lists of possible values instead of scipy.stats distributions. Pass an int for reproducible output across multiple function calls. |
None
|
error_score
|
'raise' or numeric
|
Value to assign to the score if an error occurs in forecaster fitting. If set to 'raise', the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error. |
np.nan
|
return_train_score
|
bool
|
If Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance. |
False
|
Attributes¶
| Name | Type | Description |
|---|---|---|
cv_results_ |
dict of numpy (masked) ndarrays
|
A dict with keys as column headers and values as columns, that can be
imported into a pandas For instance the below given table:: will be represented by a cv_results_ dict of:: NOTE: The key The For multi-metric evaluation, the scores for all the scorers are
available in the |
best_forecaster_ |
BaseForecaster
|
Forecaster that was chosen by the search, i.e. forecaster which gave
highest score (or smallest loss if specified) on the left out data.
Not available if See |
best_score_ |
float
|
Mean cross-validated score of the best_forecaster_. Follows sklearn's sign convention: for For multi-metric evaluation, this is present only if This attribute is not available if |
best_params_ |
dict
|
Parameter setting that gave the best results on the hold out data. For multi-metric evaluation, this is present only if |
best_index_ |
int
|
The index (of the The dict at For multi-metric evaluation, this is present only if |
scorer_ |
BaseScorer or dict
|
Scorer function(s) used on the held out data to choose the best parameters for the model. For multi-metric evaluation, this attribute holds the validated
|
n_splits_ |
int
|
The number of cross-validation splits (folds/iterations). |
refit_time_ |
float
|
Seconds used for refitting the best forecaster on the whole dataset. This is present only if |
multimetric_ |
bool
|
Whether or not the scorers compute several metrics. |
n_features_in_ |
int
|
Number of features seen during |
feature_names_in_ |
ndarray of shape (n_features_in_,)
|
Names of features seen during |
See Also¶
GridSearchCV: Exhaustive search over specified parameter values.ExpandingWindowSplitter: Cross-validation with expanding training windows.SlidingWindowSplitter: Cross-validation with sliding fixed-size windows.MeanAbsoluteError: Mean absolute error scorer.RootMeanSquaredError: Root mean squared error scorer.
Notes¶
The parameters selected are those that maximize the score of the left out data, unless an explicit scorer is passed in which case it is used instead.
If n_jobs was set to a value higher than one, the data is copied for each
parameter setting (and not n_jobs times). This is done for efficiency
reasons if individual jobs take very little time, but may raise errors if
the dataset is large and not enough memory is available. A workaround in
this case is to set pre_dispatch. Then, the memory is copied only
pre_dispatch many times. A reasonable value for pre_dispatch is
2 * n_jobs.
RandomizedSearchCV is particularly useful when the parameter space is large or when evaluating each parameter setting is expensive. By sampling a fixed number of settings, you can control the computational budget while still exploring the parameter space effectively. For high-dimensional parameter spaces, random search can be more efficient than grid search at finding good parameter settings.
Examples¶
>>> from yohou.point import PointReductionForecaster
>>> from yohou.model_selection import RandomizedSearchCV
>>> from yohou.metrics import MeanAbsoluteError
>>> from scipy.stats import uniform, randint
>>> import polars as pl
>>> from datetime import datetime, timedelta
>>> # Create sample data
>>> dates = [datetime(2020, 1, 1) + timedelta(days=i) for i in range(100)]
>>> y = pl.DataFrame({"time": dates, "value": range(100)})
>>> # Define parameter distributions
>>> param_distributions = {
... "estimator__alpha": uniform(0.01, 10.0),
... "feature_transformer__lags": [[1], [1, 2], [1, 2, 3]],
... }
>>> # Single-metric search
>>> search = RandomizedSearchCV(
... forecaster=PointReductionForecaster(),
... param_distributions=param_distributions,
... n_iter=20,
... scoring=MeanAbsoluteError(),
... cv=3,
... random_state=42,
... )
>>> search.fit(y, forecasting_horizon=5)
>>> search.best_params_
>>> y_pred = search.predict(forecasting_horizon=5)
>>>
>>> # Multi-metric search with custom refit strategy
>>> from yohou.metrics import RootMeanSquaredError
>>> scoring = {"mae": MeanAbsoluteError(), "rmse": RootMeanSquaredError()}
>>> # Custom refit: Choose parameters with best MAE, but also consider RMSE
>>> def refit_strategy(cv_results):
... # Find candidates where MAE rank is in top 5
... mae_ranks = cv_results["rank_test_mae"]
... top_mae_mask = mae_ranks <= 5
... # Among those, pick the one with best RMSE
... rmse_scores = cv_results["mean_test_rmse"]
... rmse_scores_masked = np.ma.array(rmse_scores, mask=~top_mae_mask)
... return np.ma.argmax(rmse_scores_masked) # Higher RMSE score is better
>>> search = RandomizedSearchCV(
... forecaster=PointReductionForecaster(),
... param_distributions=param_distributions,
... n_iter=20,
... scoring=scoring,
... refit=refit_strategy,
... cv=3,
... random_state=42,
... )
>>> search.fit(y, forecasting_horizon=5)
>>> search.best_params_
>>> # cv_results_ contains both metrics
>>> search.cv_results_["mean_test_mae"]
>>> search.cv_results_["mean_test_rmse"]
Source Code¶
Show/Hide source
1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 | |
Tutorials¶
The following example notebooks use this component:
-
How to Run Hyperparameter Search
Evaluation-Search
Tune forecaster hyperparameters with GridSearchCV and RandomizedSearchCV using temporal cross-validation splitters and result scatter visualisation.
-
Quickstart
Quickstart
Comprehensive end-to-end tour of yohou beyond the Getting Started tutorials, covering data loading, baseline forecasting, preprocessing pipelines, decomposition, cross-validation search, and interval prediction.