btb.tuning.tuners.base module

Package where the BaseTuner class and BaseMetaModelTuner are defined.

class btb.tuning.tuners.base.BaseMetaModelTuner(tunable, maximize=True, num_candidates=1000, min_trials=5)[source]

Bases: btb.tuning.tuners.base.BaseTuner, btb.tuning.metamodels.base.BaseMetaModel, btb.tuning.acquisition.base.BaseAcquisition

BaseMetaModelTuner class.

BaseMetaModelTuner class is the abstract representation of a tuner that is based on a model and an Acquisition. This model will try to predict the score that will be obtained with the proposed parameters by being trained over the self.trials and self.raw_scores recorded by the user.

tunable

Instance of a tunable class containing hyperparameters to be tuned.

Type

btb.tuning.tunable.Tunable

trials

A numpy.ndarray with shape (n, self.tunable.dimensions) where n is the number of trials recorded.

Type

numpy.ndarray

scores

A numpy.ndarray with shape (n, 1) where n is the number of scores recorded.

Type

numpy.ndarray

Parameters

  • tunable (btb.tuning.tunable.Tunable) – Instance of a tunable class containing hyperparameters to be tuned.

  • num_candidates (int) – Number of samples to generate and select the best of it for each proposal. Defaults to 1000.

  • maximize (bool) – If True the model will understand that the score bigger is better, if False the smaller is better. Defaults to True.

  • min_trials (int) – Number of recorded trials needed to perform a fitting over the model. Defaults to 5.

record(trials, scores)[source]

Record one or more trials with the associated scores and re-fit the model.

Trials are recorded with the associated scores to them. The amount of trials must be equal to the amount of scores recived and vice versa. Once recorded, the model is being fitted with self.trials and self.raw_scores that contain any previous records and the ones that where just recorded.

Parameters

  • trials (pandas.DataFrame, pandas.Series, dict, list(dict), 2D array-like) – Values of shape (n, len(self.tunable.hyperparameters)) or dict with keys that are self.tunable.names.

  • scores (single value or array-like) – A single value or array-like of values representing the score achieved with the trials.

Raises

ValueError – A ValueError exception is being produced if len(trials) is not equal to len(scores).

Example

The example below shows simple usage case where an UniformTuner is being imported, instantiated with a tunable object and it’s method record is being called two times with valid trials and scores.

>>> from btb.tuning.tunable import Tunable
>>> from btb.tuning.hyperparams import BooleanHyperParam
>>> from btb.tuning.hyperparams import CategoricalHyperParam
>>> from btb.tuning.tuners import UniformTuner
>>> bhp = BooleanHyperParam()
>>> chp = CategoricalHyperParam(['cat', 'dog'])
>>> tunable = Tunable({'bhp': bhp, 'chp': chp})
>>> tuner = UniformTuner(tunable)
>>> tuner.record({'bhp': True, 'chp': 'cat'}, 0.8)
>>> trials = [{'bhp': False, 'chp': 'cat'}, {'bhp': True, 'chp': 'dog'}]
>>> scores = [0.8, 0.1]
>>> tuner.record(trials, scores)
class btb.tuning.tuners.base.BaseTuner(tunable, maximize=True)[source]

Bases: object

BaseTuner class.

BaseTuner class is the abstract representation of a tuner that is not based on a model.

tunable

Instance of a tunable class containing hyperparameters to be tuned.

Type

btb.tuning.tunable.Tunable

trials

A numpy.ndarray with shape (n, self.tunable.dimensions) where n is the number of trials recorded.

Type

numpy.ndarray

raw_scores

A numpy.ndarray with shape (n, 1) where n is the number of scores recorded.

Type

numpy.ndarray

scores

A numpy.ndarray with shape (n, 1) where n is the number of normalized scores recorded.

Type

numpy.ndarray

Parameters

  • tunable (btb.tuning.tunable.Tunable) – Instance of a tunable class containing hyperparameters to be tuned.

  • maximize (bool) – If True the scores are interpreted as bigger is better, if False then smaller is better. Defaults to True.

propose(n=1, allow_duplicates=False)[source]

Propose one or more new hyperparameter configurations.

Validate that the amount of proposals requested is valid when allow_duplicates is False and raise an exception in case there is any missmatch between n, unique self.trials and self.tunable.cardinality. Call the implemented _propose method and convert the returned data in to hyperparameter space values.

Parameters

  • n (int) – Number of candidates to create. Defaults to 1.

  • allow_duplicates (bool) – If it’s False, the tuner will propose trials that are not recorded. Otherwise will generate trials that can be repeated. Defaults to False.

Returns

If n is 1, a dict will be returned containing the hyperparameter names and values. Otherwise, if n is bigger than 1, a list of such dicts is returned.

Return type

dict or list

Raises

  • ValueError – A ValueError exception is being produced if the amount of requested proposals is bigger than the possible combinations and allow_duplicates is False.

  • ValueError – A ValueError exception is being produced if the unique amount of recorded trials is the same as the amount of combinations available for self.tunable.

  • ValueError – A ValueError exception is being produced if the unique amount of recorded trials is the same as the amount of combinations available for self.tunable.

Example

The example below shows simple usage case where an UniformTuner is being imported, instantiated with a tunable object and it’s method propose is being called three times, first with a single proposal, a second with two proposals forcing them to be different and once where the values can be repeated.

>>> from btb.tuning.tunable import Tunable
>>> from btb.tuning.hyperparams import BooleanHyperParam
>>> from btb.tuning.hyperparams import CategoricalHyperParam
>>> from btb.tuning.tuners import UniformTuner
>>> bhp = BooleanHyperParam()
>>> chp = CategoricalHyperParam(['cat', 'dog'])
>>> tunable = Tunable({'bhp': bhp, 'chp': chp})
>>> tuner = UniformTuner(tunable)
>>> tuner.propose(1)
{'bhp': True, 'chp': 'dog'}
>>> tuner.propose(2)
[{'bhp': True, 'chp': 'cat'}, {'bhp': True, 'chp': 'dog'}]
>>> tuner.propose(2, allow_duplicates=True)
[{'bhp': False, 'chp': 'dog'}, {'bhp': False, 'chp': 'dog'}]
record(trials, scores)[source]

Record one or more trials with the associated scores.

Trials are recorded with their associated scores. The amount of trials must be equal to the amount of scores recived and vice versa.

Parameters

  • trials (pandas.DataFrame, pandas.Series, dict, list(dict), 2D array-like) – Values of shape (n, len(self.tunable.hyperparameters)) or dict with keys that are self.tunable.names.

  • scores (single value or array-like) – A single value or array-like of values representing the score achieved with the trials.

Raises

ValueError – A ValueError exception is being produced if len(trials) is not equal to len(scores).

Example

The example below shows simple usage case where an UniformTuner is being imported, instantiated with a tunable object and it’s method record is being called two times with valid trials and scores.

>>> from btb.tuning.tunable import Tunable
>>> from btb.tuning.hyperparams import BooleanHyperParam
>>> from btb.tuning.hyperparams import CategoricalHyperParam
>>> from btb.tuning.tuners import UniformTuner
>>> bhp = BooleanHyperParam()
>>> chp = CategoricalHyperParam(['cat', 'dog'])
>>> tunable = Tunable({'bhp': bhp, 'chp': chp})
>>> tuner = UniformTuner(tunable)
>>> tuner.record({'bhp': True, 'chp': 'cat'}, 0.8)
>>> trials = [{'bhp': False, 'chp': 'cat'}, {'bhp': True, 'chp': 'dog'}]
>>> scores = [0.8, 0.1]
>>> tuner.record(trials, scores)
exception btb.tuning.tuners.base.StopTuning[source]

Bases: Exception