API

class gfsopt.GFSOptimizer(pp=None, space=None, solver_epsilon=None, relative_noise_magnitude=None, fname=None, save=False)

Global Function Search (GFS) Optimizer. Creates a GFS optimizer for optimizing a set of hyperparameters. Supports parallel optimization runs and averaging of stochastic optimization runs, as well as saving/restoring both settings and progress to file.

Specify ‘pp’+’space’ and/or ‘fname’. If ‘fname’ is given, attempt to restore progress and settings from file. If restoring fails, continue with specified/default settings i.e. (pp, space, solver_epsilon, relative_noise_magnitude). If restoring succeeds, then any setting passed as argument in addition to the file name will be compared to the setting restored from the file, and you will be given the choice of which one to use.

Parameters:
  • pp (dict) – Problem parameters. All hyperparameters and their values for the objective function, including those not being optimized over. E.g: {'beta': 0.44}. Can be an empty dict. Can include hyperparameters being optimized over, but does not need to. If a hyperparameter is specified in both ‘pp’ and ‘space’, its value in ‘pp’ will be overridden.
  • space (dict) – Hyperparameters to optimize over. Entries should be of the form: parameter: (Low_Bound, High_Bound) e.g: {'alpha': (0.65, 0.85), 'gamma': (1, 8)}. If both bounds for a parameter are Ints, then only integers within the (inclusive) range will be sampled and tested.
  • fname (str) – File name for restoring and/or saving results, progress and settings.
  • save (bool) – (optional) Save settings and progress periodically, on user quit (CTRL-C), and on completion.
  • solver_epsilon (float) – (optional) The accuracy to which local optima are determined before global exploration is resumed. See Dlib for further documentation. Default: 0.0005
  • relative_noise_magnitude (float) –

    (optional) Should be increased for highly stochastic objective functions. Deterministic and continuous functions can use a value of 0. See Dlib for further documentation. Default: 0.001
run(obj_func, n_concurrent=None, n_avg=1, n_sims=1000, save_iter=30)

Run optimization.

Parameters:
  • obj_func (func) – function to maximize. Must take as argument every parameter specified in both ‘pp’ and ‘space’, in addition to ‘pid’, and return the result as float. ‘pid’ specifies simulation run number. If you want to minimize instead, simply negate the result in the objective function before returning it.
  • n_concurrent (int) – (optional) Number of concurrent procs. If ‘None’ or unspecified, then use as all logical cores
  • n_avg (int) – (optional) Number of runs to average over.
  • n_sims (int) – (optional) Number of times to sample and test params.
  • save_iter (int) – (optional) How often to save progress.
gfsopt.print_best(fname, n=1, minimum=False)

Load results from file specified by file name ‘fname’ and print the ‘n’ best results, where best means maximum by default and minimum if ‘minimum’ is specified.

Parameters:
  • fname (str) – File name
  • n (int) – (optional) Number of results to print
  • minimum (bool) – (optional) If lower result is better