Hyperparameters

From Helpful
(Redirected from Hyperparameter)
Jump to navigation Jump to search
This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)


In machine learning, 'hyperparameter' often means "a value you can change that controls the learning process, point is this isn't part of the training data".

This can include

  • the selection of which model to use,
  • things like the the learning rate,
  • the amount of iterations,
  • the depth of a neural net (if this is configurable).

...though it has to be said people are fuzzy with the term.


Might be split into:

  • model hyperparameters - chosen rather than inferred, often because it's a model selection thing
e.g. NN topology, NN size
  • algorithm hyperparameters - have no influence on the learner, just the speed/quality of the learning
e.g. learning rate, batch size


The name seems to be pointing out that this is not a parameter of the model itself, but one step outside, e.g. used to instantiate the learner, not to run that instantiated learner.


This sometimes includes parameters that might be traininable in general, but which for a given task you want to fix in place, e.g. because it won't converge for the given training data.


https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)