Softmax: Difference between revisions
Jump to navigation
Jump to search
mNo edit summary |
mNo edit summary |
||
Line 1: | Line 1: | ||
{{stub}} | |||
<!-- | <!-- | ||
softmax (a.k.a. softargmax, normalized exponential function) | softmax (a.k.a. softargmax, normalized exponential function) | ||
* takes a vector of numbers | * takes a vector of numbers | ||
* provides a vector of probabilities | * provides a vector of probabilities | ||
:: all in 0..1 | :: all in 0 .. 1 | ||
:: | :: that sum to 1.0 | ||
Many | Many references you will find ''now'' are its use in neural nets, | ||
where they take activation on any sort, and put them into 0..1 scale sensibly, | |||
as a normalization step that is often used at least in the final layer, and sometimes at the end of smaller building blocks as well. | |||
When using nets as multiclass classifiers, you would need ''something'' like softmax to be able to respond on all the labels, | |||
and in a way that looks like probabilities. | |||
In part it's just a choice of what you want to show (you could output classification margin scores instead), | |||
in part it's a choice that | |||
Note that it is ''not'' just normalization. | |||
Nor is just a way to bring out the strongest answer. | |||
Both its exponent internals and the "will sum to 1.0 part" will mean things shift around in a non-linear way, | |||
so even relative probabilities already in in 0..1 and summing to 1.0 will change, e.g. | |||
: softmax([1.0,0.5,0.1]) ~= 0.5, 0.3, 0.2, | |||
: softmax([0.5, 0.3, 0.2]) ~= 0.4, 0.31, 0.28 | |||
While the exponent makes it look like some choices of sigmoid functions, | |||
And it isn't directly comparable to transfer functions, and you can't get an easy graph of it, exactly ''because'' it takes multiple inputs. | |||
But also, it's a more general mathematical tool, even if it's mostly seen in machine learning. | |||
Revision as of 15:22, 28 November 2023
✎ This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.