Archives

Consider the Group Sequence Policy Optimization (GSPO) objective for reinforcement learning with large language models.

The trick that you need when you need to differentiate through a sampling process -or in other words- through a stochastic node.

In this post I show that the softmax function which is used to convert logits to probabilities, is actually utility maximization(with specific type of error on the utility).

Just a simple chain-rule we need to encapsulate in a name.

Introducing the gumbel max trick and connecting it with the literature.

ow many different ways can we think of to interpret softmax function?

What kind of probability distribution you get when you softmax your logits?