Archives

Advanced AI could push interest rates significantly lower over the next decade, not higher.

Consider the Group Sequence Policy Optimization (GSPO) objective for reinforcement learning with large language models.

We call it 'importance,' but the algorithm never picks 'important' points, it re-weights ordinary ones.

The trick that you need when you need to differentiate through a sampling process -or in other words- through a stochastic node.

In this post I show that the softmax function which is used to convert logits to probabilities, is actually utility maximization(with specific type of error on the utility).

Just a simple chain-rule we need to encapsulate in a name.

Introducing the gumbel max trick and connecting it with the literature.

ow many different ways can we think of to interpret softmax function?

What kind of probability distribution you get when you softmax your logits?