#probability
3 posts · all tags
-
· From Entropy to Epiplexity
In this paper, Finzi et al. argue that classical information theory (Shannon entropy and Kolmogorov complexity) is insufficient for understanding modern AI because it assumes an observer with unlimited computational power. Under classical theory, deterministic transformations cannot create new information, yet empirically, processes like self-play (AlphaZero) and synthetic data generation clearly improve model capabilities.
-
· Probability Paradigms
When we think about probability, there are three main philosophical approaches: frequentist, Bayesian, and propensity. Each offers a different lens for understanding uncertainty and probability.
-
· Some common divergences
Hey. usually on machine learning we have to consider distances between different probability distributions. This is in fact a hard problem and there is a misconception that the most common way to do this is using the kl divergence. I believe that this is not true.