3Blue1Brown prediction is compression
by Nate: Claude Shannon
3Blue1Brown has done it again. Language compressibility isn't just a neat mathematical trick,it is the fundamental engine behind modern large language models. Grant's latest video distills Shannon's entropy into one powerful concept: prediction is compression. If you can predict the next word better, you use fewer bits. Check out the video here: Original tweet:
