Patterns that Connect: AI, Management, Metaverse, Quantum, Philosophy, and Physics

  A compression trick from 2015 became the subject of a joint NSA, CISA and FBI advisory. The path between those two facts is worth walking slowly. A thing or two about distillation! Copyright: Sanjay Basu The mythical digit Start with the experiment that named the thing. In 2015, Hinton, Vinyals and Dean trained a large net with two hidden layers of 1,200 rectified linear units on MNIST, heavily regularized with dropout and weight constraints, and got 67 test errors. A smaller net with 800 units per layer and no regularization got 146 errors. When that same small net was trained to match the soft target distribution of the large net at a temperature of 20, it got 74 errors. Half the error rate, same architecture, same parameter count, different supervision signal. Then they did the part I keep coming back to. They removed every example of the digit 3 from the transfer set, so the student had never seen a 3 in training. It still made only 206 test errors, 133 of them on the 1,010 t...

                                  Forgetting used to be free. Remembering was the expensive thing. Philosophy Newsletter / №XXVII / Monday, 21 September 2026 A man spent four years trying to make one page about him harder to find. He won, and became the most findable version of himself that has ever existed. I n January 1998, the Barcelona newspaper La Vanguardia printed a short legal notice. A property was going to auction to settle social security debts. The notice named the debtor, Mario Costeja Gonzalez. It was the kind of announcement that exists in order to be published and then ignored, a procedural artifact with a natural half-life of about a day. Then the newspaper digitized its back catalogue. Then Google indexed it. And a decade after the auction, if you searched his name, the notice about his debt was one of the first things you found. Not a summary of his career. Not his professional listin...