Thanks to Hao Zhang, Kayvon Fatahalian, and Jean-Stanislas Denain for helpful discussions and comments. Addendum and erratum. See here [https://kipp.ly/blog/transformer-inference-arithmetic/] for an excellent discussion of similar ideas by Kipply

Last week [https://bounded-regret.ghost.io/early-2022-paper-round-up/], I talked about six recent papers from our group, and discussed the first two in detail. This week, I'll discuss the remaining four. They

My students and collaborators have been doing some particularly awesome work over the past several months, and to highlight that I wanted to summarize their papers here, and explain why I’m excited

In Future ML Systems Will Be Qualitatively Different [https://bounded-regret.ghost.io/p/1527e9dd-c48d-4941-9b14-4f7293318d5c/], I argued that we should expect ML systems to exhibit emergent capabilities. My main support for this was four

Previously, I argued [https://bounded-regret.ghost.io/p/1527e9dd-c48d-4941-9b14-4f7293318d5c/] that emergent phenomena in machine learning mean that we can't rely on current trends to predict what the future of ML will