[AI Compute Symposium 2022] Machine learning for real: Thinking more carefully about efficiency, loss functions, and GANs

Deep learning seems to touch every discipline these days, but behind its startling magic tricks, it is surprisingly primitive. It is concerning to note the extent to which today’s deep learning relies on folklore: on recipes and anecdotes, rather than on scientific principles and explanatory mathematics. Think of how much more trustworthy, robust, compact, and power-efficient our models would be if we designed them more rigorously. Our CEO Steve Teig’s assertions are accompanied by some motivating (and occasionally humorous) examples.