Probabilistic programming languages are valuable because they allow domain experts to express probabilistic models and inference algorithms without worrying about irrelevant details. However, for decades there remained an important and popular class of probabilistic inference algorithms whose efficient implementation required manual low-level coding that is tedious and error-prone. They are algorithms whose idiomatic expression requires random array variables that are latent or whose likelihood is conjugate. Although that is how practitioners communicate and compose these algorithms on paper, executing such expressions requires eliminating the latent variables and recognizing the conjugacy by symbolic mathematics. Moreover, matching the performance of handwritten code requires speeding up loops by more than a constant factor.
We show how probabilistic programs that directly and concisely express these desired inference algorithms can be compiled while maintaining efficiency. We introduce new transformations that turn high-level probabilistic programs with arrays into pure loop code. We then make great use of domain-specific invariants and norms to optimize the code, and to specialize and JIT-compile the code per execution. The resulting performance is competitive with manual implementations.
Tue 20 AugDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 15:00
|Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator
Fei Wang , Dan Zheng Purdue University, Google Brain, James Decker , Xilun Wu Purdue University, Gregory Essertel Purdue University, Tiark Rompf Purdue UniversityPre-print
|Efficient Differentiable Programming in a Functional Array-Processing Language
|From high-level inference algorithms to efficient code
Rajan Walia Indiana University, Praveen Narayanan Indiana University, USA, Jacques Carette McMaster University, Sam Tobin-Hochstadt Indiana University, Chung-chieh Shan Indiana University, USAPre-print
|Sound and robust solid modeling via exact real arithmetic and continuityDistinguished Paper
Benjamin Sherman Massachusetts Institute of Technology, USA, Jesse Michel Massachusetts Institute of Technology, Michael Carbin Massachusetts Institute of TechnologyDOI Pre-print Media Attached