Probabilistic programming languages are valuable because they allow domain experts to express probabilistic models and inference algorithms without worrying about irrelevant details. However, for decades there remained an important and popular class of probabilistic inference algorithms whose efficient implementation required manual low-level coding that is tedious and error-prone. They are algorithms whose idiomatic expression requires random array variables that are latent or whose likelihood is conjugate. Although that is how practitioners communicate and compose these algorithms on paper, executing such expressions requires eliminating the latent variables and recognizing the conjugacy by symbolic mathematics. Moreover, matching the performance of handwritten code requires speeding up loops by more than a constant factor.
We show how probabilistic programs that directly and concisely express these desired inference algorithms can be compiled while maintaining efficiency. We introduce new transformations that turn high-level probabilistic programs with arrays into pure loop code. We then make great use of domain-specific invariants and norms to optimize the code, and to specialize and JIT-compile the code per execution. The resulting performance is competitive with manual implementations.
Tue 20 Aug
|13:30 - 13:52|
Fei Wang, Dan ZhengPurdue University, Google Brain, James Decker, Xilun WuPurdue University, Gregory EssertelPurdue University, Tiark RompfPurdue UniversityPre-print
|13:52 - 14:15|
|14:15 - 14:37|
Rajan WaliaIndiana University, Praveen NarayananIndiana University, USA, Jacques CaretteMcMaster University, Sam Tobin-HochstadtIndiana University, Chung-chieh ShanIndiana University, USAPre-print
|14:37 - 15:00|
Benjamin ShermanMassachusetts Institute of Technology, USA, Jesse MichelMassachusetts Institute of Technology, Michael CarbinMassachusetts Institute of TechnologyDOI Pre-print Media Attached