Lessons from building a succinct blockchain with OCaml

This talk is a dissection of building a new blockchain technology with OCaml, including unique applications of the language and lessons learned from mistakes made in a large OCaml codebase.

Project and presentation overview

Building an new, experimental blockchain is no easy task, and as with any complex problem with various areas of concern (security, speed, portability, etc…), language choice is key. In this talk, we will discuss why we chose OCaml as the programming language for building a succinct decentralized blockchain heavily utilizing zk-SNARKS. An overview of the goals behind the project will be laid out, that those goals will be used to examine the choice of OCaml over other languages. The presentation will focus on problems we have encountered as a team while moving quickly in a large OCaml codebase, as well as the boons we reaped from choosing OCaml.

Our blockchain, Coda, aims to be a fully scalable blockchain with a fixed size. This is achieved through a unique use of recursive zk-SNARKS in order to compress a proof of blockchain state transitions, eliminating the need for the blockchain to store the entire set of blocks ever produced. This enables any node on the network to validate the correctness of information with an extraordinarily small amount of data and processing time. This talk will not focus on the details of how we achieve this, but rather our experiences creating this technology using OCaml.

Why we chose OCaml

Coda was originally being implemented in Javascript in order to take advantage of its lightweight requirements and run nodes on the network inside of web browsers. However, Javascript proved to be too painful to write a program of this complexity in, so another language was needed. OCaml was chosen for its strong type system (to help ensure correctness), focus on functional programming, overall performance, easy to use FFI, and its portability both across operating systems and architectures and its ability to run on the browser.

One of the notable items that will be discussed thoroughly regarding our decision of OCaml is our use of the language to embed a monadic DSL for specifying SNARKS as programs. This proved to be a key tool in our project, enabling us to quickly and confidently write programs which are translated into a rank 1 constraint system. This embedded language, which we coin “snarky”, lets anyone treat constraint system programming as a more familiar programming problem, and provides tools to evaluate and debug the programs that are constructed.

Lessons learned

As part of this presentation, I wish to impart a some organizational and structural problems we ran into with our architectural patterns in OCaml. Here are some of the items which will be discussed:

Mistakes in our use of functors and module signatures Lack of clearly defined boundaries between libraries Use of a poorly defined, overly flexible asynchronous communication primitive

For each of these items, we will go over the negative effects which were incurred due to the problem, and discuss how we the issues. The hope is that other developers can be more informed when starting up their own large OCaml projects, and to provide a rough guideline to help developers use functors effectively without making their lives harder down the road.