Introduction

Preprocessing in OCaml

OCaml doesn't have a macro system, that is, there is no official part of the OCaml language that will be executed at compile time in order to generate or alter the source code. However, OCaml does have an official part of its syntax dedicated to this: extension nodes and attributes; both of them introduced in OCaml 4.02. The preprocessing itself, though, is left to external programs, written by the community and specialised for their own tasks. However, without a unification framework, the following issues arise:

• Ambiguity when using several preprocessors due to lacking clear composition semantics
• Duplication of code and efforts on the different preprocessors
• Performance loss with many phases (parsing, pretty-printing, etc.) being executed multiple times
• Lack of cross-compiler compatibility
• Incompatibility among the different preprocessor rather than one homogeneous preprocessor ecosystem

ppxlib

The goal of ppxlib is to solve these problems, by providing a unifying framework for writing preprocessors. It sits in between the OCaml compiler and toolchain, and the PPX authors provide an API for them. One could sum up the ppxlib features as:

• It deals with all boilerplate, such as parsing the input, outputting the rewritten output, generating an executable, etc.
• It generates a single executable for multiple transformations and defines clear composition semantics for local transformations
• It integrates well with Dune and Merlin
• It provides a more stable API than the compiler for manipulating the AST
• A single PPX codebase usually works on several OCaml versions of the AST
• It defines restricted rewriters whose semantic ensure better confidence for the user and better compositional semantics
• It provides many helpers to pattern-match and generate AST nodes, as well as traverse the AST.

This Guide

This guide is separated into several parts.

First, we focus on the driver that performs the AST transformations: how it is generated, executed, and more importantly, what it does exactly, from migrating the AST to the different rewriting phases it goes through.

After that, we explain the different kinds of transformations that ppxlib supports, and how to register them to the driver. This section only describes the transformations and their properties, not how to actually manipulate the AST.

The part where we discuss how to manipulate the AST is split in three pages: generating AST nodes to generate OCaml code, destructing AST nodes to extract information and act differently depending on what is extracted, and traversing the AST to use fold, iter, and map on the AST. This code-manipulation part explains using the modules Ast_builder, Ast_pattern, and the ppxlib's PPX Metaquot.

We finally discuss several good practices, such as how to properly report errors, how to test your PPX, or how to migrate from other PPX libraries, such as OMP and ppx_deriving.

We end by including some examples, which you can also find in the examples folder of ppxlib's repository.

PPXs and ppxlib History

The preprocessing history of OCaml started long before ppxlib. However, this section is not yet written. You can find more information in these resources: 1 2 3 5. You can also contribute to the ppxlib documentation by opening a pull request in the repository.