Tree-based Federated Learning

Federated learning has become a powerful way to train machine learning models across many devices without collecting raw data in one place. But most federated systems rely on deep neural networks, which are often black boxes. That makes them hard to interpret, computationally heavy, and sometimes a poor fit for tabular data where decision trees usually shine.

The central challenge preventing the widespread adoption of tree-based models in federated learning is that decision trees are discrete, symbolic objects. They do not have the smooth gradients that neural networks use for optimization, so standard federated training does not directly apply and ad hoc methods are required to train a shared tree across many clients.

To address this, I have worked on TreeClimber (Augello et al., 2026), a neurosymbolic framework that enables federated learning of decision trees by mapping them to a continuous latent space. Trees are encoded as vectors using a randomized Weisfeiler-Lehman embedding and a neural network is trained to learn the inverse mapping on a large corpus of randomly generated trees. The availability of an invertible vector representation allows for continuous optimization of tree structures in a federated setting, while still producing a final model that is a single interpretable decision tree.

At the current stage, given that decision trees are discrete and not directly differentiable, the local optimization of the tree embedding is performed using zeroth-order optimization, which only requires function evaluations and does not rely on gradients.