Continuous-time models & neural ODEs

Most deep networks are discrete: an input passes through a fixed stack of layers and an output comes out. Continuous-time models take a different view — they define the hidden state as something that evolves over time, governed by a differential equation, rather than hopping through a set number of layers.

Neural ODEs

The foundational idea is the neural ODE (neural ordinary differential equation). Instead of learning what each layer does, the network learns the rate of change of its hidden state, and a solver integrates that to produce the output. "Depth" becomes the length of time you integrate — a continuous quantity you can adjust, not a fixed count.

Why this is more than elegant

The payoff is data that arrives irregularly. Real-world signals — sensor readings, medical records, financial ticks — don't come at neat, evenly spaced steps. A continuous-time model handles gaps naturally, because it is defined over continuous time in the first place, instead of pretending every step is the same size.

The speed problem — and the fix

The cost is that solving a differential equation at inference is slow. The closed-form continuous-time (CfC) models were the breakthrough here: they approximate the same continuous dynamics with an explicit formula, so no solver runs in the loop. That made continuous-time models fast enough to be practical.

The connection to "liquid"

Liquid neural networks are continuous-time models with an extra twist: their time-constants depend on the input, so the dynamics adapt as data flows. Continuous time is the foundation; liquid is what you build on top. Combine that with attention, and you reach the liquid transformer.

Related reading: What is a liquid transformer? · Liquid neural networks, explained