Guest lecture · Timothy Oh · arXiv 2603.18066

A chip design where learning is the physics.

Timothy Oh's single-author paper implements predictive coding networks as synthesizable RTL. No backward pass. No instruction stream. The task enters the machine as boundary conditions on a fixed dynamical fabric.

Timothy Oh, sole author · arXiv 2603.18066v2 · open SystemVerilog on GitHub · eleven slides, narrated

The problem

Guest lecture · full write-up

Slide 2 of 11 · Q: Why can't you just put backprop in hardware?

Backprop needs a global conductor. Distributed silicon has none.

Constraint 1

Global error propagation

Errors travel backward through every layer
Dependency chain spans the whole network

Constraint 2

Phase separation

Forward, backward, update: three synced phases
Activations stored until the backward pass ends

Constraint 3

Centralized memory

Heavy data movement through shared memory
Fine on GPUs, hostile to embedded substrates

The alternative Predictive coding: inference and learning both fall out of local prediction-error dynamics between adjacent layers.

The dynamics

Guest lecture · full write-up

Slide 3 of 11 · Q: What replaces the backward pass?

One quadratic energy. Every gradient term is local.

Prediction-error energy · each layer predicts the layer below

Per-neuron updates · state step, Hebbian weight step, bottom-up term

Locality A neuron touches only: its own state and error, activity from the layer above, weighted errors from the layer below. Nothing global survives.

The hardware

Guest lecture · full write-up

Slide 4 of 11 · Q: What does one neuron look like in silicon?

One neuron, one core, six FSM stages, every tick.

The schedule

PRED → ERR → BACKSUM → BACKVEC → WUP → STATE · predict, form error, gather lower errors, emit products upward, Hebbian update, Euler state step

What a core holds

Its scalar state, its error, its own weight row plus a bias lane. No shared parameter memory anywhere.

Arithmetic

IEEE-754 float32, HardFloat recFN, one sequential MAC reused across all indices. Point-to-point wiring, adjacent layers only.

Per-tick cost · linear in fan-in N and back-error fan-in M

The primitive

Guest lecture · full write-up

Slide 5 of 11 · Q: How does one substrate both train and infer?

Same schedule always. The task is a boundary condition.

Training

Clamp input and target layers

Per-neuron clamp: x_set_en, x_obs, CLAMP_HARD
Errors flow between the pinned boundaries
Weights move via the local Hebbian rule

Inference

Clamp input only, read the free output

Set learning rate to zero, run a tick budget
State settles under the same six stages
No mode bit, no separate datapath

Mortal computation He places this next to Hinton's frame: the FSM schedule and datapath are the process. No separation between hardware and software.

The evidence

Guest lecture · full write-up

Slide 6 of 11 · Q: Does it actually learn?

Everything converges or stagnates. Nothing ever blows up.

0.341→0.006

Teacher-student ReLU regression MSE, 25 epochs. The tanh net drops two orders of magnitude in three epochs.

verilator sim of the real RTL

64/64

Points in the (α, γ) sweep with zero divergence and zero oscillation. Convergence gate: γ ≥ 0.07.

stability phase diagram

T≈50

Inference ticks per sample where gains saturate. One tick barely learns; fifty gets MSE 0.0077.

tick-budget knee

Scaling Three architectures up to 8→16→8 train with zero RTL changes. Only compile-time dimensions differ.

The comparison

Guest lecture · full write-up

Slide 7 of 11 · Q: What does it give up against backprop?

Trades a little accuracy for a lot of consistency.

PC network, online

Fast, tight across seeds

Converges within 100-200 samples, one pass, no replay
Test MSE ≈ 0.055, seed spread under 0.003

Backprop MLP, matched

Better mean, wild variance

Mean ≈ 0.045, slower per sample
One seed stalls at 0.094, another reaches 0.014

Fidelity Float64 reference vs float32 emulation over 500 ticks: peak gap 4.77×10^-8, one float32 ULP, flat through both phases. Rounding never compounds.

Act 2 · a shared line of work

Guest lecture · full write-up

Slide 8 of 11 · Q: Who else has attacked this problem?

2021: Bhargava, Rezaei & Lankarany deleted the gradient entirely.

The design

Every synapse is an RL agent

Three actions: nudge up, nudge down, do nothing
State: its own last two actions plus two global rewards
Reward: one bit, did total loss go down
One shared policy for every synapse in the net

The result

20,000 synapses, converged

notMNIST: 88.45% vs 89.56% for gradient descent
At zero hidden units it beat gradient descent
Frozen policy transferred across shapes and tasks
Cost: 5.5 hours vs 20 minutes

Their closing line The discussion ends on cellular automata: Wolfram, Conway, complex behavior from simple local rules.

Act 2 · the middle setting

Guest lecture · full write-up

Slide 9 of 11 · Q: What sits between those two?

A scotopic-vision NCA is the same clamping, on a grid.

The setup: train a cellular automaton to predict video pixels with 99% of pixels randomly deleted. Same code in every cell, neighbors only.

The state layout: one channel holds the cell's prediction, one is the input register that latches an arriving observation, the rest the cells allocate themselves.

The mechanism: an arriving pixel pins the cell, unobserved cells run free, and the gap between observation and prediction drives the update. That is Oh's clamping primitive.

The next step it named: put it in silicon. A PixelRNN-style retina with Piotr Dudek. Oh built the layered version of that silicon.

The difference Backprop is still there, but only at design time. What ships is a purely local rule.

Act 2 · the synthesis

Guest lecture · full write-up

Slide 10 of 11 · Q: What do the three works say together?

One dial: how much global machinery does learning need?

Work

What stays global · and what it costs

Synaptic RL · 2021

One scalar bit per step. Did loss go down. Cost: sixteen times slower, simulated on one CPU.

Scotopic NCA · 2024

One offline training run. Backprop through time, at design time only. Cost: the loss must be differentiable.

PCN in RTL · 2026

A clock edge. Plus the choice of which layers to clamp. Cost: latency linear in fan-in.

Where the intelligence lives Learned by RL, then frozen. Learned by gradients, then frozen. Or derived from theory and fixed in the wiring. Oh sits at the far end.

The read

Guest lecture · full write-up

Slide 11 of 11 · Q: What is the open question?

Learned, or derived? The three papers disagree.

Where this leaves things

What the RTL paper does well: the equations map one-to-one onto FSM stages, stability is characterized instead of assumed, and every figure regenerates from the open repo.
Where reviewers will push: toy scale (max 8→16→8, regression only), simulation only, no synthesis, area, timing, or power numbers yet.
The experiment nobody has run: the 2021 paper's compute objection was an artifact of simulating twenty thousand agents on one CPU. On a substrate where each synapse is its own core, a three-action lookup policy is nearly free, and the sixteen-times-slower line inverts.
The live disagreement: should the local rule be learned by reinforcement, learned by gradients, or derived from theory? The three works take all three bets, and none of them is obviously right.

Sources: arXiv 2603.18066v2 (Oh) · arXiv 2105.14383 / AppliedMath 2(2):11 (Bhargava, Rezaei & Lankarany) · aman-bhargava.com, 2024