Sakhanda Wire
NVDA $153.63 -1.61% MSFT $506.84 -2.15% GOOGL $280.27 -1.22% META $619.77 -1.06% AMZN $256.65 -0.43%
← Back to the news

PC-ALM brings layer-local learning close to backpropagation

Sakana AI researchers Jeffrey Seely and Julian Gould have introduced Augmented Lagrangian Predictive Coding, or PC-ALM, a training method designed as a layer-local alternative to backpropagation. The approach adds a Lagrange multiplier to each layer’s constraint while retaining predictive coding’s local updates. The researchers say these multipliers converge to exact backpropagation adjoints in linear networks under a stability condition.

PC-ALM alternates local activation updates with dual updates that accumulate prediction errors. In the reported experiments, residual MLPs with widths and depths from 8 to 128 were trained for one epoch on MNIST and Fashion-MNIST. With an inference budget of T = 2L, PC-ALM matched backpropagation across the tested widths, depths and identity, tanh and ReLU activations, while standard predictive coding degraded in deep, narrow networks.

A reference configuration on Fashion-MNIST achieved 77.75% test accuracy for PC-ALM, compared with 78.66% for backpropagation and 68.13% for predictive coding. The method also trained 1000-layer residual MLPs on MNIST to within roughly 2 percentage points of backpropagation. An MIT-licensed JAX implementation runs on CPU, but the method has so far been evaluated only on small image benchmarks, including ResNet-18 tests on CIFAR-10 and Tiny ImageNet.

Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.

Originally published by MarkTechPost on

Read the original on MarkTechPost ↗

The reporting and the image belong to MarkTechPost and are credited and linked above; the text on this page is Sakhanda Wire's own summary.

← Back to the news

More stories

All the latest news