Optimizing Physical Neural Networks through Device Variable Selection
This paper improves the efficiency and accuracy of physical neural networks by leveraging device physics and trainable routing between signals.
This paper introduces a novel approach to optimizing physical neural networks by leveraging induced biases of broken-isomorphism PNN architectures.
Before reading this…
Applications
- →Image classification
To understand this paper, make sure you know these concepts first:
- Neural networksfind papers →
- Device physicsfind papers →
Abstract
More Like ThisPhysical neural networks (PNNs) leverage device physics in electronics, optics, and related domains to enable energy-efficient neural networks. Broken-isomorphism PNNs replace linear operations with multivariate physical transformations, modifying how input signals are mapped to device variables. We exploit network architectures emerging from these mappings by treating the routing between signals and input variables of physical devices as trainable parameters. This approach expands device-level tunability of the physical transformations while, at the system level, demonstrating improved expressivity of PNNs with negligible additional inference overhead. This discrete routing is optimized by training a categorical distribution using Gumbel-Softmax reparametrization. We evaluate this approach using experimentally calibrated differentiable models of multivariate electronic RNPUs for MNIST and CIFAR-10 classification. Using variable selection, we improve the MNIST classification accuracy for a small CNN model from 94.5% to 96.9% and the accuracy of CIFAR-10 from 62.8% to 68.0%. Our method uniquely leverages the induced biases of broken-isomorphism PNN architectures, paving the way towards more efficient physics-based neural network inference.