This paper investigates limitations of learning tanh neural networks under finite-precision computations and Lp accuracy guarantees.

This paper proposes Supervised Memory Training (SMT), a method for training nonlinear RNNs that sidesteps recurrent credit propagation entirely.

This paper proves that the maximum of up to 10 real numbers can be exactly represented by a ReLU network with two hidden layers, and shows that the same depth bound holds for all continuous piecewise-…

This paper shows that under standard assumptions, there is no polynomial-time verifier for exact verification of ReLU networks in an adversarial smoothed model.

This paper introduces a mechanistic neuronal network model for multilayer learning, offering biological insights and an alternative to backpropagation.

This paper proposes Hierarchical Block-Local Learning (HBLL), a framework for training deep neural networks without full end-to-end backpropagation, achieving $\mathcal{O}(\log N)$ parallel time compl…

The paper provides a unified algebraic framework to determine the formal language expressivity of recurrent neural language models, resolving conflicts in existing literature by linking expressivity t…

The paper proposes a kernel-based, polynomial approximation of the ReLU activation function to enable the use of non-linear deep learning models, such as LLMs, within the constraints of Homomorphic En…

This paper compares and optimizes CUDA strategies for a shallow neural network, achieving a 1.41x speedup on a large dataset.

The paper establishes that the training process of fully connected deep neural networks (DNNs) on exponential family data is mathematically equivalent to performing a Renormalization Group (RG) calcul…

The paper proposes Locality-Aware Redundancy Pruning (LoRP), a training-free method that prunes LLM layers by exploiting localized inter-layer redundancy, leading to improved efficiency while maintain…

This paper theoretically justifies the strong performance of linear recurrent neural networks as memory units in partially observable reinforcement learning by constructing specific linear filters tha…

The paper proposes a decision-aware quadratic replacement for the ReLU activation function, enabling low-degree, calibration-lossless polynomial approximations for neural network inference under Fully…

A new error backpropagation method called supervised counterstream learning is proposed for deep associative networks, which only requires recognition of errors during training and backpropagates corr…

This paper characterizes the gap between neural network performance and their neural tangent kernel limit on compositional tasks, attributing it to a mismatch between kernel smoothness bias and target…

This paper proposes FPRM, a Transformer-based model using fixed-point convergence as an end-to-end halting mechanism in a looped architecture to address signal propagation issues in looped architectur…

This paper characterizes continual classification in homogeneous models as sequential projections and identifies regularity properties for local linear convergence.

The paper analyzes congruence-based neural architectures for classifying positive-definite matrices, demonstrating that common semi-orthogonality constraints severely limit the model's expressivity.

The paper introduces partial multi-neuron relaxation, a novel verification technique that selectively computes tight linear bounds for a small subset of neurons to improve the efficiency and tightness…