This paper proposes Supervised Memory Training (SMT), a method for training nonlinear RNNs that sidesteps recurrent credit propagation entirely.

This paper proves that the maximum of up to 10 real numbers can be exactly represented by a ReLU network with two hidden layers, and shows that the same depth bound holds for all continuous piecewise-…

This paper shows that under standard assumptions, there is no polynomial-time verifier for exact verification of ReLU networks in an adversarial smoothed model.

This paper compares and optimizes CUDA strategies for a shallow neural network, achieving a 1.41x speedup on a large dataset.

The paper proposes a kernel-based, polynomial approximation of the ReLU activation function to enable the use of non-linear deep learning models, such as LLMs, within the constraints of Homomorphic En…

This paper introduces a mechanistic neuronal network model for multilayer learning, offering biological insights and an alternative to backpropagation.

This paper proposes Hierarchical Block-Local Learning (HBLL), a framework for training deep neural networks without full end-to-end backpropagation, achieving $\mathcal{O}(\log N)$ parallel time compl…

The paper proposes a decision-aware quadratic replacement for the ReLU activation function, enabling low-degree, calibration-lossless polynomial approximations for neural network inference under Fully…

The paper introduces Graph Cascades, a mesoscopic rewiring technique that enhances Graph Neural Networks by promoting node pairs with strong multi-hop connections to direct edges, improving performanc…

肖代替了视觉令牌的永久删除，通过可恢复的路由来改进视觉语言模型的性能

A new error backpropagation method called supervised counterstream learning is proposed for deep associative networks, which only requires recognition of errors during training and backpropagates corr…

The elasticAI.explorer is an extensible, unified Python framework that simplifies hardware-aware Neural Architecture Search (NAS) by decoupling search space definition from model implementation and de…

The paper provides a unified algebraic framework to determine the formal language expressivity of recurrent neural language models, resolving conflicts in existing literature by linking expressivity t…

The paper proposes Locality-Aware Redundancy Pruning (LoRP), a training-free method that prunes LLM layers by exploiting localized inter-layer redundancy, leading to improved efficiency while maintain…

The paper introduces Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) to achieve massive, structured compression of deep neural networks, demonstrating compression ratios up to 77,000x…

This paper theoretically justifies the strong performance of linear recurrent neural networks as memory units in partially observable reinforcement learning by constructing specific linear filters tha…

The paper introduces a novel, non-deep neural network architecture that achieves the performance of LLMs by finding the global optimum of the loss function in a single, closed-form iteration, eliminat…

The paper proposes two novel multi-column RBFN architectures, MC-PSO and MC-APSO, that combine parallel RBFN structures with swarm optimization to significantly outperform existing methods in accuracy…