This paper proposes Supervised Memory Training (SMT), a method for training nonlinear RNNs that sidesteps recurrent credit propagation entirely.

The paper provides a unified algebraic framework to determine the formal language expressivity of recurrent neural language models, resolving conflicts in existing literature by linking expressivity t…

This paper introduces a sinusoidal recurrence mechanism for harmonic spectral enrichment in implicit neural representations, validated against various models and tasks.

The paper introduces Transformers with Temporal Middle-Layer Recurrence (T2MLR), a transformers-based latent reasoning architecture that enables abstract intermediate computation to persist across dec…

The paper demonstrates that the location and nature of state encoding in sequence models are not fixed architectural traits but are highly dependent on the specific task, showing that the encoding pro…

CART introduces a parameter-efficient recurrent transformer architecture that reuses a core block multiple times, but its performance does not surpass a dense baseline, suggesting that weight sharing…

This paper theoretically justifies the strong performance of linear recurrent neural networks as memory units in partially observable reinforcement learning by constructing specific linear filters tha…

The paper proposes a unified framework for designing efficient and expressive token mixing layers by separating the direct and recurrent influences of inputs, allowing for a principled trade-off betwe…

This paper argues for the importance of modularity and heterogeneity in AI architectures, contrasting the Transformer model with the structure of the cortex.

MemPro introduces a system-level evolution framework that treats the entire memory construction-retrieval pipeline as an evolvable program, significantly improving long-horizon agent performance over…

The paper introduces HOLA (Hippocampal Linear Attention), a semiparametric test-time memory system that combines a compressive linear-attention state with a bounded exact cache for key-value associati…

This paper proposes FPRM, a Transformer-based model using fixed-point convergence as an end-to-end halting mechanism in a looped architecture to address signal propagation issues in looped architectur…

This paper proposes RecRec, a recursive recommendation model that maintains a compact latent state and updates it through a shared recursive module conditioned on interaction evidence, using an eviden…

This paper proposes a context engineering approach that scales memory with distinct information instead of tokens, using a novelty-gated cache and state-space summary.

The paper introduces AGENTCL, a rigorous evaluation framework that uses controlled task streams to accurately measure an agent's ability to accumulate and reuse knowledge across multiple tasks, thereb…

Introduces Looped World Models, a looped architecture for world modelling that iteratively refines latent environment states for up to 100x parameter efficiency.

The paper introduces HORIZON, a self-evolving agent framework for hardware design using git operations.

The paper proposes the Intelligent Computing Architecture Model (ICAM), a six-layer framework that unifies disparate concepts in model-native computing by viewing the LLM stack through a dual-plane ar…

The paper analyzes a fragment of Higher-Order Datalog, showing that restricting recursion to a linear form shifts its expressive power from time complexity to space complexity, specifically capturing…

This paper proposes RPPNet, a two-stage deep learning architecture for music generation with variable structural boundaries, which automatically derives grouping of Rhythm-Pitch Primitive sequences fr…