20 results for “Small Language Models”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
This paper evaluates the performance of twelve Small Language Models on Arabic natural language processing tasks.
Divya Tadimeti, Shawn Pan, Sameera Lanka, Chenghui Zhou +1 more
This paper demonstrates that targeted adaptation of the small language model Phi Silica, using dataset curation and fine-tuning, significantly improves its performance in short-form text rewriting, na…
This paper demonstrates that fine-tuning small language models (SLMs) on a synthetic, solution-rich Windows event log dataset allows them to outperform larger LLMs in identifying issues and providing…
Develops specialized Japanese-English translation models for real-world translation tasks.
This paper introduces a pipeline to extract grammatical rules, example sentences, and lexicons from grammar books and generates synthetic parallel corpora for fine-tuning machine translation models on…
ProbScale is a novel framework that combines neural scaling laws and language model probing to identify highly efficient, task-specific subnetworks within pre-trained Small Language Models, achieving…
This paper investigates the performance of small language models in relation extraction tasks, showing that task-specific adaptation can enable them to outperform larger, general-purpose models.
The paper proposes an aggressive, parameter-efficient method to prune non-essential experts from Mixture-of-Experts (MoE) LLMs, significantly compressing the model while maintaining high machine trans…
Zhilin Guo, Boqiao Zhang, Hakan Aktas, Kyle Fogarty +13 more
This paper introduces a Telescopic Language Model (TLM) that can serve multiple compute budgets with a single training run.
The paper introduces PortBERT, a family of RoBERTa-based language models for Portuguese, which achieves competitive performance while explicitly balancing efficiency and accuracy.
This paper introduces Weight Pair Encoding, a method to explicitly finetune neural network weights to admit a smaller grammar.
The paper investigates reducing the memory cost of n-gram embedding tables in open-weight language models.
Marek Šuppa, Andrej Ridzik, Daniel Hládek, Natália Kňažeková +1 more
This paper introduces SkMTEB, a comprehensive text embedding benchmark for Slovak, and develops efficient, locally-deployable Slovak embeddings.
Minhua Lin, Zijun Wang, Zewen Liu, Zhaotian Weng +6 more
This paper proposes a method, LM2LM, to train large language models to generate effective improvement proposals using reinforcement learning and a research world model.
Boqian Wu, Qiao Xiao, Patrik Okanovic, Tomasz Sternal +5 more
This paper introduces a new scaling law for sparse language models trained with limited data, demonstrating that sparsity can significantly improve performance and delay data saturation during multi-e…
Ziyi Cai, Shuangping Li, Yiheng Shen, Kangning Wang +1 more
This paper introduces a framework for language generation in the limit, focusing on the guarantees of validity, novelty, and output coverage.