20 results for “large-scale model training”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
Megan Frisella, Shubham Tiwari, Andy Ruan, Yi Pan +4 more
Piper is a user-controllable distributed training system that decouples the strategy from the runtime implementation, allowing users to declare a comprehensive distributed training strategy with a sma…
Mingshu Zhai, Minkun Guo, Hongrui Lian, Shuangyu Li +6 more
BaGuaLu-2 proposes a flexible memory provisioning system for large language models
Lu Liu, Huiyu Duan, Chenxin Zhu, Jintong Lu +5 more
The paper introduces LL-Bench, a comprehensive benchmark for evaluating large-scale generative models on low-level vision tasks, and proposes LL-Score, an MLLM-based evaluator that better aligns quali…
Peng Yu, Yuankai Fan, Yang Qiu, Tian Li +3 more
This paper introduces Arachne, a framework for efficient Text-to-Video model training at scale, reducing iteration time by up to 65% over leading frameworks.
This paper investigates the application of Parameter-Efficient Fine-Tuning (PEFT) methods, specifically adapters and LoRA, to large pretrained models for instance segmentation, demonstrating that thes…
Yiming Zhang, Zhonghan Zhao, Wenwei Zhang, Haiteng Zhao +12 more
This paper presents the benefits of visual pretraining for foundation model intelligence, outperforming text-only pretraining on multiple backbones and benchmarks.
The paper presents a new text-to-image generation model, Halcyon, that can be trained from scratch without internet-scale language pretraining.
The paper introduces a retraining-free method to elastify pre-trained transformers into a single model that adapts to varying computational budgets.
The paper introduces GPIC, a massive, permissively licensed, and safety-filtered image corpus of 28 trillion pixels, designed to serve as a stable and accessible benchmark for large-scale visual gener…
The paper presents a framework to model and predict energy consumption of Transformer training on multiple GPUs.
Xuan-Phi Nguyen, Shrey Pandit, Yiran Zhao, Semih Yavuz +2 more
This paper presents a memory-efficient training stack for Mixture-of-Experts (MoE) models, combining and specializing parallelism techniques for maximal efficiency.
Minhua Lin, Zijun Wang, Zewen Liu, Zhaotian Weng +6 more
This paper proposes a method, LM2LM, to train large language models to generate effective improvement proposals using reinforcement learning and a research world model.
FLORO is a multimodal geospatial foundation model that learns transferable remote sensing representations from a small, diverse corpus, achieving strong performance across various sensor types and res…
The paper introduces CalArena, a large-scale, standardized benchmark covering nearly 2000 experiments to comprehensively evaluate post-hoc calibration methods, finding that smooth calibration function…
This paper provides a theoretical explanation for why low-rank adaptation (LoRA) is effective in fine-tuning large pretrained models.
This paper analyzes the poor performance of Meta-learning for Training-data Selection (MTS) and proposes that increasing the batch size and incorporating informative features can significantly improve…