20 results for “Pretrained models”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
Megan Frisella, Shubham Tiwari, Andy Ruan, Yi Pan +4 more
Piper is a user-controllable distributed training system that decouples the strategy from the runtime implementation, allowing users to declare a comprehensive distributed training strategy with a sma…
Tong Xie, Yuanhao Ban, Yunqi Hong, Sohyun An +2 more
This paper reinterprets supervised fine-tuning as target distribution design and proposes a new method called Target-SFT that constructs the training objective directly from the desired target distrib…
Senmiao Wang, Tiantian Fang, Haoran Zhang, Yushun Zhang +3 more
This paper proposes a preconditioning layer for stable weight conditioning in LLM training.
The paper investigates the performance improvement of transfer learning in high-dimensional regression.
This paper investigates the application of Parameter-Efficient Fine-Tuning (PEFT) methods, specifically adapters and LoRA, to large pretrained models for instance segmentation, demonstrating that thes…
The paper presents a new text-to-image generation model, Halcyon, that can be trained from scratch without internet-scale language pretraining.
Yiming Zhang, Zhonghan Zhao, Wenwei Zhang, Haiteng Zhao +12 more
This paper presents the benefits of visual pretraining for foundation model intelligence, outperforming text-only pretraining on multiple backbones and benchmarks.
This paper analyzes the poor performance of Meta-learning for Training-data Selection (MTS) and proposes that increasing the batch size and incorporating informative features can significantly improve…
This paper studies the effectiveness of pretraining the Q-function in value-based reinforce learning and finds that naive Q-function pretraining often provides little benefit over random initializatio…
Andrej Tschalzev, Nick Erickson, Yuyang Wang, Huzefa Rangwala +3 more
The paper introduces TabPrep, a feature engineering pipeline that systematically improves performance across various tabular machine learning models by addressing structural data patterns ignored by c…
The paper proposes pretraining a Perceiver-style in-context learner on synthetic data to solve Multiple Instance Learning (MIL) tasks efficiently in the low-label regime.
This paper demonstrates that in-domain pretraining of BERT significantly improves the detection of DNS exfiltration, particularly in maintaining a low false positive rate.
Zakk Heile, Hayden McTavish, Varun Babbar, Margo Seltzer +1 more
The paper introduces PRAXIS, a novel algorithm that efficiently approximates the computation of 'Rashomon sets' for decision trees, significantly reducing memory and runtime complexity.
Gabriel Clark, Sofian Mejjoute, Mohamed Osman, George Close +1 more
The authors present ZONOS2 8B, a TTS model with improved naturalness, prosody, and voice cloning fidelity, achieved through scaling, data expansion, and simplification.
The paper introduces TSVD, a novel framework that efficiently pre-trains LLMs by enforcing both low rank and strict weight orthonormality, achieving performance comparable to full-parameter models wit…