Efficient ML
Model compression, pruning, quantization, distillation
20 papers indexed
Efficient Audio-Visual Event Recognition via Knowledge Distillation and Dynamic INT8 Quantization of a Hybrid Cross-Attention Network
This paper proposes a compression framework for efficient audio-visual event recognition using transformer-based models, combining architectural compression, knowledge distillation, and dynamic INT8 q…
Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models
Jinyang Du, Shenghao Jin, Ziqian Xu, Ruihao Gong +4 more
The paper proposes a compression pipeline combining few-step distillation and low-bit quantization to significantly reduce the deployment cost and parameter footprint of large dual-expert video diffus…
Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
Yujia Tong, Yuxi Wang, Yunyang Wan, Tian Zhang +2 more
This paper investigates whether model compression techniques (like quantization and pruning) preserve a Large Language Model's ability to quantify its own uncertainty, finding that accuracy-only evalu…
ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
Yuhao Wang, Mu Qiao, Haiwen Diao, Yunzhi Zhuge +4 more
This paper proposes ERA, a framework for efficient multimodal large language models using entropy-guided visual token pruning, rectified attention, and bias-aware token recycling.
VertMark: A Unified Training-Free Robust Watermarking Framework for Vertical Domain Pre-trained Language Models
Cong Kong, Xin Cheng, Zhaoxia Yin, Shuai Li +2 more
VertMark introduces a novel, unified, and training-free framework to embed robust watermarks into vertical domain pre-trained language models (VPLMs) for copyright protection across multiple specializ…
Pruning and Distilling Mixture-of-Experts into Dense Language Models
Junhyuck Kim, Jihun Yun, Haechan Kim, Gyeongman Kim +2 more
The paper introduces a systematic framework to convert large Mixture-of-Experts (MoE) models into memory-efficient, fully dense architectures, achieving superior performance compared to traditional pr…
Token Optimization Strategies for LLM-Based Oracle-to-PostgreSQL Migration
This paper formalizes token optimization as a multi-objective constrained transformation problem for LLM-based Oracle-to-PostgreSQL migration, demonstrating that adaptive routing offers the best balan…
CRIMP: Compact & Reliable DNN Inference on In-Memory Processing via Crossbar-Aligned Compression and Non-ideality Adaptation
Shuo Huai, Hao Kong, Xiangzhong Luo, Shiqing Li +4 more
This paper addresses the obstacles of using Crossbar-based In-Memory Processing (IMP) accelerators for deep neural networks (DNNs) by reusing bit-shift units for multiplication, applying pruning metho…
Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
Luyang Fang, Yongkai Chen, Jiazhang Cai, Ping Ma +1 more
The paper proposes Multi-Teacher Bayesian Knowledge Distillation (MT-BKD), a framework that uses Bayesian inference and teacher-informed priors to improve model compression, enhance predictive accurac…
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
Sangjin Kim, Yuseon Choi, Jungjun Oh, Byeongcheol Kim +1 more
This paper introduces LightRot, a lightweight rotation scheme and dedicated hardware accelerator for energy-efficient and accurate low-bit large language model inference.
Taming the Security-Energy Paradox: A Green AI Approach to Optimized Android Malware Detection
This paper tests different INT8 quantized Multi-Layer Perceptron models for malware detection on Android devices, achieving high accuracy with reduced energy consumption.
CIVIC: End-to-End Sequence Compactness for Efficient Vision-Language Models
Fengze Yang, Bo Yu, Xuewen Luo, Cathy Liu +1 more
CIVIC is a path-consistent compact visual inference framework that achieves genuine hardware efficiency in Vision-Language Models by maintaining contiguous sequence representations across all inferenc…
VibeVoice-ASR-BitNet Technical Report
Songchen Xu, Ting Song, Shaohan Huang, Zhiliang Peng +6 more
The paper introduces VibeVoice-ASR-BitNet, a compressed real-time speech recognition model optimized for edge CPUs using heterogeneous quantization and custom SIMD kernels.
It Takes a MAESTRO To Prune Bad Experts
MAESTRO is a structured pruning framework designed for MoE language models that models autoregressive expert activation trajectories as Ergodic Markov chains, yielding a globally aware importance heur…
MiCU: End-to-End Smart Home Command Understanding with Large Language Model
Haowei Han, Kexin Hu, Weiwei Cai, Debiao Zhang +5 more
The paper introduces MiCU, a domain-specific LLM that significantly improves smart home command understanding, especially for ambiguous commands, by synthesizing training data and optimizing the model…
AttnDiff: Attention-based Differential Fingerprinting for Large Language Models
Haobo Zhang, Zhenhua Xu, Junxian Li, Shangfeng Sheng +2 more
AttnDiff introduces a data-efficient white-box framework that extracts intrinsic attention-based fingerprints to verify the provenance and detect unauthorized derivation of large language models (LLMs…
OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
Geng Li, Guohao Chen, Ting Chen, Shilin Shan +5 more
OccamToken introduces a training-free, adaptive token pruning framework that replaces fixed token budgets with relative evidence testing against a register-based reference, significantly improving VLM…
Trust Region On-Policy Distillation
Xingrun Xing, Haoqing Wang, Boyan Gao, Ziheng Li +1 more
The paper introduces Trust Region On-Policy Distillation (TrOPD), a robust method that stabilizes the on-policy distillation of large language models by restricting training to regions where teacher s…
Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models
This paper proposes a method for quantizing large language models with cross-layer error compensation and finite-sample feature-statistics matching.
Importance-Aware Resource Allocation for Collaborative Task-Oriented Semantic Communication
The paper proposes iCoTASC, a framework for real-time, channel-adaptive semantic resource allocation in collaborative multi-device systems using importance-aware embedding selection, data-driven utili…