9 papers
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models
Wenjin Ke, Zhe Li, Dong Li +2
Improving the efficiency of inference in Large Language Models (LLMs) is a critical area of research. Post-training Quantization (PTQ) is a popular technique, but it often faces ch…
MonoGS++: Fast and Accurate Monocular RGB Gaussian SLAM
Renwu Li, Wenjing Ke, Dong Li +2
We present MonoGS++, a novel fast and accurate Simultaneous Localization and Mapping (SLAM) method that leverages 3D Gaussian representations and operates solely on RGB inputs. Whi…
Partial Convolution Meets Visual Attention
Haiduo Huang, Fuwei Yang, Dong Li +5
Designing an efficient and effective neural network has remained a prominent topic in computer vision research. Depthwise onvolution (DWConv) is widely used in efficient CNNs or Vi…
MSWA: Refining Local Attention with Multi-ScaleWindow Attention
Yixing Xu, Shivank Nag, Dong Li +2
Transformer-based LLMs have achieved exceptional performance across a wide range of NLP tasks. However, the standard self-attention mechanism suffers from quadratic time complexity…
EGSRAL: An Enhanced 3D Gaussian Splatting based Renderer with Automated Labeling for Large-Scale Driving Scene
Yixiong Huo, Guangfeng Jiang, Hongyang Wei +9
3D Gaussian Splatting (3D GS) has gained popularity due to its faster rendering speed and high-quality novel view synthesis. Some researchers have explored using 3D GS for reconstr…
FTP: A Fine-grained Token-wise Pruner for Large Language Models via Token Routing
Zekai Li, Jintu Zheng, Ji Liu +9
Recently, large language models (LLMs) have demonstrated superior performance across various tasks by adhering to scaling laws, which significantly increase model size. However, th…