95 citations · 97 across the 4 of their papers we have counts for
4 papers
Context Compression for Auto-regressive Transformers with Sentinel Tokens
Siyu Ren, Qi Jia, Kenny Q. Zhu
The quadratic complexity of the attention module makes it gradually become the bulk of compute in Transformer-based LLMs during generation. Moreover, the excessive key-value cache…
Unleash the Potential of 3D Point Cloud Modeling with A Calibrated Local Geometry-driven Distance Metric
Siyu Ren, Junhui Hou
Quantifying the dissimilarity between two unstructured 3D point clouds is a challenging task, with existing metrics often relying on measuring the distance between corresponding po…
Pruning Pre-trained Language Models with Principled Importance and Self-regularization
Siyu Ren, Kenny Q. Zhu
Iterative pruning is one of the most effective compression methods for pre-trained language models. We discovered that finding the optimal pruning decision is an equality-constrain…
CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondence
Siyu Ren, Yiming Zeng, Junhui Hou +1
Motivated by the intuition that the critical step of localizing a 2D image in the corresponding 3D point cloud is establishing 2D-3D correspondence between them, we propose the fir…