232 citations · 400 across the 19 of their papers we have counts for
21 papers
PLSQLBench: Benchmarking LLM Systems for Executable Procedural Database Programming
Marianne Menglin Liu, Leonid Boytsov, Daniel W. Peterson +13
We present PLSQLBench, to our knowledge the first benchmark for evaluating whether LLMs can write executable PL/SQL programs, with correctness measured through execution-based test…
Constrained Decoding with Speculative Lookaheads
Nishanth Nakshatri, Shamik Roy, Rajarshi Das +3
Constrained decoding with lookahead heuristics (CDLH) is a highly effective method for aligning LLM generations to human preferences. However, the extensive lookahead roll-out oper…
KazQAD: Kazakh Open-Domain Question Answering Dataset
Rustem Yeshpanov, Pavel Efimov, Leonid Boytsov +2
We introduce KazQAD -- a Kazakh open-domain question answering (ODQA) dataset -- that can be used in both reading comprehension and full ODQA settings, as well as for information r…
A Curious Case of Remarkable Resilience to Gradient Attacks via Fully Convolutional and Differentiable Front End with a Skip Connection
Leonid Boytsov, Ameya Joshi, Filipe Condessa
We experimented with front-end enhanced neural models where a differentiable and fully convolutional model with a skip connection is added before a frozen backbone classifier. By t…
InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers
Leonid Boytsov, Preksha Patel, Vivek Sourabh +4
We carried out a reproducibility study of InPars, which is a method for unsupervised training of neural rankers (Bonifacio et al., 2022). As a by-product, we developed InPars-light…
Positional Bias in Long-Document Ranking: Impact, Assessment, and Mitigation
Leonid Boytsov, David Akinpelu, Nipun Katyal +5
We tested over 20 Transformer models for ranking long documents (including recent LongP models trained with FlashAttention and RankGPT models "powered" by OpenAI and Anthropic clou…