HuggingFace's Transformers: State-of-the-art Natural Language Processing

Abstract

Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. \textit{Transformers} is an open-source library with the goal of opening up these advances to the wider machine learning community. The library consists of carefully engineered state-of-the art Transformer architectures under a unified API. Backing this library is a curated collection of pretrained models made by and available for the community. \textit{Transformers} is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments. The library is available at \url{https://github.com/huggingface/transformers}.

arXiv comment8 pages, 4 figures, more details at https://github.com/huggingface/transformers

References in corpus (10)

• Decoupled Weight Decay Regularization
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Longformer: The Long-Document Transformer
• NLTK: The Natural Language Toolkit
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Supervised Multimodal Bitransformers for Classifying Images and Text
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
• exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models

Cited by in corpus (523)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• fastai: A Layered API for Deep Learning
• CamemBERT: a Tasty French Language Model
• ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Contrastive Learning of Medical Visual Representations from Paired Images and Text
• Revisiting Few-sample BERT Fine-tuning
• FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems
• Transformers for Modeling Physical Systems
• A Simple Language Model for Task-Oriented Dialogue
• The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models
• OpenFL: An open-source framework for Federated Learning
• COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
• BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis
• Molecular representation learning with language models and domain-relevant auxiliary tasks
• Social Media Polarization and Echo Chambers in the Context of COVID-19: Case Study
• Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
• Revisiting Deep Learning Models for Tabular Data
• The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges
• The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
• Advances of Machine Learning in Materials Science: Ideas and Techniques
• Aligning AI With Shared Human Values
• Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch
• Generative Data Augmentation for Commonsense Reasoning
• An Empirical Study on the Usage of Transformer Models for Code Completion
• Deep Open Intent Classification with Adaptive Decision Boundary
• On the Effect of Dropping Layers of Pre-trained Transformer Models
• Temporal Graph Networks for Deep Learning on Dynamic Graphs
• It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
• An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers
• GREEK-BERT: The Greeks visiting Sesame Street
• Discovering New Intents with Deep Aligned Clustering
• Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
• Hopfield Networks is All You Need
• Multi-Head Attention: Collaborate Instead of Concatenate
• Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference
• Recommending Metamodel Concepts during Modeling Activities with Pre-Trained Language Models
• Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data
• BanFakeNews: A Dataset for Detecting Fake News in Bangla
• Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
• Linguistic Features for Readability Assessment
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Models
• Financial Document Causality Detection Shared Task (FinCausal 2020)
• A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods
• Classification Aware Neural Topic Model and its Application on a New COVID-19 Disinformation Corpus
• T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition
• FlauBERT: Unsupervised Language Model Pre-training for French
• Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
• RepBERT: Contextualized Text Embeddings for First-Stage Retrieval
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing
• InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining
• Information disorders on Italian Facebook during COVID-19 infodemic
• Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?
• Exploring Software Naturalness through Neural Language Models
• Few-shot Text Classification with Distributional Signatures
• ktrain: A Low-Code Library for Augmented Machine Learning
• Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks
• Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study
• Learning from others' mistakes: Avoiding dataset biases without modeling them
• Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages
• Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling
• medigan: a Python library of pretrained generative models for medical image synthesis
• Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain
• Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality
• Unsupervised Paraphrase Generation using Pre-trained Language Models
• Explaining Question Answering Models through Text Generation
• BERT Loses Patience: Fast and Robust Inference with Early Exit
• EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems
• FoodSAM: Any Food Segmentation
• Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition
• Explaining Explanations: Axiomatic Feature Interactions for Deep Networks
• Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domain
• Transformer-based Conditional Variational Autoencoder for Controllable Story Generation
• Variational Information Bottleneck for Effective Low-Resource Fine-Tuning
• A Study of Social and Behavioral Determinants of Health in Lung Cancer Patients Using Transformers-based Natural Language Processing Models
• MMM : Exploring Conditional Multi-Track Music Generation with the Transformer
• The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic
• IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Research
• Structured Prediction as Translation between Augmented Natural Languages
• The Lottery Ticket Hypothesis for Pre-trained BERT Networks
• Structural Text Segmentation of Legal Documents
• Task-Oriented Dialogue System as Natural Language Generation
• A Frustratingly Easy Approach for Entity and Relation Extraction
• Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Models
• Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned
• Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling
• What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning
• Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models
• How does this interaction affect me? Interpretable attribution for feature interactions
• ABNIRML: Analyzing the Behavior of Neural IR Models
• Movement Pruning: Adaptive Sparsity by Fine-Tuning
• Improving Broad-Coverage Medical Entity Linking with Semantic Type Prediction and Large-Scale Datasets
• CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization
• Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
• Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
• A Distributional Approach to Controlled Text Generation
• Characterization of anomalous diffusion through convolutional transformers
• Clinical Concept and Relation Extraction Using Prompt-based Machine Reading Comprehension
• Are we pretraining it right? Digging deeper into visio-linguistic pretraining
• FitVid: Overfitting in Pixel-Level Video Prediction
• TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
• Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation
• Pair-Level Supervised Contrastive Learning for Natural Language Inference
• Can neural networks acquire a structural bias from raw linguistic data?
• Measuring Systematic Generalization in Neural Proof Generation with Transformers
• Federated Few-Shot Learning for Mobile NLP
• Multi-Modal Deep Learning for Credit Rating Prediction Using Text and Numerical Data Streams
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes
• Augmented Datasheets for Speech Datasets and Ethical Decision-Making
• Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling
• A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks
• Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond
• CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims
• Towards Better Chinese-centric Neural Machine Translation for Low-resource Languages
• Semi-automatic Generation of Multilingual Datasets for Stance Detection in Twitter
• Training Large Neural Networks with Constant Memory using a New Execution Algorithm
• Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses
• Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT
• Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection
• MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection
• Mask and Cloze: Automatic Open Cloze Question Generation using a Masked Language Model
• Cross-domain Named Entity Recognition via Graph Matching
• Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition
• Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog
• AsPOS: Assamese Part of Speech Tagger using Deep Learning Approach
• FarsTail: A Persian Natural Language Inference Dataset
• GMAT: Global Memory Augmentation for Transformers
• Pile-Up Mitigation using Attention
• Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions
• An Explanation of In-context Learning as Implicit Bayesian Inference
• FQuAD: French Question Answering Dataset
• USTC-NELSLIP at SemEval-2022 Task 11: Gazetteer-Adapted Integration Network for Multilingual Complex Named Entity Recognition
• Statistically significant detection of semantic shifts using contextual word embeddings
• CATBERT: Context-Aware Tiny BERT for Detecting Social Engineering Emails
• Output Format Biases in the Evaluation of Large Language Models for Code Translation
• KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs
• EduBERT: Pretrained Deep Language Models for Learning Analytics
• Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization
• BERT-like Pre-training for Symbolic Piano Music Classification Tasks
• Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
• ORCAS-I: Queries Annotated with Intent using Weak Supervision
• Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges
• Embedding-based Recommender System for Job to Candidate Matching on Scale
• Scaling up DNA digital data storage by efficiently predicting DNA hybridisation using deep learning
• EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
• Comparison of State-of-the-Art Deep Learning APIs for Image Multi-Label Classification using Semantic Metrics
• On the distance between two neural networks and the stability of learning
• C5T5: Controllable Generation of Organic Molecules with Transformers
• Open-Vocabulary Object Detection Using Captions
• CoCon: A Self-Supervised Approach for Controlled Text Generation
• Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2
• Learning Sparse Prototypes for Text Generation
• The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers
• AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning
• COSPLAY: Concept Set Guided Personalized Dialogue Generation Across Both Party Personas
• Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking
• LLM-based feature generation from text for interpretable machine learning
• Finding Experts in Transformer Models
• Question Answering over Knowledge Base using Language Model Embeddings
• The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
• A Comparative Study of Feature Types for Age-Based Text Classification
• Plato Dialogue System: A Flexible Conversational AI Research Platform
• IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformers
• Hate-Alert@DravidianLangTech-EACL2021: Ensembling strategies for Transformer-based Offensive language Detection
• Autoregressive Linguistic Steganography Based on BERT and Consistency Coding
• Ad Headline Generation using Self-Critical Masked Language Model
• A Neural Span-Based Continual Named Entity Recognition Model
• FILM: Following Instructions in Language with Modular Methods
• Sinkformers: Transformers with Doubly Stochastic Attention
• PharmKE: Knowledge Extraction Platform for Pharmaceutical Texts using Transfer Learning
• EIGEN: Event Influence GENeration using Pre-trained Language Models
• Enhancing Factual Consistency of Abstractive Summarization
• Towards Understanding and Answering Multi-Sentence Recommendation Questions on Tourism
• Pre-training Text-to-Text Transformers for Concept-centric Common Sense
• Clinical Relation Extraction Using Transformer-based Models
• Taming Pretrained Transformers for Extreme Multi-label Text Classification
• Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness
• Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement
• MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
• Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives
• Response Generation with Context-Aware Prompt Learning
• Natural-Language Multi-Agent Simulations of Argumentative Opinion Dynamics
• NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution
• RecJPQ: Training Large-Catalogue Sequential Recommenders
• A Universal Adversarial Policy for Text Classifiers
• Investigation of Self-supervised Pre-trained Models for Classification of Voice Quality from Speech and Neck Surface Accelerometer Signals
• DeepSI: Interactive Deep Learning for Semantic Interaction
• Clickbait Headline Detection in Indonesian News Sites using Multilingual Bidirectional Encoder Representations from Transformers (M-BERT)
• Deep Extrapolation for Attribute-Enhanced Generation
• Split-NER: Named Entity Recognition via Two Question-Answering-based Classifications
• SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval
• GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
• Exploring and Predicting Transferability across NLP Tasks
• Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task
• Indic-Transformers: An Analysis of Transformer Language Models for Indian Languages
• Learning to Reason for Text Generation from Scientific Tables
• Modeling Task Effects on Meaning Representation in the Brain via Zero-Shot MEG Prediction
• Accelerating Real-Time Question Answering via Question Generation
• COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List
• Real-Time Execution of Large-scale Language Models on Mobile
• The Turking Test: Can Language Models Understand Instructions?
• Identifying COVID-19 Fake News in Social Media
• Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models
• Environment Inference for Invariant Learning
• Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
• Enabling Language Models to Fill in the Blanks
• Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization
• Evolving Attention with Residual Convolutions
• Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring
• Load What You Need: Smaller Versions of Multilingual BERT
• Procedural Reading Comprehension with Attribute-Aware Context Flow
• Improving Information Extraction on Business Documents with Specific Pre-Training Tasks
• Automatic Expansion of Domain-Specific Affective Models for Web Intelligence Applications
• FairyTailor: A Multimodal Generative Framework for Storytelling
• Doing Good or Doing Right? Exploring the Weakness of Commonsense Causal Reasoning Models
• Trusting RoBERTa over BERT: Insights from CheckListing the Natural Language Inference Task
• DocVQA: A Dataset for VQA on Document Images
• Multi-Modal Detection of Alzheimer's Disease from Speech and Text
• DualVoice: Speech Interaction that Discriminates between Normal and Whispered Voice Input
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference
• Literature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization
• Give your Text Representation Models some Love: the Case for Basque
• Detection and Classification of mental illnesses on social media using RoBERTa
• Sentence Embeddings using Supervised Contrastive Learning
• Automated Essay Scoring Using Transformer Models
• Condenser: a Pre-training Architecture for Dense Retrieval
• Negative Confidence-Aware Weakly Supervised Binary Classification for Effective Review Helpfulness Classification
• Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
• Predicting Directionality in Causal Relations in Text
• VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
• Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks
• Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline
• Experiencers, Stimuli, or Targets: Which Semantic Roles Enable Machine Learning to Infer the Emotions?
• Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
• WLV-RIT at HASOC-Dravidian-CodeMix-FIRE2020: Offensive Language Identification in Code-switched YouTube Comments
• Explaining The Efficacy of Counterfactually Augmented Data
• Precise Length Control in Large Language Models
• Neural Class Expression Synthesis
• Long Document Re-ranking with Modular Re-ranker
• Detecting Narrative Elements in Informational Text
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
• Order in the Court: Explainable AI Methods Prone to Disagreement
• Joint Emotion Label Space Modelling for Affect Lexica
• Multimodal Conditionality for Natural Language Generation
• Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks
• Contextual Argument Component Classification for Class Discussions
• Counterfactual Variable Control for Robust and Interpretable Question Answering
• Towards Semi-Supervised Semantics Understanding from Speech
• Sensitivity of BLANC to human-scored qualities of text summaries
• The ADAPT Enhanced Dependency Parser at the IWPT 2020 Shared Task
• Local and Global Context-Based Pairwise Models for Sentence Ordering
• Named Entity Recognition in the Style of Object Detection
• Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices
• Temporal Probability Calibration
• The Dark Side of the Language: Pre-trained Transformers in the DarkNet
• DaCy: A Unified Framework for Danish NLP
• Open and Sustainable AI: challenges, opportunities and the road ahead in the life sciences (October 2025 -- Version 2)
• A Clarifying Question Selection System from NTES_ALONG in Convai3 Challenge
• Extracting Angina Symptoms from Clinical Notes Using Pre-Trained Transformer Architectures
• Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
• Transformer-based Multi-task Learning for Disaster Tweet Categorisation
• Human Language Modeling
• Grid Search Hyperparameter Benchmarking of BERT, ALBERT, and LongFormer on DuoRC
• LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
• HASOCOne@FIRE-HASOC2020: Using BERT and Multilingual BERT models for Hate Speech Detection
• MedGPT: Medical Concept Prediction from Clinical Narratives
• Investigating the Effects of Sparse Attention on Cross-Encoders
• Proto: A Neural Cocktail for Generating Appealing Conversations
• RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators
• REX: Revisiting Budgeted Training with an Improved Schedule
• Leveraging Event Specific and Chunk Span features to Extract COVID Events from tweets
• Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning
• A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining
• Improving Portuguese Semantic Role Labeling with Transformers and Transfer Learning
• Selective Classification Can Magnify Disparities Across Groups
• verBERT: Automating Brazilian Case Law Document Multi-label Categorization Using BERT
• Direction is what you need: Improving Word Embedding Compression in Large Language Models
• The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations
• A Comparison of Supervised Learning to Match Methods for Product Search
• Cross-Domain Sentiment Classification with In-Domain Contrastive Learning
• Knodle: Modular Weakly Supervised Learning with PyTorch
• Misspelling Correction with Pre-trained Contextual Language Model
• Fine-tune the Entire RAG Architecture (including DPR retriever) for Question-Answering
• Do Explicit Alignments Robustly Improve Multilingual Encoders?
• A Generalizable Approach to Learning Optimizers
• Train and You'll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings
• SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services
• TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment
• Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
• LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation
• PhilaeX: Explaining the Failure and Success of AI Models in Malware Detection
• PyTorrent: A Python Library Corpus for Large-scale Language Models
• Prior Art Search and Reranking for Generated Patent Text
• FrameFinder: Explorative Multi-Perspective Framing Extraction from News Headlines
• FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation
• Deep Natural Language Feature Learning for Interpretable Prediction
• Interpretable bias mitigation for textual data: Reducing gender bias in patient notes while maintaining classification performance
• Exemplar Guided Active Learning
• Training Neural Networks with Fixed Sparse Masks
• PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction
• Low-Resource Multi-Granularity Academic Function Recognition Based on Multiple Prompt Knowledge
• Language Models as a Knowledge Source for Cognitive Agents
• BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation
• Towards Accurate and Reliable Energy Measurement of NLP Models
• Named Entities in Medical Case Reports: Corpus and Experiments
• Investigation on Data Adaptation Techniques for Neural Named Entity Recognition
• Machine Learning Meets Natural Language Processing -- The story so far
• SportsSum2.0: Generating High-Quality Sports News from Live Text Commentary
• Cutting through the noise to motivate people: A comprehensive analysis of COVID-19 social media posts de/motivating vaccination
• BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context
• A Survey on Awesome Korean NLP Datasets
• Exploring Transformer Based Models to Identify Hate Speech and Offensive Content in English and Indo-Aryan Languages
• AutoKG: Constructing Virtual Knowledge Graphs from Unstructured Documents for Question Answering
• Quantification of BERT Diagnosis Generalizability Across Medical Specialties Using Semantic Dataset Distance
• How Does Adversarial Fine-Tuning Benefit BERT?
• An Empirical Study of Cross-Lingual Transferability in Generative Dialogue State Tracker
• Text Data Augmentation: Towards better detection of spear-phishing emails
• MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
• Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
• IrEne: Interpretable Energy Prediction for Transformers
• UzBERT: pretraining a BERT model for Uzbek
• Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
• Detecting Hostile Posts using Relational Graph Convolutional Network
• BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?
• Pre-Training a Language Model Without Human Language
• Multilingual Speech Recognition for Low-Resource Indian Languages using Multi-Task conformer
• A Novel Incremental Cross-Modal Hashing Approach
• OpenMatch: An Open Source Library for Neu-IR Research
• Identifying Introductions in Podcast Episodes from Automatically Generated Transcripts
• Few-Shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning
• Sentiment-based Candidate Selection for NMT
• A Transformer-based Math Language Model for Handwritten Math Expression Recognition
• Self-supervised Answer Retrieval on Clinical Notes
• SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking
• TEASEL: A Transformer-Based Speech-Prefixed Language Model
• Leveraging Pre-trained Language Model for Speech Sentiment Analysis
• conSultantBERT: Fine-tuned Siamese Sentence-BERT for Matching Jobs and Job Seekers
• Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in Public Cloud
• On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation
• Low Rank Factorization for Compact Multi-Head Self-Attention
• PSG: Prompt-based Sequence Generation for Acronym Extraction
• Stress Test for BERT and Deep Models: Predicting Words from Italian Poetry
• NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback
• Lost In Translation: Generating Adversarial Examples Robust to Round-Trip Translation
• Faster Convergence for Transformer Fine-tuning with Line Search Methods
• Generative AI Takes a Statistics Exam: A Comparison of Performance between ChatGPT3.5, ChatGPT4, and ChatGPT4o-mini
• On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing
• Divergences between Language Models and Human Brains
• End-to-End Entity Detection with Proposer and Regressor
• Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models
• DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
• Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks
• Linguistic Characterization of Divisive Topics Online: Case Studies on Contentiousness in Abortion, Climate Change, and Gun Control
• High throughput screening with machine learning
• Enhancing Transformers with Gradient Boosted Decision Trees for NLI Fine-Tuning
• RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters
• GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
• Improving Diversity of Neural Text Generation via Inverse Probability Weighting
• Speech Recognition by Simply Fine-tuning BERT
• Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning
• M2FN: Multi-step Modality Fusion for Advertisement Image Assessment
• SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline
• MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation
• Exploring the Promises of Transformer-Based LMs for the Representation of Normative Claims in the Legal Domain
• Lexicon-constrained Copying Network for Chinese Abstractive Summarization
• Neural Language Modeling for Contextualized Temporal Graph Generation
• N-LTP: An Open-source Neural Language Technology Platform for Chinese
• Answering Any-hop Open-domain Questions with Iterative Document Reranking
• AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation
• An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
• Context-based Transformer Models for Answer Sentence Selection
• MLE-guided parameter search for task loss minimization in neural sequence modeling
• Lessons Learned from Applying off-the-shelf BERT: There is no Silver Bullet
• LayoutReader: Pre-training of Text and Layout for Reading Order Detection
• Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning
• SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation
• Embedded Emotions -- A Data Driven Approach to Learn Transferable Feature Representations from Raw Speech Input for Emotion Recognition
• Annotating Scientific Uncertainty: A comprehensive model using linguistic patterns and comparison with existing approaches
• ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser
• Deceptive Deletions for Protecting Withdrawn Posts on Social Platforms
• CollaborER: A Self-supervised Entity Resolution Framework Using Multi-features Collaboration
• De-Anonymizing Text by Fingerprinting Language Generation
• TextAttack: Lessons learned in designing Python frameworks for NLP
• Extracting Implicitly Asserted Propositions in Argumentation
• Lookup or Exploratory: What is Your Search Intent?
• Unstructured Knowledge Access in Task-oriented Dialog Modeling using Language Inference, Knowledge Retrieval and Knowledge-Integrative Response Generation
• Does constituency analysis enhance domain-specific pre-trained BERT models for relation extraction?
• Deep Contextual Embeddings for Address Classification in E-commerce
• AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
• Increasing The Performance of Cognitively Inspired Data-Efficient Language Models via Implicit Structure Building
• Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins
• BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification
• Gestalt: a Stacking Ensemble for SQuAD2.0
• To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs
• Neural semi-Markov CRF for Monolingual Word Alignment
• Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training
• INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers
• Is the User Enjoying the Conversation? A Case Study on the Impact on the Reward Function
• Lightweight Transformer in Federated Setting for Human Activity Recognition
• Advancing biological super-resolution microscopy through deep learning: a brief review
• Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language
• Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance
• Cross-Modal Retrieval and Synthesis (X-MRS): Closing the Modality Gap in Shared Representation Learning
• Generating Major Types of Chinese Classical Poetry in a Uniformed Framework
• Classifying Textual Data with Pre-trained Vision Models through Transfer Learning and Data Transformations
• Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need
• A Dual-Decoder Conformer for Multilingual Speech Recognition
• Novel Aficionados and Doppelgängers: a referential task for semantic representations of individual entities
• Self-Attention Networks Can Process Bounded Hierarchical Languages
• Self-conditioning pre-trained language models
• KoreALBERT: Pretraining a Lite BERT Model for Korean Language Understanding
• Toward Improving Coherence and Diversity of Slogan Generation
• Hierarchical Multi Task Learning with Subword Contextual Embeddings for Languages with Rich Morphology
• BERTnesia: Investigating the capture and forgetting of knowledge in BERT
• Multi-Task Learning of Query Intent and Named Entities using Transfer Learning
• Self-attention Comparison Module for Boosting Performance on Retrieval-based Open-Domain Dialog Systems
• Spinning Sequence-to-Sequence Models with Meta-Backdoors
• DDSupport: Language Learning Support System that Displays Differences and Distances from Model Speech
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• EdinSaar@WMT21: North-Germanic Low-Resource Multilingual NMT
• CompRes: A Dataset for Narrative Structure in News
• UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation
• Transfer Learning in Visual and Relational Reasoning
• Improving Neural Model Performance through Natural Language Feedback on Their Explanations
• Neural Architecture Search for Sentence Classification with BERT
• NatCat: Weakly Supervised Text Classification with Naturally Annotated Resources
• Hidden Biases in Unreliable News Detection Datasets
• Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models
• A deep Natural Language Inference predictor without language-specific training data
• WASSA@IITK at WASSA 2021: Multi-task Learning and Transformer Finetuning for Emotion Classification and Empathy Prediction
• An Interpretable and Uncertainty Aware Multi-Task Framework for Multi-Aspect Sentiment Analysis
• A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification
• Probing Language Models for Understanding of Temporal Expressions
• On Inductive Biases for Machine Learning in Data Constrained Settings
• Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation
• Investigating Robustness of Dialog Models to Popular Figurative Language Constructs
• Semi-Supervised Text Classification via Self-Pretraining
• Syntax Customized Video Captioning by Imitating Exemplar Sentences
• Text Classification for Predicting Multi-level Product Categories
• Semi-Automating Knowledge Base Construction for Cancer Genetics
• Unifying Few- and Zero-Shot Egocentric Action Recognition
• ConQX: Semantic Expansion of Spoken Queries for Intent Detection based on Conditioned Text Generation
• Can Question Generation Debias Question Answering Models? A Case Study on Question-Context Lexical Overlap
• Distilling Linguistic Context for Language Model Compression
• Segmentation Approach for Coreference Resolution Task
• View Distillation with Unlabeled Data for Extracting Adverse Drug Effects from User-Generated Data
• Normalizador Neural de Datas e Endereços
• Causal Abstractions of Neural Networks
• Improvement of a dedicated model for open domain persona-aware dialogue generation
• Corpus-level and Concept-based Explanations for Interpretable Document Classification
• Multi-Pair Text Style Transfer on Unbalanced Data
• Noise Stability Regularization for Improving BERT Fine-tuning
• Neural Search: Learning Query and Product Representations in Fashion E-commerce
• Open Aspect Target Sentiment Classification with Natural Language Prompts
• Neural Optimization Kernel: Towards Robust Deep Learning
• Towards Zero-Shot Knowledge Distillation for Natural Language Processing
• Priberam Labs at the NTCIR-15 SHINRA2020-ML: Classification Task
• Political Ideology and Polarization of Policy Positions: A Multi-dimensional Approach
• A cost-benefit analysis of cross-lingual transfer methods
• Dispatcher: A Message-Passing Approach To Language Modelling
• When to Fold'em: How to answer Unanswerable questions
• CausalNLP: A Practical Toolkit for Causal Inference with Text
• TensorX: Extensible API for Neural Network Model Design and Deployment
• Autoregressive Reasoning over Chains of Facts with Transformers
• Word class flexibility: A deep contextualized approach
• CarneliNet: Neural Mixture Model for Automatic Speech Recognition
• ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning
• On the Significance of Question Encoder Sequence Model in the Out-of-Distribution Performance in Visual Question Answering
• The Topic Confusion Task: A Novel Scenario for Authorship Attribution
• From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension
• Speak or Chat with Me: End-to-End Spoken Language Understanding System with Flexible Inputs
• Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation
• Text Simplification by Tagging
• A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions
• Improving Similar Language Translation With Transfer Learning
• Subgoal Search For Complex Reasoning Tasks
• STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
• PicTalky: Augmentative and Alternative Communication Software for Language Developmental Disabilities
• Text-to-hashtag Generation using Seq2seq Learning
• Pretrained Language Models are Symbolic Mathematics Solvers too!
• Have Attention Heads in BERT Learned Constituency Grammar?
• Importance Estimation from Multiple Perspectives for Keyphrase Extraction
• Open4Business(O4B): An Open Access Dataset for Summarizing Business Documents
• Answer Generation for Questions With Multiple Information Sources in E-Commerce
• Using Deep Learning to Identify Patients with Cognitive Impairment in Electronic Health Records
• IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages
• Pruning Attention Heads of Transformer Models Using A* Search: A Novel Approach to Compress Big NLP Architectures
• Understanding Model Robustness to User-generated Noisy Texts
• From #Jobsearch to #Mask: Improving COVID-19 Cascade Prediction with Spillover Effects
• How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task
• Customizing Contextualized Language Models forLegal Document Reviews
• Do We Still Need Automatic Speech Recognition for Spoken Language Understanding?
• Domain-Transferable Method for Named Entity Recognition Task
• Turn-level Dialog Evaluation with Dialog-level Weak Signals for Bot-Human Hybrid Customer Service Systems
• Robust Document Representations using Latent Topics and Metadata
• diagNNose: A Library for Neural Activation Analysis
• User-friendly automatic transcription of low-resource languages: Plugging ESPnet into Elpis
• Learning Structured Representations of Entity Names using Active Learning and Weak Supervision
• How does a Pre-Trained Transformer Integrate Contextual Keywords? Application to Humanitarian Computing
• Detecting Attackable Sentences in Arguments
• RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
• Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method
• Converting the Point of View of Messages Spoken to Virtual Assistants
• Deep Learning Meets Projective Clustering
• Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset
• Refinement of genetic variants needs attention
• A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing
• Hierarchical Summarization for Longform Spoken Dialog
• Structural analysis of an all-purpose question answering model
• Calibrate your listeners! Robust communication-based training for pragmatic speakers
• Semantic maps and metrics for science Semantic maps and metrics for science using deep transformer encoders
• Continuous Entailment Patterns for Lexical Inference in Context
• BERT Embeddings Can Track Context in Conversational Search
• CoRGi: Content-Rich Graph Neural Networks with Attention
• BERT based Transformers lead the way in Extraction of Health Information from Social Media
• Probabilistic Predictions of People Perusing: Evaluating Metrics of Language Model Performance for Psycholinguistic Modeling
• Debiasing Sentence Embedders through Contrastive Word Pairs