13 citations · 32 across the 19 of their papers we have counts for
22 papers
Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts
Soumajyoti Sarkar, Yuxin Tang, Sheng Zha
In large-scale pretraining, the algorithm, architecture, and systems decisions are conventionally made in disconnected stages. A scaling law stage selects an architecture and train…
Tokens-per-Parameter Coverage Is Critical for Robust LLM Scaling Law Extrapolation
Joshua Shay Kricheli, Alexander Lawrence Reid, Soumajyoti Sarkar +2
Neural scaling laws approximate a language model's loss as a power-law function of parameter count and token count . Following Chinchilla-style compute-optimal training, man…
Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
Soumajyoti Sarkar, Leonard Lausen, Volkan Cevher +3
Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. These models use conditionally activated feedforward subnetworks…
EMC: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence
Chung-Yiu Yau, Hoi-To Wai, Parameswaran Raman +2
A key challenge in contrastive learning is to generate negative samples from a large sample set to contrast with positive samples, for learning better encoding of the data. These n…
Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Tasks
Soumajyoti Sarkar, Leonard Lausen
Tables stored in databases and tables which are present in web pages and articles account for a large part of semi-structured data that is available on the internet. It then become…
HYTREL: Hypergraph-enhanced Tabular Data Representation Learning
Pei Chen, Soumajyoti Sarkar, Leonard Lausen +4
Language models pretrained on large collections of tabular data have demonstrated their effectiveness in several downstream tasks. However, many of these models do not take into ac…
An Algorithm For Adversary Aware Decentralized Networked MARL
Soumajyoti Sarkar
Decentralized multi-agent reinforcement learning (MARL) algorithms have become popular in the literature since it allows heterogeneous agents to have their own reward functions as…
Parameter and Data Efficient Continual Pre-training for Robustness to Dialectal Variance in Arabic
Soumajyoti Sarkar, Kaixiang Lin, Sailik Sengupta +3
The use of multilingual language models for tasks in low and high-resource languages has been a success story in deep learning. In recent times, Arabic has been receiving widesprea…
Bandit based centralized matching in two-sided markets for peer to peer lending
Soumajyoti Sarkar
Sequential fundraising in two sided online platforms enable peer to peer lending by sequentially bringing potential contributors, each of whose decisions impact other contributors…
Bandits in Matching Markets: Ideas and Proposals for Peer Lending
Soumajyoti Sarkar
Motivated by recent applications of sequential decision making in matching markets, in this paper we attempt at formulating and abstracting market designs for P2P lending. We descr…
Mitigating Bias in Online Microfinance Platforms: A Case Study on Kiva.org
Soumajyoti Sarkar, Hamidreza Alvari
Over the last couple of decades in the lending industry, financial disintermediation has occurred on a global scale. Traditionally, even for small supply of funds, banks would act…
A Feature-Driven Approach for Identifying Pathogenic Social Media Accounts
Hamidreza Alvari, Ghazaleh Beigi, Soumajyoti Sarkar +4
Over the past few years, we have observed different media outlets' attempts to shift public opinion by framing information to support a narrative that facilitate their goals. Malic…
Can social influence be exploited to compromise security: An online experimental evaluation
Soumajyoti Sarkar, Paulo Shakarian, Mika Armenta +2
Social media has enabled users and organizations to obtain information about technology usage like software usage and even security feature usage. However, on the dark side it has…
Mining user interaction patterns in the darkweb to predict enterprise cyber incidents
Soumajyoti Sarkar, Mohammad Almukaynizi, Jana Shakarian +1
With rise in security breaches over the past few years, there has been an increasing need to mine insights from social media platforms to raise alerts of possible attacks in an att…
Use of a controlled experiment and computational models to measure the impact of sequential peer exposures on decision making
Soumajyoti Sarkar, Ashkan Aleali, Paulo Shakarian +3
It is widely believed that one's peers influence product adoption behaviors. This relationship has been linked to the number of signals a decision-maker receives in a social networ…
Understanding Information Flow in Cascades Using Network Motifs
Soumajyoti Sarkar, Hamidreza Alvari, Paulo Shakarian
A growing set of applications consider the process of network formation by using subgraphs as a tool for generating the network topology. One of the pressing research challenges is…
Less is More: Semi-Supervised Causal Inference for Detecting Pathogenic Users in Social Media
Hamidreza Alvari, Elham Shaabani, Soumajyoti Sarkar +2
Recent years have witnessed a surge of manipulation of public opinion and political events by malicious social media actors. These users are referred to as "Pathogenic Social Media…
Using network motifs to characterize temporal network evolution leading to diffusion inhibition
Soumajyoti Sarkar, Ruocheng Guo, Paulo Shakarian
Network motifs are patterns of over-represented node interactions in a network which have been previously used as building blocks to understand various aspects of the social networ…
Detection of Violent Extremists in Social Media
Hamidreza Alvari, Soumajyoti Sarkar, Paulo Shakarian
The ease of use of the Internet has enabled violent extremists such as the Islamic State of Iraq and Syria (ISIS) to easily reach large audience, build personal relationships and i…
Leveraging Motifs to Model the Temporal Dynamics of Diffusion Networks
Soumajyoti Sarkar, Hamidreza Alvari, Paulo Shakarian
Information diffusion mechanisms based on social influence models are mainly studied using likelihood of adoption when active neighbors expose a user to a message. The problem aris…
Predicting enterprise cyber incidents using social network analysis on the darkweb hacker forums
Soumajyoti Sarkar, Mohammad Almukaynizi, Jana Shakarian +1
With rise in security breaches over the past few years, there has been an increasing need to mine insights from social media platforms to raise alerts of possible attacks in an att…
Understanding and forecasting lifecycle events in information cascades
Soumajyoti Sarkar, Ruocheng Guo, Paulo Shakarian
Most social network sites allow users to reshare a piece of information posted by a user. As time progresses, the cascade of reshares grows, eventually saturating after a certain t…