3 citations · 3 across the 5 of their papers we have counts for
6 papers
Hopscotch: Discovering and Skipping Redundancies in Language Models
Mustafa Eyceoz, Nikhil Shivakumar Nayak, Hao Wang +2
Modern causal language models stack many attention blocks to improve performance, but not all blocks are necessary for every task. We propose Hopscotch, a simple yet effective meth…
Mathematical Modeling of Option Pricing with an Extended Black-Scholes Framework
Nikhil Shivakumar Nayak
This study investigates enhancing option pricing by extending the Black-Scholes model to include stochastic volatility and interest rate variability within the Partial Differential…
Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
Nikhil Shivakumar Nayak, Krishnateja Killamsetty, Ligong Han +8
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. E…
Graph Attention for Heterogeneous Graphs with Positional Encoding
Nikhil Shivakumar Nayak
Graph Neural Networks (GNNs) have emerged as the de facto standard for modeling graph data, with attention mechanisms and transformers significantly enhancing their performance on…
Towards Interpretable Soft Prompts
Oam Patel, Jason Wang, Nikhil Shivakumar Nayak +2
Soft prompts have been popularized as a cheap and easy way to improve task-specific LLM performance beyond few-shot prompts. Despite their origin as an automated prompting method,…
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs
Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang +10
The rise of large language models (LLMs) has created a significant disparity: industrial research labs with their computational resources, expert teams, and advanced infrastructure…