5 citations · 10 across the 12 of their papers we have counts for
10 papers · 1 filter
CoPeP: Benchmarking Continual Pretraining for Protein Language Models
Darshan Patil, Pranshu Malviya, Mathieu Reymond +2
Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, structure, and function from evolutionary sta…
Torque-Aware Momentum
Pranshu Malviya, Goncalo Mordido, Aristide Baratin +4
Efficiently exploring complex loss landscapes is key to the performance of deep neural networks. While momentum-based optimizers are widely used in state-of-the-art setups, classic…
Manifold Metric: A Loss Landscape Approach for Predicting Model Performance
Pranshu Malviya, Jerry Huang, Aristide Baratin +2
Determining the optimal model for a given task often requires training multiple models from scratch, which becomes impractical as dataset and model sizes grow. A more efficient alt…
Lookbehind-SAM: k steps back, 1 step forward
Gonçalo Mordido, Pranshu Malviya, Aristide Baratin +1
Sharpness-aware minimization (SAM) methods have gained increasing popularity by formulating the problem of minimizing both loss value and loss sharpness as a minimax objective. In…
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Pranshu Malviya, Gonçalo Mordido, Aristide Baratin +5
Adaptive gradient-based optimizers, notably Adam, have left their mark in training large-scale deep learning models, offering fast convergence and robustness to hyperparameter sett…
Feature diversity in self-supervised learning
Pranshu Malviya, Arjun Vaithilingam Sudhakar
Many studies on scaling laws consider basic factors such as model size, model shape, dataset size, and compute power. These factors are easily tunable and represent the fundamental…
An Introduction to Lifelong Supervised Learning
Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta +4
This primer is an attempt to provide a detailed summary of the different facets of lifelong learning. We start with Chapter 2 which provides a high-level overview of lifelong learn…
A Causal Approach for Unfair Edge Prioritization and Discrimination Removal
Pavan Ravishankar, Pranshu Malviya, Balaraman Ravindran
In budget-constrained settings aimed at mitigating unfairness, like law enforcement, it is essential to prioritize the sources of unfairness before taking measures to mitigate them…
TAG: Task-based Accumulated Gradients for Lifelong learning
Pranshu Malviya, Balaraman Ravindran, Sarath Chandar
When an agent encounters a continual stream of new tasks in the lifelong learning setting, it leverages the knowledge it gained from the earlier tasks to help learn the new tasks b…
Contextual Care Protocol using Neural Networks and Decision Trees
Yash Pratyush Sinha, Pranshu Malviya, Minerva Panda +1
A contextual care protocol is used by a medical practitioner for patient healthcare, given the context or situation that the specified patient is in. This paper proposes a method t…