4 papers
Balancing Sparse RNNs with Hyperparameterization Benefiting Meta-Learning
Quincy Hershey, Randy Paffenroth
This paper develops alternative hyperparameters for specifying sparse Recurrent Neural Networks (RNNs). These hyperparameters allow for varying sparsity within the trainable weight…
Principled Curriculum Learning using Parameter Continuation Methods
Harsh Nilesh Pathak, Randy Paffenroth
In this work, we propose a parameter continuation method for the optimization of neural networks. There is a close connection between parameter continuation, homotopies, and curric…
Solo Connection: A Parameter Efficient Fine-Tuning Technique for Transformers
Harsh Nilesh Pathak, Randy Paffenroth
Parameter efficient fine tuning (PEFT) is a versatile and extensible approach for adapting a Large Language Model (LLM) for newer tasks. One of the most prominent PEFT approaches,…
Rethinking the Relationship between Recurrent and Non-Recurrent Neural Networks: A Study in Sparsity
Quincy Hershey, Randy Paffenroth, Harsh Pathak +1
Neural networks (NN) can be divided into two broad categories, recurrent and non-recurrent. Both types of neural networks are popular and extensively studied, but they are often tr…