3 citations · 3 across the 7 of their papers we have counts for
7 papers
Surprisal-Rényi Free Energy
Shion Matsumoto, Raul Castillo, Benjamin Prada +1
The forward and reverse Kullback-Leibler (KL) divergences arise as limiting objectives in learning and inference yet induce markedly different inductive biases that cannot be expla…
Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learning
Ankur Mali, Lawrence Hall, Jake Williams +1
Activation functions govern the expressivity and stability of neural networks, yet existing comparisons remain largely heuristic. We propose a rigorous framework for their classifi…
Realizable Circuit Complexity: Embedding Computation in Space-Time
Benjamin Prada, Ankur Mali
Classical circuit complexity characterizes parallel computation in purely combinatorial terms, ignoring the physical constraints that govern real hardware. The standard classes $\m…
Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT
Rushitha Santhoshi Mamidala, Anshuman Chhabra, Ankur Mali
Prompt-based reasoning strategies such as Chain-of-Thought (CoT) and In-Context Learning (ICL) have become widely used for eliciting reasoning capabilities in large language models…
Bridging Predictive Coding and MDL: A Two-Part Code Framework for Deep Learning
Benjamin Prada, Shion Matsumoto, Abdul Malik Zekri +1
We present the first theoretical framework that connects predictive coding (PC), a biologically inspired local learning rule, with the minimum description length (MDL) principle in…
TeLU Activation Function for Fast and Stable Deep Learning
Alfredo Fernandez, Ankur Mali
We propose the Hyperbolic Tangent Exponential Linear Unit (TeLU), a neural network hidden activation function defined as TeLU(x)=xtanh(exp(x)). TeLU's design is grounded in the cor…
Exploring Learnability in Memory-Augmented Recurrent Neural Networks: Precision, Stability, and Empirical Insights
Shrabon Das, Ankur Mali
This study explores the learnability of memory-less and memory-augmented RNNs, which are theoretically equivalent to Pushdown Automata. Empirical results show that these models oft…