5 papers
DREG: A Layer-Wise Jacobian Regularization as a General-Purpose Penalty
Rowan Martnishn
We present a large-scale empirical study isolating the contributions of the Derivative Regularization penalty (DREG). Across a fully-crossed factorial sweep of 960 experiments span…
Layer-wise Derivative Controlled Networks Achieve Competitive Accuracy and Gradient Stability Across Data Regimes
Rowan Martnishn
Derivative-controlled networks based on ChainzRule (CR) combine cubic polynomial layers with a lightweight forward-mode per-layer Jacobian penalty (DREG). In this second paper of a…
Pre-Warm: Initializing Convolutional Filters from First-Batch Patch Dictionaries
Rowan Martnishn
Random initialization of convolutional filters does not use the training images. Previous work has shown that image patches can be copied into the first layer, and that k-means or…
ChainzRule: Sample-Efficient, Robust Deep Learning Across Tabular, NLP, and Vision Tasks
Rowan Martnishn
Production deep learning systems across enterprise domains operate under constraints that academic benchmarks routinely obscure: labeled data is expensive, inference budgets are ti…
Layer-wise Derivative Controlled Networks
Rowan Martnishn, Sean Anderson
As machine learning models grow in complexity, they increasingly struggle with three conflicting demands: the need for high accuracy, the requirement for hardware efficiency, and t…