4 papers · 1 filter
Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedures
Simon Dennis, Kevin Shabahang, Hao Guo +1
Parameter-efficient fine-tuning methods like LoRA have become the default for adapting large language models, succeeding across instruction following, style transfer, and factual a…
When Mean CE Fails: Median CE Can Better Track Language Model Quality
Hao Guo, Simon Dennis, Rivaan Patil +1
Mean cross-entropy is the standard validation metric for language models, but it can fail to track model quality during training. We examine this in two common scenarios. First, in…
Beyond Inference-Only Deployment: Comparing Weight-Based Consolidation Against Cascading Compaction
Simon Dennis, Kevin Shabahang, Hao Guo +1
Major LLM platforms deploy models in an inference-only configuration: the model serves requests but never updates per-user weights. Users must repeatedly re-teach preferences, corr…
In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks
Simon Dennis, Michael Diamond, Rivaan Patil +2
Agent orchestration frameworks -- LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, and others -- place an external orchestrator above the LLM, tracking state and injecting routing…