24 citations · 31 across the 8 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2023
Structured Packing in LLM Training Improves Long Context Utilization
Konrad Staniszewski, Szymon Tworkowski, Sebastian Jaszczur +4
Recent advancements in long-context large language models have attracted significant attention, yet their practical applications often suffer from suboptimal context utilization. T…
cs.CL2023★ 1 cited
Mixture of Tokens: Continuous MoE through Cross-Example Aggregation
Szymon Antoniak, Michał Krutul, Maciej Pióro +7
Mixture of Experts (MoE) models based on Transformer architecture are pushing the boundaries of language and vision tasks. The allure of these models lies in their ability to subst…