4 papers · 1 filter
WNet: Discrete Wavelets Transform for Efficient Token Mixing
Rana Aref Salama, Abdou Youssef, Mona Diab
In a Transformer, token mixing is the step that lets each token draw information from other tokens, and it dominates the cost of encoding long sequences. Self-attention does this m…
DWTSumm: Discrete Wavelet Transform for Document Summarization
Rana Salama, Abdou Youssef, Mona Diab
Summarizing long, domain-specific documents with large language models (LLMs) remains challenging due to context limitations, information loss, and hallucinations, particularly in…
Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding
Rana Salama, Abdou Youssef, Mona Diab
Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in Image and Signal processing suggest that wavelets can be effectiv…
Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform
Rana Aref Salama, Abdou Youssef, Mona Diab
Wavelet transforms, a powerful mathematical tool, have been widely used in different domains, including Signal and Image processing, to unravel intricate patterns, enhance data rep…