86 citations · 105 across the 10 of their papers we have counts for
10 papers
Compute-in-Memory Attention: A Time-Domain Analog Softmax Circuit with RC-Tunable Temperature
Ankur Singh, Ashish Gautam, Shruti R. Kulkarni +1
Softmax is a key operation in Transformer attention, but its exponentiation and normalization add significant overhead in compute-in-memory (CIM) accelerators, especially when anal…
VS-Graph: Scalable and Efficient Graph Classification Using Hyperdimensional Computing
Hamed Poursiami, Shay Snyder, Guojing Cong +2
Graph classification is a fundamental task in domains ranging from molecular property prediction to materials design. While graph neural networks (GNNs) achieve strong performance…
HyperGraphX: Graph Transductive Learning with Hyperdimensional Computing and Message Passing
Guojing Cong, Tom Potok, Hamed Poursiami +1
We present a novel algorithm, \hdgc, that marries graph convolution with binding and bundling operations in hyperdimensional computing for transductive graph learning. For predicti…
Transductive Spiking Graph Neural Networks for Loihi
Shay Snyder, Victoria Clerico, Guojing Cong +4
Graph neural networks have emerged as a specialized branch of deep learning, designed to address problems where pairwise relations between objects are crucial. Recent advancements…
Comparative Study of Large Language Model Architectures on Frontier
Junqi Yin, Avishek Bose, Guojing Cong +2
Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the d…
Optimizing Distributed Training on Frontier for Large Language Models
Sajal Dash, Isaac Lyngaas, Junqi Yin +5
Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Recent studies on loss sca…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…
Prediction of Adsorption in Nano-Pores with Graph Neural Networks
Guojing Cong, Anshul Gupta, Rodrigo Neumann +3
We investigate the graph-based convolutional neural network approach for predicting and ranking gas adsorption properties of crystalline Metal-Organic Framework (MOF) adsorbents fo…
AI-aided multiscale modeling of physiologically-significant blood clots
Yicong Zhu, Changnian Han, Peng Zhang +5
We have developed an AI-aided multiple time stepping (AI-MTS) algorithm and multiscale modeling framework (AI-MSM) and implemented them on the Summit-like supercomputer, AIMOS. AI-…
On the convergence properties of a -step averaging stochastic gradient descent algorithm for nonconvex optimization
Fan Zhou, Guojing Cong
Despite their popularity, the practical performance of asynchronous stochastic gradient descent methods (ASGD) for solving large scale machine learning problems are not as good as…