New Method Significantly Boosts AI Speed, Energy Efficiency on Existing Hardware

Artificial intelligence relies on power-hungry hardware, driving companies to build massive data centers. New research led by the University of California, Davis, shows AI models can maintain response quality while being faster and using far less energy on existing hardware. 

“Rather than just focus on making existing mathematical operations faster, we’re thinking about how we can tweak and change the model’s operations so that they fit more naturally with the hardware,” said Cameron Shinn, an electrical and computer engineering Ph.D. student working in Child Family Professor of Engineering and Entrepreneurship John Owens’ research group. 

Shinn and a team of researchers developed a method allowing AI models to skip low-value computations, detailed in a paper published on arXiv. Called Dynamic Blocked Attention Sparsity via Softmax Thresholding, or BLASST, the method makes models up to 50% faster without needing to retrain models or introduce new hardware. 

The work received the Best Paper Award during MLSys 2026, a leading machine learning and AI systems conference, and NVIDIA is already integrating the method into its AI software. 

BLASST Processing 

When an AI model answers a prompt, it compares relationships between words (represented by numbers known as tokens) to provide a response. It does this with an attention mechanism, which allows the system to assess how each word pairing is or is not relevant to the prompt. The longer the prompt becomes, the amount of computation grows quadratically, placing more pressure on the hardware. 

With BLASST, the model can skip attention scores that contribute little value to its response. 

Take a book group discussing the morality of a book’s main character. Members would likely cite scenes where the character made a decision (high attention score) and avoid sections focusing on the decisions of side characters (low attention score). While the side characters’ decisions are important for the overall story, they aren’t very useful in providing an answer to the protagonist’s moral character. 

BLASST works in much the same way. It helps the AI model understand what information is important for the question at hand and what is not. With fewer computations, the hardware is faster, and less energy is consumed. 

“You don’t always need all of that earlier context to understand the current moment,” explained Shinn. 

Working Smarter, Not Harder 

The BLASST method is a sparse attention approach, which focuses on only the most important relationships between words.  

Previous sparse attention methods have required significant computational power to identify what is high-value and what is low-value, reducing their efficiency and speedup. They also often require retraining the AI model or reconfiguring the model’s architecture.  

BLASST works around previous efficiency issues by reusing information the model has already computed. 

"We're trying to grab as much of the existing work and math that's already being done by the model and use that information instead of trying to come up with our own complex calculations,” Shinn said. 

BLASST also calibrates itself to the hardware powering the AI model, meaning it requires no retraining or adjustments to function. 

Innovation Takes Collaboration 

The idea for BLASST grew out of Shinn's two internships at NVIDIA. One of his supervisors suggested he continue to work on an idea about making AI models more efficient, which led Shinn and other interns to continue developing the concept into BLASST.  

Throughout the process, the method evolved into a collaborative effort bringing together researchers from UC Davis, Rice University, NVIDIA and Meta. 

"The project was a very big team effort," Shinn said. "We had researchers and product people helping out. It was a lot of people from multiple different teams coming together to work on a common idea." 

Shifting toward AI Efficiency 

Many advances in artificial intelligence have come from more training or adding more computational power. Shinn believes the future of AI innovation will come from finding ways to make models more efficient. 

“The idea is: Can we achieve the same capabilities with less compute, less power, less energy, less time and less money?” Shinn said. “BLASST shows that you can reliably get about a 50% speedup with existing hardware. When companies are spending hundreds of millions of dollars on computing, that could represent an enormous amount of money, energy and time saved.”