Cassia.ai: 20x Lower Power AI, Energy-efficient Language Models, using Approximate Computing

The AI industry is in a "gold rush," with heavy investment in compute power for advanced AI. This race mirrors the early internet, where giants like Google arose. However, this pursuit of powerful AI is costly, both financially and environmentally, due to the energy consumption of data centers.

The interview features James Tandon, the CEO and founder of https://www.Cassia.ai and Mark Greenberg, the VP of Products, Cassia.ai is Patent Pending Technology. They discuss the current state of the AI industry, comparing it to the early days of the internet. Like the internet boom, there’s a race to capture users and market share, with companies investing heavily in AI technology and infrastructure. Despite the vast amounts of money being poured into AI development, the focus remains on acquiring users first, with monetization strategies to follow later.

Research paper: https://ieeexplore.ieee.org/document/...

Cassia.ai offers a solution: approximate computation. This reduces AI's computational needs without significant accuracy loss, using fewer transistors and logic gates for calculations, thus lowering power use and improving efficiency. While not for everything, it's promising for reducing AI's environmental impact and making it more accessible for edge devices.

Cassia.ai's core technology lies in its approach to numbers. By treating floating-point numbers as fixed-point and using logarithmic multiplication, it achieves significant efficiency improvements, enabling faster, less power-hungry calculations ideal for AI algorithms.

Cassia.ai's techniques might slightly reduce accuracy, but this is often negligible in AI. Neural networks are resilient to minor errors, making them suitable for approximate computation. Any accuracy loss can be recovered through further training.

Cassia.ai's technology is adaptable across AI applications. Whether for cloud-based AI or tinyML chips, its techniques can be integrated into various platforms. This benefits chip makers and AI developers aiming for efficiency.

Cassia.ai's benefits extend beyond power reduction. It enables faster processing and reduced latency, crucial for real-time applications like autonomous driving. Furthermore, it makes AI more accessible in areas with limited resources.

Cassia.ai envisions partnering with others to bring approximate computation to a wider audience, driving a more sustainable and efficient future for AI, and offering a compelling solution as AI demand grows.

The AI industry is booming, but it's facing a major power problem. We explore how approximate computing could be the solution to this problem, allowing us to run AI models more efficiently and on smaller devices.
They explain that the AI industry is experiencing a "gold rush" as companies race to develop and deploy the most advanced models, fueled by massive investments. However, this rapid growth comes at a cost: power consumption.
● Training and running these large language models (LLMs) requires immense computational power, leading to a surge in demand for high-performance computing hardware and data centers.
● This, in turn, is pushing the limits of global power consumption and infrastructure, with companies like OpenAI proposing data centers that would consume a staggering 1% of humanity's total power usage.
This situation presents a challenge and an opportunity. They discuss approximate computing as a potential solution to the AI power problem.
● Approximate computing involves using simplified mathematical operations, such as replacing complex multiplications with additions, which are less computationally intensive.
● This approach introduces a small degree of error into the calculations, but the sources argue that neural networks are inherently resilient to such errors, and the overall accuracy loss is minimal.
They highlight Cassia.ai's technology, which utilizes a logarithmic-based multiplication approximation technique. This approach significantly reduces the power consumption of AI computations, particularly for matrix multiplication operations, which are fundamental to LLMs.
● The sources claim that Cassia.ai's technology can achieve a 10x reduction in power consumption for 16-bit floating-point multiplications with less than 1% accuracy loss in neural network output.
● This technology is compatible with existing AI hardware and software ecosystems, including quantization and sparsity techniques, further enhancing its potential for power savings.
They acknowledge that approximate computing may not be suitable for all applications, particularly those requiring absolute precision. However, for many AI use cases, the trade-off between accuracy and power efficiency is highly favorable, especially for edge devices and resource-constrained environments.
They conclude that approximate computing offers a promising path forward for the AI industry, enabling the development and deployment of more powerful and energy-efficient AI models.