Technow: Block sparsity by Meta, RAPIDS cuDF by Nvidia, efficient-kan
Unlocking faster AI performance is the focus of today’s post! Discover how block sparsity speeds up Vision Transformers (ViTs) by 1.46x with minimal accuracy loss, potentially benefiting large language models too. Learn about RAPIDS cuDF integration in Google Colab, offering up to 50x acceleration for pandas code on GPU instances. Plus, dive into the efficient implementation of Kolmogorov-Arnold Network (KAN) that reduces memory costs and enhances computation efficiency.
Speeding up ViTs with Block sparsity
They show promising results of up to a 1.46x speedup with <2% drop in accuracy on float32 Vision Transformers on A100 GPUs by applying block sparsity on MLP module’s weights. This approach can potentially be applied to other types of transformers including large language models.
RAPIDS cuDF 50x pandas in Colab notebook
At Google I/O’24, Laurence Moroney, head of AI Advocacy at Google, announced that RAPIDS cuDF is now integrated into Google Colab. Developers can now instantly accelerate pandas code up to 50x on Google Colab GPU instances, and continue using pandas as data grows—without sacrificing performance.
efficient-kan
This repository contains an efficient implementation of Kolmogorov-Arnold Network (KAN), a new neural network architecture that was introduced by a recent paper.
The performance issue of the original implementation comes from expanding all intermediate variables to perform the different activation functions. This repository implements a solution to this efficiency gap by formulating all activation functions as a linear combination of a fixed set of basis functions.
This reformulation can significantly reduce the memory cost and make the computation a straightforward matrix multiplication, and works with both forward and backward pass naturally.
