Deepdive: half memory with sequential backward calls, SaySelf, Diffusion On Syntax Trees

Unlock transformative advancements in AI with these three cutting-edge techniques. First, learn how to slash your GPU memory usage by up to 50% with a simple PyTorch trick, allowing you to double your batch size by calling backward() on each loss separately. Next, discover SaySelf, a revolutionary framework for Large Language Models (LLMs) that drastically improves confidence estimation by 30%, providing more reliable self-reflective rationales and reducing errors. Finally, dive into the world of neural diffusion models with a technique that edits syntax trees directly, boosting code generation efficiency by 20% and enhancing debugging accuracy. These innovations are poised to redefine AI performance, making your models faster, more efficient, and safer.

half memory with sequential backward calls

Using this simple PyTorch trick, you can cut your GPU memory usage by up to 50% or effectively double your batch size. Instead of summing multiple losses and then calling backward(), you should call backward() on each loss separately.

# Don't do this
loss1 = compute_loss1(model, images)
loss2 = compute_loss2(model, images)

loss = loss1 + loss2
loss.backward()


# Do
this
loss1 = compute_loss1(model, images)
loss1.backward()

loss2 = compute_loss2(model, images)
loss2.backward()

Why It Works
This approach frees the computational graph after each backward() call, leading to substantial memory savings. For example, if your model originally used 8GB of GPU memory with a batch size of 32, applying this technique could reduce memory usage to 4GB, allowing you to double your batch size to 64. The results will remain identical to the traditional method.

Limitations
This technique only works if you have multiple losses with disentangled computational graphs.

Bonus
The more losses you have, the more memory you will save.

SaySelf: Teaching LLMs to Express Confidence

Large language models (LLMs) often generate inaccurate information and fail to provide reliable confidence estimates, limiting their applications. Previous methods for confidence estimation, including prompting and training-based approaches, yield suboptimal or binary confidence scores.

Solution
SaySelf, a training framework, improves LLM confidence estimates and generates self-reflective rationales. It uses multiple reasoning chains, semantic clustering, and reinforcement learning with a custom reward function to fine-tune LLMs.

Results
SaySelf reduces confidence calibration error by 30% and maintains task performance on both in-distribution and out-of-distribution datasets. Generated rationales further enhance calibration accuracy. The code is publicly available on GitHub.

Diffusion On Syntax Trees

Autoregressive language models generate code line by line without runtime feedback, making debugging difficult and inefficient.

Solution

Shreyas Kapur’s neural diffusion model edits syntax trees directly, introducing a “noise” process inspired by fuzzing techniques. This model iteratively refines and debugs code while ensuring syntactic validity.

Results

The model improves efficiency by 20% in inverse graphics tasks, requires 30% fewer nodes, and achieves 99% accuracy for TinySVG with a pixel matching tolerance of 0.005. This approach allows faster, more accurate, and resource-efficient code generation and debugging.