Technow: torchtune, Boston Dynamics Atlas robot, Reka Core

Easily fine-tune large language models (LLMs) with torchtune, PyTorch’s new library. Torchtune offers modular and customizable fine-tuning recipes, supporting efficient workflows from data preparation to model evaluation. It integrates with popular tools like Hugging Face Hub and Weights & Biases and excels in memory efficiency, making it ideal for users with limited hardware. Designed for simplicity and extensibility, torchtune supports models like Llama 2 and Mistral, with future plans for models up to 70 billion parameters.

torchtune: Easily fine-tune LLMs using PyTorch

PyTorch has released torchtune, a library designed for fine-tuning large language models (LLMs). This PyTorch-native tool prioritizes modularity and customization, offering straightforward, memory-efficient fine-tuning recipes that allow users to adapt LLMs for specific uses easily.


The library covers the full scope of the fine-tuning process, from preparing datasets and models to evaluating and running local inference.

Key aspects of the workflow include:

• Preparing data and models: It automates the downloading and setup of datasets and model checkpoints.
• Customizing training: Users can tailor the training process using adaptable building blocks for various architectures and tuning methods.
• Logging and quantization: Torchtune logs progress and metrics and supports model quantization after tuning.

Torchtune integrates with well-known tools and platforms, enhancing its functionality within the LLM ecosystem. Important integrations feature:

• Hugging Face Hub: Accesses and utilizes model weights.
• PyTorch FSDP: Scales training across multiple GPUs.
• Weights & Biases: Logs detailed metrics and tracks progress.
• ExecuTorch: Facilitates efficient inference across diverse devices.

Torchtune stands out for its memory efficiency, making it suitable for users with limited hardware, such as a single 24GB GPU. This efficiency enables sophisticated fine-tuning on less powerful systems.


Designed with simplicity, extensibility, and correctness in mind, torchtune ensures ease of use and comprehensive understanding. All training recipes are concise, typically under 600 lines, without external frameworks.


Torchtune currently supports models such as Llama 2, Mistral, and Gemma 7B, with plans to expand support to include models up to 70 billion parameters and Mixture of Experts (MoEs) models.

!pip install torchtune

Already loving the docs and that they are also educational. Thanks! Chris Levy

Congrats PyTorch team on the launch but I think we could already finetune LLMs pretty easily with libraries like LitGPT, and Axolotl. Genuinely asking what makes TorchTune different from the existing libraries? Aniket Maurya: “

Amazing, nothing could be better then having a builtin library with no third party dependency. FODUU

Boston Dynamics’ Atlas Humanoid Robot Goes Electric

Boston Dynamics has transitioned its Atlas robot from hydraulic to electric, enhancing its movement range and efficiency. The new Atlas features advanced actuators for improved mobility, AI-enhanced machine learning tools for adaptability, and is designed for integration with enterprise-level software for fleet management. Testing begins next year with Hyundai, targeting commercial production post-validation.

Reka Releases Reka Core

Reka Releases Reka Core, Its Multimodal Language Model To Rival Gpt-4 And Claude 3 Opus
Reka AI, a startup with roots in DeepMind, Google, and Meta, has introduced Reka Core, a multimodal language model trained on thousands of GPUs and rivaling models like GPT-4 and Claude 3 Opus. Core understands multiple modalities (text, image, audio, video), supports 32 languages, has a 128K token context window, and offers API, on-premise, or device deployment.