Training

Track Your Experiments in SuperGradients Using Weights & Biases

As a deep learning practitioner, navigating the vast landscape of models, selecting the ideal one for your use case, and optimizing it to achieve the best possible results for your dataset can be challenging. 

That’s where SuperGradients comes in — an open source PyTorch-based library that gives you easy access to cutting-edge models, already fine-tuned and optimized to implement with just a few lines of code.

The SuperGradients model zoo is a great starting point for your projects, allowing you to develop a strong baseline and jumping-off point. Even with a strong baseline, you still have to run experiments.

Conducting experiments is core to your work as a deep learning practitioner. 

From fine-tuning hyperparameters to selecting the right architecture, running experiments helps you evaluate various aspects of your model and determine the best configurations, improving performance and accuracy.

Experiment tracking can be challenging due to the sheer volume of information.

As deep learning projects often involve numerous hyperparameters, model architectures, and data variations, it becomes increasingly difficult to keep track of everything, leading to disorganization and lost insights.

Reproducibility is a huge challenge deep learning, as it helps ensure your work is accurate and reliable. 

Maintaining a detailed record of your experiments enables you and others to validate your findings, recreate your models, and build upon your successes. This ultimately fosters collaboration and accelerates the overall progress of AI research.

Weights & Biases (WandB) became a popular choice of tooling for streamlined experiment tracking.

WandB is an experiment management platform designed to help you efficiently monitor and analyze your experiments, making optimizing your models, reproducing results, and collaborating with fellow researchers easier than ever.

By combining the powerful SuperGradients library with Weights & Biases, you’ll have access to an unparalleled deep learning toolkit. 

This blog post will show you how to seamlessly  integrate SuperGradients with WandB.

What is SuperGradients?

Deci’s SuperGradients Training Library.

SuperGradients is Deci’s free open-source library to train PyTorch-based deep learning models. 

It supports many common computer vision tasks, including object detection, image classification, and semantic segmentation.

SuperGradients streamlines the training process through a single, versatile script, allowing you to easily load and fine-tune pre-trained models, many of which offer improved accuracy compared to existing training libraries.

This powerful tool enables you to train or fine-tune state-of-the-art (SOTA) computer vision models using just one training library. 

SuperGradients significantly shortens the training process, facilitates result reproduction, and eliminates the uncertainty surrounding your model’s training outcomes.

However, it lacks an integrated visual monitoring solution. That’s where Weights & Biases (W&B) comes into play—an intuitive tool designed to easily track, compare, and visualize your machine learning experiments.

By combining the prowess of SuperGradients with the experiment tracking and model monitoring capabilities of Weights & Biases, you can create a seamless and effective deep learning workflow for your computer vision projects.

A Peek into Weights and Biases (WandB)

WandB is a tool that makes it easy to track, compare, and visualize ML experiments using just a few lines of code. It’s free for academic projects and is currently used by many big-league players to optimize their ML activities. 

WandB has some cool features that make it easy to get live metrics showing how your models and tuning are progressing. 

You will see graphs illustrating how the model improved (or didn’t) from one version to the other, which parameters helped improve performance, and whether your tweaking is moving you in the right direction or away from your target goals. Each time you try out a new version, you will see the results streamed live to the dashboard. 

It is ideal for monitoring, tracking, and managing your experiments as it provides full visualization for the training statistics, automated our experiments, and made it easy to share the results with colleagues. 

Seamless Integration With WandB

WandB can be thought of as an “extended” TensorBoard, where you can manage experiments, create reports, upload files, etc.

To set up the environment:

  1. Go to your WandB settings, and create an API Key.
  2. On the machine, run pip install wandb
  3. Once WandB is installed, run wandb login and use your username and the generated key. There should be a .netrc file created.
  4. Run cat ~/.netrc and make sure you see something like the following:

machine **wandb.your_env**
  login <your user>
  password <your key>

If you don’t see this, just create it manually.

  1. To enable WandB, log in via SuperGradients and add the following code to your .yaml training file:

training_params:
  # Your training params
  sg_logger: "wandb_sg_logger" # Weights&Biases Logger, see class super_gradients.common.sg_loggers.wandb_sg_logger.WandBSGLogger for details
  sg_logger_params: # Params that will be passed to __init__ of the logger super_gradients.common.sg_loggers.wandb_sg_logger.WandBSGLogger
    project_name: "project_name" # W&B project name
    save_checkpoints_remote: True
    save_tensorboard_remote: True
    save_logs_remote: True
    entity: "<YOUR-ENTITY-NAME>" # username or team name where you're sending runs
    api_server: "<OPTIONAL-WANDB-URL>" # Optional: In case your experiment tracking is not hosted at wandb servers

As you can see, there are two “parameters” – project_name and entity.

The hierarchy of WandB is: entity -> project_name -> experiment_name.

6. Launch your training and watch for the results in WandB.

SuperGradients + WandB = Your Power Multiplier

Harness the full power of SuperGradients by pairing it with Weights & Biases, creating a synergy that elevates your deep learning experience and streamlines your training process. 

SuperGradients provides a comprehensive training library equipped with scripts, sample code, and configurations to select and train your model on a dataset efficiently.

Meanwhile, Weights & Biases offers robust visualization and automation capabilities to monitor, track, and plot your training data.

When united, SuperGradients and Weights & Biases form an unbeatable combination, optimizing your deep learning workflow and driving results.

Don’t wait any longer—start exploring the potential of SuperGradients and Weights & Biases today!

You May Also Like

Top 10 List of Large Language Models Reshaping the Open-Source Arena

From Top-k to Beam Search: Everything You Need to Know About LLM Decoding Strategies

High Performance AI - Deci Gen AI Deve Platform and Deci Nano

Introducing Deci’s Gen AI Development Platform and Deci-Nano

The latest deep learning insights, tips, and best practices delivered to your inbox.

Share
Add Your Heading Text Here
				
					from transformers import AutoFeatureExtractor, AutoModelForImageClassification

extractor = AutoFeatureExtractor.from_pretrained("microsoft/resnet-50")

model = AutoModelForImageClassification.from_pretrained("microsoft/resnet-50")