Algorithms

13 Free Resources and Model Zoos for Deep Learning and Computer Vision Models

Graphic depicting different neural networks across colorful background

Computer vision is a fast-growing subfield of AI and deep learning. From cashierless stores in retail to crop detection in agriculture, there’s an increasing interest in CV applications. This has created a vibrant community that gladly shares architectures, codes, pre-trained models, and even tips for every stage of the development cycle.

Starting a CV project from scratch takes time. So, the usual process is, given a problem or a use case, you look for models that partially solve it. If one already exists, then training a new model is no longer necessary. Otherwise, you use the publicly available models to guide you through which architecture to choose as a base for fine-tuning or transfer learning.

 In this post, we list 13 free resources and model zoos for deep learning and computer vision models that can hopefully help to simplify your work.

1. Papers with Code

Papers With Code is a community-led open resource that puts together machine learning papers, codes, datasets, methods, and evaluation tables. While the core team of the project is based in Meta AI Research, everyone across academia and industry can contribute.

a screenshot of papers with code

Source: Papers with Code

You can filter the database and browse state-of-the-art models, datasets, tasks, methods, and trending research. Moreover, you can see popular benchmarks, submissions, and developments made on each task and dataset.

a screenshot of papers with code

Source: Papers with Code

In addition to computer vision, the resource is updated regularly with the latest papers and implementations in other subfields of AI such as natural language processing (NLP), reinforcement learning, audio, and more. There are also portals for other disciplines including physics, astronomy, and statistics.

2. Open Model Zoo 

OpenVINO™ Toolkit’s Open Model Zoo is a repository of pre-trained deep learning models and demos that developers can use to speed-up the development and production deployment process.

 The models in the resource can be divided into two categories: Intel pre-trained models and public pre-trained models. In total, there are over 200 models in the repository covering a variety of tasks such as object detection, place recognition, text to speech, colorization, and more. Users can also leverage dozens of demo application templates.

a screenshot of openvino toolkit

Source: OpenVINO Toolkit GitHub

To make it fast and easy, the community can use automation tools to download and convert the models into the OpenVINO™ IR format. There are tools for quantizing models, printing information, and copying datasets from an installed location. Open Model Zoo also provides an accuracy checker to validate model accuracy.

3. Deci’s Hardware Aware Model Zoo

Deci is an end-to-end deep learning development platform that enables data scientists and deep learning engineers to build, optimize, and deploy models to any environment and hardware. It includes a model zoo with PyTorch-based computer vision models for all the main tasks. Unlike other model zoos, the models in Deci’s zoo are pre-trained and optimized (compiled and quantized) for a wide range of hardware types.

In addition to displaying the accuracy per model, you can easily select a target inference hardware and see the model’s runtime performance results such as throughput, latency, model size, and memory footprint for various hardware.

This hardware aware model zoo is designed to help select the best deep neural network (DNN) for your task and ensure that it will run well on your inference hardware, even before you start training it. Having this information in advance can help you not only to save time and resources for model training, but also significantly reduce the friction of deploying the model once it’s ready.

a gif of deci model zoo

The models’ accuracy and runtime performance results are plotted on an “Efficient Frontier” graph which enables you to easily see the expected “accuracy-latency” trade-off on the selected hardware. You can also click each model and compare it to others.

 You can download the models for free or deploy with Infery, a Python runtime engine, when you sign up to Deci’s deep learning platform.

Currently, we support a range of Intel CPUs (Cascade Lake, Skylake, Broadwell, and Intel NUC Tiger Lake) and NVIDIA GPUs (K80, V100, T4, Jetson Xavier, Jetson Nano, and Jetson Xavier AGX).

TRY DECI’S MODEL ZOO

4. Jetson Model Zoo and Community Projects 

The Jetson Zoo is an open-source catalog containing guidelines for installing add-on packages and frameworks on NVIDIA Jetson. It also includes DDN models for inferencing on Jetson with support for TensorRT.

a screenshot of jetson model zoo

Source: eLinux.org

The Model Zoo includes classification, object detection, segmentation, and pose estimation models, code samples, and links to the original sources. You can also see resources for performance benchmarks.

a screenshot of nvidia community projects

Source: NVIDIA

In addition, NVIDIA features community projects created for Jetson developer kits. These projects contain codes, videos, and instructions, which you can use for inspiration or modify according to your own needs.

5. Model Zoo

Model Zoo curates pre-trained deep learning models for a variety of frameworks and domains. Built by Jing Yu Koh, who is now a machine learning researcher at Google, the models are well-tagged, making it easy for researchers and practitioners to filter and find models that suit what they’re looking for.  

a screenshot of model zoo

Source: ModelZoo 

Some of the frameworks include TensorFlow, PyTorch, Keras, Caffe, and more. You can browse computer vision, NLP, reinforcement learning, unsupervised learning, audio and speech, and generative models. In addition, each model or library has in-depth implementation details.

6. TensorFlow Hub

TensorFlow is a popular library for training and inference of deep neural networks. With the TensorFlow Hub, deep learning practitioners can simply download and reuse assets for machine learning.

a screenshot of tensorflow hub

Source: TensorFlow

The pre-trained SavedModels are ready for solving new tasks with less training time and data. Downloading them involves detailed documentation and minimal coding.

7. TensorFlow Model Garden

Maintained by the TensorFlow team, the Model Garden contains a variety of different implementations of SOTA models, example implementations, and best practices for modeling.

The repository features four main directories:

  • Official is the collection of SOTA model implementations using the latest TensorFlow 2 APIs.
  • Research contains model implementations in TensorFlow 1 or 2 that are in the research phase. 
  • Community includes a curated list of TensorFlow 2-powered GitHub repositories.
  • Orbit is a library that enables users to write customized training loop code in TensorFlow 2.
a screenshot of tensorflow model garden

Source: TensorFlow GitHub

Unlike TensorFlow Hub, the Model Garden provides a solid foundation of configurations and datasets for model training or fine-tuning.

8. MediaPipe Models 

MediaPipe is an open-source cross-platform that offers customizable machine learning solutions for live and streaming media. Some solutions include instant motion tracking, face mesh, KNIFT, and selfie segmentation. While still in alpha, products such as Google Nest and Ariel AI (acquired by Snap) are already using MediaPipe solutions.

a screenshot of mediapipe

Source: MediaPipe

On its GitHub repository, developers can see how to implement the models in Python, JavaScript, C++, Android, and iOS. Each model also has a card that includes all details regarding its specifications, applications, limitations, and ethical considerations.

9. Awesome TensorFlow Lite

TensorFlow Lite is specifically built for running inference on devices with limited compute power such as phones and embedded devices.

a screenshot of magenta github

Source: Magenta GitHub

Awesome TensorFlow Lite is a comprehensive repository for TensorFlow Lite. It includes models with samples ranging from computer vision to recommendation systems. It also gathers resources for tutorial projects, plugins and SDKs, and other learning resources such as books, podcasts, and videos, which can be helpful for beginners.

10. Awesome CoreML Models

Core ML is a framework released by Apple to enable developers to integrate machine learning into applications. With Awesome CoreML Models, you get the largest list of models in Core ML (iOS 11+) that you can use to experiment and implement on iOS, macOS, tvOS, and watchOS.

a screenshot of awesome coreml github

Source: Awesome CoreML GitHub

In addition to the different models that process image and text, the repository also provides users with visualization tools. Supported model formats that are convertible to Core ML with examples include Caffe, Keras, XGBoost, Scikit-learn, and more.

11. Pinto Model Zoo

Created by Katsuya Hyodo, the Pinto Model Zoo is a repository that shares tuning results of trained models generated by TensorFlow or Keras. To date, it contains over 260 optimized models. Post-training quantization techniques used include weight quantization, integer quantization, full integer quantization, Float16 quantization), and quantization-aware training.

a screenshot of pinto model zoo

Source: Pinto Model Zoo GitHub

The models are also available for TensorFlow Lite, OpenVINO, CoreML, TensorFlow.js, TF-TRT, MediaPipe, and ONNX format. Moreover, you can see various sample implementations of the trained models with specifications, codes, and video files.

12. ONNX Model Zoo

Open Neural Network Exchange (ONNX) is an open-source format for representing machine learning models. With the ONNX Model Zoo, you can explore and use pre-trained, SOTA models in ONNX that the community contributed. There are models for various vision, language, and other interesting tasks.

a screenshot of onnx model zoo

Source: ONNX GitHub

Each model has a corresponding Jupyter notebook for model training and running inference. Written in Python, the notebooks contain links to the datasets and original papers. There are also instructions for using Git LFS command line when downloading multiple ONNX models, as well as for configuring starter Python code when validating your ONNX model with test data.

Bonus:

13. CatalyzeX 

CatalyzeX is a small browser extension that allows you to get implementation code for machine learning research papers directly on Google, ArXiv, GitHub, Twitter, and other websites.

a screenshot of catalyzex on twitter

Source: CatalyzeX

As you browse online, the extension provides you with an in-line link next to papers in the fields of AI, NLP, computer vision, deep learning, and reinforcement learning. On the CatalyzeX website, you can also see a collection of models, code, and papers for popular deep learning tasks.

Conclusion

Beyond providing the building blocks for deep learning projects, the vibrant, open-source community is instrumental in the continuous progress and increasing accessibility of AI. Take advantage of these free resources and deep learning model zoos so you can speed up your development and production deployment process.

If the Deci hardware-aware model zoo sparked your interest, you should request free trial to the Deci Platform today.

You May Also Like

Top 10 List of Large Language Models Reshaping the Open-Source Arena

From Top-k to Beam Search: Everything You Need to Know About LLM Decoding Strategies

High Performance AI - Deci Gen AI Deve Platform and Deci Nano

Introducing Deci’s Gen AI Development Platform and Deci-Nano

The latest deep learning insights, tips, and best practices delivered to your inbox.

Share
Add Your Heading Text Here
				
					from transformers import AutoFeatureExtractor, AutoModelForImageClassification

extractor = AutoFeatureExtractor.from_pretrained("microsoft/resnet-50")

model = AutoModelForImageClassification.from_pretrained("microsoft/resnet-50")