Improve latency and throughput and reduce model size by up to 10x while maintaining your models’ accuracy.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
An automotive company running a U-Net based segmentation model on a NVIDIA Jetson Xavier NX struggled to achieve the target throughput in production.
Using Deci’s AutoNAC engine, a faster and smaller model was generated. Latency was reduced by 2.1X, model size was reduced by 3X, and memory footprint was reduced by 67% – all while maintaining the original accuracy.
A defense company developing electro-optics solutions for space, airborne, ground, and maritime applications was looking to improve the throughput of an image-denoising model for video stream analysis to deliver real-time insights on the edge. The team was also looking for a way to free up GPU resources to support additional parallel tasks on the same edge device.
Using the Deci platform and its NAS-based engine, the team built a new architecture that delivered a 1.58x acceleration of throughput while maintaining the original model’s accuracy. The team achieved the desired performance within 10 days. Once the team trained the model, they easily compiled and quantized it to TensorRT FP16 using Deci’s platform.
A defense company needed to process high resolution
images for an object detection and tracking task on an
NVIDIA Jetson Xavier NX device. In order for the system to
become operational, the customer they needed to run in a
10 watt mode and achieve a throughput of 10 frames per
second.
Using Deci’s AutoNAC engine, the customer was able to
increase throughput by 3.1X, and run smooth object
tracking, and unlock a new security application.
Watch a quick walkthrough of how you can use Deci to accelerate your models’ inference performance.
Vadim Zhuk, Senior Vice President
RingCentral
from transformers import AutoFeatureExtractor, AutoModelForImageClassification
extractor = AutoFeatureExtractor.from_pretrained("microsoft/resnet-50")
model = AutoModelForImageClassification.from_pretrained("microsoft/resnet-50")