Deploy and Serve
Your Deep Learning Models
in an Optimized Container-based Server

Deci’s Runtime Inference Container (RTiC)

RTiC, as a standard docker container, has its own file system plus dedicated inference server software and packages, all bundled together within the container. RTiC maximizes the utilization of the underlying hardware while enabling the inference of multiple models on the same hardware. You get to leverage best-of-breed open source optimization compilers, such as TensorRT and OpenVino. With RTiC, you can use standard container orchestration applications such as Kubernetes to deploy, manage, and scale microservices up or down.

Benefits:

• Profit from a highly efficient inference server that works on most common cloud environments, including GCP, AWS, and Azure, orchestrated using standard tools including Kubernetes, EKS, AKS, and GKS
• RTiC optimizes performance and resource utilization by leveraging best-of-breed graph compilers such as TensorRT and OpenVino, integrated into the container for easy use
• Standard API communication to/ from the models, including functions for inference requests, loading models (hot swap), and measuring or monitoring model performance in production