Author: Scott Bair, who is a key voice at Intel Labs, sharing insights into innovative research for inventing tomorrow’s technology.
A collaboration between Intel Labs and Deci, a start-up company focused on deep learning development, is accelerating the journey toward more scalable AI. By combining Deci’s algorithmic acceleration technology with Intel chip architectures, the two companies have successfully—and dramatically—optimized deep learning inferencing. Thanks in no small part to this collaboration, deep learning can now be used to build, optimize, and deploy more accurate and efficient models for cloud environments.
Intel’s journey with Deci began in 2019 when the two joined forces to optimize deep learning inference on Intel Architecture (IA) CPUs. A year later in 2020, Deci submitted inference results achieved with its proprietary neural architecture search engine called AutoNAC (Automated Neural Architecture Construction) and 2nd Gen Intel® Xeon® Scalable processors to the open division of the MLPerf v0.7 benchmark suite. The submission used the Intel® Distribution of the OpenVINO™toolkit to cut the inference latency and boost the throughput of the ResNet-50 deep neural network model. The results showed a whopping 11.8x reduction in latency and an 11x increase in throughput, all while preserving the model’s accuracy within 1%. (Additional details about the submission can be found on Intel Communities Blog.)
“This was a major win for hardware-aware acceleration of computer vision models with neural architecture search and 8–bit quantization running on Intel hardware,” said Guy Boudoukh, a deep learning researcher within Intel’s AI Lab. Guy is also a mentor to Deci under Intel® Ignite, a global corporate program for early-stage start-ups, launched in 2019 by Intel.
Deci was one of the first companies to join Intel Ignite, a program designed to help turn the cutting-edge ideas of ambitious entrepreneurs into industry-disrupting technologies. The program is focused on deep tech innovations. Working with leading mentors and industry experts, each Ignite start-up receives a tailor-made program to meet its specific business needs. The program covers a wide range of fields, including founders’ dynamics, go-to-market strategy, marketing and sales, product development and management, recruitment, and more. DECI is a batch 1 alumni of the Intel Ignite Tel Aviv program.
Deci and Intel Boost Computer Vision (CV) and Natural Language Processing (NLP) Models
Over the last two years, the Deci/Intel collaboration has continued, resulting in another compelling MLPerf submission, this time for both Computer Vision (CV) and Natural Language Processing (NLP) inference models.
The CV submission accelerated ResNet50 with Deci’s AutoNAC to generate DeciNets deep neural networks. The optimized DeciNets models were submitted in the ResNet50 category in an offline scenario on two different hardware platforms: a 12-core 2nd Gen Intel® Xeon® processor and two 3rd Gen Intel Xeon Scalable processors with different configurations of 4 and 32-cores. The models were optimized on a batch size of 32 and quantized to INT8 using OpenVINO. Compared to the 8-bit ResNet50 model, Deci delivered a+1.74% improvement in accuracy and a 4x improvement in throughput. The submission demonstrated a 37% improvement in throughput performance, as well as a notable improvement in accuracy, compared to their previous submission in 2020.
The NLP submission accelerated the Bert-Large model by implementing Deci’s Automated Neural Architecture Construction (AutoNAC) to generate the Intel-optimized DeciBERT models. The Deci Bert models were quantized with OpenVINO to 8-bit precision. Benchmarked on 32-core 3rd Gen Intel Xeon Scalable processors and 16-core 2nd Gen Intel Xeon C processors, the .results demonstrated up to a 5x acceleration in throughput performance and +1.03 improvement in accuracy.
“These recent MLPerf submissions mark another significant milestone in our collaboration with Intel to deliver efficient deep learning inference on Intel CPUs. The major increase delivered both in accuracy and throughput directly translates into a larger number of the resource-intensive tasks and applications that can now be carried out on Intel CPUs,” said Yonatan Geifman, Deci CEO and Co-founder.
More details can be found about the NLP and CV submissions in these articles:
- Intel and Deci Boost NLP Models at MLPerf — Reaching Faster and More Accurate Inference Performance
- How Deci and Intel Achieved up to 16.8x throughput increase and +1.74% accuracy improvement at MLPerf
The Intel-Deci collaboration continues. Just recently Deci joined the Intel Disruptor Program and currently, Deci is working with Intel to demonstrate AutoNAC technology performance on 4thGen Intel Scalable processors, codenamed Sapphire Rapids.
