From state-of-the-art hardware to large language models, we’ve seen tremendous growth in the deep learning space this year. Fueling transformations across all industries, the breakthroughs in several tasks, including speech recognition and image recognition, are beginning to drive value for early adopters, and it shows no signs of stopping.
With the deep learning market expected to register a CAGR of 42.56% from 2020 to 2025, where is the industry headed next? What should practitioners and enthusiasts watch out for in 2023? Our Developer Advocate, Harpreet Sahota, asked his favorite deep learning experts their predictions for 2023. Here’s what they have to say.
1. Tristan Behrens
Dr. Tristan Behrens is an AI music composer and accredited expert in deep learning at AI Guild. He’s one of the most creative people in deep learning, using his knowledge and skills to create amazing audio and visual art.
“In 2022, we saw that the container of Generative Deep Learning was wide open. In 2023, we will see that has not been a Pandora’s box, which is a good thing, but that it also poses as a challenge to the status quo and a strong disruptor when it comes to human productivity and creativity.”
2. Glenn Jocher
Glenn is a pioneer in the field of antineutrino analysis, having led the US National Geospatial-Intelligence Agency’s efforts and published the world’s first Global Antineutrino Map in Nature. He’s on a mission to build the world’s best vision AI and work on the future of AGI.
AI advances will continue in 2023 along the same trendlines as 2022. Architecture and loss function updates will produce the most performant YOLO models ever in 2023. New hardware arrivals and quantization and pruning methods will streamline deployment on an ever larger array of higher TOPs edge devices, and more real-world use cases will emerge with AI as a force for good.
3. Tim Scarfe
Dr. Tim Scarfe is an expert in machine learning and AI. He is the CEO of the Machine Learning Street Talk YouTube channel and podcast, and is also the CTO of XRAI.
The past five years have witnessed a surge in the deployment of pointillistic models using traditional ML devops. However, this trend is set to be replaced by zero-shot, multi-task inference on massive monolithic models hosted in the cloud by tech giants. Consequently, the average machine learning practitioner will no longer be responsible for training and deploying their own models.
4. Aladdin Persson
Aladdin Persson is a machine learning engineer. He believes AI will transform our societies in a really impactful way. Through his YouTube channel and GitHub, he shares with other people that new things he learns about ML and deep learning.
We’ve seen a lot of impressive applications and progress such as Alphatensor, stable diffusion, chatGPT, whisper, but fundamentally there hasn’t been huge architectural changes for a long time. Transformers have been dominant for the last five years and I wonder if we will see a change in this, and my guess is that it will be in the graph domain.
I also think about combining various input sources, and finding clever models that use both text, audio, video for example.
5. Andreas Welsch
Andreas Welsch shares his expertise of the last 6 years working in the AI space and the last 21 years in the IT industry. At SAP, he has piloted AI with customers, built and led an AI Center of Excellence, and defined the AI go-to-market strategy.
We’ll see Generative AI systems negotiating a business transaction with each other on their users’ behalf.
6. Jon Krohn
Jon Krohn is Co-Founder and Chief Data Scientist at Nebula.io. In addition to his work at Nebula.io, he’s the author Deep Learning Illustrated, host of SuperDataScience, presenter of ML tutorials on Udemy, and adviser of tech startup boards.
Leveraging “foundational” Large Language Models (LLMs) with orders of magnitude more parameters than previous models, 2022 was the most extraordinary year for A.I. ever. Specifically, we witnessed a “big bang” in the emergence of generative models with staggering, expert human-level creative capacity on tasks as diverse as artwork (e.g., DALL-E 2), long-form conversational text (e.g., ChatGPT), and language-based gameplay negotiation (i.e., CICERO). In 2023, with the release of next-generation LLMs like GPT-4, expect existing human-level generative capabilities to become markedly more refined and new human-level generative capacities to emerge (e.g., w.r.t. video).
7. Miguel Fierro
Miguel Fierro leads the personalization team at Microsoft. He is an expert on recommender systems and works on NLP, computer vision, and other ML solutions. In addition, Miguel helps people and companies understand and apply AI.
Who has the decision power in a company? the roles that generate revenue, and who generate revenue? the business roles. Even at tech companies like Google, Microsoft, or Meta, whose leaders are technical, they are business people. One big problem outside these tech companies is that business people don’t understand AI. They know that if they don’t use AI in their companies, they are going to be left behind (they know what happened 20 years ago with those that didn’t understand the internet). A person who can teach AI in simple terms to business people will create a lot of value.
8. Jazmia Henry
Jazmia Henry is a Senior Applied AI Engineer at Microsoft. Through her growing community, she shares tips about data science and machine learning, strategies to make complex problems more simple, evangelizing about data modelling and parallelism, and more.
Deep Learning and IoT become friends. I would posit that the greatest advancements in Deep Learning this year were language models: BLOOM, ChaptGPT, and Dalle-2/Stable Diffusion. These advancements required a LOT of compute to handle billions of parameters. Once trained, these models were deployed to websites for people to play with but not integrated into advanced systems that had the ability to make decisions, improve day to day processes, or do anything physically actionable. Deploying such models on IoT edge for IoT integration could have outsized effects. Imagine some chatbot trained using BLOOM that is integrated into some autonomous system using IoT? Imagine you being able to play around and make art images with Stable Diffusion on your phone outside of a webpage? Imagine ChatGPT writing scripts on a tablet or checking code written in an open source IoT device for errors on the go?
That’s high level impact. This may not happen in 2023, however, the possibilities of taking Deep Learning models and turning them into autonomous systems is pretty badass.
9. Reem Mahmoud
Reem Mahmoud builds data and ML solutions at an HR tech startup, redefining and revolutionizing the hiring process. She loves to share what she learns about the ML and data space and how she gets the solutions she creates of the ground and up and running.
December 2022 saw a rush of excitement with the launch of ChatGPT. A few days later, the tone shifted with the same enthusiastic people starting to notice the limitations, flaws, and risks of ChatGPT.
Generative deep models brought and will continue to bring incredible advancements to the AI arena.
2023 will move us towards a better understanding of such models and their alleged consequences – both positive and negative. We will see more efforts to answer questions on WHY and HOW should these models be built and by WHOM. We will see a massive interest in investigating, setting, and enforcing governance of such models.
10. Brandeis Marshall
Brandeis Marshall is a data equity strategist. She is the Founder and CEO of DataedX Group, a social impact business that provides educational resources and training in recognizing algorithmic harms and humanizing data for data educators, scholars, and practitioners.
The tech community will continue to be addicted to releasing DL-laced products and services. But the pushback on these products to be responsible, worthy of people’s trust and data conscience will be faster and louder in 2023. Performative internal ethical review of a new tech product or service is hitting its end. I’m excited to see external third-party data/AI/tech ethics companies start to replace ineffective internal ethics review teams. Also I’m excited to witness the surge of public interest in tech adopting my phrase of 2023: “move slower and build better (humanity-driven) tech.
11. Santona Tuli
Santona Tuli loves to building end-to-end data science pipelines: from architecting, testing, and productionizing statistical data analyses to informing good decision making. She is an expert in strategic data featurization and explainable ML and result validation.
The state of generative models, in the form of large language and computer vision models, today is extremely impressive. But they are not yet “useful”. I hope we find applications for them, but I also hope that we continue to push the research. We didn’t get these models in a day, and in the pursuit of monetization we may just sideline the scientific curiosity that is necessary for such innovation. I hope we don’t.
12. Kristen Kehrer
Kristen Kehner is a Developer Advocate at Comet. She creates content about machine learning projects, how to create effective data presentations, data science trends, and products for data professionals and decision makers.
2023 will be an exciting year for deep learning. As someone who has built a number of practical object detection projects without formal computer vision training, what strikes me is how much more accessible this field is becoming.
In addition, big new advancements in AI are transcending the data community and I’m seeing people use the term ‘machine learning’. Earlier this year I was seeing art subreddits discussing the ethics of AI generated art and the popularity of ChatGPT extended far beyond just the data community within a day or two of launch.
But my favorite AI conversation this year was with a group of 8 year olds. I gave a presentation on ‘What is AI?’ and couldn’t believe their ability to come up with interesting use cases of AI that they see in their lives that were not explicitly covered.
In 2023 I hope to share with more children about how computers use historical data of all different formats to make a decision and champion the idea to my technical friends that they do not need a formal background in deep learning to apply it to their own personal projects at home. Let’s build AI literacy for all.
13. Yonatan Geifman, PhD
After completing his PhD at the Technion, Yonatan Geifman co-founded Deci. Its mission is to enable deep learning models to run efficiently on any hardware in real-time, solving one of the most challenging problems in AI.
As AI inference keeps on increasing, enterprises must pay more attention to how they can properly productize and scale AI without hurting their bottom line. This means figuring out strategies on how to best develop, deploy, and scale AI cost efficiently and in shorter time frames.
More Value from Deep Learning in 2023
Deep learning offers unique and powerful ways to address many of the challenges we’re facing today. Its wide range of use cases provide opportunities especially for organizations looking to drive cost-efficiency.
But to truly reap the benefits and unlock long-term value, there needs to be a careful orchestration of the various factors that impact its successful implementation. In addition to the predictions mentioned, this year, we’re seeing not just better streamlining among the hardware, software, and algorithms used to make deep learning models work, but also novel approaches and adoption of development tools to ensure more successful and continuous deployments in any production environment – from the cloud to the edge.
