Deep Learning Inference at Scale with AI Readiness

Deep Learning Inference at Scale with AI Readiness

Deep Learning Inference at Scale with AI Readiness

2 CONTENTS Introduction: Deep learning is coming of age ............ 2 Determining AI readiness ...................................................... 4 Developing and deploying data governance and security policies .......................................................................... 5 Infrastructure strategies for the shift to deep learning inference at scale ...................................................... 6 The magnifying impact of optimized software ........... 9 Next steps: Breaking barriers between model and reality ................................................................................... 11 Further reading ........................................................................ 12 Skip to discover the benefits of the latest generation Intel® Xeon® Scalable processors and Intel® Deep Learning Boost 1. Deep learning is coming of age… and it’s happening fast By 2020, deep learning will have reached a fundamentally different stage of maturity. Deployment and adoption will no longer be confined to experimentation, becoming a core part of day-to-day business operations across most fields of research and industries. Why? Because advancements in the speed and accuracy of the hardware and software that underpin deep learning workloads have made it both viable and cost-effective. Much of this added value will be generated by deep learning inference – that is, using a model to infer something about data it has never seen before. Models can be deployed in the cloud or data center, but more and more we will see them on end devices like cameras and phones. Intel predicts that there will be a shift in the ratio between cycles of inference and training from 1:1 in the early days of deep learning, to well over 5:1 by 2020¹. Intel calls this the shift to ‘inference at scale’ and, with inference also taking up almost 80 percent of artificial intelligence (AI) workflows (Figure 1, Page 3), it follows that the path to true AI readiness starts with selecting hardware architectures that are well-suited to this task. However, as the AI space is becoming increasingly complex, a one-size-fits-all solution cannot address the unique constraints of each environment across the AI spectrum. In this context, critical hardware considerations include availability, ease of use, and operational expense. What type of infrastructure do you use for your edge devices, workstations or servers today? Do you want to deal with the complexities of multiple architectures? Exploring these challenges is the subject of this paper. Figure 1: A typical AI workflow label data 15% load data 15% Labor-intensive augment data 23% Experiment with topologies 15% tune hyper- parameters 15% Compute-intensive (Model Training) support inference 8% share results 8% Labor-intensive Source Data Development Cycle 15% 15% 23% 15% 15% 8% 8% Inference Inference within broader application 3 1. Opportunity 2. Hypotheses 3. Data Prep 4. Modeling 5. Deployment 6. Iteration Time To Solution (TTS) 7. Evaluation 2. Determining AI readiness Understanding where your organization sits in terms of AI readiness is critical to prioritizing actions and smoothing the path from experimenting with AI to real- world deployments. Organizations can be grouped into three categories of AI readiness: foundational, operational or transformational, and progressing to the next stage or achieving ongoing success depends on having the right elements in place across skills and resources, infrastructure and technology, processes and models. Among the defining characteristics of operational and transformational AI-ready enterprises is – to varying degrees – their ability to support better decision making or automate business processes/responses with AI through inference at scale. At the foundational stage, enterprises should be prioritizing developing and deploying proof of concepts (PoCs) to establish and build the infrastructure, skills and executive buy-in required to scale with AI. These topics are dealt Read the full Deep Learning Inference at Scale with AI Readiness.

By 2020, deep learning will have reached fundamentally different stage of maturity. Deployment and adoption will no longer be confined to experimentation, becoming a core part of day-to-day business operations across most fields of research and industries.

However, as the AI space is becoming increasingly complex, a one-size-fits-all solution cannot address the unique constraints of each environment across the AI spectrum. In this context, critical hardware considerations include availability, ease of use, and operational expense. What type of infrastructure do you use for your edge devices, workstations or servers today? Do you want to deal with the complexities of multiple architectures?

Exploring these challenges is the subject of this guide.

Sections include:

Determining AI readiness

Developing and deploying data governance and security policies

Infrastructure strategies for the shift to deep learning inference at scale

The magnifying impact of optimized software

Next steps: Breaking barriers between model and reality

Benchmarks Validate CPUs for Deep Learning Inference

Intel® Xeon® Scalable processors can be an effective option for data scientists looking to run multiple workloads on their infrastructure without investing in dedicated hardware.

Learn more

AI Proof of Concept in Five Steps

A five-step approach to success with AI proof of concepts.

Learn more

Learn About Intel® Deep Learning Boost (Intel® DL Boost)

Intel® Xeon® Scalable processors take AI performance to the next level with Intel® Deep Learning Boost (Intel® DL Boost).

Learn more