What Are the Different Types of AI Workloads?

Key Takeaways

• AI workloads differ by data, latency, cost, accuracy, and infrastructure needs.
• Training builds models, but inference usually dominates production operations.
• RAG, agents, multimodal AI, and edge AI are changing capacity planning.

AI Workloads Are Now an Infrastructure Category

As of June 4, 2026, AI workloads no longer fit inside a single label such as “training” or “inference.” The practical categories now include data preparation, model training, fine-tuning, retrieval, real-time inference, batch inference, agentic workflows, multimodal processing, edge deployment, monitoring, and safety evaluation. Google Cloud distinguishes training, fine-tuning, inference, and serving as separate activities with different objectives and business priorities.

An AI workload is the unit of work required to build, adapt, run, test, or manage an artificial intelligence system. It may involve cleaning millions of records, training a model on graphics processing units, producing a response to a user question, searching a private document collection, generating images, classifying satellite imagery, ranking social media posts, translating speech, monitoring model drift, or evaluating output quality. The workload matters because the same AI model can behave like several different infrastructure products depending on how it is used.

The machine learning lifecycle provides a useful starting point. Amazon Web Services describes the lifecycle as a cyclic process with phases such as business goal identification, machine learning problem framing, data processing, model development, deployment, and monitoring. That structure shows why AI is more than one compute job. A production AI service includes the work that happens before model training, during model development, after deployment, and during the long operating life of the service.

The most useful classification separates AI workloads by their operating requirement. Some jobs need maximum throughput and can run overnight. Others need sub-second response times. Some need large clusters connected by high-speed networking. Others run on phones, cars, satellites, cameras, factory devices, or small servers near the user. Some workloads process text; others process images, video, audio, geospatial data, code, sensor streams, or scientific data.

The infrastructure impact now appears in energy planning, site selection, chip procurement, and data center finance. The International Energy Agency projects that global data center electricity consumption will roughly double to around 945 terawatt-hours by 2030 in its base case, with accelerated servers driven mainly by AI adoption projected to grow faster than conventional servers. That projection does not mean every AI workload has the same energy profile. It means workload mix now shapes data center design, grid planning, procurement, and capital allocation.

Space economy analysis reaches the same lesson through a different route. New Space Economy’s coverage of NVIDIA space computing notes that space-based compute can make sense for narrow, high-value, time-sensitive workloads before it makes sense for mainstream Earth-based demand. That distinction is useful for terrestrial AI planning as well. Workload type determines where compute belongs, how much latency matters, and what cost structure can be tolerated.

The main AI workload categories can be organized by the function they perform.

Data Preparation Workloads Shape AI Quality Before Models Run

Data preparation is the least glamorous AI workload and often one of the most expensive. It includes collecting data, removing duplicates, converting formats, labeling examples, resolving conflicting records, extracting text from documents, splitting documents into passages, converting images into features, validating metadata, and creating training or retrieval datasets. Weak data preparation can make a powerful model unreliable, expensive, or unusable in production.

Traditional machine learning often depends on structured data such as rows in a database. Generative AI expands the data problem to documents, software code, emails, chat transcripts, diagrams, audio, video, satellite imagery, medical images, engineering drawings, and sensor readings. Each data type creates a different workload. Text needs tokenization and document chunking. Images need annotation and quality screening. Video needs frame extraction, object tracking, and temporal alignment. Audio needs transcription, speaker separation, and noise handling.

The AWS Machine Learning Lens separates general machine learning workloads from generative AI workloads because generative systems add foundation model selection, prompt engineering, retrieval-augmented generation architectures, and governance challenges. That distinction matters because generative AI often needs two data layers: data used to train or adapt the model, and data retrieved at run time to ground responses.

Labeling workloads vary by domain. A customer-service classifier might need thousands of labeled examples. A medical imaging model may need expert annotation from clinicians. A satellite-image model may need geospatial labels tied to coordinates and timestamps. A fraud-detection model may need historical transaction outcomes, privacy controls, and careful sampling to avoid teaching the system from outdated behavior. Each labeling process adds labor, review, dispute resolution, and audit requirements.

Data cleaning also has commercial value. Many organizations discover that their first AI workload is data remediation rather than model deployment. Customer records may be inconsistent, product catalogs may contain duplicate names, scanned documents may contain optical character recognition errors, and internal knowledge bases may mix current policy with obsolete material. Without cleanup, downstream AI workloads can produce confident responses from stale or conflicting inputs.

Synthetic data is another category. It refers to artificial data created to supplement or replace real data for training, testing, simulation, or privacy protection. Synthetic data can help when real examples are scarce, sensitive, or unsafe to collect. It can also introduce hidden bias if the generator reflects the same gaps found in the original data. The workload includes generation, validation, filtering, and testing against real-world performance.

Space and defense markets show why data preparation is a workload rather than an administrative step. Earth observation systems create large volumes of imagery, radar returns, spectral measurements, and metadata. New Space Economy’s coverage of the space data economy describes a shift from raw data delivery toward analytics and decision support. That shift depends on AI workloads that turn raw inputs into classified objects, detected changes, forecasts, alerts, and usable products.

Training and Fine-Tuning Workloads Build and Adapt Models

Training is the process of building model capability from data. In classical machine learning, training might mean teaching a model to predict equipment failure, detect fraud, recommend products, or classify images. In deep learning, training can involve large neural networks with many layers. In foundation models, training can involve huge datasets, specialized accelerator clusters, high-speed networking, and long-running jobs that consume large amounts of electricity and engineering effort.

Training workloads differ by model size, data volume, target accuracy, and tolerance for delay. A small forecasting model may train on a standard server. A computer vision model may require graphics processing units. A frontier language model may require thousands of accelerators connected through high-bandwidth networking. The workload includes data loading, model computation, checkpointing, error recovery, experiment tracking, and evaluation.

Pretraining is the broadest form of training for modern foundation models. It teaches a model general patterns from large datasets before the model is adapted for a specific product or domain. Pretraining is expensive because it processes enormous volumes of text, code, images, audio, video, or mixed data. It is usually performed by companies, research organizations, or state-backed programs with access to large compute budgets.

Fine-tuning is narrower. It adapts an existing model to a specific task, style, domain, or output format. Microsoft’s Foundry fine-tuning documentation states that fine-tuning can improve task results, permit more training examples than a prompt can contain, reduce prompt length, and lower request latency in some cases. Microsoft also describes low-rank adaptation as a way to fine-tune a smaller set of parameters, which can make adaptation more manageable than full model training.

Post-training is a broader term that includes supervised fine-tuning, preference tuning, safety tuning, tool-use training, and policy alignment. A base model may know language patterns but still need work to follow instructions, refuse unsafe requests, use tools, cite documents, operate in a company’s tone, or format answers correctly. These post-training workloads often depend more on data curation and evaluation than on raw compute scale.

Reinforcement learning workloads teach systems from rewards, preferences, simulations, or feedback. They are common in robotics, game playing, control systems, and some language model alignment workflows. The compute profile can differ from standard supervised training because the system may run repeated trials, score the results, update policy behavior, and test again. Simulation can become the expensive part, especially in robotics, autonomous systems, and industrial control.

Evaluation workloads sit beside training. Model developers need tests for accuracy, bias, security, latency, refusal behavior, factuality, reasoning, multilingual capability, and domain performance. Evaluation can be cheap for a small classifier, but expensive for a general-purpose model with many use cases. It can require human review, automated testing, red-team testing, benchmark runs, and production sampling.

Training has attracted much of the public attention because it creates frontier models. Production economics often shift attention to post-training and inference. A company may train rarely, fine-tune occasionally, and run inference constantly. That distinction explains why hardware and cloud buyers increasingly ask which workloads will dominate their spending over the operating life of a system.

Inference Workloads Turn Models Into Products

Inference is where a trained model processes new inputs and produces outputs. Google Cloud describes AI inference as the execution phase that uses a trained or fine-tuned model to make predictions on new data. Google also separates inference from serving, which involves deploying and managing the model endpoint that handles requests.

Inference can look small at the level of one request. A user asks a question, a model returns an answer. A camera sends an image, a model detects a vehicle. A satellite processes a radar scene, a model identifies possible changes. The economic challenge appears when the same system must handle millions or billions of requests with predictable latency, uptime, security, and cost control.

Real-time inference serves users or machines that expect an immediate response. Chatbots, translation systems, voice assistants, fraud scoring, ad ranking, industrial controls, and autonomous navigation all depend on latency. A delay of several seconds may be acceptable for an internal drafting tool. It may be unacceptable for high-frequency fraud detection or robotic motion planning. Real-time inference pushes organizations toward optimized serving software, model compression, caching, autoscaling, and careful routing.

Batch inference processes large groups of inputs without an immediate user waiting for each result. Customer segmentation, document classification, product catalog enrichment, invoice extraction, offline image analysis, and nightly recommendation updates can often run as batch workloads. OpenAI’s Batch API is an example of an asynchronous approach for jobs that do not need immediate responses, with a separate processing pool and lower cost for eligible use cases.

Streaming inference processes continuous flows. Examples include live video analytics, speech recognition, industrial sensor monitoring, network intrusion detection, and vehicle perception. Streaming workloads add a timing problem. The system needs to process data as it arrives, avoid backlog, and preserve sequence. It may also need to combine fast local decisions with slower central analysis.

Serving infrastructure has its own workload profile. NVIDIA Triton Inference Server supports AI inference across cloud, data center, edge, and embedded devices, and it can deploy models from multiple frameworks. That kind of serving layer matters because inference workload management includes model loading, batching, versioning, routing, resource allocation, scaling, and monitoring.

Cost per inference has become a central planning measure. A model that is impressive in a demo can be too expensive for a high-volume service. Smaller models, quantization, distillation, caching, speculative decoding, retrieval design, prompt reduction, and hardware selection can cut cost without destroying value. The best architecture often uses different models for different tasks: a small model for routine classification, a larger model for difficult reasoning, and a retrieval system to supply current facts.

New Space Economy’s article on the AI bubble debate makes the same point from the market side. AI infrastructure requires spending on chips, data centers, electricity, cooling, and networking. Those assets need revenue-producing workloads, not just technical excitement. Inference is where many AI systems either become useful products or expose weak unit economics.

RAG and Agentic Workloads Add Search, Memory, and Tool Use

Retrieval-augmented generation (RAG) combines a generative model with an information retrieval system. Instead of relying only on the model’s internal training, a RAG system searches documents, databases, or knowledge stores and supplies relevant material to the model at run time. That design helps when answers must reflect current, private, or specialized information.

RAG is not one workload. It includes document ingestion, chunking, embedding, vector indexing, keyword indexing, hybrid search, reranking, prompt assembly, generation, citation handling, access control, and monitoring. Microsoft’s Azure AI Search RAG documentation describes both classic RAG and agentic retrieval. It also identifies content preparation tasks such as chunking, optical character recognition, image analysis, vectorization, synonym handling, and semantic ranking.

Embedding workloads deserve separate attention. An embedding model converts text, images, code, audio, or other content into numerical representations that support similarity search. The workload may run once for a static archive or continuously for changing documents. The index must handle updates, deletions, access permissions, version control, and quality checks. Poor embedding design can create slow search, irrelevant retrieval, or accidental exposure of restricted information.

Agentic AI adds orchestration. An agentic system can break a task into steps, search for information, call tools, write or run code, ask another model to check the answer, update a plan, and repeat. NVIDIA’s 2026 discussion of AI factories describes agentic workloads as always-on inference that can reason, plan, search, use tools, retrieve data, write code, and take action. NVIDIA also states that multi-agent systems make AI workloads longer, deeper, and more compute-intensive.

Agentic workloads are hard to estimate because the number of model calls can change at run time. A simple prompt may trigger one model response. A research agent may trigger searches, document reads, code execution, tool calls, model self-checks, and follow-up calls. Cost depends on task length, retrieval breadth, model size, tool latency, and stopping rules. Without controls, an agent can spend more compute than the value of the task justifies.

Memory workloads sit between RAG and agents. A system may remember user preferences, case history, previous decisions, or workflow state. Memory can improve continuity, but it adds privacy, retention, consent, and deletion requirements. The technical workload includes deciding what to store, how long to keep it, how to retrieve it, and how to prevent irrelevant memory from contaminating current work.

Security workloads become more demanding with RAG and agents. A system that can retrieve private data or use tools needs permission enforcement, audit logs, prompt injection defenses, data loss prevention, and safe execution environments. A standard chatbot may only produce text. An agent tied to email, code repositories, calendars, customer systems, or procurement tools can create operational risk if its workload design lacks boundaries.

Space-based AI gives a practical example of why RAG and agents will not always run in one place. A satellite may process imagery on board, summarize important changes, transmit a smaller result, and allow a ground system to perform heavier retrieval or human review. New Space Economy’s article on orbital data center failure modes emphasizes that moving compute into orbit changes risks involving heat, radiation, networking, autonomy, and debris. Workload placement becomes a design decision, not just a cost preference.

Multimodal and Physical AI Workloads Expand the Data Types

Multimodal AI handles more than one input or output type. A multimodal system may process text and images, text and audio, video and sensor data, or documents containing diagrams, tables, signatures, and scanned text. Multimodal workloads are often heavier than text-only workloads because they process larger files, use specialized encoders, and may need multi-stage pipelines.

Computer vision workloads classify images, detect objects, segment scenes, recognize defects, read text from images, or track movement in video. They are common in manufacturing, retail, logistics, health care, agriculture, autonomous vehicles, insurance, and defense and security. Some computer vision jobs can run at the edge on cameras or embedded devices. Others need central processing when the images are large, the model is complex, or the analysis can wait.

Video AI is especially demanding. A video is a time-ordered sequence of images, often paired with audio and metadata. Video workloads may require frame sampling, object tracking, activity recognition, event detection, summarization, and storage management. A real-time safety system in a factory has different latency and reliability needs from a media archive that summarizes recordings overnight.

Speech and audio workloads include speech-to-text transcription, speaker identification, translation, call-center scoring, meeting summarization, acoustic anomaly detection, and voice interfaces. Audio workloads are sensitive to noise, accents, microphones, latency, and privacy. A call-center assistant may require real-time transcription and response suggestions; a legal archive may need slower but more accurate batch transcription.

Code workloads have become a large category. They include code completion, bug detection, automated testing, documentation generation, migration support, software agent workflows, and security scanning. Code generation changes infrastructure demand because it can involve long context windows, repository retrieval, test execution, and repeated model calls. The workload may combine language modeling, search, program analysis, and sandboxed execution.

Physical AI connects models to robots, vehicles, drones, sensors, industrial equipment, or scientific instruments. The workload may include perception, planning, control, simulation, reinforcement learning, digital twins, and safety monitoring. Physical AI workloads often need real-time decisions, local fallback behavior, and extensive testing before deployment. Failure costs can be higher because the model affects physical equipment rather than only a screen response.

Geospatial AI is an important subset for the space economy. Earth observation systems use optical imagery, synthetic aperture radar, hyperspectral data, radio-frequency data, and weather observations. AI workloads can classify land cover, detect ships, identify flood extent, monitor crops, track infrastructure change, or support defense and security analysis. New Space Economy’s article on orbital data centers and Earth observation argues that the stronger commercial test is whether compute can lower total cost, reduce latency, improve resilience, or enable better products.

Edge and Device AI Workloads Move Compute Closer to Data

Edge AI runs models near the source of data instead of sending every input to a central cloud. The edge may be a phone, laptop, vehicle, camera, industrial controller, medical device, satellite, ship, aircraft, or local server. Edge workloads matter because bandwidth, latency, privacy, reliability, and cost can make central processing impractical.

Device AI includes model execution on consumer hardware. Examples include local transcription, photo search, language translation, accessibility features, image enhancement, personal assistants, and document summarization. The model must fit within strict limits for memory, battery life, heat, and response time. That creates demand for smaller models, neural processing units, model compression, and operating-system integration.

Industrial edge workloads often involve sensors, cameras, and control systems. A factory defect detector may need to classify parts as they pass through a production line. A mining system may monitor equipment vibration. A power grid tool may identify anomalies in sensor streams. These workloads cannot always depend on continuous cloud connectivity. Local inference can keep operations running during network outages and reduce data transfer.

Automotive and robotics workloads combine perception, prediction, planning, and control. The system must interpret sensor data, understand nearby objects, estimate movement, choose an action, and monitor safety. Training can happen in centralized systems, but inference often occurs on the vehicle or robot. Simulation adds another workload because teams need to test rare events before deployment.

Satellite edge workloads process data in orbit. A satellite may run computer vision or signal processing before downlinking results. This can reduce bandwidth demand and speed time-sensitive alerts. The tradeoff is that spacecraft compute faces power, radiation, thermal, maintenance, and communications constraints. New Space Economy’s coverage of space-based compute infrastructure describes early commercial systems that combine communications networks with onboard accelerated compute.

Edge AI does not replace cloud AI. It redistributes work. A common pattern uses edge devices for immediate filtering, central systems for model improvement, and cloud services for heavy training or broad coordination. The right split depends on latency, bandwidth cost, privacy, safety, and how often the model needs updates.

The workload categories differ most clearly by latency, data location, and cost pressure.

Organizations Need Workload-Specific AI Planning

Procurement fails when organizations buy generic AI capacity without mapping workloads. Training, inference, RAG, agents, video analytics, and device AI do not consume resources in the same way. A training cluster may need high-speed accelerator networking. A chatbot service may need predictable token throughput. A RAG system may depend on search quality and document permissions. A video system may need storage and streaming pipelines. A monitoring system may need audit logs and evaluation datasets more than larger models.

Capacity planning should begin with workload questions. How many users or devices will send requests? How fast must the system respond? How large is each input? Does the model need private data at run time? Is the workload real-time, batch, streaming, or event-driven? Does the organization need to train models, fine-tune models, or use external models? What happens if the system is unavailable? What data must remain on premises, inside a country, or under a specific compliance regime?

Benchmarking helps, but benchmark results require care. MLCommons develops benchmarks that measure AI systems under defined rules, including model behavior, datasets, quality metrics, and performance. These benchmarks support comparison, but they do not replace workload testing with an organization’s own data, security requirements, latency targets, and operating constraints.

The Stanford AI Index tracks data on AI research, development, deployment, and societal effects. Its 2026 report page describes the project as an effort to collect and visualize AI-related data for policymakers, researchers, executives, journalists, and the public. Such broad tracking is useful because AI workload demand affects more than model performance; it affects talent, power, capital spending, safety, and regulation.

Model selection is part of workload planning. The best model is not always the largest one. A small model may handle classification, extraction, translation, or routing at lower cost. A larger model may be reserved for complex reasoning, ambiguous requests, or high-value tasks. Mixture-of-model architectures can send routine jobs to smaller models and escalate harder tasks only when needed.

Data governance is also workload planning. A RAG workload over confidential contracts needs access control and retrieval logs. A fine-tuning workload needs rights to use the training data. A safety evaluation workload needs examples of failure modes. An edge AI workload may need local update policies and fallback behavior. These requirements belong in architecture and procurement decisions, not after deployment.

The economics of AI workloads will keep changing, but the categories are stable enough to guide decisions. Training builds capability. Fine-tuning adapts capability. Inference delivers capability. Retrieval grounds capability. Agents coordinate capability. Edge AI places capability near data. Monitoring protects capability after deployment. Organizations that plan at that level can compare vendors, data centers, chips, cloud services, and software platforms with far more discipline than organizations that treat AI as one generic technology purchase.

Summary

AI workload planning begins with a simple separation: building models, adapting models, running models, grounding models, coordinating multi-step systems, moving AI closer to data, and operating the full service safely. These categories explain why AI infrastructure discussions now include accelerators, memory, storage, networking, search systems, observability, safety testing, energy supply, and deployment location.

The most visible workload may be training, but production value usually depends on inference, retrieval, monitoring, and workflow integration. A model that cannot be served affordably, grounded in trusted data, monitored for failure, and matched to the right deployment environment remains a technical asset rather than a reliable product.

For the space economy, the same logic applies with sharper constraints. On-orbit AI, satellite edge processing, and orbital data centers will succeed first where the workload benefits from location, latency reduction, bandwidth savings, resilience, or mission-specific data. For terrestrial organizations, the lesson is broader: the workload defines the system.

Appendix: Useful Books Available on Amazon

• AI Engineering
• Designing Machine Learning Systems
• Hands-On Machine Learning With Scikit-Learn, Keras, and TensorFlow
• Deep Learning
• Machine Learning Design Patterns
• Practical MLOps
• Designing Data-Intensive Applications

Appendix: Top Questions Answered in This Article

What Is an AI Workload?

An AI workload is a defined unit of work needed to build, adapt, run, test, or manage an artificial intelligence system. Examples include preparing data, training a model, fine-tuning it for a task, running inference for users, searching documents for RAG, coordinating agents, and monitoring production behavior.

How Is Training Different from Inference?

Training builds model capability from data. Inference uses a trained or fine-tuned model to produce outputs from new inputs. Training is often compute-heavy and periodic, but inference can dominate operating cost because production systems may run continuously for large user populations.

Why Does Fine-Tuning Matter?

Fine-tuning adapts an existing model to a narrower task, domain, or output style. It can improve task performance, reduce prompt length, and make smaller deployed models practical. It still requires clean data, evaluation, cost control, and lifecycle management.

What Is Batch Inference?

Batch inference processes many inputs together when no immediate response is required. It suits document classification, catalog enrichment, analytics, offline recommendations, and large archive processing. The main benefits are higher throughput, lower unit cost, and easier scheduling.

What Is Real-Time Inference?

Real-time inference produces outputs fast enough for an active user, machine, or service. Chat assistants, fraud scoring, vehicle perception, live translation, and industrial controls use this pattern. The main design pressures are latency, reliability, scaling, and predictable cost.

What Is RAG?

Retrieval-augmented generation combines a generative model with a search or retrieval system. The model receives relevant material from documents, databases, or knowledge stores at run time. RAG helps systems answer from current, private, or domain-specific information.

What Are Agentic AI Workloads?

Agentic AI workloads involve systems that plan steps, retrieve information, call tools, use memory, check results, and take actions. They can consume more compute than a single prompt because one task may trigger many model calls and external operations.

Why Are Multimodal AI Workloads Harder to Run?

Multimodal AI handles text, images, audio, video, documents, or sensor data in the same system. These workloads often require larger files, specialized processing, storage pipelines, and more complex evaluation. Video and physical AI can be especially demanding because time and sequence matter.

Why Does Edge AI Matter?

Edge AI runs models near the data source, such as on phones, cameras, vehicles, factory equipment, or satellites. It can reduce latency, protect privacy, save bandwidth, and support offline operation. The tradeoff is that edge devices have limited memory, power, and cooling.

How Should Organizations Plan AI Infrastructure?

Organizations should classify workloads before choosing chips, cloud services, data centers, or model providers. They should separate training, fine-tuning, inference, RAG, agents, monitoring, and edge deployment. Each category has different needs for latency, cost, security, data location, and reliability.

Appendix: Glossary of Key Terms

AI Workload

A defined unit of work required to build, adapt, run, test, or manage an artificial intelligence system. The term covers data preparation, training, fine-tuning, inference, retrieval, agent orchestration, edge deployment, monitoring, and safety evaluation.

Machine Learning Lifecycle

The full sequence of activities used to create and operate a machine learning system. It usually includes problem framing, data processing, model development, deployment, monitoring, feedback, and retraining when data or business needs change.

Training

The process of teaching a model from data so it can identify patterns, make predictions, or generate outputs. Training can range from small statistical jobs to large accelerator-cluster workloads for foundation models.

Fine-Tuning

The process of adapting an existing model to a specific task, domain, style, or output format. Fine-tuning usually uses a smaller dataset than pretraining and can improve performance for targeted production use.

Inference

The process of running a trained model on new inputs to produce predictions, classifications, recommendations, text, images, audio, code, or other outputs. Inference is the operational phase where many AI services deliver user value.

Batch Inference

A form of inference that processes many inputs together without requiring immediate responses. It fits jobs such as document classification, customer segmentation, offline recommendations, image archive analysis, and bulk data enrichment.

Real-Time Inference

A form of inference where the system must respond quickly enough for a live user, machine, or process. It places strong pressure on latency, uptime, throughput, model size, and serving architecture.

Retrieval-Augmented Generation

A design pattern that connects a generative model to a search or retrieval system. It supplies relevant information at run time so outputs can reflect current, private, or specialized material.

Embedding

A numerical representation of text, image, audio, code, or other data. Embeddings support similarity search, clustering, recommendation, retrieval, and RAG systems by placing related items near each other in mathematical space.

Agentic AI

An AI system design in which models plan steps, use tools, search information, call other systems, evaluate intermediate results, and take actions. Agentic workloads can be harder to estimate because task length can change during execution.

Multimodal AI

AI that handles more than one data type, such as text and images, audio and documents, or video and sensor streams. Multimodal systems often need larger pipelines and more complex testing than text-only systems.

Edge AI

AI that runs near the source of data rather than only in a central cloud. Edge deployment can reduce latency, improve privacy, save bandwidth, and support offline operation, but it faces limits in power, memory, and maintenance.

Model Serving

The process of packaging, deploying, scaling, and managing a model so applications can use it for inference. Serving includes routing requests, managing versions, monitoring performance, and controlling reliability.

Model Drift

A decline in model performance caused by changes in data, user behavior, operating conditions, or business rules after deployment. Drift monitoring helps teams decide when to update data, retrain models, or change system behavior.

AI Accelerator

A processor designed to speed AI computations. Examples include graphics processing units, tensor processing units, neural processing units, and other specialized chips used for training, inference, or device-level AI.