AI Production Deployment Checklist: 40 Points Before You Go Live

Why Most AI Models Never Reach Production

In short

Only 22% of AI models reach production. The primary failure modes are not technical — they are operational: missing monitoring, undefined rollback procedures, and governance gaps that stop deployment at the approval stage.

According to Hakia's December 2025 analysis of real production deployments, 78% of AI models never make it to live systems. That number should recalibrate how teams approach deployment preparation.

The failure is rarely about model quality. Teams that build accurate, well-tested models still fail to deploy them because the operational layer — monitoring, governance, rollback — was never built.

Three failure categories account for the majority of stalled deployments seen across our 100+ enterprise AI implementations at Alice Labs:

• Monitoring and observability not designed before launch: Teams discover post-launch that they cannot detect model drift, silent failures, or data quality degradation in real time.
• Governance and approval blockers: Legal, compliance, or procurement flags the deployment after engineering is complete — adding weeks or months to the timeline.
• Infrastructure underestimation: Staging environment performance does not replicate production load, causing failures that only surface at scale.

KPMG's April 2026 findings project average AI spend at $207 million per US organization over the next 12 months — nearly double the prior year. The cost of a failed deployment at that investment level is not recoverable.

TechRepublic reported in January 2026 that as of mid-2025, nearly two-thirds of organizations remained stuck in the pilot stage. The pattern is consistent: teams without a structured pre-launch checklist address blockers in the wrong sequence, compounding delays.

This checklist covers six readiness domains in the order they should be validated:

1. Model Validation — performance, fairness, and latency under realistic conditions
2. Infrastructure & Scalability — compute, storage, and networking at peak load
3. Security & Access Control — hardening against adversarial inputs and data leakage
4. Monitoring & Observability — real-time detection of drift and failure
5. Governance & Compliance — data lineage, model cards, audit trails, and regulatory alignment
6. Rollout Strategy — shadow mode, canary releases, and staged expansion

The 6 Readiness Domains This Checklist Covers

Each domain maps to a distinct failure mode. Skipping or deferring any one of them increases the probability of a production incident or a blocked go-live.

• Model Validation: Confirming the model performs correctly, fairly, and within latency budgets under realistic conditions — not just on the holdout set.
• Infrastructure & Scalability: Ensuring compute, storage, and networking can sustain peak production load with acceptable degradation curves.
• Security & Access Control: Hardening the deployment against adversarial inputs, prompt injection, unauthorized access, and data leakage vectors.
• Monitoring & Observability: Instrumentation to detect performance drift, data distribution shift, and infrastructure failure in real time before users are impacted.
• Governance & Compliance: Data lineage documentation, model cards, audit trails, and alignment with the EU AI Act compliance checklist for regulated environments.
• Rollout Strategy: Shadow mode validation, canary releases at 5%, and staged traffic expansion to contain the blast radius of early failures.

Understanding why deployments fail is the first step. For a broader view of failure patterns across the full AI project lifecycle, see our analysis of why AI projects fail.

Domain 1: Model Validation (8 Checklist Points)

In short

Model validation before production must go beyond holdout accuracy. You need out-of-distribution testing, fairness audits, latency benchmarking under peak load, and documented performance thresholds with defined acceptable ranges.

Validation in a notebook is not production validation. The model has passed development testing — now it needs to be stress-tested against production realities, edge cases, and business expectations.

These 8 validation checkpoints must be completed and documented before any model advances to infrastructure review:

1. Holdout set performance meets defined minimum threshold. Specify the metric (F1, AUC, RMSE) and the threshold agreed with business stakeholders in writing. "Good enough" is not a threshold.
2. Out-of-distribution (OOD) test results documented. Test on data that falls outside the training distribution. Document edge-case behavior explicitly — including failure modes.
3. Fairness and bias audit completed. Test model outputs across demographic or categorical subgroups relevant to the use case. For high-risk applications, this is non-negotiable under the EU AI Act.
4. Latency benchmarked at P50, P95, and P99 under simulated peak load. Average latency misleads. P99 latency is what your slowest 1% of users experience — and often what SLAs are measured against.
5. Model outputs spot-checked by domain experts. Someone who knows the business domain — not just data scientists — has reviewed a minimum of 200 real predictions and signed off.
6. Model card written and approved. Documents intended use, limitations, training data, performance characteristics, and known failure modes. See the subsection below for required fields.
7. Baseline comparison completed. The AI model must demonstrably outperform the current solution — whether a manual process, rule-based system, or previous model — on a defined metric agreed in advance.
8. Rollback model version archived and tested. If this deployment fails, the previous model version must be restorable in under 30 minutes. Test this before go-live, not during an incident.

What a Production-Ready Model Card Must Include

A model card is not a nice-to-have — it is a governance document. Under the EU AI Act, high-risk AI systems require technical documentation that maps directly to what a model card contains.

Every model card at Alice Labs across our 100+ implementations includes these seven sections as a standard deliverable:

• Model description and intended use cases: What the model does, what it is designed for, and — critically — what it is not designed for.
• Training data sources, date range, and known biases: Where the data came from, when it was collected, and any documented distributional gaps or biases identified during preparation.
• Performance metrics with confidence intervals: Not just point estimates — include variance measures so stakeholders understand result reliability.
• Known limitations and out-of-scope use cases: Explicit documentation of scenarios where the model should not be used or will underperform.
• Fairness evaluation results: Subgroup performance metrics for any demographic or categorical variables relevant to the deployment context.
• Deployment environment requirements: Hardware, software dependencies, input data format requirements, and integration constraints.
• Contact person and review/update schedule: Who owns the model post-launch, and when the next scheduled performance review is due.

For AI systems deployed in regulated EU contexts, cross-reference the EU AI Act compliance guide to confirm your model card satisfies Annex IV technical documentation requirements.

Domain 2: Infrastructure & Scalability (7 Checklist Points)

In short

Infrastructure readiness means your production environment can handle peak load, not just average load. Seven checkpoints cover compute provisioning, auto-scaling, latency SLAs, storage, network configuration, dependency health, and disaster recovery.

The most common infrastructure failure in AI production deployment is not a hardware outage — it is a staging environment that does not accurately replicate production load patterns. Teams test at 10% of real traffic and discover the gap only after go-live.

These 7 infrastructure checkpoints must be validated against production-equivalent load, not staging approximations:

1. Compute resources provisioned for peak load plus 30% headroom. Baseline your compute against peak concurrent request volume, not average. The 30% buffer covers traffic spikes without requiring manual intervention.
2. Auto-scaling policies configured and load-tested. Define scale-up and scale-down thresholds. Verify that auto-scaling responds within your defined SLA window under a simulated traffic spike.
3. Model serving latency meets defined SLA under load. Run a load test at 150% of expected peak traffic. Document P50, P95, and P99 response times. Any breach of P99 SLA is a deployment blocker.
4. Storage and database connections validated at scale. Confirm that database connection pools, vector store query times, and storage I/O do not degrade past acceptable thresholds under concurrent load.
5. Network configuration reviewed for egress costs and latency. For cloud deployments, cross-region data transfer and egress costs under production volume are frequently underestimated. Validate routing paths and associated costs before go-live.
6. All upstream API and model dependencies have SLA documentation. If your production model calls an external LLM API, a third-party data source, or an internal service, each dependency's availability SLA must be documented and factored into your own uptime commitment.
7. Disaster recovery and failover procedures tested. Simulate a primary region failure or primary model serving endpoint failure. Verify that failover completes within the defined RTO (recovery time objective) — and document that RTO explicitly.

Load Testing Requirements Before Production

Load testing for AI systems differs from standard application load testing. Model inference is computationally expensive — latency degrades non-linearly as concurrency increases, not linearly.

• Test at 100%, 150%, and 200% of expected peak concurrent load to understand the degradation curve, not just the pass/fail threshold.
• Use production-equivalent request payloads — not synthetic minimal inputs. Real inputs are often longer, more varied, and more computationally demanding than test stubs.
• Measure cold-start latency separately from warm inference latency. For serverless or auto-scaled deployments, cold starts can add hundreds of milliseconds to P99 — a frequent user-facing surprise.
• Run load tests for a minimum of 30 minutes continuously to surface memory leaks, connection pool exhaustion, and gradual performance degradation that only appear under sustained load.
• Document all load test results and include them in the deployment approval package — not just a pass/fail summary.

For teams building on agentic AI architectures where multiple model calls chain together, load-test the full pipeline — not individual model endpoints in isolation. See our guide to AI agent architecture patterns for pipeline-specific infrastructure considerations.

Domain 3: Security & Access Control (7 Checklist Points)

In short

AI production security requires hardening beyond standard application security. Seven checkpoints cover input validation, prompt injection defenses, access controls, data encryption, PII handling, adversarial input testing, and dependency vulnerability scanning.

AI systems introduce attack surfaces that standard application security practices do not cover. Prompt injection, model inversion attacks, and training data extraction are AI-specific threats that require explicit mitigation before production.

Complete all 7 security checkpoints before infrastructure review is finalized:

1. Input validation and sanitization implemented for all model inputs. Define allowable input formats, length limits, and character sets. Reject or sanitize inputs that fall outside defined parameters before they reach the model.
2. Prompt injection defenses tested for LLM-based systems. For any system using an LLM, run a structured prompt injection test suite covering direct injection, indirect injection via retrieved context, and jailbreak attempts. Document results and implemented mitigations.
3. Role-based access control (RBAC) implemented and reviewed. Define who can query the model, who can access outputs, and who can modify model configuration or replace the serving artifact. No shared credentials.
4. Data in transit and at rest encrypted to current standards. TLS 1.2 minimum for data in transit. AES-256 for data at rest. Verify that model artifacts, input logs, and output logs are all covered — not just the primary data pipeline.
5. PII handling procedures documented and technically enforced. If the model processes personal data, document retention periods, anonymization procedures, and deletion workflows. Verify these are technically implemented, not just written in a policy document.
6. Adversarial input testing completed. Test model behavior under intentionally malformed, adversarial, or boundary-case inputs. Document failure modes — graceful degradation is acceptable, silent incorrect outputs are not.
7. Dependency vulnerability scan completed with no critical findings open. Run a software composition analysis (SCA) scan across all model serving dependencies. No critical or high-severity CVEs may remain unresolved at deployment time.

AI-Specific Security Threats to Test Before Go-Live

Beyond the 7 checklist items, validate your system against these AI-specific threat categories:

• Model inversion attacks: Can an adversary reconstruct training data from model outputs? Particularly relevant for models trained on sensitive proprietary or personal data.
• Membership inference: Can an attacker determine whether a specific record was in the training data? Relevant for healthcare, financial, and HR AI applications.
• Output manipulation: Can carefully crafted inputs cause the model to produce outputs that appear valid but are systematically incorrect in ways that benefit the attacker?
• Supply chain attacks: Are your model weights, base models, or third-party components sourced from verified, integrity-checked repositories?

For deployments subject to EU regulation, these security requirements map directly to the risk management obligations under the EU AI Act. The EU AI Act risk categories guide clarifies which security controls are mandatory versus recommended at each risk tier.

Domain 4: Monitoring & Observability (7 Checklist Points)

In short

AI monitoring must cover four dimensions: model performance drift, data distribution shift, infrastructure health, and business KPI alignment. Seven checkpoints ensure you can detect failures in real time — not in the next quarterly review.

Monitoring is the single most common gap in failed AI deployments. A model that performs well at launch can degrade silently over weeks as real-world data drifts away from the training distribution.

These 7 monitoring checkpoints must be operational before the first production request is served:

1. Model performance metrics logged and baselined. Capture prediction confidence scores, output distributions, and task-specific metrics (accuracy, F1, RMSE) continuously. Establish a baseline during shadow mode so you have a comparison point from day one.
2. Data distribution monitoring implemented. Monitor input feature distributions against training baselines. Statistical drift in incoming data — measured via PSI (Population Stability Index) or similar — is the leading indicator of model degradation before performance metrics show it.
3. Infrastructure health dashboards live and alerting configured. CPU utilization, memory, GPU utilization (if applicable), request queue depth, error rates, and latency percentiles must have defined alert thresholds with named on-call owners.
4. Business KPI tracking connected to model outputs. Define at least one business metric that the model is expected to move — and instrument it. If the model drives revenue, track revenue. If it drives deflection, track deflection. Disconnected from business KPIs, technical monitoring alone will not catch the case where the model is technically healthy but business-wrong.
5. Prediction logging implemented with sampling strategy defined. Log a statistically representative sample of inputs, outputs, and confidence scores. Define the sampling rate (1%, 10%, 100% — depending on volume and privacy constraints) before go-live.
6. Alerting runbooks written for the top 5 alert scenarios. Alerts without runbooks generate panic, not resolution. Document the investigation steps and escalation path for at minimum: high error rate, latency SLA breach, data drift detected, model performance drop, and infrastructure failure.
7. Retraining trigger criteria defined in writing. Specify the exact conditions — drift threshold, performance degradation percentage, time-based schedule, or business metric breach — that will trigger a model retraining or replacement cycle.

The 4 Dimensions of AI Production Monitoring

Standard infrastructure monitoring covers only one of the four dimensions required for AI systems. All four must be instrumented independently.

The operational practices that keep models healthy after launch fall under MLOps. For teams building their MLOps function, our guide to what MLOps is covers the tooling, processes, and team structures needed to sustain production AI systems.

Domain 5: Governance & Compliance (6 Checklist Points)

In short

Governance checkpoints are not bureaucratic overhead — they are deployment blockers in regulated environments. Six checkpoints cover data lineage, model cards, audit trails, regulatory alignment, human oversight, and incident response procedures.

Governance is where technically complete deployments most frequently stall. Engineering finishes, then legal, compliance, or procurement reviews the system and finds documentation that was never created.

Address these 6 governance checkpoints in parallel with engineering work — not after it:

1. Data lineage documentation complete. Every data source feeding the model must be documented: origin, collection method, consent basis (if personal data), transformation history, and current version. This is a regulatory requirement under GDPR and the EU AI Act for high-risk systems.
2. Model card approved by required stakeholders. The model card written in Domain 1 must be reviewed and signed off by legal, compliance, and the business owner — not just the data science team. Document the approval with names and dates.
3. Audit trail instrumentation live before go-live. For regulated industries, every model decision must be traceable: which model version made the decision, on what input, at what time, with what confidence. Implement immutable logging before the first production request.
4. Human oversight procedures defined for high-risk outputs. For any AI system making decisions with material consequences — credit, hiring, medical, legal — define explicitly when a human must review the model's output before it takes effect. Document the review threshold and the handoff process.
5. Regulatory alignment confirmed for the deployment jurisdiction. For EU deployments, confirm EU AI Act risk classification and required controls. For US deployments, confirm NIST AI RMF alignment if applicable. Document the regulatory framework and your compliance status in writing. See the NIST AI RMF guide for US-specific requirements.
6. Incident response plan documented and tested. Define who is notified, in what sequence, when the model produces harmful outputs, is exploited, or causes a material business incident. Run a tabletop exercise before go-live. The AI incident response plan template covers the required structure.

EU AI Act Requirements for Production Deployment

For organizations deploying AI in the EU, the EU AI Act imposes specific pre-deployment requirements that map directly to this checklist. High-risk systems — as defined under Annex III — require all six governance checkpoints to be satisfied and documented before placing the system on the market or putting it into service.

• Technical documentation (Annex IV): Covers model card, data lineage, and system architecture documentation.
• Logging and record-keeping (Article 12): Requires automatic logging of events enabling post-market monitoring and incident investigation.
• Human oversight (Article 14): Requires that high-risk systems can be monitored, overridden, and halted by a named human operator.
• Conformity assessment: For certain high-risk categories, requires third-party assessment before deployment — not self-declaration.

For a complete mapping of EU AI Act requirements to deployment practices, use the EU AI Act compliance checklist alongside this deployment checklist.

Domain 6: Rollout Strategy (5 Checklist Points)

In short

A phased rollout — shadow mode, then canary at 5%, then staged expansion — limits the blast radius of failures that only surface under real production conditions. Five checkpoints define the gate criteria for each phase.

Even a fully validated, infrastructure-ready, security-hardened model will encounter surprises in production. The rollout strategy determines whether those surprises affect 5% of users or 100%.

These 5 rollout checkpoints define a sequence that contains failure risk at each stage:

1. Shadow mode deployment completed and results reviewed. Run the new model in parallel with the current system for a minimum of 5 business days. The new model receives real inputs but its outputs are logged — not served. Compare output distributions between the shadow model and the live system. Any material divergence requires investigation before advancing.
2. Canary release gate criteria defined before canary starts. Define in writing: what percentage of traffic starts in the canary (recommended: 5%), what metrics are monitored, and what thresholds trigger an automatic rollback. Criteria must be defined before the canary launches — not evaluated subjectively after.
3. Canary phase completed with no gate criteria breaches. Run the canary for a minimum of 48 hours at 5% traffic. Advance to the next stage only if all defined gate criteria are met. Document the canary results formally.
4. Staged expansion plan documented with explicit rollback triggers. Define the traffic expansion sequence: 5% → 20% → 50% → 100%, with dwell time at each stage and the specific conditions that trigger a rollback at each gate. "Roll back if it looks bad" is not a rollback trigger.
5. Rollback procedure tested and timed. Execute a full rollback drill before go-live. Confirm the previous model version can be restored and traffic can be redirected within the defined RTO. Document the actual time the drill took — not the theoretical time.

Shadow Mode vs. Canary: When to Use Each

Shadow mode and canary releases are complementary, not interchangeable. They test different failure modes and should both be used in sequence.

The phased rollout approach described here is a core component of a mature AI implementation process. For the full implementation lifecycle from pilot to production, see the AI implementation roadmap.

Post-Launch: The First 30 Days in Production

In short

The 30 days after go-live are the highest-risk period for a new AI deployment. Establish a structured review cadence — daily for the first week, then weekly — with explicit criteria for escalation and defined owners for each monitoring dimension.

Go-live is not the finish line. The first 30 days of production operation are when the failure modes that evaded pre-launch testing most commonly surface: unusual user input patterns, edge cases not present in test data, and business KPI divergences that only become visible over time.

Structure the post-launch period with explicit review cadences:

• Days 1–7: Daily structured reviews. Review all four monitoring dimensions each day. Any alert from the preceding 24 hours is discussed, root-caused, and assigned a resolution owner before the review ends.
• Days 8–14: Daily reviews continue. Shift focus from reactive alert response to pattern identification. Are specific input types generating higher error rates? Are latency patterns changing at specific times of day?
• Days 15–30: Bi-weekly reviews. If the first two weeks have been clean, move to bi-weekly cadence. Compile a 30-day production health report against the pre-launch baseline.
• Day 30: Formal production health review. Compare 30-day actual performance against the pre-launch projections. Decide whether the deployment is stable enough to transition to standard operational monitoring cadences.

Model Drift Detection: When to Retrain vs. When to Rollback

Not all performance degradation requires rollback. The decision tree between retraining, parameter adjustment, and rollback should be defined before any degradation occurs — not when the team is under pressure responding to an incident.

• Gradual performance drift (<10% over 30 days): Investigate data distribution shift. If confirmed, initiate retraining on updated data. No rollback required if business KPIs remain above threshold.
• Sudden performance drop (>15% in under 48 hours): Immediate rollback to the previous model version. Investigate whether the cause is a data pipeline failure, upstream API change, or adversarial input pattern before re-deploying.
• Data distribution shift with stable performance: Monitor closely. Retraining is recommended within 30 days even if current performance is acceptable — the model is operating outside its validated distribution envelope.
• Business KPI breach with stable technical metrics: This indicates the model is technically correct but business-wrong. Requires stakeholder review and possible reframing of the model's objective function — not a standard retraining cycle.

The operational discipline required to manage models after go-live is the domain of LLMOps for language-model-based systems. For teams deploying LLMs in production, see our guide to what LLMOps is and how it extends MLOps practices for generative AI.

The Complete 40-Point AI Production Deployment Checklist

In short

This consolidated 40-point checklist covers all 6 domains. Use it as a sign-off document before any AI system goes live in production.

Use this consolidated checklist as a pre-launch sign-off document. Each item should be checked with a named owner and a completion date — not just ticked as "done."

Domain 1: Model Validation (8 Points)

• ☐ Holdout set performance meets stakeholder-agreed minimum threshold (metric and threshold documented)
• ☐ Out-of-distribution test results documented with explicit failure modes recorded
• ☐ Fairness and bias audit completed across all relevant subgroups
• ☐ Latency benchmarked at P50, P95, and P99 under simulated peak load
• ☐ Minimum 200 predictions spot-checked by domain expert (not just data scientists)
• ☐ Model card written and approved by legal, compliance, and business owner
• ☐ Baseline comparison showing model outperforms current solution on defined metric
• ☐ Rollback model version archived and restoration tested — confirmed restorable in under 30 minutes

Domain 2: Infrastructure & Scalability (7 Points)

• ☐ Compute provisioned for peak load plus 30% headroom
• ☐ Auto-scaling policies configured and validated under simulated traffic spike
• ☐ P99 latency SLA confirmed met under 150% of expected peak load
• ☐ Storage and database connections validated at scale — no degradation past threshold
• ☐ Network routing and egress costs validated for production volume
• ☐ All upstream API and model dependency SLAs documented
• ☐ Disaster recovery and failover tested — RTO documented and confirmed

Domain 3: Security & Access Control (7 Points)

• ☐ Input validation and sanitization implemented for all model inputs
• ☐ Prompt injection test suite completed for LLM-based systems — mitigations documented
• ☐ RBAC implemented and reviewed — no shared credentials
• ☐ Data in transit (TLS 1.2+) and at rest (AES-256) encrypted
• ☐ PII handling procedures documented and technically enforced
• ☐ Adversarial input testing completed — failure modes documented
• ☐ Dependency vulnerability scan complete — no critical or high-severity CVEs open

Domain 4: Monitoring & Observability (7 Points)

• ☐ Model performance metrics logged and baselined during shadow mode
• ☐ Data distribution monitoring implemented with drift detection thresholds defined
• ☐ Infrastructure dashboards live — alert thresholds set with named on-call owners
• ☐ Business KPI tracking connected to model outputs and baselined
• ☐ Prediction logging implemented with sampling strategy and retention period defined
• ☐ Alerting runbooks written for top 5 alert scenarios
• ☐ Retraining trigger criteria documented in writing

Domain 5: Governance & Compliance (6 Points)

• ☐ Data lineage documentation complete for all training and inference data sources
• ☐ Model card approved by legal, compliance, and business owner — names and dates recorded
• ☐ Audit trail instrumentation live — immutable logging confirmed before first production request
• ☐ Human oversight procedures defined and documented for high-risk outputs
• ☐ Regulatory alignment confirmed for deployment jurisdiction — compliance status documented
• ☐ Incident response plan documented and tabletop exercise completed

Domain 6: Rollout Strategy (5 Points)

• ☐ Shadow mode deployment completed — minimum 5 business days — results reviewed and approved
• ☐ Canary gate criteria defined in writing before canary launch
• ☐ Canary phase completed at 5% traffic for minimum 48 hours — no gate criteria breaches
• ☐ Staged expansion plan documented — explicit rollback triggers at each traffic gate
• ☐ Rollback procedure drilled and timed — actual restoration time documented

Frequently Asked Questions: AI Production Deployment

In short

Answers to the most common questions about AI production deployment checklists, timelines, and failure modes.

How long does AI production deployment typically take?

For enterprise AI systems, the deployment phase — from completed model to stable production — typically takes 6–14 weeks. This includes 2–3 weeks for infrastructure and security validation, 1–2 weeks for governance approvals, and 2–4 weeks for phased rollout including shadow mode and canary phases.

Teams that begin governance documentation in parallel with engineering — rather than after it — reduce total deployment time by 3–6 weeks on average, based on patterns across Alice Labs' 100+ enterprise implementations.

What are the most common causes of AI deployment failure?

According to Hakia's December 2025 analysis of real production deployments, the three most common failure causes are: missing monitoring and observability (teams cannot detect when models degrade), governance and compliance blockers surfacing after engineering is complete, and infrastructure that was not load-tested at production-equivalent scale.

Model quality — the focus of most pre-deployment effort — is rarely the primary failure cause. Operationally, teams that build monitoring first and model last have significantly better deployment outcomes.

What is shadow mode deployment and when should I use it?

Shadow mode deployment runs the new model in parallel with the current production system. The new model receives real inputs but its outputs are logged rather than served to users. It is used to validate output distributions, latency behavior, and infrastructure performance under real production inputs — before any user is exposed to the new model.

Shadow mode should be used for any user-facing AI system. The minimum recommended shadow mode period is 5 business days. It can only be skipped for non-critical internal tools where incorrect outputs carry no material consequence.

What is a model card and is it required?

A model card is a structured document that describes an AI model's intended use cases, training data, performance characteristics, limitations, and fairness evaluation results. It is required — not optional — for high-risk AI systems under the EU AI Act (Annex IV technical documentation requirements).

For lower-risk systems, model cards are strongly recommended as a governance artifact that accelerates approval processes and provides a reference for post-launch monitoring decisions. At Alice Labs, model cards are a standard deliverable on every AI implementation.

How do I detect model drift in production?

Model drift is detected through two primary signals: data distribution shift in incoming inputs (measured via PSI or KL divergence against training distribution) and degradation in model performance metrics over time (accuracy, F1, confidence score distributions). Data drift typically precedes performance drift by days to weeks, making input monitoring the earlier warning signal.

A PSI value above 0.2 on any top-10 feature is a standard trigger for drift investigation. Define your specific thresholds before go-live and document them in the monitoring runbook.

What is a canary deployment for AI models?

A canary deployment routes a small percentage of production traffic — typically 5% — to the new model version while the remaining 95% continues on the current model. It allows real-world validation with limited user impact. Gate criteria for advancing or rolling back must be defined before the canary starts.

The term comes from the historical practice of using canaries in coal mines as early warning systems. The principle is the same: expose the new model to real conditions at small scale before committing fully.

What does the EU AI Act require before deploying AI in production?

For high-risk AI systems as defined under Annex III of the EU AI Act, pre-deployment requirements include: technical documentation (Annex IV), a conformity assessment, registration in the EU database, implementation of a risk management system, data governance procedures, logging and record-keeping systems, transparency obligations, and human oversight measures.

The EU AI Act compliance obligations are mapped directly to Domains 1, 3, 4, and 5 of this checklist. For a complete regulatory mapping, use the EU AI Act compliance checklist for 2026.

Can I use a shorter checklist for low-risk AI deployments?

For genuinely low-risk internal tools — recommendation systems, content classifiers, internal search — a simplified checklist covering the highest-priority items from each domain is defensible. The minimum viable checklist for low-risk deployments covers: holdout accuracy threshold, latency benchmark at P95, basic RBAC, infrastructure health monitoring, prediction logging, and a rollback procedure.

However, risk classification must be made explicitly and documented — not assumed. A system that appears low-risk at design time can have higher-risk implications once it influences business decisions at scale. When in doubt, apply the full 40-point checklist.