Photo by Vishnu Mohanan on Unsplash

Why Enterprise AI Agents Cost More Than You Budgeted: Four Myths That Drain Your Q2 Spend

The Slack message came in at 6:19 AM: “Why did our optimization agent run 14,000 API calls overnight?” The data product manager who asked that question had launched an AI agent three weeks earlier to automate database query optimization across the organization’s analytics stack. The agent was supposed to reduce manual review time. Instead, it racked up $18,000 in compute costs in a single weekend by recursively triggering its own optimization suggestions. According to Deloitte’s 2026 State of AI in the Enterprise report, 43% of organizations deploying autonomous AI agents in production reported unplanned cost overruns exceeding 200% of initial budget estimates within the first quarter of operation. That’s not a rounding error. That’s a pattern.

David Ohnstad has seen this failure mode from both sides—as a Senior Data Product Manager shipping AI integrations at Veeam Software, and as the person debugging runaway agent behavior at 2 AM when cloud bills spike. The problem isn’t that AI agents don’t work. The problem is that most enterprise teams deploy them with assumptions borrowed from static automation playbooks—assumptions that break the moment an agent starts making decisions without human checkpoints. What follows are four myths that persist across enterprise AI implementations, why they survive despite mounting evidence, and what actually happens when you replace them with operational reality.

Myth One: “If the Agent Passes QA in Staging, It’s Safe in Production”

This is the most expensive myth in enterprise AI deployment. Teams run agents through staging environments, watch them perform the intended task correctly, and assume production behavior will mirror those results. It won’t. Staging environments are smaller, slower, and—critically—bounded. Production environments are not. An agent that optimizes three database queries in staging might attempt to optimize 3,000 in production if no one has defined a rate limit, a cost ceiling, or a scope boundary.

Why does this myth persist? Because it worked for traditional automation. A script that runs successfully in staging will behave identically in production as long as the data structure matches. But AI agents aren’t scripts. They make decisions based on context, and production context is always richer—more data sources, more users, more edge cases—than staging. According to McKinsey’s 2024 Global AI Survey, 68% of organizations that experienced AI deployment failures cited “unexpected agent behavior at scale” as the primary failure mode. That’s not a technical bug. That’s a conceptual misunderstanding of what an agent does when it encounters a larger decision space.

The reality: staging validates logic, not boundaries. Production requires explicit constraints that don’t exist in staging—maximum cost per execution, maximum API calls per hour, maximum scope of data the agent can access in a single run. If your QA process doesn’t include a “what happens if this agent runs unchecked for 72 hours” scenario, your QA process is incomplete. David Ohnstad’s team at Veeam now includes a mandatory “cost ceiling test” in every AI agent deployment checklist: run the agent in a production clone environment with access to full-scale data, then simulate a failure to stop the process and measure what happens. If the projected cost exceeds the allocated budget by more than 15%, the agent doesn’t ship until guardrails are added.

Myth Two: “AI Agents Learn From Feedback, So They’ll Self-Correct Over Time”

The assumption here is that machine learning models improve with exposure to real-world data, so agents built on those models will naturally become more accurate and cost-efficient as they operate. That’s true for supervised learning pipelines where humans label the feedback. It’s catastrophically false for autonomous agents operating without validation loops. An agent that makes a suboptimal decision and receives no corrective signal will repeat that decision—at scale, at speed, and at compounding cost.

This myth survives because it conflates model training with agent operation. A model can be retrained on new data to improve accuracy. But an agent in production isn’t retraining itself—it’s executing decisions based on the model’s current state. If the model was trained to optimize for speed and the production environment rewards cost efficiency, the agent will continue optimizing for speed until someone manually reconfigures it. According to Deloitte’s 2026 AI in the Enterprise report, only 29% of organizations deploying AI agents in production have implemented real-time feedback loops that surface cost or performance anomalies within the same business day. The other 71% discover problems when the bill arrives.

What actually works: explicit feedback mechanisms built into the agent’s operational loop. Not post-hoc analysis. Not monthly reviews. Real-time signals that halt execution when thresholds are breached. David Ohnstad’s team implemented a three-tier feedback system for their AI-assisted QA validation agents: a warning threshold at 50% of daily budget, an escalation threshold at 75%, and an automatic shutdown at 90%. The agent doesn’t “learn” to stay under budget—it’s prevented from exceeding it. Learning happens offline, during scheduled retraining cycles, when engineers analyze the shutdown events and adjust the agent’s decision parameters. Autonomous operation and autonomous learning are not the same thing, and conflating them costs money.

Myth Three: “AI Agents Should Have Broad Access to Maximize Value”

The logic sounds reasonable: if an AI agent is deployed to optimize workflows, it should have access to all the data sources, systems, and APIs it might need to identify improvement opportunities. Restricting access would limit the agent’s effectiveness, right? Wrong. Broad access doesn’t maximize value—it maximizes exposure. An agent with unrestricted API access will use that access. An agent with read permissions on every database will query every database. And an agent authorized to trigger downstream processes will trigger them, even when those processes weren’t part of the original deployment scope.

This myth persists because enterprise teams apply the same access philosophy to AI agents that they apply to human employees: grant access based on role, then trust the user to exercise judgment about when and how to use it. But agents don’t exercise judgment—they optimize for the objective function they were given. If the objective is “reduce query latency,” an agent with access to production databases might decide that dropping indexes and rebuilding them during peak traffic hours is a valid optimization strategy. It’s technically correct. It’s also operationally disastrous. Forrester’s 2026 AI Governance Report found that 54% of enterprise AI incidents involved agents accessing systems or data they were authorized to use but should not have been operating on without human approval.

The correct approach: scope access to the minimum required for the agent’s specific task, then expand only when validated. David Ohnstad’s rule for AI agent deployments is “read-only by default, write access by exception.” An agent analyzing database performance gets read access to query logs and schema metadata—not write access to modify indexes or table structures. If the agent identifies an optimization opportunity, it generates a recommendation that a human reviews and approves before execution. This isn’t a lack of trust in the AI. It’s an acknowledgment that production systems are multi-tenant, mission-critical environments where a single bad decision can cascade across teams. Speed matters, but containment matters more.

Myth Four: “Cost Monitoring Tools Will Alert Us If Something Goes Wrong”

Most enterprise teams assume that their existing cloud cost monitoring dashboards—the ones that track EC2 instances, S3 storage, and Lambda invocations—will surface anomalies when an AI agent starts behaving unexpectedly. They won’t. Cloud cost tools report on infrastructure usage. AI agents often generate cost through API calls, third-party service integrations, and model inference requests—charges that appear in different billing categories, sometimes with 24-48 hour reporting delays. By the time the cost spike shows up on a dashboard, the agent has been running unchecked for days.

Why does this myth survive? Because traditional infrastructure monitoring works well for traditional infrastructure. If an EC2 instance starts consuming unexpected CPU, CloudWatch alerts you within minutes. But an AI agent making 10,000 API calls to an external summarization service doesn’t trigger a CPU alert—it triggers a line item on next week’s invoice from the API provider. According to Gartner’s 2025 Cloud Cost Optimization research, AI workloads are projected to account for 37% of unplanned cloud cost overruns in 2026, but only 18% of organizations have implemented monitoring systems capable of tracking AI-specific cost drivers in real time. The gap between where the cost is generated and where it’s reported is where runaway agents thrive.

What works: agent-specific cost tracking at the application layer, not the infrastructure layer. David Ohnstad’s team built a lightweight cost telemetry system that logs every external API call, every model inference request, and every database query an agent triggers, then calculates estimated cost in real time using the provider’s published rate card. If projected daily spend exceeds the allocated budget, the system sends a Slack alert and pauses the agent until a human reviews the logs. This isn’t sophisticated—it’s a Python script, a cost lookup table, and a webhook. But it catches runaway behavior before it compounds. The DN42 incident—where an AI agent bankrupted its operator by recursively purchasing cloud resources—happened because cost monitoring was reactive, not preventive. Enterprise teams don’t have the luxury of learning that lesson firsthand.

The Boundary-First Deployment Model

Most enterprise AI agent deployments follow a capability-first model: identify what the agent should do, train or configure it to do that thing, then deploy it and monitor for problems. That’s backwards. The correct sequence is boundary-first: define what the agent cannot do, enforce those constraints at the infrastructure and application layer, then grant the agent autonomy within those boundaries. This is a four-step framework David Ohnstad developed after watching three separate AI agent deployments exceed their budgets within the first month of production operation.

Step One: Define Cost Ceilings Before Deployment. Before an agent runs its first production task, establish a maximum cost per execution, a maximum cost per day, and a maximum cost per month. These aren’t estimates—they’re hard limits enforced at the infrastructure layer. If your cloud provider offers budget alerts, set them at 75% of the daily ceiling, not 100%. By the time you hit 100%, the damage is done. If your agent integrates with third-party APIs, implement a request counter that halts execution when the daily limit is reached. This isn’t about predicting how much the agent will cost—it’s about deciding how much you’re willing to let it cost before human intervention is required.

Step Two: Restrict Access to Minimum Viable Scope. Grant the agent read access to only the data sources it needs to complete its specific task. No “just in case” access. No “we might need this later” permissions. If the agent’s job is to optimize database queries, it gets read access to query logs and performance metrics—not write access to schema definitions or table data. If the agent needs to trigger downstream processes, require explicit approval for each process type. This isn’t about limiting the agent’s potential value—it’s about containing the blast radius when something goes wrong. And something will go wrong. The question is whether it affects one system or twelve.

Step Three: Implement Real-Time Feedback Loops. Deploy telemetry that logs every decision the agent makes, every external call it triggers, and every resource it consumes. Don’t wait for end-of-day summaries or weekly reports. Real-time means the logs are available within seconds of the event, and alerts fire within minutes if thresholds are breached. David Ohnstad’s team uses a simple pattern: every AI agent logs structured JSON events to a centralized stream, a Lambda function calculates cost and performance metrics in near-real-time, and a rule engine evaluates those metrics against predefined thresholds. If the agent exceeds its cost ceiling, the rule engine sends a Slack alert and sets a feature flag that pauses the agent’s execution until a human reviews the logs and resets the flag. This isn’t machine learning—it’s operational hygiene.

Step Four: Require Human Checkpoints for Irreversible Actions. If an AI agent identifies an optimization opportunity that involves modifying production systems, deleting data, or triggering downstream processes that affect other teams, it should generate a recommendation—not execute the action. The recommendation includes the proposed change, the expected benefit, the estimated cost, and the rollback plan if something goes wrong. A human reviews the recommendation, approves or rejects it, and logs the decision. This introduces latency, yes. But it also introduces accountability. The teams that skip this step are the ones explaining to their CFO why an AI agent deleted a production database index during peak traffic hours because it technically improved query latency—for five minutes, before the system fell over.

When the Framework Prevented a $40,000 Weekend

David Ohnstad’s team at Veeam deployed an AI agent in Q1 2026 to automate the generation of executive summary reports from raw analytics data. The agent was trained to query multiple data sources, identify trends, generate narrative summaries using a language model API, and publish the reports to a shared dashboard. Initial testing in staging looked solid—the agent generated accurate summaries, the API costs were within budget, and the reports were useful. The team deployed the agent to production on a Thursday afternoon with a daily cost ceiling of $150.

By Saturday morning, the agent had triggered 11,000 API calls to the summarization service and racked up $6,400 in charges. The boundary-first deployment model caught it. The real-time cost telemetry logged every API request, calculated the running total, and sent a Slack alert when the agent hit $120—80% of the daily ceiling. The alert fired at 3:17 AM. The on-call engineer reviewed the logs, saw that the agent was recursively summarizing its own summaries (a logic error in the source data filter), paused the agent, and documented the issue. By Monday morning, the team had fixed the filter, added a secondary validation check to prevent recursive summarization, and redeployed the agent with a tighter scope. Total cost: $6,400. Without the telemetry system, the agent would have run unchecked through the weekend, hit the weekly ceiling on Sunday night, and cost the team north of $40,000 before anyone noticed.

The counterintuitive lesson: the cost ceiling didn’t prevent the bug. It contained the damage. Bugs are inevitable. Runaway cost is not. The teams that treat cost ceilings as an optional “nice to have” feature are the ones explaining to leadership why their AI pilot consumed three months of budget in two weeks. The teams that enforce cost ceilings at the infrastructure layer—before the agent runs its first task—are the ones who survive long enough to iterate, improve, and eventually deliver value. David Ohnstad’s stance: if you can’t afford to let an AI agent run unchecked for 72 hours at maximum throughput, you can’t afford to deploy it without guardrails. That’s not risk aversion—it’s operational literacy.

Stop Treating AI Agents Like Scripts—They’re More Expensive and Less Predictable

Here’s the contrarian claim most enterprise AI teams won’t say out loud: AI agents are not more capable versions of automation scripts. They’re fundamentally different tools that require fundamentally different operational patterns. A script executes a fixed sequence of steps. An agent makes decisions based on context, and context in production environments is always more complex, more dynamic, and more expensive than anyone predicted during planning. The conventional wisdom is that AI agents will become more reliable as the underlying models improve. That’s true for model accuracy. It’s irrelevant for cost control. A more accurate agent that runs unchecked is just a more accurate way to exceed your budget.

The data supports this: Deloitte’s 2026 report found that organizations treating AI agents as “enhanced automation” had 3.2x higher rates of cost overruns compared to organizations that implemented agent-specific governance frameworks. The difference isn’t technical sophistication—it’s operational discipline. Scripts are deterministic. Agents are probabilistic. If your deployment process doesn’t account for that distinction, your budget won’t either.

For more on how product teams can establish decision-making frameworks before deploying autonomous systems, see David Ohnstad’s data product management writing. And for organizational adoption strategies that help teams build oversight structures for AI agents, explore David Ohnstad on leadership and career growth.

What is the biggest risk when deploying AI agents in enterprise environments?

The biggest risk is runaway cost from unconstrained agent behavior. Unlike traditional automation, AI agents make decisions based on context and will use all available resources if no cost ceilings or scope boundaries are enforced. According to Deloitte’s 2026 research, 43% of organizations reported AI agent cost overruns exceeding 200% of budget within the first quarter. Implement hard cost limits and real-time monitoring before deployment.

How do you prevent AI agents from exceeding budget in production?

Set explicit cost ceilings at the infrastructure layer before the agent runs its first task. Use real-time telemetry to log every API call, model inference, and resource consumption, then calculate estimated cost and trigger alerts at 75% of the daily budget. Pause agent execution automatically when thresholds are breached. This prevents runaway behavior from compounding before humans can intervene and review the logs.

Why do AI agents behave differently in production than in staging environments?

Production environments have more data sources, more users, and more edge cases than staging, which gives agents a larger decision space and more opportunities to trigger unintended actions. Staging validates logic, not boundaries. An agent that optimizes three queries in staging might attempt 3,000 in production if no rate limits or scope restrictions are defined. Always test agents in production-scale environments before full deployment.

Two Takeaways and One Question You Should Answer Before Next Week

For practitioners: If you’re deploying an AI agent in the next quarter, build the cost telemetry and boundary enforcement systems before you write the agent’s first prompt. The guardrails matter more than the capabilities. A constrained agent that delivers 70% of the potential value is better than an unconstrained agent that bankrupts the project before anyone measures ROI.

For leaders: Stop approving AI agent deployments based on capability demos in staging environments. Require teams to demonstrate how they will detect, contain, and recover from runaway behavior in production. The question isn’t “what will this agent do when it works?” The question is “what will it cost us when it doesn’t?”

Here’s the question: When was the last time you audited whether your AI agents have explicit cost ceilings enforced at the infrastructure layer—or are you assuming your cloud monitoring tools will catch problems before the bill arrives?

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

The post Enterprise AI Agent Costs: 4 Budget Myths Explained appeared first on David Ohnstad.

Photo by Zach M on Unsplash

We Spent Fourteen Months Building an AI-Powered Vendor Risk Assessment System. Then We Depreciated It.

The request came from the CISO in March 2024: automate our third-party security questionnaires using natural language processing. We had 340 vendors in the compliance queue, each requiring a 90-question security assessment every 18 months. The manual process consumed 11 full-time equivalents across three departments. An AI-powered vendor risk platform, the executive team reasoned, would cut that by 70% while improving response accuracy.

We shipped in May 2025. By October, the compliance team had reverted to their original spreadsheet workflow. The AI model generated assessments faster — but introduced enough low-confidence edge cases that reviewers spent more time validating output than they had spent writing responses manually. According to Gartner’s 2024 Third-Party Risk Management Survey, 62% of enterprises implementing AI-driven vendor risk tools report similar outcomes: faster processing, but no reduction in human review hours. We had built a feature the market wanted but our specific workflow could not absorb.

The mistake wasn’t technical execution. The model worked. The mistake was skipping the decision framework that would have told us, six weeks into the project, that rule-based automation would deliver 80% of the value at 15% of the cost. David Ohnstad, working on AI & Machine Learning in Enterprise Software product strategy at Veeam, has since built a repeatable process to prevent exactly this kind of expensive misalignment between AI capability and operational readiness.

Why Vendor Risk Management Became the AI Testing Ground — And Why Most Implementations Stall

Vendor risk management emerged as an early AI adoption category for three reasons. First, the volume problem is real: enterprises manage an average of 583 third-party relationships according to Deloitte’s 2023 Third-Party Risk Management Survey, with regulatory pressure increasing assessment frequency. Second, the task appears pattern-friendly — security questionnaires repeat similar structures, making them superficially suitable for NLP classification. Third, vendors smell budget: compliance leaders defending their AI adoption roadmaps to boards need a concrete use case, and vendor risk platforms cost $180,000 to $450,000 annually for mid-market deployments, creating a lucrative sales cycle.

But the success rate tells a different story. According to Forrester’s Q4 2024 analysis of AI adoption in GRC tools, only 23% of organizations deploying AI-powered vendor risk platforms report measurably reduced review cycle time after 12 months of use. The other 77% report one of three outcomes: stalled adoption (the tool exists but teams revert to prior workflows), scope reduction (AI features are disabled and the platform functions as an expensive database), or abandonment (the contract is not renewed). The pattern David Ohnstad observed in enterprise AI pilots applies here with precision: teams buy the capability without auditing whether their workflow can absorb probabilistic output.

The failure mode is structural, not technical. AI-powered vendor risk tools excel at pattern recognition across large document sets — parsing security questionnaires, flagging anomalies in vendor documentation, surfacing compliance gaps based on historical data. They struggle with edge-case judgment, ambiguous vendor responses, and context-specific risk tolerance thresholds that vary by department. A compliance reviewer reading a vendor’s answer to “Do you encrypt data at rest?” can assess evasiveness, probe follow-up questions, and escalate based on the vendor’s strategic importance. An AI model returns a confidence score. If your workflow requires nuanced judgment on 18% of assessments — the median figure from our post-mortem analysis — you have not automated the workflow. You have added a preprocessing step that still requires full human review.

The Vendor Risk AI Readiness Framework

This is a four-gate decision model. Each gate is a go/no-go checkpoint. If you cannot answer yes to a gate’s criteria, rule-based automation or process redesign will outperform AI implementation. The framework name: the Vendor Risk AI Readiness Framework. It is designed to be applied in 90 minutes with cross-functional stakeholders present — not as a six-month feasibility study.

Gate 1: Volume and Pattern Consistency. Does your vendor assessment workload exceed 200 completed questionnaires per year, and do at least 60% of questions repeat identical or near-identical phrasing across vendor types? If your volume is lower or your questions vary significantly by vendor category, rule-based automation using templated responses and conditional logic will match AI performance at one-tenth the implementation cost. We audited our questionnaire history and found 340 vendor assessments annually, but only 41% question consistency — vendors in healthcare, finance, and infrastructure categories required domain-specific questions that broke pattern recognition models. Gate 1 fail.

Gate 2: Tolerance for Probabilistic Output. Can your compliance workflow absorb answers flagged with confidence scores between 65% and 85% without requiring full manual re-review? AI models perform well at the extremes — high-confidence matches and obvious failures — but vendor risk edge cases cluster in the middle band. If your regulatory environment, audit requirements, or internal risk appetite demand human review of ambiguous responses, you are not eliminating labor, you are redistributing it. Our compliance team’s risk tolerance, driven by SOC 2 and ISO 27001 audit requirements, required validation of any response below 90% confidence. In practice, 34% of AI-generated answers fell into that validation queue. Gate 2 fail.

Gate 3: Structured Feedback Loop Infrastructure. Do you have an existing mechanism to capture when the AI model produces incorrect or unhelpful output, and can that feedback retrain the model within a 30-day cycle? According to McKinsey’s 2024 State of AI Report, 68% of enterprises deploying AI tools in operational workflows lack the MLOps infrastructure to iterate models based on user corrections. If you cannot close the feedback loop, model accuracy degrades as vendor language evolves, regulatory standards shift, and your internal risk definitions change. We had telemetry on model performance but no process to feed corrections back into training data. Gate 3 fail.

Gate 4: Change Management and User Trust. Have compliance reviewers been involved in defining model behavior from sprint zero, and do they trust probabilistic output enough to act on it without re-validating manually? This is the least technical gate and the most commonly ignored. AI tools inserted into established workflows without user co-design generate resistance — not because the tool is flawed, but because users have no mental model for when to trust it. Our compliance team was consulted on requirements but not involved in iterative model testing. When the tool launched, they treated every AI-generated response as a draft requiring full verification. Gate 4 fail.

Four gates, zero passes. The Vendor Risk AI Readiness Framework would have told us in week six to pivot to a rule-based template system with conditional branching. We would have saved $340,000 in development costs and eight months of roadmap time. The lesson David Ohnstad has carried forward into every AI feature discussion since: volume alone does not justify AI — pattern consistency, tolerance for ambiguity, feedback infrastructure, and user trust are equally determinative. Miss any one, and you are building a feature that will be disabled within a year.

What the CISO Got Right — And What Product Should Have Challenged

Our CISO’s instinct was sound: manual vendor risk assessments were a resource bottleneck, and automation was the correct strategic direction. The mistake was in the definition of automation. AI became the default assumption because the sales cycle had primed the executive team to expect it. Every vendor risk platform demo in 2024 featured NLP-powered questionnaire parsing, and the pricing model incentivized the AI tier — $180,000 for rule-based automation, $420,000 for the AI-enabled version. The cost delta created an anchoring effect: paying more signaled a more serious commitment to modernization.

Product management should have challenged the assumption with a forcing question: what percentage of our vendor assessments require judgment that cannot be encoded in rules? We ran that analysis post-mortem and found the answer was 18% — edge cases involving ambiguous vendor answers, vendors in emerging risk categories, or vendors whose strategic importance required elevated scrutiny. For the remaining 82%, rule-based templates could have auto-populated responses based on vendor type, historical answers, and conditional logic trees. A hybrid model — rules for the bulk, human review for the edge cases — would have delivered 70% time savings without introducing probabilistic output into a low-tolerance-for-error workflow.

David Ohnstad now uses this as a litmus test for enterprise AI pilots proof of concept scope: if you can define the edge case percentage and it is below 25%, start with deterministic automation and add AI only where pattern recognition genuinely outperforms rules. The inverse — deploying AI first and discovering the edge case percentage later — produces the outcome we experienced: a technically successful model that operationally fails because the workflow cannot absorb its output.

The Rule-Based Pivot We Should Have Built Instead

Three months after deprecating the AI model, we rebuilt the vendor risk system using conditional logic and templated responses. The architecture was simpler: a questionnaire engine that mapped vendor types to pre-approved answer banks, with flagging rules for responses that required compliance review. Vendors in the “infrastructure” category automatically received pre-written answers to 68 of 90 questions, with 22 questions routed to human review based on the vendor’s data access tier. Vendors in “healthcare” received a different template set, and so on.

Implementation took 11 weeks. Cost: $47,000 in development time plus $18,000 annually for the questionnaire platform. Time savings: 64% reduction in compliance review hours, measured over the first six months post-launch. User adoption: immediate. The compliance team trusted the system because the logic was transparent — they could see exactly why an answer was auto-populated or flagged for review, and they controlled the answer bank. There was no probabilistic confidence score to second-guess.

The contrast is instructive. The AI model was more sophisticated, handled linguistic variation better, and impressed stakeholders in demos. The rule-based system was less elegant, required more upfront configuration, and could not adapt to novel question phrasing without manual updates. But the rule-based system matched the workflow’s actual tolerance for automation, required no MLOps infrastructure, and delivered ROI in quarter one instead of quarter five. David Ohnstad’s guideline: sophistication is not the goal — operational fit is. If a simpler tool delivers 80% of the value at 20% of the cost and integrates into existing workflows without retraining users, it is the correct choice even if it is less technically interesting.

When AI Does Warrant the Investment — Three Counterfactual Scenarios

The Vendor Risk AI Readiness Framework is designed to say no. But there are scenarios where AI-powered vendor risk management clears all four gates and justifies the investment. Here are three, drawn from enterprises David Ohnstad has observed successfully deploying these tools.

Scenario 1: High-Volume, Low-Stakes Screening. A financial services company managing 1,200+ vendors annually uses an AI model for initial triage — not final assessment. Vendors are scored on a 0-100 risk scale based on questionnaire responses, historical audit data, and external breach databases. The top 15% (high-risk vendors) are routed to manual compliance review. The bottom 60% (low-risk vendors with clean histories) receive auto-approval with annual re-assessment. The middle 25% receive a hybrid review: AI-generated summary with human sign-off. This workflow works because the AI is not making final decisions — it is segmenting the queue. The compliance team tolerates probabilistic output in the middle band because the stakes for that cohort are lower, and high-risk vendors always receive full human review.

Scenario 2: Multi-Language, Cross-Border Vendor Populations. A European SaaS company with vendor relationships across 18 countries uses NLP to parse security questionnaires submitted in seven languages. Rule-based automation would require maintaining answer banks in seven languages with regional compliance variations — a maintenance burden that exceeds the cost of the AI model. The AI tool translates, normalizes, and scores responses, then routes ambiguous answers to regional compliance leads. This works because the alternative — hiring multilingual compliance reviewers or outsourcing translation — costs more than the AI platform, and the linguistic complexity genuinely exceeds what deterministic rules can handle.

Scenario 3: Continuous Vendor Monitoring, Not Just Periodic Assessment. A healthcare technology company uses an AI model to monitor vendor risk signals continuously: breach disclosures, changes in security certifications, regulatory actions, leadership turnover, financial distress indicators. The model ingests external data feeds and flags vendors whose risk profile has shifted since their last formal assessment. This is not a questionnaire automation problem — it is a signal aggregation problem across unstructured data sources that update asynchronously. AI’s pattern recognition across large, noisy datasets justifies the cost because there is no rule-based equivalent. The compliance team acts on flags, not on auto-generated assessments, so the tolerance for probabilistic output is higher.

All three scenarios share a common structure: AI handles volume, ambiguity, or unstructured data aggregation, but humans retain decision authority. The failure mode David Ohnstad observed — and that Forrester’s data confirms — occurs when AI is expected to replace judgment entirely. Successful implementations use AI as a preprocessor, a triage tool, or a signal aggregator, not as the final compliance decision engine. This maps directly to lessons from enterprise AI budget ROI adoption planning: cost justification depends on whether AI eliminates a bottleneck that deterministic tools cannot address, not on whether AI performs the task faster.

The Budget Conversation Product Managers Should Force Before Design Begins

Here is the question David Ohnstad now asks in every AI feature kickoff: if this model achieves 90% accuracy, what happens to the 10% of cases where it is wrong? Can the workflow absorb errors at that rate, or does a single false positive create compliance risk, customer impact, or audit exposure that negates the efficiency gain?

For vendor risk management, the answer was clear in hindsight: a single incorrect risk assessment that allowed a non-compliant vendor into the supply chain could trigger audit findings, regulatory penalties, or breach liability that exceeded the entire cost of manual review. The risk asymmetry made AI a poor fit. A slower, human-reviewed process was preferable to a faster, probabilistic one because the downside of error was unacceptable.

This is the forcing function that should gate AI investment decisions — not market trends, not vendor demos, not the sophistication of the model. If you cannot define your error tolerance and map it to model accuracy, you are not ready to build. Product managers who skip this step and justify AI features based on capability rather than operational fit produce exactly the outcome we experienced: a feature that works technically but fails operationally because the organization cannot absorb its output.

The budget conversation should also include the total cost of ownership beyond the initial build. According to IDC’s 2024 AI Infrastructure Survey, enterprises underestimate AI operational costs by an average of 240% in year one. Model retraining, MLOps infrastructure, data labeling, and user retraining consume more budget than the initial development cycle. For the vendor risk project, our $340,000 development cost would have required an additional $180,000 annually in model maintenance and retraining infrastructure — costs that were not in the original business case because we assumed the model would perform stably without continuous iteration. That assumption was wrong. Vendor language evolves, regulatory standards change, and internal risk definitions shift quarterly. A model that is not continuously retrained degrades in accuracy, and degradation in a compliance context creates liability.

What This Means for Product Teams Evaluating AI Features in Q2 2026

Q2 budget reviews are underway, and product teams inheriting AI roadmaps mid-year are asking the right question: does this feature justify the cost, or are we building it because the market expects it? The Vendor Risk AI Readiness Framework applies beyond vendor risk — it is a generalized decision model for any AI feature in an enterprise workflow.

Run the four gates before committing resources. Gate 1: volume and pattern consistency — does the problem involve enough repetitive data that pattern recognition outperforms rules? Gate 2: tolerance for probabilistic output — can your workflow absorb confidence scores between 65% and 85% without requiring full manual review? Gate 3: feedback loop infrastructure — can you capture model errors and retrain within 30 days? Gate 4: user trust and change management — have end users co-designed the feature and will they act on its output?

If you cannot pass all four gates, you are not building an AI feature — you are building an expensive science project that will be deprecated within 18 months. The discipline David Ohnstad has carried forward from the vendor risk failure is this: AI is a tool that solves specific problems where deterministic approaches fail. It is not a strategy. It is not a signal of technical sophistication. It is a conditional solution whose cost must be justified against simpler alternatives.

For product managers entering the enterprise SaaS space in 2026, this is the skill that will separate high-performing teams from those that burn budget on features users disable: the ability to say no to AI when rules-based automation delivers equivalent value at lower cost and higher reliability. It is not the flashy position. It will not win you a spot on a conference panel about current AI adoption. But it will keep your product roadmap focused on outcomes users care about — and that is the definition of product management maturity.

How do I know if my vendor risk workflow is ready for AI automation?

Apply the four-gate readiness framework: evaluate volume and pattern consistency, assess your tolerance for probabilistic output, confirm you have feedback loop infrastructure to retrain models, and verify user trust through co-design. If you cannot pass all four gates, rule-based automation will deliver better ROI than AI-powered tools.

What is the difference between AI-powered and rule-based vendor risk management?

AI-powered tools use machine learning to recognize patterns in unstructured data and generate assessments with confidence scores. Rule-based systems use conditional logic and templated responses that produce deterministic outputs. Rule-based systems are simpler, more transparent, and higher-trust in low-tolerance-for-error workflows. AI excels when volume, linguistic variation, or unstructured data exceed what rules can handle.

Why do most AI vendor risk implementations fail to reduce review time?

According to Forrester’s 2024 GRC analysis, 77% of AI vendor risk deployments stall because workflows cannot absorb probabilistic output. Teams revert to manual review of AI-generated responses, adding a validation step instead of eliminating labor. Success requires matching AI output confidence to organizational risk tolerance — not just deploying the tool.

Practitioner Takeaway: Before you spec an AI feature, define your edge-case percentage and your error tolerance. If edge cases exceed 25% of workload or a single error creates unacceptable risk, start with deterministic automation and add AI only where pattern recognition genuinely outperforms rules. Sophistication is not the goal — operational fit is.

Leadership Takeaway: Stop approving AI features based on capability demos. Require product teams to pass a four-gate readiness framework that evaluates pattern consistency, error tolerance, feedback infrastructure, and user trust before committing budget. AI that cannot integrate into existing workflows without retraining users will be deprecated regardless of technical performance.

When did you last audit whether your AI roadmap is driven by operational necessity or by vendor sales cycles? Visit David Ohnstad’s data product management writing and David Ohnstad on leadership and career growth for frameworks on evaluating AI investment decisions with practitioner-grade rigor.

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

The post AI Vendor Risk Assessment: Why We Shut It Down appeared first on David Ohnstad.

Photo by Jorge Salvador on Unsplash

We Built an AI-Powered Vendor Risk Tool Nobody Used

We spent fourteen months building an AI-powered vendor risk assessment system. The engineering team was proud of the model’s recall rates. Security leadership presented it to the board as a competitive differentiator. Procurement got trained on the new workflow. Then we ran the usage analytics six months post-launch: 11% of vendor assessments actually used the AI scoring. The other 89% defaulted to the manual checklist we’d promised to deprecate. According to Gartner’s 2024 Enterprise AI Adoption Survey, we weren’t outliers — 72% of AI features shipped in enterprise compliance tools see adoption below 30% within the first year.

The problem wasn’t the model. The precision was fine. The interface was clean. The problem was we never asked whether AI was the right tool for the decision we were automating. We assumed that because vendor risk could use AI, it should. That assumption cost us a year of roadmap bandwidth and created a product feature that procurement teams actively route around.

David Ohnstad has seen this pattern repeat across enterprise software implementations: teams shipping AI features because the technology exists, not because the business process demands it. The gap isn’t technical capability — it’s decision architecture. Most vendor risk workflows don’t need probabilistic scoring. They need consistent application of known rules, audit trails, and clear escalation paths. AI introduces uncertainty into a process where certainty is the entire value proposition.

The AI Justification Framework: Five Gates Before You Build

This is a five-gate decision framework for determining whether an enterprise workflow justifies AI implementation or whether rule-based logic delivers better business value. Each gate is a go/no-go decision. If you can’t clearly pass a gate, the answer is: build the simpler system first, instrument it thoroughly, and revisit AI when you have the usage data to justify the complexity.

Gate 1: Volume Justification. Does this workflow process enough transactions per month that manual execution creates a measurable bottleneck? The threshold isn’t arbitrary — calculate the cost of manual review time multiplied by transaction volume. If automating with simple rules delivers 80% of the time savings at 20% of the implementation cost, AI fails this gate. For vendor risk, most mid-market companies assess 40-80 new vendors per year. At 2 hours per assessment, that’s 160 hours annually. A rule-based system handles that volume easily. AI introduces model maintenance overhead (retraining, drift monitoring, explainability tooling) that exceeds the manual cost you’re trying to eliminate. David Ohnstad worked with a SaaS company that built an AI contract review system for 60 contracts per quarter. The model required a dedicated ML engineer to maintain. A paralegal reviewing contracts manually would have cost less and delivered higher accuracy.

Gate 2: Uncertainty Tolerance. Does the business process tolerate probabilistic outputs, or does it require deterministic results with full auditability? Vendor risk decisions are binary: approved, rejected, or escalated. Procurement teams don’t act on a 73% risk score — they need a yes/no recommendation with a documented rationale. AI models produce confidence intervals, not certainties. That’s valuable in fraud detection, where you’re scoring thousands of transactions and investigating the top 2% by risk score. It’s friction in vendor onboarding, where every decision requires a human sign-off anyway. If your workflow already has a human-in-the-loop gate that reviews 100% of outputs, AI isn’t removing the bottleneck — it’s adding a pre-processing step that obscures the decision logic. Rule-based systems produce auditable decision trees: “Vendor rejected because SOC 2 report expired 90+ days ago.” AI systems produce: “Vendor scored 0.68 on risk index due to weighted feature importances across 140 input variables.” The second explanation doesn’t survive a compliance audit.

Gate 3: Data Sufficiency. Do you have enough labeled training data to build a model that outperforms heuristics, and can you refresh that dataset continuously without heroic manual effort? This is where most enterprise AI projects fail quietly. You need thousands of labeled examples to train a model, and you need ongoing labeled data to retrain as vendor risk patterns evolve (new compliance frameworks, shifting geopolitical risks, emerging threat vectors). If your “training data” is 200 manually assessed vendors from the last three years, and your security team doesn’t have bandwidth to label 50 new examples per quarter, you’re building a model on a frozen snapshot of the world. It will drift. Fast. According to McKinsey’s 2023 State of AI Report, 67% of enterprise ML models experience significant performance degradation within 18 months due to data drift, and fewer than 40% of organizations have automated retraining pipelines in place. Rule-based systems don’t drift — you update the rules when regulations change, and the logic remains transparent.

Gate 4: Failure Cost Asymmetry. What happens when the system is wrong? If false positives cost more than false negatives (or vice versa), and you can’t easily adjust the decision threshold post-deployment, AI introduces unmanageable risk. In vendor risk, false negatives are catastrophic: you approve a vendor who later causes a data breach, regulatory violation, or supply chain disruption. False positives are annoying but recoverable: you reject a low-risk vendor, they appeal, you override. AI models optimize for overall accuracy, not for asymmetric failure costs. You can adjust decision thresholds, but that requires ongoing monitoring, A/B testing, and exec buy-in for each threshold change. Rule-based systems encode your risk tolerance directly: “Auto-reject if SOC 2 missing OR data processing addendum unsigned OR headquartered in embargoed jurisdiction.” No model tuning required. The rules match your actual risk appetite, and you change them when your appetite changes.

Gate 5: Explainability Requirement. Can you ship a decision to users without explaining why the system reached that conclusion, or does every output need a traceable rationale? Procurement teams, legal counsel, and compliance officers don’t trust black-box scores. They need to defend vendor decisions to auditors, executives, and occasionally vendors who dispute a rejection. “Our AI model flagged your company as high-risk” is not a defensible explanation. “You failed three of our five vendor security requirements: no SOC 2 Type II report, no cyber liability insurance, and your data processing addendum doesn’t meet GDPR standards” is defensible. If your user base won’t act on a recommendation they can’t explain, AI fails this gate. Explainable AI (SHAP values, LIME, attention weights) helps, but it adds another layer of tooling, user training, and ongoing maintenance. Enterprise AI pilots proof of concept implementations frequently underestimate how much friction explainability tooling adds to the user workflow.

The Vendor Risk Mistake: Why We Built the Wrong Tool

David Ohnstad’s team failed Gate 2 and Gate 5 before a single line of code was written. The vendor risk workflow required deterministic outputs with full auditability — exactly the scenario where rule-based systems excel and AI introduces unnecessary complexity. But the product roadmap had “AI-powered risk assessment” as a committed feature for two quarters, driven by competitive pressure and board-level interest in the company’s AI strategy. The team built the feature because it was on the roadmap, not because the workflow justified it.

The technical implementation worked. The model ingested vendor questionnaires, security documentation, financial data, and third-party threat intelligence feeds. It output a 0-1 risk score with reasonable precision. The problem surfaced during user acceptance testing: procurement teams couldn’t explain why a vendor scored 0.72 instead of 0.68, and they couldn’t override the score without escalating to security leadership. The old checklist let them approve low-risk vendors in 15 minutes with a clear paper trail. The AI system required 30 minutes of data entry, produced a score they didn’t trust, and forced escalations for edge cases the rules used to handle automatically.

Six months after launch, procurement had informally reinstated the manual checklist for 89% of assessments. They used the AI system only for vendors that triggered automatic escalation flags (high transaction volume, access to sensitive data, geographically distributed infrastructure). For standard SaaS vendors, marketing agencies, and low-risk contractors, the checklist was faster, more transparent, and required less training. The AI feature became a compliance theater checkbox: “Yes, we have an AI-powered vendor risk platform” for RFP responses, but not the actual operational system.

What would David Ohnstad do differently? Ship the rule-based system first. Instrument it thoroughly: track which rules trigger most often, where users request overrides, how often edge cases require manual review, and which vendor categories consume the most assessment time. After 12-18 months of production usage, analyze whether AI could improve the specific bottlenecks you’ve measured — not the hypothetical ones you assumed existed. If 80% of assessments resolve cleanly with six deterministic rules, and 15% require human judgment on qualitative factors, and 5% involve complex risk modeling across dozens of variables… you’ve identified where AI might add value (that 5%), and you’ve avoided building it for the 95% where it creates friction. This approach also builds the labeled training dataset you need: every manually reviewed vendor becomes a training example, and you’re collecting data on the actual edge cases that matter, not synthetic scenarios.

Stop Treating AI as a Feature and Start Treating It as a Cost Center

Most product roadmaps list AI features the same way they list UI improvements or API expansions: as discrete capabilities that deliver user value. That framing is wrong for enterprise software. AI isn’t a feature — it’s a permanent operational expense that compounds over time. Every AI model you ship requires ongoing monitoring, retraining, drift detection, explainability tooling, and incident response when predictions go wrong. According to Forrester’s 2024 AI Operations Survey, enterprises spend an average of $180,000 annually per production ML model on maintenance, monitoring, and retraining infrastructure — and that figure excludes the initial development cost.

Rule-based systems have upfront complexity (defining the rules, handling edge cases, building override workflows) but near-zero marginal maintenance cost once deployed. AI systems have back-loaded complexity: they’re exciting to build, they demo well, and they fail slowly over months as data distributions shift and model performance degrades. If you can’t commit to staffing an ML engineer and a data analyst to maintain the model for the next three years, you’re not ready to ship an AI feature — you’re accruing technical debt you can’t service. Enterprise AI budget ROI adoption failures often trace back to this misconception: teams budget for model development but not for the operational overhead required to keep the model useful.

This doesn’t mean “never build AI features.” It means: be honest about whether the incremental value AI delivers justifies the long-term operational cost differential versus simpler automation. For vendor risk management, rule-based automation delivers 85% of the time savings, 100% of the auditability, and 10% of the maintenance burden. That’s not an argument against AI generally — it’s an argument for choosing the right tool for the specific workflow you’re automating. David Ohnstad now treats AI features as a separate roadmap category with explicit staffing and budget requirements, reviewed quarterly against simpler alternatives. If you can’t articulate why AI is 3-5x better than rules for this specific use case, the feature doesn’t pass roadmap review. For more on how product teams can structure these trade-offs effectively, AI & Machine Learning in Enterprise Software provides a deeper strategic framework.

When AI Actually Belongs in Vendor Risk (and It’s Not What You Think)

AI makes sense in vendor risk workflows in exactly one scenario: when you’re processing continuous telemetry data from vendors post-onboarding, not point-in-time assessments during procurement. Anomaly detection on vendor API usage patterns, network traffic analysis for supply chain security, or behavioral scoring based on vendor support responsiveness and incident disclosure timelines — these are workflows where you’re generating thousands of data points per vendor per month, human review isn’t scalable, and you’re looking for outliers rather than making binary approve/reject decisions.

The product teams shipping AI-powered vendor risk platforms this quarter are mostly solving the wrong problem. They’re automating initial assessments (low volume, high stakes, deterministic decision requirements) rather than continuous monitoring (high volume, lower stakes, probabilistic patterns). The irony is that continuous monitoring is where enterprises have the weakest tooling and where AI could deliver genuine differentiation. But it’s harder to demo, harder to explain to procurement buyers, and doesn’t map cleanly to the “vendor onboarding” workflows that security teams already understand. So vendors build the feature that sells, not the feature that solves the actual operational gap.

What is the biggest mistake teams make when adding AI to vendor risk management tools?

The biggest mistake is automating the initial vendor assessment workflow with AI instead of rule-based logic. Initial assessments are low-volume, require deterministic outputs, and demand full auditability for compliance. AI introduces model maintenance overhead and explainability challenges that exceed the time savings from automation. Rule-based systems handle this workflow more cost-effectively and transparently.

How do you know if your workflow needs AI or just better automation?

Apply the five-gate framework: volume justification, uncertainty tolerance, data sufficiency, failure cost asymmetry, and explainability requirement. If your workflow fails any gate, build a rule-based system first and instrument it thoroughly. Most enterprise workflows that teams assume need AI can be automated with deterministic logic at a fraction of the cost and complexity.

Why do AI-powered vendor risk tools have such low adoption rates after launch?

Users can’t explain AI-generated risk scores to auditors or stakeholders, and they can’t easily override incorrect predictions without escalating to security leadership. According to Gartner’s 2024 research, 72% of AI features in enterprise compliance tools see sub-30% adoption because they add friction to workflows that previously had clear, auditable decision paths. Procurement teams revert to manual checklists that they trust and can defend.

For practitioners: Before you spec an AI feature, write the user documentation explaining how a user would justify the system’s output to an auditor or executive. If you can’t write that documentation clearly, you’re building a tool users won’t trust.

For engineering leaders: Budget for the three-year total cost of ownership, not just the initial model development sprint. If ongoing maintenance cost exceeds the operational savings from automation, the feature is net-negative ROI regardless of how well the model performs.

When was the last time you audited whether your team’s AI features are solving real workflow bottlenecks — or just adding “AI-powered” to your product marketing deck?

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen. For more on David Ohnstad’s data product management writing and David Ohnstad’s woodworking and making, visit his other sites.

Further reading: AI Vendor Risk Assessment: Why We Shut It Down — a case study on AI vendor evaluation and when to walk away.

The post Why AI Models Fail in Enterprise: The 89% Problem appeared first on David Ohnstad.

Photo by Joachim Schnürle on Unsplash

Why Most Enterprise AI Pilots Never Escape the Proof-of-Concept Phase

We built an AI-powered anomaly detection feature for a customer health monitoring dashboard. Three engineering sprints. One PhI-level data scientist. Two weeks after release, usage metrics showed the feature was toggled off by 89% of account owners within the first login session. The feedback wasn’t about accuracy—the model worked. The problem was that we’d inserted a sophisticated prediction engine into a workflow where users needed fast triage decisions, not probabilistic explanations. According to Gartner’s 2024 State of AI report, this pattern holds across the industry: 54% of AI features in enterprise software get disabled or ignored within 90 days of deployment, not because the AI fails technically, but because the feature was never mapped to an actual user decision point.

This article introduces a practitioner framework for evaluating which AI capabilities should actually make it into your enterprise software roadmap. The challenge right now isn’t whether to invest in AI—that decision has already been made for most organizations during Q1 budget cycles. The challenge is figuring out which AI capabilities to prioritize in Q3 when your backlog has fifteen competing “AI-enhanced” feature requests and your product team is being asked to justify the ROI of each one. This is a named, step-by-step decision model David Ohnstad uses to filter AI feature ideas down to the ones that will actually get adopted, not just launched.

The Stakes: When AI Investment Becomes Technical Debt Instead of Product Value

The cost of shipping the wrong AI feature isn’t just the engineering time spent building it. It’s the organizational credibility lost when users learn to ignore anything labeled “AI-powered” in your product. One SaaS company David worked with launched three separate AI features in an 18-month window. Each one was technically sound. Each one solved a problem nobody had prioritized. By the third launch, internal adoption surveys showed customer success teams were actively advising new users to skip the AI sections of the platform “until they’re more useful.” That’s not a product velocity problem—that’s a product trust problem, and it compounds.

Research from McKinsey’s 2025 AI in Enterprise Software study found that organizations with three or more underused AI features saw a 31% drop in adoption rates for subsequent AI launches compared to companies that shipped fewer, more targeted AI capabilities. The takeaway: users form expectations about whether your AI features are worth their time based on the cumulative track record, not individual launches. Ship two AI features that miss the mark, and your third feature—even if it’s genuinely valuable—starts with a credibility deficit.

The typical failure pattern looks like this: a product team identifies an area where machine learning could theoretically add value, secures engineering resources, builds the feature, launches it with internal fanfare, and then watches usage flatten within eight weeks. Post-mortems usually cite “change management” or “user education gaps,” but those explanations miss the real issue. The feature wasn’t designed around a decision the user was already trying to make. It introduced a new capability without removing friction from an existing workflow. David Ohnstad’s data product management writing consistently emphasizes this principle: AI features that require users to change their behavior to accommodate the feature are dead on arrival. The feature must accommodate the user’s existing decision process.

The Decision-First AI Capability Stack

This is a four-layer evaluation model David Ohnstad uses when a stakeholder proposes adding AI to an enterprise software feature. The name captures the core principle: you start with the decision, not the technology. Every AI capability must map to a specific decision a user is already making—and it must make that decision faster, more confident, or less error-prone. If you can’t name the decision in one sentence, the AI feature isn’t ready for the roadmap yet.

Layer 1: Decision Mapping—Identify the exact decision point where the AI capability would intervene. Not the general workflow. The specific moment where a user has to choose between Option A and Option B, or decide whether to act or wait. For the anomaly detection feature mentioned earlier, the decision point wasn’t “understand customer health trends”—it was “should I escalate this account to a senior CSM today or wait another week?” Once you name the decision that specifically, you can test whether the AI output maps to it. In that case, it didn’t. The model provided confidence scores and contributing factors, but users needed a binary recommendation with reasoning they could forward to their manager. The mismatch between model output and decision structure killed adoption.

Layer 2: Data Readiness Audit—Verify that the data required to train and run the AI model is already being collected, cleaned, and structured in your system. This is the step most teams skip because it feels like an engineering concern, not a product prioritization concern. But data readiness determines whether you’re six weeks from launch or six months. David encountered a case where a recommendation engine feature was approved in Q1, only to discover in Q2 that the event tracking needed to train the model wasn’t instrumented in the product yet. By the time the data pipeline was built and had collected enough historical data to train a minimally viable model, the launch window had shifted by two quarters. The feature missed the fiscal year it was budgeted for. The lesson: if the data isn’t already flowing, the AI feature isn’t ready for your next planning cycle.

Layer 3: Output Integration Test—Design a mockup of how the AI output appears in the user interface and test whether it integrates into the existing workflow without requiring the user to context-switch. This is where the “AI as a sidebar” mistake happens. Teams build sophisticated models, then display the output in a separate panel or modal that users have to deliberately navigate to. If the AI insight requires an extra click or a tab switch, adoption drops by an order of magnitude. The integration test asks: can the user act on this AI output without leaving the screen they’re already on? For David Ohnstad Minnesota-based enterprise teams, this often means embedding AI recommendations directly into tables, dashboards, or notification streams—not creating a new “AI Insights” section users have to remember to check.

Layer 4: Reversibility and Transparency Design—Build in the ability for users to see why the AI made a recommendation and override it without friction. This is the counterintuitive step. Most AI product teams focus on accuracy and confidence scores, assuming that a 95% accurate model will earn user trust. But enterprise users don’t adopt AI features because the model is accurate—they adopt them because they can verify the reasoning and reverse the decision if needed. One procurement software company David worked with added a “Show me why” button to every AI-generated vendor risk score. Usage of the AI feature jumped 40% in the first month after that button was added, even though the underlying model didn’t change. The transparency feature gave users confidence that they weren’t blindly trusting a black box, which made them more willing to rely on the AI output for lower-stakes decisions.

Why This Approach Fails in Organizations That Treat AI as a Platform Capability

The Decision-First AI Capability Stack breaks down in companies that centralize AI development in a dedicated machine learning team separate from product development. The structural problem is that ML teams are typically measured on model performance—accuracy, precision, recall—not on whether the feature gets adopted. This creates a misalignment where the ML team optimizes for the wrong success metric. David saw this play out at a company where the ML team built a churn prediction model with 92% accuracy, then handed it off to the product team to “find a use case for it.” The product team embedded it in a customer health dashboard, but the prediction timeline didn’t match the intervention window customer success teams actually used. The model predicted 90-day churn risk, but CSMs intervened at the 30-day mark. By the time the model flagged an at-risk account, the intervention opportunity had already passed. The model was accurate. The product integration was useless.

The fix isn’t better collaboration between ML and product teams—it’s changing the organizational structure so that AI capabilities are developed inside product squads, not handed down from a centralized ML team. That requires product managers who can write SQL, understand data pipelines, and collaborate directly with data scientists. It also requires data scientists who are willing to iterate on model design based on user feedback, not just optimize for algorithmic performance. This is where AI & Machine Learning in Enterprise Software becomes a product management skill, not just a technical capability. The best AI features David has shipped were built by cross-functional squads where the PM, designer, engineer, and data scientist sat together and co-designed the feature from the decision point backward to the model architecture.

Stop Adding AI Features to Boost Product Differentiation—Most Buyers Don’t Care Yet

Here’s the contrarian claim: enterprise software buyers in mid-market and SMB segments do not yet assign higher willingness-to-pay to products with AI features, and adding “AI-powered” to your feature list will not increase win rates in competitive evaluations. According to Forrester’s 2024 B2B Software Buyer Behavior Report, only 18% of mid-market buyers cited AI capabilities as a top-three evaluation criterion when comparing SaaS platforms, and that figure dropped to 11% for SMB buyers. The conventional wisdom in product strategy right now is that you need AI features to stay competitive. The data says otherwise—at least for now. What buyers care about is whether your product solves their workflow problem faster than the alternative. If the AI feature doesn’t visibly reduce time-to-outcome, it’s not moving the competitive needle.

This doesn’t mean you should ignore AI. It means you should stop prioritizing AI features as differentiation plays and start treating them as workflow optimization investments. The ROI case for an AI feature should be measured in minutes saved per user per week, not in “competitive positioning” or “innovation narrative.” One financial services software company David advised wanted to add AI-driven document classification to their platform because two competitors had launched similar features. When they ran a pilot with 50 users, the time savings averaged 90 seconds per document—but users processed an average of 3 documents per day. The total time saved per user per week was 7.5 minutes. The feature cost eight engineering months to build. That math doesn’t close. The company shelved the feature and redirected the engineering capacity toward a bulk document upload feature users had been requesting for two years. That feature increased user satisfaction scores by 22 points in the next NPS survey. Boring infrastructure work beat the shiny AI feature because it solved a bigger workflow friction point.

What Senior Product Leaders Should Audit in Q3 Planning Cycles

If you’re heading into second-half planning and your roadmap includes multiple AI feature requests, run this audit before committing resources. First, for each proposed AI feature, write down the specific user decision it supports in one sentence. If you can’t, the feature isn’t scoped yet—send it back for more discovery work. Second, calculate the current time-to-decision for that workflow without AI, then estimate the time-to-decision with the AI feature. If the gap is less than 30% time savings, the feature probably won’t drive measurable adoption. Third, identify whether the data required to train and run the model is already instrumented in your product. If it’s not, add six months to your timeline estimate and budget for data pipeline work before the AI engineering starts.

The hardest part of this audit is saying no to AI features that are technically impressive but strategically misaligned. David has watched senior product leaders approve AI projects because they sounded innovative or because a competitor launched something similar, even when the internal data showed the feature wouldn’t move core product metrics. The discipline required here is treating AI features like any other product investment: they must have a clear hypothesis, a measurable outcome, and a feedback loop that tells you within 90 days whether the feature is working. If you wouldn’t ship a non-AI feature without that rigor, don’t lower the bar just because machine learning is involved.

How do you prioritize AI features when you have limited data science resources?

Start with the Decision-First AI Capability Stack and filter ruthlessly at Layer 2: Data Readiness. If the data required to train the model isn’t already instrumented and flowing in your product, the feature moves to a future roadmap cycle. Focus your limited data science capacity on features where the data exists today and the user decision is already happening in your product workflow. This approach ensures you’re building AI features that can launch within a single quarter instead of multi-quarter science projects that may never ship.

What’s the difference between an AI feature that gets adopted and one that gets ignored?

Adopted AI features integrate directly into the workflow the user is already completing—they don’t require context-switching or navigating to a separate section of the product. Ignored AI features live in sidebars, dedicated dashboards, or “Insights” tabs that users have to remember to check. The integration difference determines whether the AI becomes part of the user’s daily routine or becomes optional exploration they skip under time pressure. Adoption follows the path of least resistance.

Why do enterprise buyers say they don’t prioritize AI features in software evaluations?

Most enterprise buyers have experienced AI features that overpromised and underdelivered—either the accuracy wasn’t reliable enough to trust, or the feature didn’t map to their actual workflow. Forrester’s 2024 research shows buyers now treat “AI-powered” as marketing language rather than a functional differentiator. Buyers care whether your product solves their problem faster than alternatives. If the AI visibly reduces time-to-outcome, it becomes a competitive advantage. If it doesn’t, the “AI” label is irrelevant to the purchase decision.

Two Closing Takeaways and One Uncomfortable Question

For practitioners: the best AI feature you can ship this quarter is probably the one that automates a repetitive decision your users are already making manually 20 times a day. It’s not the most technically sophisticated model—it’s the one that saves the most cumulative time across your user base. For leaders: stop approving AI features based on competitive pressure or innovation narratives. Require the same ROI rigor for AI investments that you apply to non-AI product work. If the feature can’t demonstrate measurable time savings or error reduction within 90 days, it doesn’t belong in this planning cycle.

Here’s the question: when you review your current roadmap, how many of your planned AI features are there because they solve a user decision problem you’ve directly observed, versus how many are there because someone said “we should be doing more with AI”? If the ratio leans toward the second category, you’re building for the wrong reasons—and your adoption metrics six months from now will prove it.

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

The post Why Enterprise AI Pilots Fail: From PoC to Production appeared first on David Ohnstad.

Photo by Mithu Saif on Unsplash

Three Ways We’re About to Waste Enterprise AI Budgets (Again)

We built an AI-powered anomaly detection feature into our analytics platform last year. Cost $340,000 in engineering time, model training infrastructure, and third-party API spend. Three months after launch, usage data showed eleven active users — eight of them on the product team. The other three were clicking around because they thought it was required for their quarterly review.

The model worked. The predictions were accurate. The interface was clean. And nobody needed it because we never asked what decision it would change or what workflow it would improve. According to Gartner’s 2024 AI Readiness Assessment, 72% of enterprise AI features launched in the previous year had adoption rates below 15% — not because the technology failed, but because the business case was never validated before engineering started building.

We’re entering the second-half budget planning cycle right now, and I’m watching enterprise teams make the same mistake in slow motion. The pitch decks all say “AI strategy.” The roadmaps all have LLM integrations and predictive analytics modules. But when you ask what specific user problem each feature solves, you get vague statements about “enabling data-driven decisions” or “improving efficiency.” That’s not a product strategy. That’s a press release looking for a justification.

Why AI Feature Selection Breaks Down at Enterprise Scale

The failure mode isn’t technical — it’s organizational. Most enterprise AI initiatives start with a CTO or VP who attended a conference, saw a demo, and came back convinced the company needs “more AI.” Engineering gets a mandate to explore machine learning capabilities. Product gets asked to identify use cases. And nobody stops to build the connective tissue between what’s technically possible and what would actually move a business metric.

The gap shows up in three places. First, engineering teams optimize for technical sophistication rather than user workflow integration. A recommendation engine that requires six clicks and a data export to use will lose to a simple dropdown filter every time, even if the ML model is objectively better. Second, product teams treat AI features as additions rather than replacements — they stack new capabilities on top of existing workflows instead of asking what manual steps the AI should eliminate. Third, executive sponsors measure success by launch announcements rather than sustained usage or measurable ROI.

According to McKinsey’s 2024 State of AI Report, organizations that tie AI investments directly to specific operational metrics see 3.2x higher adoption rates than those pursuing general “innovation” mandates. The difference isn’t the quality of the models — it’s whether anyone defined what success looks like before the first sprint started.

David Ohnstad has seen this pattern repeat across multiple enterprise software rollouts, and the warning signs are always the same: roadmap items phrased as capabilities rather than outcomes, user research that asks “would you use this?” instead of “what decision would this change?”, and business cases built on TAM expansion rather than workflow replacement. When those three show up together, you’re about to spend six months building something impressive that nobody will open twice.

The Feature Justification Stack: A Pre-Build Validation Framework

Before any AI feature gets engineering resources, it needs to pass through four validation layers. This isn’t a prioritization exercise — it’s a go/no-go filter. If a feature can’t clear all four layers with specific, documented answers, it doesn’t belong on the roadmap yet. Most enterprise teams skip straight to technical feasibility and wonder why adoption is weak. The framework forces you to validate the problem before you validate the solution.

Layer 1: Workflow Disruption Mapping. Identify the exact manual steps the AI feature will replace or eliminate. Not “improve” — replace. Draw the current user workflow with specific actions: opens report, filters by region, exports to Excel, builds pivot table, emails summary to stakeholders. Now draw the workflow with the AI feature active. If the new workflow still has more than two manual steps, you haven’t disrupted the workflow — you’ve added a detour. The feature fails Layer 1.

Layer 2: Decision Delta Quantification. Name the specific decision that will change because of this feature, and quantify what “better” looks like. “Faster insights” is not a decision delta — it’s a hope. “Reduces time from data request to vendor selection from 11 days to 3 days, enabling quarterly contract reviews instead of annual reviews” is a decision delta. You need a number, a timeframe, and a named business process that will operate differently. If you can’t write that sentence with specifics, the feature isn’t ready for engineering yet.

Layer 3: Data Readiness Audit. This is where most teams discover they’re not as ready as they thought. Map every data input the AI feature needs: source system, refresh frequency, schema stability, historical depth, and known quality issues. Then validate that all of those inputs exist in production today — not “we could build a pipeline” or “we’re planning to integrate that system next quarter.” If the data doesn’t exist now, the feature launch date is fantasy. According to Forrester’s 2024 Data Infrastructure Survey, 58% of enterprise AI delays are caused by data pipeline gaps discovered after development starts, not model performance issues.

Layer 4: Feedback Loop Definition. Before you build the feature, design how you’ll know if it’s working. Not vanity metrics like “daily active users” — those measure curiosity, not value. Define the instrumentation that will tell you whether the feature is changing the decision you identified in Layer 2. If the goal was reducing vendor selection time, you need telemetry that captures: how many users complete the full workflow, how often they override the AI recommendation, and whether contract review frequency actually increases. The feedback loop is part of the feature scope. If it’s not in the requirements doc, the feature isn’t done.

The counterintuitive part: Layer 3 should often kill features you’re excited about. If the data isn’t production-ready, no amount of model sophistication will save the launch. David Ohnstad has watched teams burn four months on a predictive analytics feature before discovering that the source system’s timestamps were in local time without timezone metadata, making historical trend analysis impossible. The model was brilliant. The data was garbage. Layer 3 would have caught that in week one.

When I Watched a $280K AI Feature Die in User Testing

We built a natural language query interface for our data warehouse. The pitch was straightforward: business users could ask questions in plain English instead of writing SQL or waiting for analyst support. The engineering team trained a fine-tuned model on our schema, built a clean chat interface, and launched a beta to 50 users. The demo was flawless. Executives loved it. We planned a full rollout for Q3.

Then we ran structured user observation sessions — not surveys, actual watch-them-work sessions. What we discovered: users weren’t asking questions because they didn’t know what questions to ask. The analysts who used the system regularly already knew SQL and preferred it because the query was explicit and reviewable. The business users who couldn’t write SQL also couldn’t formulate questions specific enough for the model to return useful results. They’d type “show me sales trends” and get frustrated when the system asked them to specify a date range, product category, and region. The feature assumed a level of data literacy that didn’t exist in the target user base.

The failure wasn’t the NLP model — it was the workflow assumption. We thought the barrier was SQL syntax. The actual barrier was not knowing what analysis to run in the first place. That’s a training problem and a data discovery problem, not a natural language interface problem. If we’d run Layer 1 validation — mapping the actual user workflow, not the idealized one — we would have seen that users didn’t start with questions. They started with reports someone else built, then modified filters. The AI feature we should have built was report recommendation based on role and recent activity, not a query interface.

We deprecated the feature eight months after launch. The $280,000 in sunk cost hurt, but the bigger cost was the eight months of roadmap space we gave to something that failed Layer 1. David Ohnstad still uses that launch as the reference case when product teams ask why the Feature Justification Stack requires workflow disruption mapping before technical feasibility. If you can’t draw the before/after workflow with specific steps, you don’t understand the problem well enough to build a solution yet.

Stop Prioritizing AI Features by Technical Wow Factor

Most enterprise roadmaps rank AI features by model sophistication or competitive positioning: “Company X launched a recommendation engine, so we need one too.” That’s not product strategy — that’s feature parity theater. The features that drive adoption and ROI are rarely the most technically impressive. They’re the ones that eliminate the most manual steps in the highest-frequency workflows.

A simple AI feature that auto-tags incoming support tickets and routes them to the right team will deliver more measurable value than a sophisticated sentiment analysis dashboard that requires manual CSV uploads and ten minutes of configuration before it shows anything useful. The tagging feature disappears into the workflow. The dashboard sits in a tab users open during quarterly reviews to check a box. According to Harvard Business Review’s 2023 analysis of enterprise software adoption, features that require zero behavior change see 4.1x higher sustained usage than features that require new habits, regardless of technical sophistication.

This runs counter to how most AI budgets get allocated. Engineering teams want to work on interesting problems. Executives want to announce current capabilities. And product managers get caught in the middle, trying to justify why the boring automation feature should get resources instead of the flashy predictive analytics module. But boring automation is what changes workflows. Flashy analytics is what gets screenshotted for LinkedIn.

The test: if you removed the AI feature tomorrow, would a specific operational process break or slow down? If the answer is no — if users would just go back to the manual workflow without meaningful friction — you’ve built a novelty, not a product. Novelties get demoed. Products get used daily without anyone thinking about them. David Ohnstad’s data product management writing consistently returns to this distinction: the best data products are the ones users forget exist because they’ve become invisible infrastructure.

Where Enterprise AI Budgets Should Actually Go in Second-Half Planning

If you’re finalizing AI investments for the rest of 2026, shift budget away from net-new features and toward feature integration depth. The AI capabilities that will move business metrics are the ones that eliminate steps in existing workflows, not the ones that add new dashboards or analysis tools. That means prioritizing AI features that reduce manual data preparation, auto-generate reports users already create weekly, and surface recommendations at decision points users already hit in their normal work.

Three investment categories that consistently outperform exploratory AI initiatives: workflow automation within existing product surfaces (not new standalone AI tools), predictive pre-population of form fields or configurations based on historical patterns, and automated quality checks that block bad data before it enters downstream systems. None of these are exciting conference talk material. All of them save users 10-30 minutes per day and reduce error rates by measurable percentages.

The budget allocation mistake David Ohnstad sees most often: dedicating 70% of AI investment to new capabilities and 30% to instrumentation, training, and feedback loops. That ratio should be inverted. If you can’t measure whether an AI feature is changing decisions, you can’t optimize it or justify continued investment. And if users don’t understand what the feature does or how to integrate it into their workflow, adoption will plateau at 15-20% regardless of model quality. The Feature Justification Stack forces those considerations upfront, before engineering resources get committed to features that will launch to silence.

The other under-invested area: data pipeline resilience and schema stability monitoring. AI features fail more often from upstream data issues than from model drift. If your source systems change schemas without notification or your ETL pipelines silently drop records when they hit edge cases, even a perfect ML model will return garbage. Budget 25% of AI investment toward data infrastructure observability — not sexy, but it’s what keeps the features you’ve already built working reliably.

How to Run a Feature Justification Stack Audit This Week

Pull your current AI roadmap. For each feature in active development or planned for the next six months, write down four things: the specific manual workflow it replaces (Layer 1), the measurable decision delta it enables (Layer 2), the production data sources it requires with current status (Layer 3), and the instrumentation plan for measuring whether it’s working (Layer 4). If you can’t write all four with specifics, the feature isn’t ready for engineering yet — it needs more product definition work.

The features that pass the audit with complete answers in all four layers are the ones that should get prioritized. The ones that have gaps in Layer 1 or Layer 2 need user research and workflow validation before they get resources. The ones that fail Layer 3 need data pipeline work before they get model development time. And the ones missing Layer 4 need instrumentation design as part of the feature scope, not as a “phase two” add-on.

This audit typically kills 30-40% of the features on an enterprise AI roadmap — not because they’re bad ideas, but because they’re not ready yet. That’s a feature, not a bug. Better to discover that in planning than in month six of development when you realize the data doesn’t exist or the workflow assumption was wrong. According to IDC’s 2024 Enterprise Software Development study, organizations that implement formal validation gates before engineering sprints start see 47% fewer late-stage scope changes and 31% faster time-to-adoption for features that do launch.

What is an AI feature prioritization framework for enterprise software?

An AI feature prioritization framework evaluates potential AI capabilities against business workflow integration, data readiness, and measurable outcomes before engineering resources are allocated. The Feature Justification Stack specifically requires each feature to clear four validation layers: workflow disruption mapping, decision delta quantification, data readiness audit, and feedback loop definition. Features that can’t pass all four layers aren’t ready for development yet.

How do you validate AI feature ideas before development starts?

Map the exact manual workflow the AI feature will replace, quantify the specific decision that will change, audit whether the required data exists in production today, and design the instrumentation that will measure whether the feature is working. If any of those four validations reveal gaps, the feature needs more product definition work before engineering starts building. This prevents investing months in technically sound features that users don’t adopt.

Why do enterprise AI features fail after launch?

Most enterprise AI features fail because they’re designed as additions to existing workflows rather than replacements for manual steps. Features that require users to change habits or learn new tools see adoption rates below 20%, even when the underlying models are accurate. Successful AI features eliminate steps in high-frequency workflows and surface recommendations at decision points users already encounter in their normal work, requiring zero behavior change.

Two Takeaways for Different Roles

For practitioners: Run the Feature Justification Stack audit on your current roadmap this week. Any feature that can’t clear all four layers with specific, documented answers needs more product definition work before it gets engineering time. Prioritize AI features by workflow disruption and decision delta, not by technical sophistication or competitive positioning. The boring automation that saves users 15 minutes daily will outperform the impressive analytics dashboard that gets opened quarterly.

For leaders: Reallocate second-half AI budgets away from exploratory features and toward integration depth, data pipeline observability, and instrumentation. The ROI comes from features that disappear into existing workflows, not standalone AI tools that require training and behavior change. Measure success by sustained usage and measurable decision deltas, not by launch announcements or feature count. If your teams can’t quantify the decision that will change because of an AI feature, don’t fund it yet.

When was the last time you audited whether your AI roadmap is full of features users will actually adopt, or capabilities engineering teams want to build? The difference between those two lists is where budgets go to die. For more on building AI & Machine Learning in Enterprise Software that users actually need, the validation framework matters more than the model architecture.

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

The post Enterprise AI Budget Waste: Three Costly Mistakes appeared first on David Ohnstad.

Photo by Florian Olivo on Unsplash

Why Most Enterprise Organizations Should Not Build AI Agents Right Now

We launched an autonomous agent to handle internal IT ticket routing at a 2,000-person SaaS company. Three weeks in, engineers were getting Slack notifications every eleven minutes asking for permission to escalate, reassign, or close tickets the system had already categorized. The agent wasn’t broken—it was working exactly as designed. The problem was that nobody had mapped the actual decision tree before automating it, so every edge case became a permission request. According to Gartner’s 2024 AI deployment survey, 55% of organizations are piloting or deploying AI agents right now. Based on what David Ohnstad has observed shipping data products and AI integrations at Veeam, most of them should stop.

The Hacker News community surfaced this exact failure mode this week with a game called “Continue? Y/N” that satirizes the permission fatigue autonomous agents introduce into workflows. The game forces players to approve every trivial micro-decision an AI agent makes—precisely the operational chaos enterprises are discovering after deployment. The comment thread drew 64 upvotes and dozens of practitioners sharing stories of agent implementations that created more coordination overhead than the manual processes they replaced. With Snowflake Summit approaching and every vendor pitching agent frameworks, the search intent is shifting from “how to implement AI agents” to “should we implement AI agents.” This article gives decision-makers permission to be strategic rather than reactive.

The Readiness Gap Nobody Is Measuring

AI agents fail not because the technology is immature, but because the organizational infrastructure required to support them doesn’t exist. Most enterprises lack three foundational elements: process documentation granular enough to automate, feedback mechanisms fast enough to catch agent errors before they cascade, and data quality mature enough to trust unsupervised decisions. According to McKinsey’s 2024 State of AI report, only 28% of organizations have established data governance frameworks solid enough to support production AI systems—yet agent adoption is running at double that rate.

The failure pattern is predictable. A VP sees a demo where an agent summarizes customer feedback, identifies trends, and drafts response templates in seconds. The team gets budget approval, integrates the agent with Zendesk and Slack, and launches to a pilot group. Within two weeks, customer success managers are manually reviewing every agent-generated response because the system occasionally hallucinates product features that don’t exist or misclassifies urgent escalations as routine inquiries. The agent didn’t break—the underlying data it was trained on included outdated documentation, inconsistent tagging, and no ground truth for what constitutes an escalation. The organization spent six months building the agent and zero months auditing whether their support ticket taxonomy was even coherent.

David Ohnstad has watched this pattern repeat across data product deployments: teams automate before they standardize. The correct sequence is to document your process, measure it, improve it to the point where exceptions are genuinely rare, and only then consider automation. Most organizations skip directly to the automation step because it’s more exciting than the taxonomy audit. The result is an agent that amplifies every inconsistency in your existing workflow at machine speed.

The Agent Readiness Stack

Before deploying any autonomous agent, organizations need to pass a four-layer readiness audit. This is not a maturity model—it’s a binary assessment. If you can’t answer “yes” to every question in a layer, you are not ready for agents at that scope. The Agent Readiness Stack works from the foundation up: data integrity, process documentation, feedback infrastructure, and human escalation protocols. Most organizations fail at layer one and never check the other three.

Layer 1: Data Integrity Audit. Can you programmatically validate that your data is complete, consistent, and current enough to make unsupervised decisions? This means schema enforcement, null-handling rules, and a defined SLA for how stale data can be before it’s unusable. If your CRM has duplicate customer records, conflicting address fields, or tags that mean different things across teams, an agent will make decisions based on whichever record it encounters first. The audit question is not “Is our data clean?”—it’s “Do we have automated checks that would catch data quality issues before an agent acts on them?” If the answer is no, stop here. Fix the data pipeline before you deploy the agent. According to Forrester’s 2024 data quality research, 67% of enterprises report that poor data quality has caused a customer-facing error in the past year. That error rate is unacceptable for human-reviewed workflows. It’s catastrophic for autonomous ones.

Layer 2: Process Documentation Depth. Is your workflow documented at the decision-node level, not just the happy-path level? This is where most teams fail. They document what happens when everything goes right, but they haven’t mapped the 30+ edge cases where human judgment is currently required. An agent can’t replicate judgment—it can only follow rules. If your process relies on employees “knowing when something feels off,” that’s not a process an agent can execute. The test is simple: could a new hire follow your documentation and make the same decisions your senior team makes? If not, you haven’t documented the process—you’ve documented an outline. David Ohnstad ran this audit for a data validation workflow at Veeam and discovered that 40% of the “process” was tribal knowledge held by three engineers who had been there since launch. Automating that workflow would have meant losing the error-catching that happened when those engineers noticed anomalies that didn’t fit documented rules. The correct move was to extract that tribal knowledge into explicit validation rules first, test them for six months, and only then consider automation.

Layer 3: Feedback Infrastructure Speed. Can you detect and halt an agent error within one operational cycle—ideally within minutes? Most organizations discover agent errors days or weeks after deployment because they don’t have monitoring that distinguishes agent-generated actions from human ones. If your agent sends 200 emails with incorrect pricing before someone notices, you don’t have feedback infrastructure. You have a delayed audit trail. The requirement here is real-time error detection: anomaly alerts when agent behavior deviates from baseline, manual review queues for high-stakes actions, and a kill switch that doesn’t require an engineering deploy to activate. If you can’t answer “yes” to all three, your agent will cause damage before you know it’s malfunctioning.

Layer 4: Human Escalation Protocols. When the agent encounters something it can’t handle, does it have a defined escalation path that doesn’t create coordination overhead? This is the layer the “Continue? Y/N” game satirizes. Agents that escalate every edge case to humans for approval are worse than no automation at all—they create interruption-driven workflows where employees context-switch dozens of times per day to approve trivial decisions. The correct design is to route escalations to a review queue that gets processed in batches, not in real-time. If your agent can’t distinguish between “pause and queue for review” and “interrupt a human immediately,” it will become the thing employees route around rather than rely on.

When David Ohnstad Built Feedback Loops Instead of Agents

In 2023, David Ohnstad was leading a project to automate QA validation for data pipeline deployments at Veeam. The initial scope included an agent that would run test suites, analyze failures, and auto-generate bug tickets with root cause hypotheses. The engineering VP loved the concept. The QA team was skeptical. David ran the Agent Readiness Stack audit and discovered they failed at Layer 2: the QA process was documented for happy-path scenarios, but the decision tree for categorizing failure severity was entirely judgment-based. Senior QA engineers knew which pipeline failures were cosmetic (log a ticket, ship anyway) versus which were data-corrupting (halt the deploy, page the on-call). That knowledge wasn’t written down—it was learned over years of seeing what broke in production.

The team spent three months not building an agent. Instead, they built a feedback loop: every time a QA engineer made a severity decision, they logged the reasoning in a structured format. After 90 days, they had 340 documented severity decisions with contextual notes. They used that dataset to build a decision tree with explicit rules: if the failure affects customer-facing dashboards, it’s a blocker. If it affects internal reporting only and there’s a manual workaround, it’s a medium-priority ticket. If it’s a cosmetic issue in a deprecated feature, it’s backlog. Once the rules were explicit and tested, they automated the categorization—but not the decision. The agent categorized failures and queued them for human review in batches. The QA team reviewed the queue once per day, approved or overrode the categorization, and provided feedback that improved the decision tree.

Six months in, the agent’s accuracy hit 94%. The QA team reduced time spent on categorization by 60%, but they never handed full autonomy to the agent. Why? Because the 6% of cases the agent got wrong were the high-stakes ones—failures that looked routine in the data but had downstream implications the agent couldn’t infer. A human reviewing a batch queue could catch those. An autonomous agent making unsupervised decisions could not. The lesson David Ohnstad drew from this: agents are excellent at executing rules and surfacing patterns. They are terrible at making judgment calls in novel situations. If your process requires judgment, don’t automate it—build feedback loops that make the judgment explicit, then automate the execution once the rules are stable.

The Contrarian Position: Stop Measuring Agent Adoption, Start Measuring Process Maturity

The conventional wisdom in enterprise AI is that competitive advantage comes from deploying agents faster than your competitors. According to Harvard Business Review’s 2024 AI strategy analysis, 73% of executives report feeling pressure to deploy AI agents to avoid falling behind industry peers. David Ohnstad’s position is that this is the wrong metric entirely. Agent adoption is a lagging indicator of organizational readiness, not a leading indicator of competitive advantage. The companies that will win are the ones that spend 2025 auditing their process documentation, data quality, and feedback infrastructure—and delay agent deployment until those foundations are solid.

The reason this matters is that poorly deployed agents create technical debt that’s invisible until it’s expensive. When a dashboard goes unused, you wasted engineering time but the system itself doesn’t cause damage. When an agent makes unsupervised decisions based on bad data or incomplete rules, it creates compounding errors that require forensic audits to unwind. A customer success agent that misclassifies escalations doesn’t just waste time—it damages customer relationships in ways that take months to repair. The cost of a bad agent deployment is orders of magnitude higher than the cost of waiting until your organization is ready.

Rather than building agents immediately, leaders attending Snowflake Summit should first assess organizational readiness using the frameworks outlined in David Ohnstad’s data product management writing. The competitive advantage is not in being the first to deploy agents—it’s in being the first to deploy agents that reliably improve outcomes without requiring constant human oversight. That requires process maturity, not procurement speed.

What This Means for Practitioners and Leaders

For practitioners building AI & Machine Learning in Enterprise Software, the takeaway is to resist the pressure to deploy agents before your infrastructure is ready. The Agent Readiness Stack is not a suggestion—it’s a prerequisite. If you can’t pass all four layers, your job is to fix the foundational gaps, not to build an agent that will fail in production. Document your processes at the decision-node level. Build feedback loops that make tribal knowledge explicit. Instrument your systems so you can detect errors in minutes, not weeks. Only then should you consider automation.

For leaders setting AI strategy, the takeaway is to reframe success metrics. Stop measuring how many agents you’ve deployed and start measuring how many processes you’ve documented well enough to automate safely. The organizations that rush into agent adoption without process maturity will spend 2026 unwinding the technical debt. The organizations that invest in readiness infrastructure now will spend 2026 deploying agents that actually work. The gap between those two cohorts will define competitive advantage in enterprise AI far more than who shipped first.

Here’s the question to ask your team this week: if we deployed an autonomous agent to handle our highest-volume workflow tomorrow, how many hours would pass before it made a decision we’d have to manually reverse? If the answer is anything other than “we’re confident it wouldn’t,” you’re not ready. And that’s okay—most organizations aren’t. The companies that admit that and fix the foundational issues will be the ones still running agents successfully in three years. For more perspectives on navigating enterprise technology strategy, visit David Ohnstad Minnesota.

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

The post Enterprise AI Agents: Why Most Companies Should Wait appeared first on David Ohnstad.

Photo by Daniel Andrade on Unsplash

Most Enterprise AI Projects Are Failing Because You Built the Platform First

We greenlit a $2.3 million AI infrastructure investment in Q2 2024. By Q4, the platform was live, certified, and fully integrated with our data lake. Usage after 90 days: one exploratory proof-of-concept from a single product team, abandoned three weeks later when the sponsor moved to a different role. According to McKinsey’s 2023 State of AI Report, 68% of enterprise AI initiatives stall in pilot phase, never reaching production deployment. We didn’t even make it to pilot — we built production infrastructure for use cases that didn’t exist yet.

The pattern is everywhere right now. Oracle announces Select AI. Snowflake launches new ML features at Summit. Your VP of Engineering comes back from a conference convinced you need a vector database and a dedicated LLM fine-tuning pipeline. Procurement gets three competing proposals for AI platforms, each promising to “unlock your data’s potential.” And six months later, you’re paying six figures annually for infrastructure that supports exactly zero business decisions.

The conventional wisdom in enterprise software is to build the foundation first — get the data infrastructure right, establish the platform, then let use cases emerge organically. That approach worked when infrastructure was expensive and use case validation was cheap. In 2025, it’s backwards. You’re spending tens of thousands on platforms before spending ten hours validating whether anyone actually needs what you’re about to build.

Why Infrastructure-First AI Projects Collapse

The failure mode is specific and repeatable. A senior leader attends a conference, sees a compelling demo, and returns convinced the organization needs an AI platform. The business case gets written around projected capabilities — “imagine if our support team could query our entire knowledge base in natural language” or “picture our analysts running predictive models without writing code.” The infrastructure gets funded because the vision is compelling and the vendor pitch is polished.

Then reality arrives. The platform is ready, but nobody has defined what question it’s supposed to answer. Product teams are told they can “experiment with AI,” but they don’t have time to learn a new toolset for problems they’ve already solved a different way. Data engineers are asked to pipe data into the new system, but the data model wasn’t designed for the queries the AI needs to run. According to Gartner’s 2024 AI Hype Cycle analysis, 54% of AI projects fail to move from pilot to production due to unclear business objectives and lack of stakeholder alignment — not technical failure, but a complete absence of validated demand.

The dollars tell the story. A Fortune 500 client of David Ohnstad’s spent $4.1 million on a customer intelligence AI platform in 2023. Eighteen months later, three teams had production use cases: two were generating reports that could have been built faster with SQL, and one was running sentiment analysis that the customer success team didn’t trust and stopped referencing in their quarterly reviews. The platform wasn’t wrong — it was solving problems nobody had prioritized solving.

What went wrong wasn’t the technology. The platform worked exactly as advertised. The failure was strategic: the company bought infrastructure hoping use cases would emerge, instead of validating use cases and then procuring exactly the infrastructure those use cases required. They built a stadium before confirming anyone wanted to play the game.

The Use-Case-First AI Stack

This is a three-stage validation model that inverts the traditional enterprise AI procurement process. Instead of platform-then-use-case, you validate demand, prototype with minimal infrastructure, then scale only what’s proven. David Ohnstad has used this structure to kill two AI platform proposals that would have burned budget and validated four use cases that shipped to production with infrastructure costs under $18,000 annually.

Stage 1: Demand Validation Before Infrastructure Commitment. Identify one specific business decision that’s currently slow, expensive, or inconsistent because the data exists but isn’t accessible in the format decision-makers need. Not a general capability (“we could use AI for forecasting”) but a named decision with a named owner. Example: “The pricing team manually pulls competitor data from eight sources every Monday morning to adjust weekly campaign budgets, and the process takes four hours.” Interview the team making that decision. Confirm they would actually change their workflow if you gave them an automated version. Get them to commit: “If you build this, I’ll use it weekly and replace the current manual process.” If you can’t get that commitment, don’t build anything.

Stage 2: Prototype with Rented Infrastructure. Use API-based AI services — OpenAI, Anthropic, Google Vertex — to build a working prototype of the validated use case. No platform procurement. No infrastructure buildout. Spend $200-$2,000 on API credits to prove the concept works and delivers the business value you projected. This stage should take two to four weeks, not two to four months. If the use case is real, the prototype will get adopted immediately. If it doesn’t, you’ve spent thousands instead of hundreds of thousands discovering the demand wasn’t actually there.

Stage 3: Scale Infrastructure Only After Proven Adoption. Once the prototype is in production use for 60-90 days and you have utilization data, procurement data, and user feedback proving the value is real, then — and only then — evaluate whether you need dedicated infrastructure. Most use cases never reach this stage because API costs stay manageable. The few that do scale are now backed by real usage data, which means you can size infrastructure correctly and negotiate from a position of validated demand, not speculative ROI.

The counterintuitive part: this process eliminates most AI projects before they start, and that’s the goal. According to Forrester’s 2024 AI Strategy Report, organizations that establish use-case gatekeeping criteria before platform investments see 3.2x higher production deployment rates than those that build infrastructure first. You’re not trying to maximize the number of AI projects — you’re trying to maximize the number that actually ship and get used.

What This Looked Like at Veeam: The Backup Recommendation Engine

David Ohnstad’s team was pitched an ML platform in early 2024 with a compelling demo: predictive backup failure detection using historical metadata to flag risky configurations before they caused downtime. The vendor estimated $180,000 annually for the platform, plus integration costs. The projected ROI was based on “reducing backup failures by 15-20%,” which sounded great in a slide deck but had never been tested with actual Veeam customer data.

Instead of greenlighting the platform, David ran the Use-Case-First AI Stack. Stage 1: Demand Validation. He interviewed six customer success managers and four enterprise support engineers to confirm the failure mode was real and frequent enough to justify automation. It was — but the conversation surfaced a more specific problem: customers weren’t ignoring warnings, they were getting too many warnings and couldn’t prioritize which ones mattered. The real use case wasn’t prediction, it was triage: “Tell me which three configurations in my environment are most likely to cause a failure this week so I know where to focus.”

Stage 2: Prototype with Rented Infrastructure. David’s team used OpenAI’s API and a Python script to analyze backup metadata from 40 enterprise customers, rank risky configurations by likelihood and impact, and generate a weekly prioritized list. Total prototype cost: $340 in API credits over three weeks. They shipped it to the six CSMs who’d validated the problem. Within two weeks, four of them had changed their customer check-in process to lead with the prioritized list instead of the generic health dashboard they’d been using for years.

Stage 3: Scale Infrastructure Only After Proven Adoption. After 90 days, the prototype was running for 120 customers, API costs had reached $1,800/month, and CSMs were reporting measurably faster issue resolution. At that point — with proven adoption, real cost data, and confirmed ROI — the team evaluated infrastructure options. They didn’t need the $180,000 platform. They built a lightweight model using existing AWS infrastructure for $22,000 annually, a tenth of the original vendor proposal, sized exactly to the validated use case.

The product shipped to general availability six months after the initial vendor pitch. But the path was opposite: instead of infrastructure enabling use cases, validated use cases defined infrastructure. The CSMs didn’t adopt it because the platform was impressive — they adopted it because it solved a specific problem they’d named, and the prototype proved it worked before anyone committed budget to scale.

Stop Funding AI Platforms — Start Funding AI Pilots with Kill Criteria

Most enterprise AI governance is designed to approve projects, not kill them. You have a review process, a business case template, an executive sponsor requirement. What you don’t have: mandatory kill criteria that force teams to prove demand before they get infrastructure budget. The result is a portfolio of AI initiatives that all cleared the approval bar but half of them shouldn’t have started.

Here’s the contrarian claim: enterprise AI budgets should be split 20/80 — 20% for infrastructure, 80% for use case validation and prototyping. Right now, most organizations do the opposite. They spend heavily on platforms and tooling, then underfund the work required to prove anyone will actually use what gets built. That’s why you have a vector database running in production with three stored embeddings and a fine-tuned LLM that’s been queried twice in six months.

The fix is straightforward but requires changing how AI investments get evaluated. Proposals should be required to name the decision-maker who will use the output, the current manual process being replaced, and the commitment threshold: “If we build this, I will use it [weekly/daily/per transaction] and replace [specific current workflow].” If the team can’t get that commitment before they write code, they don’t have a use case — they have a hypothesis that hasn’t been tested. Fund the test, not the platform.

This approach kills approximately 60-70% of AI project proposals before they reach the prototype stage, and that’s a feature, not a bug. The proposals that survive are the ones backed by validated demand, which means they’re far more likely to reach production and get used. David Ohnstad has seen teams propose twelve AI use cases in a planning cycle, validate three, prototype two, and ship one to production — and that one delivered more measurable business value than the eight projects running in parallel at peer organizations, all of which were built on expensive platforms and none of which had proven adoption after twelve months.

Infrastructure-first thinking treats AI as a capability you need to have. Use-case-first thinking treats AI as a tool you deploy only when the problem is validated and the manual alternative is measurably worse. The second approach is slower to start and faster to deliver value. The first approach fills your architecture diagram with impressive components that nobody uses. For more on aligning AI investments with product strategy, see AI & Machine Learning in Enterprise Software and David Ohnstad’s data product management writing.

How do you validate an AI use case before building infrastructure?

Identify a specific business decision currently made manually, interview the decision-maker to confirm they’d change their workflow if you automated it, and get explicit commitment: “If you build this, I’ll use it weekly and replace my current process.” Without that commitment, you have a hypothesis, not a validated use case. Prototype with API-based services to prove value before committing to infrastructure investment.

What is the main reason enterprise AI projects fail to reach production?

Most failures aren’t technical — they’re due to unclear business objectives and lack of validated demand. According to Gartner’s 2024 analysis, 54% of AI projects stall because organizations build platforms before confirming anyone actually needs what they’re creating. Infrastructure-first approaches assume use cases will emerge organically, but in practice, teams default to familiar tools and workflows unless the new solution solves a proven, painful problem.

Why shouldn’t companies invest in AI platforms before validating use cases?

Platforms are expensive, slow to implement, and create sunk-cost pressure to justify the investment even when use cases don’t materialize. Validating demand first with lightweight prototypes costs thousands instead of hundreds of thousands and proves whether the problem is real before you commit to infrastructure. Forrester’s 2024 research shows organizations that require use-case validation before platform procurement achieve 3.2x higher production deployment rates than those that build infrastructure first.

Two Immediate Actions and One Uncomfortable Question

For practitioners: the next time your organization considers an AI platform investment, require the proposal to name three validated use cases with committed users before procurement begins. Not hypothetical capabilities — specific decisions, specific decision-makers, specific commitments to change workflows. If the team can’t produce that list, the platform isn’t ready to buy.

For leaders: audit your current AI portfolio and separate projects into two categories: those with measurable production adoption by named users, and those still seeking use cases. Kill or pause everything in the second category until demand is validated. Reallocate that budget to use-case discovery and prototyping work for the few initiatives that have proven they’re solving real problems. You’ll fund fewer projects, but the ones you fund will ship.

When was the last time you audited your AI investments not by how much you spent or how impressive the technology was, but by how many people changed their actual decision-making behavior because the tool existed? If the answer is “never” or “I’m not sure,” you’re not measuring what matters — and that’s the gap where most enterprise AI budgets disappear. For broader context on structuring product teams to prioritize validated demand over speculative capabilities, see David Ohnstad Minnesota.

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

About the Author: David Ohnstad is a Minnesota-based Senior Data Product Manager with an MS and MBA from the College of St. Scholastica. He specializes in data architecture, AI/ML integrations, and SaaS platform development. Outside work, he builds furniture and hikes Duluth’s trails. Find his work at davidohnstad.com and github.com/davidohnstad40-netizen.

The post Why Enterprise AI Projects Fail: Platform-First Thinking appeared first on David Ohnstad.

Photo by BoliviaInteligente on Unsplash

Stop Building AI Agents: Why Most Enterprises Should Wait

Your organization doesn’t need AI agents right now. You need better data infrastructure, clearer decision frameworks, and the discipline to admit that the promise of autonomous systems is masking operational chaos you haven’t solved yet. According to Gartner’s 2024 AI Implementation Survey, 68% of enterprises piloting AI agents report “permission fatigue” and workflow disruption as primary barriers to adoption—not technical limitations. The problem isn’t the technology. It’s that most companies are trying to automate decisions they haven’t properly structured in the first place.

This isn’t a call to abandon AI. David Ohnstad uses Claude daily at Veeam Software to accelerate QA validation and engineering handoffs. The difference: those are bounded tasks with clear success criteria and immediate feedback loops. Autonomous agents operating across systems, making decisions without human checkpoints, and requiring constant intervention? That’s a different category of risk, and most organizations aren’t ready for it.

The “Continue? Y/N” game that surfaced on Hacker News this week perfectly captures the dysfunction: a system that asks for permission at every step isn’t autonomous—it’s just slower manual work with extra latency. But enterprises rushing to deploy agents are building exactly that: systems that require more human oversight than the workflows they were supposed to replace.

What Happens When You Deploy Agents Too Early

A mid-market SaaS company deployed an AI agent to handle tier-1 customer support tickets in Q4 2023. Within three weeks, their support team was spending 40% of their time reviewing agent responses, correcting hallucinations, and apologizing to customers for incomplete answers. The agent wasn’t “learning”—it was creating technical debt in the form of customer trust erosion. By January 2024, they’d rolled it back entirely and rebuilt their knowledge base infrastructure instead. Total cost: $180,000 in implementation, $340,000 in customer churn from poor experience during the pilot.

According to McKinsey’s 2024 State of AI Report, 43% of enterprises that deployed AI agents in 2023 scaled them back or discontinued them within six months. The primary reason wasn’t model accuracy—it was organizational readiness. Teams didn’t have clear escalation paths. Data sources weren’t unified. Success metrics were vague. The agent became a scapegoat for underlying process failures that existed long before AI entered the picture.

The failure mode is predictable: companies treat agents as a shortcut past foundational work. You don’t have a unified customer data model? The agent will surface inconsistencies. Your approval workflows are informal and undocumented? The agent will either ignore them or halt every transaction for human review. Your team doesn’t trust the data feeding the system? They’ll override the agent’s decisions constantly, rendering it useless. AI and machine learning myths in enterprise software often stem from exactly this pattern: expecting technology to solve structural problems that require operational discipline first.

The Agent Readiness Stack: Four Prerequisites Before You Build

Before deploying any autonomous agent system, David Ohnstad uses a four-layer assessment framework called the Agent Readiness Stack. It’s not a technical checklist—it’s an organizational maturity audit. Most companies fail at layer one and don’t realize it until they’re debugging hallucinations in production.

Layer 1: Decision Authority Mapping. Can you name every decision the agent will make and the human who currently makes it? If the agent is supposed to “handle customer inquiries,” that’s not specific enough. Does it issue refunds? Escalate bugs? Promise delivery dates? Each decision needs a named owner, a documented approval threshold, and a fallback path when the agent encounters ambiguity. Most organizations discover during this exercise that their processes are far more inconsistent than they believed. That’s the point—surface the chaos before you automate it.

Layer 2: Data Source Unification. Does the agent pull from one clean source of truth, or is it querying six different systems with overlapping but conflicting data? If your customer record exists in Salesforce, Zendesk, your billing system, and a legacy database with different field names and update frequencies, the agent will make decisions based on whichever source it hits first. That’s not intelligence—it’s random selection with expensive infrastructure. Fix data architecture before adding agents on top of it. This is where David Ohnstad’s data product management writing emphasizes the non-negotiable role of unified schema design in any AI deployment.

Layer 3: Feedback Loop Infrastructure. Can you measure whether the agent’s decision was correct within 24 hours? If the agent routes a support ticket and you don’t know if the customer got a resolution until they churn three months later, your feedback loop is too slow to train or correct the system. According to Harvard Business Review’s 2023 analysis of AI adoption failures, organizations with sub-48-hour feedback mechanisms had 4.2x higher agent retention rates than those relying on quarterly reviews. Real-time feedback isn’t a nice-to-have—it’s the only way to prevent compounding errors.

Layer 4: Escalation Path Design. When the agent doesn’t know what to do, what happens? If the answer is “it asks the user,” you’ve built the Continue? Y/N game—a slower version of manual work. If the answer is “it makes its best guess,” you’ve built a liability generator. The correct answer: the agent hands off to a human specialist with full context, and that handoff is logged, measured, and analyzed to improve the agent’s confidence boundaries over time. Most enterprises skip this entirely and wonder why their agents either paralyze workflows or create messes that take weeks to untangle.

Why David Ohnstad Didn’t Deploy Agents at Veeam (Despite the Pressure)

David Ohnstad faced this exact decision in early 2024. His team at Veeam was under pressure to pilot an AI agent for internal analytics query generation—stakeholders wanted non-technical employees to “just ask questions in plain language” and get SQL results back automatically. The vendor demos looked compelling. Leadership was enthusiastic. The timeline was aggressive.

David ran the Agent Readiness Stack assessment. Layer 1 failed immediately: the team couldn’t agree on what constituted a “valid” query. Some users wanted real-time data, others needed historical snapshots, and the business definitions of core metrics—customer, active user, churn—varied across departments. An agent generating SQL from natural language would surface those inconsistencies instantly, but the organization wasn’t ready to resolve them. Deploying the agent would just formalize the confusion.

Layer 2 revealed worse problems: the data warehouse had three different customer ID schemas depending on acquisition history, and several key tables hadn’t been documented in over 18 months. The agent would generate syntactically correct SQL that returned nonsense results, and users wouldn’t know the difference. David made the call: halt the agent pilot and spend six weeks cleaning the data model and establishing shared metric definitions first. Leadership pushed back. He held the line.

The decision was validated three months later when a competitor in the same market launched an agent tool and faced exactly the predicted failure mode: users lost trust in the results within weeks, the agent was quietly deprecated, and the company spent Q2 2024 rebuilding the data infrastructure they’d skipped initially. David’s team, meanwhile, now has the foundation in place to deploy agents responsibly—when the business actually needs them, not just because the technology exists. For related insights on maturing AI & Machine Learning in Enterprise Software capabilities strategically rather than reactively, David’s pillar content explores this timing question in depth.

The Contrarian Position: Waiting Is a Competitive Advantage

Stop treating AI agent deployment as a race. The conventional wisdom in enterprise software right now is that early adopters will capture compounding advantages—better models, more training data, operational learning curves that competitors can’t match. That logic works for foundational capabilities like cloud migration or API-first architecture. It doesn’t work for agents, because agents built on broken processes just automate dysfunction faster.

According to Forrester’s 2024 AI Implementation Gap Report, enterprises that delayed agent deployment until completing data infrastructure maturity assessments saw 62% higher agent utilization rates and 71% lower rollback rates than organizations that rushed pilots in 2023. Waiting isn’t falling behind—it’s avoiding costly mistakes that early movers are quietly unwinding right now. Microsoft’s recent decision to cancel internal Anthropic licenses in favor of token-based billing, as reported across enterprise AI communities this week, signals exactly this shift: organizations are moving from “deploy everything” to “deploy what we can actually measure and control.”

The real competitive advantage isn’t having agents first—it’s having reliable data architecture, clear decision frameworks, and the operational maturity to know when automation adds value versus when it obscures problems. David Ohnstad’s position: if you can’t manually execute the workflow you want to automate with 90% consistency today, an agent won’t fix it. It will just make the inconsistency faster and harder to detect.

What to Do Instead: Build the Foundation Now

If you’re not deploying agents yet, you’re not standing still—you’re building the infrastructure that makes agents viable later. Start with data architecture: unified customer records, documented business logic, version-controlled transformations. Then move to decision mapping: which workflows have clear success criteria, fast feedback loops, and low tolerance for error? Those are agent candidates. Everything else needs human judgment, and that’s not a failure—it’s an accurate assessment of risk.

Establish feedback loops for the systems you already have. Can you tell within 48 hours whether a feature change improved user behavior? If not, you don’t have the instrumentation needed to train or correct an agent. Invest in observability, logging, and metric alignment before adding autonomous decision-making on top of blind infrastructure. For leaders attending Snowflake Summit or similar enterprise AI events in the coming weeks, the most valuable conversations aren’t about which agent framework to adopt—they’re about assessing whether your organization has completed the prerequisite work that makes agents safe to deploy.

For practitioners: document every manual workflow you’d consider automating. Write down the decision tree, the edge cases, the escalation paths. If you can’t diagram it clearly enough for a junior team member to execute it, an agent won’t figure it out either. That documentation work is valuable whether you deploy agents next quarter or next year. It forces clarity, surfaces gaps, and gives you the baseline to measure whether automation actually improved anything. David Ohnstad’s experience building data products taught him that clarity at rest is the foundation for velocity in motion—ambiguous requirements don’t get faster when you add AI, they just fail more expensively.

If you’re being pressured to pilot agents because competitors are doing it, ask one question: what decision will this agent make that we currently make inconsistently or not at all? If the answer is vague, you’re automating for the sake of automation. That’s not strategy—it’s theater. The discipline to wait, mature your infrastructure, and deploy agents when they solve a real problem is rarer and more valuable than rushing a pilot to satisfy a roadmap slide. For additional perspectives on balancing practical craft with strategic planning in technical domains, David Ohnstad’s woodworking and making projects offer analogies that apply directly: measure twice, cut once, and don’t use a power tool when hand tools give you more control.

Frequently Asked Questions

Two Takeaways and One Question

For practitioners: Treat agent deployment as a forcing function for organizational clarity, not a technology implementation. If you can’t document the workflow, define success metrics, and unify data sources manually first, automation will amplify your confusion rather than resolve it. Build the Agent Readiness Stack assessment into every AI pilot proposal—it will save you from expensive rollbacks later.

For leaders: Stop measuring AI progress by how many agents you’ve deployed. Measure it by whether your data infrastructure, decision frameworks, and feedback loops are mature enough to support autonomous systems when they’re genuinely needed. The enterprises winning in 2025 won’t be the ones who rushed agents into production in 2024—they’ll be the ones who built foundations strong enough to scale agents safely and effectively.

Here’s the question you need to answer before your next agent pilot: Can you name the last automated workflow you deployed, measure whether it’s working as intended right now, and explain what would happen if it started making incorrect decisions tomorrow? If you can’t, you’re not ready for agents—and that’s the most strategic realization you can have this quarter.

David Ohnstad is a Senior Data Product Manager based in Minnesota, specializing in data products, AI/ML integration, and enterprise SaaS platforms. Follow his work at github.com/davidohnstad40-netizen.

Further reading: AI Vendor Risk Assessment: Why We Shut It Down — a case study on AI vendor evaluation and when to walk away.

The post AI Agents in Enterprise: Why Most Organizations Should Wait appeared first on David Ohnstad.

Unlike traditional development models, where APIs were added as a final component, API-first strategies place APIs at the core of the design process. This not only ensures better scalability and integration but also positions APIs as the backbone of innovation. By embracing this strategy, businesses are able to meet the growing demand for interconnected systems and provide a smooth user experience across multiple devices and platforms.

The Historical Context of APIs

APIs have been around for decades, initially designed as simple connectors to facilitate communication between software systems. These early APIs were rudimentary, serving highly specific purposes without much flexibility or scalability. For instance, they were often built as one-off solutions tailored to individual projects, which made them difficult to adapt or reuse.

The early 2000s marked a pivotal point in API history with the rise of web APIs. Companies like Amazon and Salesforce introduced APIs that enabled developers to access functionalities remotely, leveraging the power of the internet. This shift allowed businesses to create dynamic applications that could interact with external services, ushering in an era of collaboration and scalability. These APIs set the foundation for modern API-first strategies, offering new possibilities for developers to build tools and services that worked smoothly with existing systems.

The shift to web APIs not only expanded the possibilities for integration but also fueled the rise of the app economy. Platforms like iOS and Android embraced APIs to create ecosystems where third-party developers could innovate freely, contributing to the explosive growth of mobile applications and cloud-based services.

Why API-First Matters Today

The modern digital ecosystem is more interconnected than ever, with consumers demanding products and services that work smoothly together. Whether it’s a smartphone syncing with a smart home device or a business platform integrating multiple tools, users expect flawless interoperability. This is where API-first strategies prove invaluable.

By prioritizing APIs from the outset, businesses ensure that their products are designed for integration and scalability. This approach reduces the risk of compatibility issues and allows for faster iterations, as APIs serve as a stable foundation for new features. Moreover, API-first strategies promote a modular development approach, enabling teams to work independently on different components while maintaining overall cohesion.

Another critical advantage of API-first strategies is their ability to reduce time-to-market. By establishing a clear framework through APIs, development teams can build and test features in parallel, accelerating the product development cycle. In a hyperconnected world, where speed and adaptability are key, this capability provides a significant competitive edge.

The Role of Standardization and Best Practices

Standardization is at the core of the API-first movement. Protocols like REST, GraphQL, and gRPC have become industry standards, providing developers with clear guidelines for building APIs that are both solid and user-friendly. These protocols not only streamline the development process but also ensure compatibility across diverse platforms and systems.

REST, for example, has become synonymous with simplicity and reliability, making it a popular choice for building web APIs. GraphQL, on the other hand, offers greater flexibility by allowing developers to query specific data, reducing the amount of unnecessary information transferred. Meanwhile, gRPC, with its focus on high-performance communication, is particularly suited for large-scale systems.

Best practices, such as thorough documentation, version control, and automated testing, further enhance the effectiveness of API-first strategies. Clear documentation ensures that developers can understand and use APIs effectively, while version control maintains backward compatibility as APIs evolve. Automated testing, meanwhile, helps identify and resolve potential issues early in the development process, ensuring a smoother user experience.

APIs as a Driver of Innovation

API-first strategies have become a catalyst for innovation, enabling businesses to unlock new possibilities and revenue streams. By exposing core functionalities through APIs, companies empower developers to build applications and services that extend the value of their products. This openness has given rise to ecosystems where collaboration and creativity thrive.

For instance, APIs have been instrumental in the fintech industry, where platforms like Stripe and PayPal have revolutionized online payments. By providing APIs that developers can easily integrate, these companies have enabled countless businesses to incorporate smooth payment solutions into their services. Similarly, in the healthcare sector, APIs are driving advancements in telemedicine and patient data management, improving accessibility and outcomes.

This open ecosystem fosters partnerships between businesses, allowing them to integrate their services more effectively. For example, e-commerce platforms can leverage APIs to connect with logistics providers, enabling real-time tracking and efficient inventory management. Such integrations enhance customer satisfaction and streamline operations, creating value for all stakeholders involved.

Challenges and Considerations

Despite their advantages, API-first strategies come with their own set of challenges. Developing APIs requires significant upfront investment, both in terms of time and resources. Businesses must carefully consider factors like security, scalability, and compliance, as these can impact the success and reliability of their APIs.

Security is perhaps the most critical concern. APIs often serve as gateways to sensitive data and systems, making them attractive targets for cyberattacks. To mitigate these risks, businesses must implement solid authentication and encryption protocols, as well as monitor API usage to detect and prevent malicious activity.

Scalability is another key consideration. As user demand grows, APIs must be able to handle increased traffic without compromising performance. This requires thoughtful architecture and the use of scalable technologies like microservices and cloud infrastructure.

Compliance with regulations, such as GDPR and HIPAA, adds another layer of complexity. Businesses must ensure that their APIs adhere to these requirements to protect user data and avoid legal repercussions. This often involves conducting regular audits and updating APIs to remain compliant with evolving regulations.

The Future of API-First Strategies

The future of API-first strategies is closely tied to advancements in technology. Artificial intelligence and machine learning are already transforming how APIs are designed and used, enabling them to process complex data and deliver insights in real-time. This is paving the way for innovative applications, from autonomous vehicles to personalized healthcare solutions.

The concept of “API as a product” is also gaining traction. Companies are increasingly recognizing the value of APIs as standalone offerings that can be monetized. This shift is driving the creation of API marketplaces, where businesses can buy and sell APIs to meet specific needs. Such marketplaces are fostering a new era of collaboration and efficiency, as companies leverage each other’s APIs to enhance their own offerings.

Moreover, the rise of the Internet of Things (IoT) is further emphasizing the importance of APIs. As more devices become connected, APIs will play a central role in ensuring smooth communication between these devices. This will enable new use cases, from smart homes to connected industrial systems, further expanding the scope of API-first strategies.

Final Thoughts

The evolution of API-first strategies reflects the growing complexity and interconnectedness of the digital world. By prioritizing APIs as the foundation of product development, businesses can create scalable, interoperable, and innovative solutions that meet the demands of modern users. While challenges like security, scalability, and compliance remain, the benefits of this approach far outweigh the costs. As technology continues to evolve, API-first strategies will play an even more significant role in shaping the future of connectivity and innovation.

More from David Ohnstad: David Ohnstad data product management — David Ohnstad on leadership and career

The post The Evolution of API-First Strategies in a Hyperconnected World appeared first on David Ohnstad.

For professionals who spend their careers close to the mechanics of innovation, this gradual transformation is unmistakable. David Ohnstad, whose work spans product strategy, data-informed thinking, and technology trends, has often highlighted how the most meaningful advances are not the loud, dramatic ones. They are the quiet evolutions that change the way people make decisions, communicate, collaborate, and move through their everyday responsibilities, whether at work or outdoors on a trail in Minnesota.

Because developing technology isn’t about show, this distinction is more important than ever. It has to do with practicality.

Technology’s Progress Isn’t About Hype – It’s About Usability

A common misperception is that “emerging tech” refers to anything advanced, daunting, or unrelated to daily life. In actuality, today’s most significant technologies are successful because they elegantly and simply solve issues.

When AI summarizes a complex email thread so a team can move forward without hours of backtracking, that isn’t hype – that’s operational clarity.

When automation handles repetitive steps in a workflow, it doesn’t replace expertise – it protects it by giving experts more room to think. When wearables help people track their health with quiet consistency, that isn’t about being futuristic but about being functional.

Data Shows the Way Forward – But Only When It’s Interpreted Well

We’re surrounded by data nowadays and there is no shortage of it. But again, what matters here is the way it’s interpreted and translated. Many teams misjudge this process by focusing on dashboards instead of patterns, or on volume instead of clarity.

By making interpretation easier, new technological developments are bridging that divide.

These days, tools reveal insights before they fall through the cracks, flag anomalies before they worsen, and offer context to avoid mistakes. This is important in every industry, including product strategy, outdoor leisure, and healthcare.

But the real impact is human: people make better decisions when they understand the information guiding them. The technology doesn’t replace judgment; it strengthens it.

Automation Isn’t a Shortcut – It’s a Structural Upgrade

Automation is frequently misinterpreted. It reorganizes complexity rather than eliminating it, which prevents the human brain from having to function in a chaotic manner.

In high-pressure roles, where decisions compound quickly, automation acts as a stabilizer. It ensures essential tasks are executed consistently, leaving room for more strategic thinking.

That’s why automation has become indispensable in areas like:

• Workflow coordination
• Data processing
• Quality control
• Internal communication
• User experience management

It gives the teams precision that they wouldn’t have otherwise. Precision is crucial because that protects an organization’s momentum when challenges arise.

The Rise of Predictive Tools Has Changed How We Approach Problems

Forecasting used to be a specialized discipline. Now, predictive tools quietly sit behind everything from fitness routines to inventory systems to navigation apps.

Technology allows you to consider potential outcomes rather than just responding to them. Time, effort, and needless tension are all saved. This shift has real consequences for everyday problem-solving:

• Teams identify issues earlier.
• Individuals adjust routines efficiently.
• Organizations minimize risk more strategically.
• Decisions are made with more situational awareness.

The structure behind these systems is systematic and complex, but the benefit is disarmingly simple: better preparation.

User-Friendly Innovation Is Quietly Redefining Accessibility

The most powerful emerging technologies today share one characteristic: they don’t demand technical fluency. They adapt to people, not the other way around.

Voice interfaces have transformed the way individuals with mobility impairments engage with their surroundings. People with visual strain can read information more easily thanks to adaptive displays.

Innovation isn’t defined only by speed or power. It’s defined by the extent to which it expands access.

The Real Impact Is Subtle but Profound

It is rare for new digital trends to drastically alter everyday life. Rather, they improve the mundane by streamlining chores that previously required a lot of time, focus, and imagination. They give the impression that big goals are achievable, high-pressure decisions are structured, and complexity is doable.

That’s the quiet truth: technology isn’t transforming life with grand promises; it’s transforming life with practical precision.

For professionals who care about strategy, clarity, and meaningful progress, that impact is far more valuable than any futuristic headline.

Emerging tech is doing exactly that.

More from David Ohnstad: David Ohnstad data product management — David Ohnstad on leadership and career

The post The Real Impact of Emerging Tech Trends on Everyday Problem-Solving appeared first on David Ohnstad.