We operate AI systems in production: the models, LLM features and agents your customers rely on. We find what makes them give wrong answers, act outside their limits or cost more each month, fix it in the stack you already run, and stay on to operate it.
30 minutes with DharmOps. Or describe the problem and an engineer replies within 24 hours.
Most teams don't call us because of a model. They call because an AI feature their customers use has started giving wrong answers, taking risky actions or costing more than it should. These are the situations we hear most often.
The chatbot quotes a policy that doesn't exist, the copilot invents a product spec, or search returns last quarter's price. We build a test set from your real questions and score every answer, so you know how often it's wrong and why.
Someone edited a prompt, or your model provider shipped an update, and behaviour changed with no error raised. We version prompts and models and test every change before it reaches users.
A customer reports a bad answer, and nobody can see what the model was given, which documents it pulled or which tools it called. We trace every request from prompt to answer, so a complaint becomes a specific fix.
An agent runs on a shared API key with access to production, and nothing stops it from deleting, sending or paying. We give each agent its own identity, only the access its task needs, and an approval step for anything that can't be undone.
Token, GPU or vector database spend climbs every month, and no one can say which feature is driving it. We break the cost down per feature and cut what doesn't hold its weight, without lowering answer quality.
The team that built it has moved on to the roadmap, and when it breaks at night there's no runbook and no one on call. We take on the day-to-day operation and put a named person on call.
If one of these describes your week, the next step is a 30-minute call.
Book a Discovery Call (opens in a new tab)Want to see the work first? Read two situations we've fixed, with before-and-after numbers
AIOps services keep AI systems working after launch. DharmOps measures how your models, LLM features and agents behave in production, fixes what breaks quality, safety or cost, and then runs them: evaluations, tracing, access control, cost reporting, runbooks and on-call.
The work falls into three tracks. Most clients need one or two to start:
In IT, "AIOps" often means using AI to watch servers, group alerts and find root causes across infrastructure. That's a different service. Ours operates the AI itself. Where the two meet, alerting and incident response for AI workloads, it's part of what we run.
The test results, traces, audit trails and access records we set up are the evidence an AI governance program asks for: what each model and agent did, what it could reach, and who approved each change. We support your governance and compliance work; we don't certify it. Company-wide data policy sits with Data Governance & Security.
Companies with AI live in front of customers or inside operations, or launching within the next few months, that are seeing wrong answers, risky agent behaviour, rising cost or no clear owner. It suits product companies whose engineers are focused on the roadmap, and enterprises moving AI pilots into production.
If you're still deciding whether to use AI, want a product built from scratch, or need a foundation model trained, this isn't the right service. We'll say so on the call.
Every change below is measured against the baseline we record in the diagnostic, before anything in your environment is touched.
| What changes | How it's measured | Why it matters to you |
|---|---|---|
| Answers you can stand behind | Accuracy and grounding (whether the answer is supported by your own documents), scored on a test set built from your real questions, before and after. | You can tell leadership, customers and auditors how often the AI is right, with evidence, instead of hoping. |
| Releases that don't surprise anyone | Every prompt, model or provider change runs against the test set. A release that misses the pass mark doesn't ship. | Provider updates and prompt edits stop reaching customers untested. |
| Agents that stay inside their limits | Share of agent actions that run under the agent's own identity, are logged and are traceable, with destructive actions behind approval. | An agent can't quietly delete data or send something it shouldn't, and your security team can see exactly what it did. |
| An AI bill you can explain | Cost per feature and per request, month over month. Each saving is tested against the pass mark before rollout. | You know what each AI feature costs to run and can decide which ones earn their place. |
| Faster, cheaper model training | GPU activity, retrain time and GPU hours per run. For a US retailer: GPU activity 20–30% → 65–75%, weekly retrain 18–22 hours → 5–7, GPU spend per retrain 65–70% lower. | The same hardware does more work, and models are fresher each morning. |
| Someone on call | Every failure mode has a written first response, and incidents end in a fix or a new alert. | When it breaks at night, there's a plan and a person, not a scramble. |
What cost savings look like. Caching and model routing are the usual levers. Providers advertise up to 90% off cached input tokens, but output isn't cached, so real-world savings on total cost are closer to 25–35%. Routing savings depend on how many of your requests a smaller model can handle at the same quality. We measure both against your pass mark and show you the saving and its quality cost side by side before you approve the change.
Each area below is something we measure, fix and then keep running. The diagnostic tells you which ones your system needs.
A test set built from your real user questions and known bad answers. Accuracy, grounding and task completion scored against a pass mark agreed with you in writing, and wired into your CI pipeline (the automated checks that run before code ships), so a release that misses the mark is blocked.
You receive: a versioned test set, a written pass mark, and LLM evaluation that runs on every prompt, model or provider change.
Why it matters: you find out a change made answers worse before a customer does.
Every request traced: the prompt, the model version, the documents retrieved, each tool call, tokens, latency and cost. OpenTelemetry where your stack supports it, in the tools you already run.
You receive: a trace of every request from prompt to answer, dashboards and alerts on answer quality and agent behaviour, not just uptime and errors.
Why it matters: a vague complaint becomes a specific, reproducible fix.
Each agent gets its own identity instead of a shared key, separate staging and production credentials, access cut down to what its task needs, and human approval for deletes, payments and outbound messages. MCP servers get the same treatment: each tool they expose is scoped, authenticated and logged.
You receive: an audit trail of every step and tool call that your security team can read, and findings against both OWASP lists with evidence.
Why it matters: an agent can't take an action you didn't intend, and you have the evidence when a customer's security review asks.
We check that the documents behind retrieval (RAG, the step where the AI looks up your content before answering) are current, complete and removed when deleted at the source. We fix how documents are split and indexed, tune vector storage and search, and keep the pipelines that feed the AI monitored.
You receive: a retrieval quality score kept separate from generation quality, and fixes to the data path, including pgvector in the PostgreSQL you already run.
Why it matters: most wrong answers in AI search start with the data, not the model. This is database and pipeline work, and it's where DharmOps started.
Token, GPU and vector spend broken down per feature, per customer or per request. Caching, model routing, shorter context and token budgets applied where they hold the pass mark.
You receive: LLM cost optimization with the saving and its quality cost shown side by side, plus budgets and alerts that catch a spike in days, not on the invoice.
Why it matters: you can forecast AI cost and see what each feature earns against what it costs.
Automated deployment with versioned models and controlled rollouts, drift monitoring and retraining triggers, and fixes to data loading, checkpoints and storage when GPUs wait on data.
You receive: a deployment path you can repeat, drift alerts with retraining rules, and storage and transfer cost per training workload.
Why it matters: models stay accurate as the world changes, and GPU hours go into training, not waiting.
A written report: a map of your models, prompts, agents, tools and data sources and who owns each; baseline measurements (answer quality on a starter test set, latency, error rate, cost per feature); a risk review of every agent's access; findings against both OWASP lists, each with evidence; and a ranked fix list with effort, risk and rollback for each.
Each agreed fix as a reviewed, versioned commit in your repository: test suites in CI, tracing and dashboards, agent identities and approval steps, prompt and model versioning, caching and routing changes. Before-and-after results on the same baseline.
Monitoring of quality, cost, latency and agent actions; the test set kept current as your product changes; model and provider upgrades tested before rollout; incident response and reviews; a monthly report; runbooks kept current.
| Included | Not included |
|---|---|
| LLM features, agents and trained models you already run in production, or will launch within a few months | Building the AI product or agent from scratch, or choosing the use case |
| Test sets, release gates, tracing, monitoring and alerting | Training foundation models or research work |
| Agent identities, least access, approval steps and audit trails | Writing prompts as a standalone service |
| Retrieval quality, vector storage and the pipelines that feed your AI | Warehouse and pipeline builds (Data Platform Engineering) |
| Token, GPU and vector cost per feature | Warehouse and platform spend (Data Infrastructure FinOps) |
| Model deployment, drift monitoring and the GPU data path | Buying or installing GPU hardware, or selling a tool |
| Technical evidence for audits: logs, test results, access records | Legal compliance sign-off or certification audits (we provide the evidence; your auditors sign off) |
| Everything in the agreed written scope | Anything new, which is added to the scope in writing first |
There are six stages, and you approve each one before the next starts. Nothing in your environment changes until you've seen the evidence and agreed to the change.
You tell us what your AI does, what's going wrong and what it costs: wrong answers, a change that broke something, an agent with too much access, or a bill nobody can explain. We tell you whether it looks like a model, data or agent-setup problem and what a diagnostic would examine. If the fix is small enough for your team, or we're not the right team, we tell you on the call.
30 minutes · scheduled on our booking page
We sign an NDA, then work with read-only access to logs, traces, usage data and the prompt and agent code. We map the system: models, prompts, agents, tools, data sources, and who owns each. We build a starter test set from your real questions and record a baseline: answer quality, latency, error rate and cost per feature. We review what every agent can access and assess against the OWASP Top 10 lists for LLM and agentic applications. Each finding gets a risk score, probability times impact, with the evidence behind it.
Usually 1–2 weeks for one or two AI features or agents, report included. A larger estate is split into phases, so the first phase fits that range
We walk you through the report, answer questions on each finding, and turn the fixes you choose into a written scope. The scope lists what's included and what isn't, the assumptions it rests on, what each side provides and by when, and how each fix will be accepted, for example "answer accuracy on the test set at or above the agreed pass mark".
A review meeting with your engineering and business leads, held when the report is delivered, or the next business day if your leads aren't available
We confirm scope, timeline and acceptance criteria at kickoff, then apply the fixes cheapest first. Every change is version-controlled, reviewed and tested against your test set before it reaches production, and ships with a written rollback. Larger changes, like a model swap or a prompt rewrite, run against copies of real requests first, then go to part of your traffic where your setup allows it. Agent permissions are tightened one agent at a time.
Applied in the change windows you agreed. Timing for each fix is set in the written scope
We re-run the same measurements as the baseline on production and compare them. Before handover we test what matters when something goes wrong: alerts fire, rollbacks work, and a blocked agent action really is blocked.
Straight after the last change
AI systems drift: providers update models, documents change, and new features add new risks. So we stay on and run it: monitoring of answer quality, cost per feature, latency and agent actions, the test set kept current, alert review, and a review after every incident. When a model provider changes or retires a model, we re-run your test set against the new version before you switch.
Continues after the fix, with the same engineer
| Severity | Example | First response |
|---|---|---|
| Critical | A customer-facing AI feature is down or giving harmful answers, or an agent is taking actions it shouldn't | Within 30 minutes |
| High | A problem that affects your users: answer quality below the pass mark, or cost over budget | Within 2 hours |
| Normal | Questions, planned changes, reviews | Next business day |
Estimates are given in writing in the diagnostic report and the written scope, and we tell you when one changes and why.
"We were running performance issues on a PostgreSQL database with over 300GB of data. DharmOps analysed the workload, found the bottlenecks, and handled the tuning at a scale that required genuine expertise."
More on our work with SaaS and software companies.
When AI in production becomes a problem, teams usually weigh five other routes. Each one has a place; here is what each does and doesn't cover.
| Basis | DharmOps | Observability or evaluation tool | Hire an MLOps or LLMOps engineer | Your AI engineers, part-time | Large consultancy or AI vendor | Turn the feature off |
|---|---|---|---|---|---|---|
| Time to start | A discovery call, then a diagnostic | Days to set up | Months to recruit | Already there | Weeks of scoping | Today |
| Finds the cause | Yes, with evidence in writing | Shows traces and scores; someone still has to read them | Depends on the hire | When the roadmap allows | Yes, often by a junior team after a senior pitch | No |
| Makes the fix | Yes, in the stack you already run | No | Yes | Competes with every feature they're meant to ship | Often advice tied to the vendor's own platform | Removes the risk and the value |
| Covers | Models, LLM features and agents, plus the data and databases underneath | Traces, dashboards, test runs | One person, rarely evals, agent security, data and cost together | The product they built | Scale and brand | Nothing |
| After the fix | Same engineer stays for ongoing support | Keeps alerting | Stays, but can't cover on-call alone | Back to the roadmap | Engagement ends | The feature is gone |
| Who owns the work | You: test sets, dashboards, runbooks and configuration | Vendor dashboards | You | You | Varies | n/a |
Whoever you talk to, ask who builds the test set, who holds the keys, what you keep when you leave, and what they did the last time a model provider changed something.
Measured before anything changes.A baseline on your own test set comes first, and every fix is compared against it.
Model, data or agent setup: we find out which.We score retrieval and generation separately, so you don't pay for fixes in the wrong place.
The data and databases underneath, in the same team.When the cause is a stale index or an overloaded PostgreSQL, the same team fixes it.
One team from diagnosis to ongoing support.The engineer who found the problem fixes it and stays on, with published response times.
You own everything.Test sets, dashboards, runbooks and configuration live in your repository and accounts. If you stop working with us, nothing stops working.
When to start elsewhere. If you're still building the pipelines and warehouse behind your reports, start with Data Platform Engineering. If the problem is a slow or failing database, start with Database Engineering. If business pipelines are producing stale or wrong data, start with Data Reliability Engineering.
Giving an outside team access to production AI systems, and the customer data behind them, is a serious decision. These are the rules we work by.
Company-wide data access policy is the work of data governance and access control; agent permissions follow it.
No. In IT, AIOps usually means using AI to monitor infrastructure and cut alert noise. Our AIOps means operating your AI systems in production: MLOps for the models you train, LLMOps for features built on large language models, and AgentOps for AI that takes actions. Where the two overlap, alerting and incident response for AI workloads, it's part of what we run.
Hire if the work will keep one person busy full time and you have someone to manage them. Outside help fits when the problem has to be found and fixed first, when it spans skills one person rarely has (testing answers, agent permissions, data and cost), or when you need cover outside business hours. US MLOps salaries run around $160K at the median, and hiring takes months. We work alongside the people who built it: they approve every production change, and the runbooks are written so a later hire inherits them.
Yes. That's where most of our work starts. The diagnostic is read-only: traces, logs, costs, and test runs against a sample of real requests. Fixes go out only after you approve them and they pass your test set. Larger changes, like a model swap or a prompt rewrite, run against copies of real requests first, then go to part of your traffic where your setup allows it. Every change has a written rollback.
You don't need to know before you call. The diagnostic separates them: we score retrieval and generation separately, trace which documents each answer used, and measure latency and cost at each step. If the cause is the data, the database or the infrastructure, the report says so and shows the evidence, so you don't pay for fixes in the wrong place. The database and pipeline work underneath is work we also do. Database engineering
We work inside the stack you already run and recommend a new tool only where the diagnostic shows a gap. We've run PyTorch training on Kubernetes, GPU monitoring with NVIDIA DCGM, and vector search on pgvector and Pinecone, and we work with the tracing and evaluation tools you already have. Where your stack supports it, tracing follows OpenTelemetry, so you can change tools later without redoing the setup. You own the configuration, test sets, dashboards and runbooks from day one.
Each stage is priced separately, and you see the price before you approve it. The diagnostic is quoted after the discovery call. The fixes are quoted in the written scope, broken out by phase, after you've read the report. Ongoing support is a monthly retainer. Six things set the price: how many AI features, models and agents are in scope; how many model providers you use; whether tests and tracing already exist; agent risk and compliance requirements; the size and refresh rate of the data behind retrieval; and the support hours you need, including on-call if you want it. There's no fixed price list, because a price given before anyone has looked at your system is a guess.
Because quoting a fix before anyone has confirmed the cause is how AI projects overrun. The diagnostic measures where answers go wrong, where agents can act outside their limits and where the cost comes from, then ranks the fixes by impact. That ranking is what makes the fix quote accurate. If the cause turns out to be small, you've paid for a diagnostic instead of a project, and the report stands on its own if you'd rather have your team do the fixes.
No, and we don't put a number on it before we've measured. We commit to the process: a measured baseline, a written plan before any change, and before-and-after results on the same measurements. If a change doesn't move the measurement it was meant to move, we reverse it and say so in the results.
You can. The diagnostic, the fixes and ongoing support are separate stages, and you decide at each one whether to continue. Ongoing support has a 3-month initial term, then runs month to month with 30 days' written notice. You keep everything we set up, and access is removed when the work ends.
Transparency duties under Article 50 apply from 2 August 2026, and AI systems already on the market before that date have until 2 December 2026. High-risk obligations apply from 2 December 2027 for Annex III systems and 2 August 2028 for AI in regulated products. The logging, testing, human oversight and documentation we set up are the technical evidence these rules ask for. We support your compliance work; we don't certify it. Data governance and access control
Wherever AI answers customers or takes actions, the risks and the evidence an auditor asks for depend on the industry.
An assistant that explains products or an agent that touches accounts has to be right and traceable. We test answers against approved content, log every agent action, and keep the evidence a model review or audit asks for.
AI that drafts notes or answers patients has to keep protected health information inside controlled systems. We run evaluation and tracing where the data lives, with access you approve and a log of what was touched.
Recommendation and forecasting models retrain on growing event data, and shopping assistants quote prices and stock. We keep training fast and the data behind answers current, so the assistant doesn't sell what's out of stock.
In-product copilots and agents ship on your release cycle and add cost per customer. We gate every release on a test set, trace each request and report cost per feature and per customer.
Vision and sensor models drift as lines, lighting and parts change. We monitor drift, set retraining triggers and keep agents that touch scheduling or inventory inside their limits.
Support and content assistants handle high volume, where small error rates reach many people. We measure answer quality at volume, route requests to the right model, and alert when quality or cost moves.
Bring the symptom to a 30-minute discovery call. You'll leave knowing whether the problem is in the model, the data or the agent setup, and what the diagnostic would measure.