AWS describes the new system as collaborative, AI-powered observability for applications, slated to save time for engineering teams
AWS has announced Amazon CloudWatch now offers CloudWatch Omni, which it calls an AI-powered observability experience for the applications and AI agents that run together.
AI observability is increasingly important to organisations running agentic workflows.
As news reports break of the latest models going ‘rogue’ from frontier firms, enterprises who have already embedded agentic AI into core workflows need to ensure that their models are governed and safe.
They also need to take into account a host of other issues spanning model drift to controlling token usage (cost).
Traditional metrics like latency and error rate cannot tell you whether an agent’s response was helpful, coherent, or factually correctDaniel Abib, Senior Specialist Solutions Architect at AWS
Core features
Unlike traditional AWS tools, CloudWatch Omni operates outside the AWS Management Console.
Developers can use it natively inside IDEs like VS Code and Kiro to debug agents locally before syncing telemetry to the cloud. Omni also provides a standalone web portal accessible via standard enterprise SSO without requiring an AWS account.
Built on OpenTelemetry standards (OpenInference and ADOT), Omni supports major AI frameworks including LangChain, CrewAI and the OpenAI SDK.
It features 17 built-in evaluators for scoring model coherence and tool selection, alongside an Agent Topology view to map complex, multi-agent workflows.
Observability and Omni
“Organisations deploying agentic AI systems face observability challenges that traditional monitoring can’t address. Agent behavior is non-deterministic: a prompt change can degrade response quality even when standard metrics show no errors,” says Daniel Abib, Senior Specialist Solutions Architect at AWS.
“Teams spend hours manually reviewing logs across multiple systems, unable to pinpoint what changed or why. Existing tools force teams to choose between siloed generative AI monitoring or fragmented solutions requiring constant context-switching between their coding environment and browser-based dashboards.”
CloudWatch Omni is positioned to change this by organising observability around applications rather than individual signals, and bringing a whole team into the same workspace.
“Traditional metrics like latency and error rate cannot tell you whether an agent’s response was helpful, coherent, or factually correct. Evaluators score each response against quality dimensions, letting you measure what users actually experience and catch regressions that standard monitoring misses entirely,” Daniel adds.
Key facts
- Delivers AI agent observability outside the AWS console.
- Runs natively inside IDEs like VS Code and Kiro.
- Provides standalone web portal access via enterprise SSO.
- Includes 17 built-in evaluators for model response quality.
- Built on open standards including OpenInference and ADOT.
AWS core partners
AWS operates with a variety of other key partners. Some of them include:
- NVIDIA collaborates heavily on AI infrastructure, driving high-performance computing capabilities across the cloud ecosystem. By scaling up massive deployments featuring millions of next-generation GPUs, the partnership accelerates complex machine learning models and genAI innovations.
- Intel works alongside key cloud partners to continuously optimise hardware and software performance across enterprise environments. By deeply integrating custom processor architectures with cloud infrastructure, this partnership delivers maximum efficiency, enhanced security and reliable scalability for mission-critical workloads.
- Cisco delivers integrated networking and robust security resilience solutions tailored for cloud and hybrid work environments on AWS. Partnering to secure modern digital infrastructures, Cisco seamlessly connects enterprise applications while defending against evolving cyber threats.
- NetApp drives joint hybrid cloud storage and data management innovations, empowering enterprises to seamlessly control, protect and scale their data assets.
