Cloud Application Monitoring: Best Practices, Metrics, and Tools

Applications no longer run on a single server you can walk up to. In the cloud, the containers, nodes, and functions behind a service may not exist five minutes from now, and a slow request can pass through a dozen managed services before it reaches the user. Finding out why something broke takes more than a CPU graph.

The cost of getting this wrong is high. The New Relic Observability Forecast 2025, based on a survey of more than 1,700 IT and engineering professionals in 23 countries, puts the median cost of high-impact outages at $76 million per year, or $2 million per hour.

This guide will help you build an application monitoring strategy for cloud workloads. It covers the definition, the main types of monitoring, the metrics that matter, 11 best practices, the requirements your setup should meet, and the tools available on AWS, Azure, and Google Cloud.

It is the collection and analysis of metrics, logs, traces, and user data from an application that runs in the cloud, so that the team can see its availability, performance, and errors. It answers two questions: is the application working for users right now, and if not, where is the problem?

It overlaps with two related practices, but it is not the same as either of them. Infrastructure monitoring watches hosts, containers, networks, and managed services: CPU, memory, disk, and network traffic. Application monitoring looks at the service itself: requests, transactions, errors, and the user experience. Cloud application performance monitoring (APM) is the part of application monitoring that focuses on the code, individual transactions, and response time, down to the slow database query or the function that blocks a request.

Application monitoring in the cloud has its own difficulties. Resources are ephemeral and replaced by autoscaling. Much of the stack runs on managed services where you can’t install an agent. Serverless functions live for seconds. Many applications run in several regions, and some across several providers. Monitoring has to follow services, versions, and user journeys instead of individual machines.

Cloud monitoring works differently from traditional setups. In an on-premises data center, the team knew which server ran which part of the system. In the cloud, capacity grows and shrinks with demand, services are spread across regions, and many dependencies belong to the provider. Without monitoring built for this, problems surface only when users complain.

The first reason to invest is user experience. Good monitoring catches slow features and failing transactions before support tickets and social media posts start piling up. If checkout latency rises only during a sale, the traffic and saturation signals show it first, while there is still time to add capacity or fix a limit.

The second reason is the cost of downtime. According to the same New Relic report, teams with full-stack observability pay about $1 million per hour for high-impact outages, half the overall median, and detect them 7 minutes faster.

The third reason is cloud spending. Cloud bills grow quickly when resources sit idle or are oversized. Monitoring shows which services waste resources, so you can resize them or change scaling rules.

The fourth reason is security. Unusual patterns in traffic, errors, or access often point to a problem in the environment. Catching them early makes the investigation shorter and cheaper.

No single type of monitoring covers everything. Most production teams combine several of them, depending on the architecture and on what users touch directly.

APM, logs, error tracking, and infrastructure monitoring look at the system from the inside. External application monitoring checks the system the way a user or a partner sees it: from outside your network, on a schedule, against public endpoints. RUM adds the view from real browsers and devices.

Distributed tracing ties these views together. For a microservices application, end to end application monitoring means following a single request from the browser through the API gateway, the services, the queue, and the database, so you can see which step added the delay.

Application monitoring metrics fall into four groups: the golden signals of each service, resource usage, user experience, and business outcomes. A Java service adds a fifth group for the JVM.

The core set comes from the Google SRE Book, which defines the Four Golden Signals and advises: “If you can only measure four metrics of your user-facing system, focus on these four.”

• Latency: the time it takes to service a request. Look at percentiles such as p95 and p99, not at the average, because a healthy average can hide a slow tail that a large share of users feel.
• Traffic: the demand on the system, such as requests per second, jobs, messages, or transactions. Unexpected spikes or drops often signal trouble.
• Errors: the rate of failed requests, including timeouts and wrong responses. This signal is easy to collect and still often missing from dashboards.
• Saturation: how “full” the service is. When CPU, memory, queues, connection pools, or Kubernetes limits approach their maximum, latency and errors usually follow.

For Java application monitoring, a running container says little about the service. A Java app can degrade slowly long before it looks down from outside: heap grows, garbage collection pauses get longer, threads get stuck, and every request waits for a database connection. Put those metrics next to the golden signals of the same service.

Business metrics complete the picture. A two-second delay on the checkout page costs money, while the same delay on a blog page may not matter. Showing response time next to conversion or cart abandonment makes the impact of a performance problem visible to everyone, not only to engineers.

Cloud environments give teams flexibility, but they also break the assumptions behind traditional monitoring tools. Three challenges come up most often.

Ephemeral resources. Instances, pods, and functions appear and disappear with demand. Tools designed for permanent servers lose track of them, show gaps in graphs, or keep alerting on hosts that no longer exist. Monitoring has to discover new resources automatically and group data by service, not by host name.

Finding the root cause in distributed systems. In a microservices architecture, a single click can trigger calls to many services, sometimes across several cloud providers. When users report random slowdowns, a dashboard per service is not enough. A timeout in one third-party API can ripple through the whole system, and only tracing across service boundaries shows where it started.

Alert fatigue. Distributed systems generate a lot of signals, and it is easy to turn each one into an alert. When most notifications are false alarms, engineers start ignoring all of them, including the ones that matter. The problem is not the number of metrics but the lack of rules about which conditions deserve a human.

If you work across several cloud providers or a hybrid setup, AppRecode’s managed cloud services team can help you set up monitoring that accounts for the differences between platforms.

These cloud application monitoring best practices work for most teams on AWS, Azure, or Google Cloud, whatever tools they use. They follow a simple order: decide what to measure, collect it in a consistent way, alert only on what matters, and keep the data useful and safe.

Instead of tracking hundreds of metrics that nobody reads, build each service dashboard around latency, traffic, errors, and saturation. These four signals answer the first question in any incident: is the service healthy, and if not, which way is it failing? The metrics section above explains what to watch for each one. Add resource and business metrics only after the four signals are in place for every user-facing service.

A service level indicator (SLI) is a measurement of what users experience, such as the share of checkout requests that succeed or the share of searches that return in under 500 ms. A service level objective (SLO) is the target for that indicator over a period, for example 99.9% successful checkouts over 30 days. The difference between the target and 100% is the error budget: the amount of failure you can afford.

Define SLOs for the journeys that matter most, such as signup, login, search, checkout, and payment, not for every endpoint. Then alert on the error budget burn rate. A fast burn means users are hurting right now and someone should act. A slow burn can become a ticket for the next working day.

OpenTelemetry is an open standard for instrumenting applications. It defines how code produces traces, metrics, and logs, and lets you send that data to different backends. It is not a backend itself, so you still need a place to store and view the data.

Instrumenting with OpenTelemetry keeps you independent from any single vendor. If you move from one monitoring platform to another, or send data to two of them during a migration, the code does not change. Use the same resource attributes everywhere: service name, environment, version, and region. Those labels make it possible to filter and compare data across services later.

In a microservices setup, a single user action may call many services. When something breaks, distributed tracing shows which one is responsible by following the request as it moves between services and recording the time spent at each step.

Make sure your tracing uses correlation IDs that follow requests everywhere, including queues and background jobs. You also need automatically generated dependency maps that show how services call each other. These maps often reveal bottlenecks that individual metrics hide, such as a service that calls the database far more often than expected. For Kubeshop, AppRecode integrated Datadog with Kubernetes on Amazon EKS, and troubleshooting time dropped by 20%.

Alert fatigue is dangerous because engineers who receive constant notifications start ignoring all of them. The basic rule is to page a person only for conditions that need human action right now. A useful way to apply it is three tiers:

•             Critical alerts go straight to phones, for real emergencies such as an outage, data corruption, or a security breach.
•             Warning alerts are collected into a daily digest, for trends moving in the wrong direction that are not yet urgent.
•             Informational notices go into logs for later analysis and need no immediate attention.

Each alert should explain what is happening, which users or services are affected, the likely causes, and the first steps to take, with links to the dashboard and the runbook. One of the core APM best practices is to create alerts that help solve problems, not just report them. Review noisy alerts regularly and turn them into tickets, dashboard panels, or nothing.

Strong backend monitoring does not tell you what users actually see. Your API may respond in milliseconds while a heavy script blocks page rendering in the browser, and customers still experience a slow site.

Web application monitoring best practices start with watching the frontend and the backend together. On the frontend, track page load time, time to first byte (TTFB), Core Web Vitals, JavaScript errors, and real user interactions with RUM. On the backend, track the golden signals of each service. Then link the two views with trace headers and request IDs, so a browser error can be followed to the backend trace or log line that caused it. Some problems, such as an API that stalls only for mobile users on certain networks, can be found only when both views are combined.

Real user monitoring has one limitation: it shows problems only after users run into them. Synthetic monitoring tests the application with automated scripts that act like users, on a schedule, before anyone complains.

External application monitoring runs these scripts from outside your network and from several regions, against the critical flows: login, search, checkout, and key API endpoints. Because the scripts run the same flows every time, it is easy to see performance degrade over time. They also catch issues during low-traffic periods, such as at night, before the peak user base is affected. Combine synthetic checks with RUM: synthetic tests confirm that core functions work at all times, and RUM shows how real customers experience the app.

Technical metrics in isolation are just numbers. They become useful when they are connected to business outcomes. A team can spend weeks shaving milliseconds off API calls while the real problem for the business is somewhere else, for example in search relevance.

Application performance monitoring best practices also call for connecting what machines report with what the business sees. Build dashboards that show response time next to conversion rate or cart abandonment for each important page, and failed payments next to the error rate of the payment service. When business leaders see these connections, it is easier to agree on which performance work to do first.

During an incident, engineers lose time switching between a metrics dashboard, a log search, and a tracing tool that use different names for the same service. Correlation removes that step.

Write the trace ID into every log line, so you can jump from a slow trace to the logs of that exact request. Use the same service names and labels in all three data types. Add deployment markers to charts, because many incidents start right after a change. Ideally, the team has one tool or one incident dashboard where metrics, logs, and traces for a service can be opened from the same place.

Telemetry from cloud application performance monitoring can become one of the larger lines on the cloud bill, and heavy instrumentation can also slow the application. Plan the volume from the start.

Use sampling for traces: keep a share of ordinary requests and all errors and slow requests. Watch metric cardinality, because labels such as user ID or full URL create millions of time series. Set retention by data type, for example detailed traces for days, aggregated metrics for months. Test agents and sampling settings under realistic load before a full rollout, and compare CPU, memory, and p99 latency with and without them.

Logs and traces often capture more than intended: email addresses, tokens, payment details, or full request bodies. Once that data is in a monitoring platform, it is copied, indexed, and visible to many people.

Mask or drop personal data at the source, before it leaves the application or the collector. Limit access to dashboards and log search with RBAC, and keep audit logs of who viewed or exported data. If your application is subject to GDPR or HIPAA, treat monitoring data as part of the same scope and agree the rules with your compliance team. Following these cloud monitoring best practices for data handling keeps observability from becoming a privacy risk.

Application monitoring requirements depend on the architecture, but most cloud teams need the same core capabilities. Use this list when you review your current setup or compare platforms:

☐ Auto-discovery of ephemeral resources such as instances, pods, and functions.

☐ OpenTelemetry support for traces, metrics, and logs.

☐ Correlation of logs, metrics, and traces through shared IDs and labels.

☐ Alerts that each have an owner and a runbook.

☐ Dashboards per service and per user journey.

☐ SLOs with error budget and burn rate alerts.

☐ Retention settings and cost controls for telemetry.

☐ RBAC and PII masking.

☐ Integration with incident management tools such as PagerDuty, Opsgenie, or Slack.

☐ Multi-cloud or hybrid coverage, if your workloads need it.

The best place to start is the tooling that already sees your workloads. Each major cloud provider has native monitoring, and open source and commercial platforms fill the gaps.

Start with the native tools of your cloud, instrument the code with OpenTelemetry, and add a platform only where the native tools leave a gap, such as tracing across clouds or frontend monitoring. For an AI-driven AdTech platform, AppRecode set up monitoring and alerting on CloudWatch and New Relic with autoscaling, and the platform reached 99.995% uptime with MTTR under one day. For an AI contact center platform migrating from EC2 to EKS, the team built monitoring of performance, application, and infrastructure with Datadog and Logz.io. If you need help choosing or setting up a stack, AppRecode’s application monitoring team can help.

Monitoring is easiest to get right when it is designed together with the application, not added after the first major outage. Start with the golden signals and SLOs for your most important user journeys, instrument services with OpenTelemetry, and make every alert actionable. Then connect technical data to business results, so performance work gets the priority it deserves.

These application monitoring best practices also improve everyday development. When developers see how each release changes latency and errors, they can spot a regression soon after the release that caused it. To move faster, AppRecode’s DevOps development team can help you build and run the setup.

Start with the four golden signals: latency, traffic, errors, and saturation. They are not the only metrics worth keeping, but they are the best first filter when someone asks whether the application is healthy. Look at latency through percentiles, because averages hide the slow tail that p95 and p99 reveal. Traffic is the work the app does: requests, jobs, messages, or API calls. Errors include failed requests, timeouts, retries, and dependency problems. Saturation can come from CPU, memory, disk, queue depth, database connections, thread pools, or Kubernetes limits. Add deployment markers to the same charts and one or two product metrics for critical journeys, such as failed payments for a checkout service.

Treat “the container is running” as the beginning, not the answer, because Java services often degrade slowly before they look down. Start with the JVM: watch whether heap keeps climbing, whether garbage collection takes noticeable time, and whether threads get stuck. Check connection pools too, since “random slowness” is often every request waiting for a database connection. Then connect the JVM view to the user-facing path. An API needs request volume, failed requests, slow percentiles, retries, and timeouts. A worker needs queue age and failed jobs. Finally, add traces and logs with request IDs, environment names, and release versions. OpenTelemetry is a sensible choice for instrumentation, because the same data can go to any backend later.

In the cloud, the thing you monitor keeps changing shape. On-premises, teams often know their important servers by name and where each one runs. In the cloud, the stable unit is rarely the machine: a pod is replaced, a node disappears, autoscaling adds capacity and removes it an hour later, and a managed database hides the hardware. Monitoring has to follow services, versions, regions, clusters, endpoints, and user journeys instead of familiar hosts. Releases and configuration changes also happen more often, so labels and deployment markers matter more. Provider metrics from load balancers, queues, and databases help, but the key question stays the same: can users complete the action they came for?

Yes, they can, although the overhead is not a fixed number for every stack. An agent does real work while the app serves traffic: it collects timings, creates spans, records errors, may sample profiles, and sends telemetry to another system. In a quiet internal service that cost may be invisible. In a busy checkout API during a sale, with tracing on every request, large custom attributes, and aggressive profiling, it can show up clearly. Test the agent with realistic traffic before a wide rollout, and compare CPU, memory, network egress, throughput, and p95 and p99 latency. Sampling is the main lever: start low and add detail for important services or during investigations.

Yes, but the two views must stay connected. The frontend view shows what people feel: page load, JavaScript errors, failed browser calls, mobile network problems, layout shifts, and CDN issues. The backend view shows what services do: slow APIs, failed jobs, database pressure, queue delays, and dependency errors. When the views never meet, the frontend team blames the API and the backend team sees a healthy service. Request IDs, trace headers, release versions, and environment labels link a browser error to a backend trace. Keep separate dashboards for daily work, and build one shared incident view for important journeys such as signup, search, checkout, payment, or upload.

Make alerts point only to conditions that deserve human attention, instead of muting everything. Separate symptoms from background noise: a single pod restart may be normal in Kubernetes, while a rising checkout error rate or a user-facing latency spike is not. Tie alerts to service objectives where possible, such as error budget burn, high p99 latency, or failed critical jobs. Every alert needs an owner, a clear threshold, a runbook, and enough context to start, including dashboard links, recent deployments, and the affected service and region. Review noisy alerts regularly and turn some into tickets or dashboard panels. Ten alerts that wake the right person for the right reason are better than a hundred that train the team to ignore the phone.

Monitoring tells you whether known things are healthy, while observability helps you investigate questions you did not predict. Monitoring alerts when the error rate crosses a threshold, p99 latency misses a target, or a queue stops draining. Those conditions were chosen in advance. Observability matters when a failure is messier: only one region is slow, one customer segment sees timeouts, or a deployment causes a chain reaction through a queue, a database, and a downstream API. Answering that needs metrics, logs, traces, deployment markers, and labels you can explore during the incident. The difference is not a tool category, since many platforms do both. Teams need both disciplines.

Start from where the application already runs and what the team is missing during incidents. On AWS, CloudWatch with X-Ray is the natural first stop. On Azure, Azure Monitor with Application Insights plays that role, and on Google Cloud, Cloud Monitoring with Cloud Trace. For Kubernetes, Prometheus with Grafana is a common choice for metrics and dashboards. OpenTelemetry is useful for instrumentation, so you can change backends later. A paid platform such as Datadog or New Relic may be worth it when you need tracing, profiling, and frontend monitoring in one place. The tools table above compares these options.