What Is Cloud Architecture, and Why Do Apps Fail Under Load?

This guide is for IT heads, CTOs and application owners at mid-size companies in India and the GCC who run, or are about to build, an application on AWS, Azure or both. It starts with a plain question: what is cloud architecture? It is the design of how an application’s parts are arranged and connected on cloud infrastructure, so the application can grow, recover and stay secure. The guide gives a short cloud architecture definition, names the layers, and then explains the design weaknesses that turn a traffic peak into an outage. If you want the method for keeping an application up, the links in each section lead to it.

What Is Cloud Architecture? The Short Answer

The cloud architecture definition in one line is a blueprint for an application: which parts exist, how they connect, where data lives and how the whole thing behaves when demand changes. Think of a retail app on a sale day. The blueprint decides whether the checkout, the product search and the database can each take a surge in traffic, or whether one of them gives way first.

A cloud makes capacity easy to add, but it does not change a design that can’t use it. Google’s SRE book names overload as the most common cause of cascading failures, and Microsoft’s scaling guidance says that scaling out doesn’t fix every performance issue. The rest of this article explains why, and what to check in your own design. A good design shows its benefits as growth without a rebuild and recovery from a fault, while a weak one shows its challenges as the load failures below.

What Is Cloud Architecture Made Of? The Layers in One View

This guide uses five cloud architecture layers. Providers draw them differently, so the names vary from one diagram to another.

Each of the cloud architecture layers is a place where load can meet a limit. That is why the layers matter for the rest of this article.

What Is Cloud Architecture Compared With Cloud Infrastructure?

In the cloud architecture definition, the design is the architecture and the parts are the infrastructure. Servers, storage and networks are the materials, so the architecture says how many of each, where they sit and how they depend on one another. The encyclopedia entry on cloud infrastructure covers the parts in detail.

You will also meet the service models (IaaS, PaaS, SaaS) and the deployment models (public, private, hybrid, multicloud). They describe who runs which layer and where the application lives. They are choices inside an architecture, not a replacement for one.

Why Apps Fail Under Load: Four Design Weaknesses

Load fails an application at its tightest or its shared part, and each weakness sits in one of the cloud architecture layers above. The table names four weaknesses, what you would see, and the design fix. Each fix has a fuller method in the linked posts.

A Part That Can’t Scale Out

A database or another part that holds state can be the bottleneck, so adding web servers adds load without adding capacity. Microsoft’s Azure Well-Architected scaling guidance names stateful parts as the most likely cause of bottlenecks. The same page describes large relational databases without sharding that can’t be refactored without significant impact, and says session stickiness can limit scaling and can introduce single points of failure.

Picture the web tier with spare room while the database sits at its limit. Adding web servers sends more requests to the database, which lengthens its queue instead of shortening it. The fix in one line: keep servers stateless and hold state in a data store, as Step 2 of the guide to improving application availability explains.

Scaling That Arrives Too Late or in the Wrong Order

New instances take time to start, so a sudden burst can pass before the extra capacity arrives. Services scale at different speeds as well. In Microsoft’s example, API Management instances can take up to 45 minutes to finish scaling, which can be longer than a short sale or payroll peak. A maximum number of scale units, once set, stops growth at that number.

Order matters too. If you scale an upstream part first, you can overload the part behind it. Microsoft’s guidance points to finding bottlenecks first and avoiding that overload, and the four steps below are our reading of Microsoft’s guidance, not a sequence the page prescribes.

The fix in one line: schedule capacity ahead of known peaks and watch quotas, as Step 4 of the availability guide and check 18 of the high availability checklist describe.

Failures That Spread Through Retries

A cascading failure grows through positive feedback. When one cluster is lost, the others take its load and start to fail in turn. Google’s SRE book chapter on cascading failures describes how CPU, memory, threads and file descriptors run out, slow requests pile up, and restarting tasks that fail health checks can drive the number of working tasks down further.

Naive retries make this worse. In the book’s example, three layers each retry a failed call three times, which is four attempts per layer. One user action can then become 4 × 4 × 4 = 64 attempts on the database. This is a worked example from the book, not a measured statistic.

The fix in one line: bound retries, back off with jitter and shed load. Step 3 of the availability guide and check 14 of the high availability checklist give the settings.

Limits Nobody Tested

Capacity that was never tested to failure can fail on the day of the peak. Google’s SRE book calls load testing to failure “the most important exercise” for preventing server overload, the cause it names as the most common behind cascading failures. For the method, see Step 7 of the availability guide and check 19 of the checklist, both linked above.

Four Questions to Ask About Your Own Application

Answer these from your cloud architecture layers, diagrams and metrics before the next peak. If you can’t answer one, that gap is where to look first, because the cloud architecture definition above asks you to know how the parts connect and where data lives.

1. Which part fills first when traffic rises: the web tier, the database, a queue or a third-party service?
2. Can that part scale out, or does it need a redesign?
3. What happens to the whole application when one dependency slows down?
4. When was each limit last tested to failure, and by whom?

If the answers point at a weakness, a short conversation with a specialist can help before a peak day. The design choices that fix a weak part are set out in seven design practices for a resilient cloud setup. Signisys provides cloud services and platform engineering, and can talk through your design for scalable and resilient cloud architecture. The reliability and resilience work page explains the wider approach.

For the related question of keeping an application available after a failure, see how to improve application uptime and availability. Our service page covers cloud services and platform engineering. To check a design before it is approved, use the 21 checks for reviewing a cloud design. To choose between a monolith, microservices and serverless, read how to choose between three application styles.

What Is Cloud Architecture FAQ