Key takeaways

The web is a distributed system, not “a site on a server.” Even a simple page passes through DNS, TCP/TLS, often a CDN and reverse proxy, before application code sees the request. Knowing each hop helps separate network issues from SQL problems.

Responsibilities are split on purpose. The web server (Nginx, Caddy) accepts connections and serves static files; the application (NestJS, Laravel, Go) holds business logic; the database stores state. Mixing roles is convenient early and expensive at scale.

Cache and async are not “later optimizations.” Browser cache, CDN, Redis, and queues define whether a product survives traffic spikes without latency collapse. Sending email synchronously inside an HTTP handler is an architectural anti-pattern.

2020s frontend extends the backend; it does not replace it. React, SSR, SSG, and hydration solve first paint and interactivity but do not remove APIs, auth, and server-side data. Historical context lives in our evolution of web application architecture.

You cannot operate without observability. Logs, metrics, and tracing (OpenTelemetry, Prometheus, Grafana) are part of the architecture once you have more than one service.

Introduction: why trace the whole request

A developer who only sees their layer — a React component or a SQL query — often treats symptoms, not causes. “The page is slow” might mean a missing PostgreSQL index, a CDN miss, blocking middleware, or a heavy JS bundle. A full-path map turns guesses into a checklist.

Imagine an incident: users complain checkout is slow. Frontend opens DevTools Waterfall and sees POST /orders at 4 seconds. Backend APM shows 3.8s in one SQL query without an index. DevOps notices the load balancer keeps hitting an instance at 98% CPU due to a worker memory leak. Without a shared map, everyone fixes “their” layer while the root cause remains. The full request path is the language frontend, backend, DBA, and SRE share.

We follow packets in network order: from the URL bar to JSON in DevTools and back. Examples use a typical stack (Nginx, Node/NestJS, PostgreSQL, Redis), but principles transfer to PHP, Go, and Java.

What happens after you enter a URL

The user types https://example.com/products or follows a link. The browser must answer: which IP serves the host, how to open a secure connection, and which resource to request (/products).

Until DNS returns an address, TCP cannot start. Until TLS completes, HTTP does not travel in the clear. These steps take milliseconds with warm cache and hundreds of milliseconds on a cold cross-continent path.

DNS — the internet’s phone book

Browsers connect to IP addresses, not names — e.g. 192.0.2.1 for example.com. DNS resolves the name.

Typical resolution chain:

1. Browser and OS cache — recent lookups are instant.
2. DNS resolver (ISP or 1.1.1.1, 8.8.8.8).
3. Root servers → TLD servers → authoritative nameservers for the domain.
4. A/AAAA records (often CNAME to a CDN).

DNS caching speeds repeat visits but delays propagation when you change hosting (TTL). Engineers lower TTL before migrations — from 86400 to 300 seconds a day before cutover — or some users hit the old IP for another day.

Besides A (IPv4) and AAAA (IPv6) records, production sites rely on CNAME (alias to CDN), MX (mail), and TXT (verification, SPF). A DNS mistake means “site is down” while the server is healthy: the browser talks to the wrong host. Tools like dig, nslookup, and Cloudflare dashboards debug the chain without guessing.

Establishing a connection

TCP handshake

After IP resolution, the browser opens TCP to port 443 (HTTPS): SYN, SYN-ACK, ACK. Each round-trip adds latency — on mobile RTT of 100–200ms, TCP+TLS alone can cost a quarter second before the first HTTP byte. HTTP/2 multiplexes streams; HTTP/3 over QUIC reduces handshakes and survives network changes (Wi‑Fi → LTE) better than TCP-tied connections.

Connection pooling (keep-alive) matters: without it, every asset pays full TCP+TLS again. Browsers reuse connections by default — another reason not to disable keep-alive on Nginx without cause.

TLS and HTTPS

Over TCP, TLS negotiates version, exchanges certificates, verifies the site, and sets symmetric keys: ClientHello → ServerHello + cert → chain validation → encrypted channel.

Browsers check name match (SAN), expiry, and revocation (OCSP stapling on well-tuned servers). Let's Encrypt made HTTPS mass-market via ACME automation. HTTPS protects credentials and cookies on public Wi‑Fi, ensures integrity (no injected JS), and earns browser/search trust. HSTS forces HTTPS for the domain — downgrade protection.

The HTTP request

With a ready channel, the browser sends HTTP:

GET /products HTTP/1.1
Host: example.com
Accept: text/html,application/json
Cookie: session_id=abc123
Authorization: Bearer eyJhbG...

Methods carry semantics: GET read, POST create, PUT/PATCH update, DELETE remove — affecting caching and idempotency.

Headers carry metadata: Content-Type, language, Authorization, Cookie, Cache-Control, CORS on cross-origin calls.

Body appears in POST/PUT/PATCH — JSON orders, uploads, GraphQL. GET usually uses query strings.

Responses mirror the model: status codes (200, 201, 400, 401, 404, 500), headers (Set-Cookie, Location), and body. APIs distinguish “not found” from “forbidden” without parsing HTML error pages.

HTTP/1.1 typically one request/response per connection at a time; HTTP/2 uses binary frames, multiple streams, HPACK header compression. Dozens of small assets per page hurt less than in 2010, but oversized cookies still cost bandwidth.

CDN — why sites feel fast

A CDN places edge servers globally. Static assets (images, CSS, JS) and sometimes HTML are served from the nearest PoP, not a single US datacenter.

Providers like Cloudflare, Fastly, and Akamai cache by Cache-Control and their policies. Origin stays source of truth; CDN accelerates and absorbs DDoS. Dynamic POST /orders usually hits origin; some APIs cache carefully at the edge.

Cache directives matter: max-age, s-maxage for shared caches, stale-while-revalidate to serve stale while refreshing. Purge by URL or tag after deploy — otherwise users keep old app.js with fixed bugs. Best practice: fingerprinted static assets (app.a1b2c3.js) with long max-age, short or no cache on HTML shell.

Load balancing

Production rarely runs on one VM. A load balancer — Nginx, HAProxy, AWS ALB — distributes traffic, terminates TLS, health-checks instances, routes /api vs /.

Algorithms: round robin, least connections, weighted pools. A failed node drops out; users do not see total outage if capacity remains.

Sticky sessions bind clients to one instance when session state lives in process memory — an anti-pattern at scale; prefer Redis sessions. Health checks hit GET /health every 5–10s. Cloud balancers work at L4 (TCP) or L7 (HTTP path/host routing).

Web server

Behind the balancer: Nginx, Apache, or Caddy. Accepts connections, terminates TLS (or receives decrypted traffic), serves static files, proxies dynamic requests upstream, compresses (gzip, brotli), rate-limits.

Web server ≠ application. Nginx should not encode order business rules — it moves bytes efficiently. Confusing roles leads to one process doing everything.

Nginx is event-driven: few workers serve thousands of connections. Tune worker_connections, proxy_buffer_size, proxy_read_timeout. Caddy automates HTTPS; Apache + mod_php dominated shared hosting; clouds prefer Nginx + FPM/Node upstream. Typical: location /api/ → 127.0.0.1:3000, location / → static dist or SSR upstream.

Application server

Business logic lives here:

PHP: PHP-FPM + Laravel / Symfony.

Node.js: Express, NestJS, Fastify — one language with frontend; avoid blocking the event loop.

Java: Spring Boot — enterprise scale.

.NET: ASP.NET Core.

Go: Gin, Fiber — compiled binaries, concurrency.

Apps listen on internal ports (3000, 8080) reachable only via Nginx. In containers the port lives on the docker network; only ingress is public.

Run processes via systemd, PM2, supervisor, or Kubernetes Deployments. Horizontal scale adds identical instances behind the balancer; vertical scale adds CPU/RAM. CPU-bound Node may run multiple workers per host.

Inside the application

For POST /orders:

Routing maps URL + method to a handler.

Middleware — JSON parse, CORS, logging, auth, validation (Zod, class-validator).

Controller reads the request, calls a service, returns status + JSON.

Service layer — stock rules, discounts, payment gateway.

Repository / ORM — OrderRepository.save(), Prisma, TypeORM, Eloquent.

Layering simplifies tests: mock repository in service tests, mock service in controller tests.

Step-by-step POST /orders with {"productId":42,"qty":2}: auth middleware reads JWT → userId; validate checks schema; controller calls orderService.create; service in a transaction checks stock, reserves items, charges payment, saves via repository, enqueues OrderCreated for email; returns 201 with {orderId}. Payment failure rolls back — no “created” order without charge.

Map errors to HTTP: 409 out of stock, 422 payment declined, 500 only for unexpected exceptions logged server-side.

Authentication and authorization

The server must know who calls the API and what they may do.

Sessions: server stores state; client holds session_id cookie. Revocable server-side; sticky sessions possible. See JWT security mistakes in Node.

JWT: signed self-contained token; stateless verification; harder to revoke early.

OAuth / OIDC: sign-in via Google, GitHub, Microsoft — delegate identity.

Authorization (roles, RBAC) often lives in separate middleware or policy engines.

Session cookies need HttpOnly, Secure, SameSite. JWT in localStorage is XSS-sensitive. Compromise: short access token + HttpOnly refresh. See Node.js security risks.

Caching

Modern web does not scale without cache layers:

Browser cache — Cache-Control, ETag.

CDN cache — static and cacheable API at edge.

Reverse proxy cache — Nginx caches upstream responses.

Redis (application cache) — hot profiles, catalogs, heavy aggregates. Invalidation is hard; patterns in application-level caching in Node.

Good cache speeds the system 10–100× and protects the database; bad cache serves stale prices.

Cache stampede: key expires, thousands hit the DB at once. Fix with TTL jitter, single-flight, Redis mutex. Cache-aside: read Redis, on miss read SQL, then populate Redis.

Patterns in application-level caching.

Message queues

Not every operation belongs in the HTTP request. Email, PDF generation, image processing, CRM sync — use RabbitMQ, Kafka, Redis queues (Bull/BullMQ), SQS.

Pattern: API enqueues, returns 202 Accepted with job_id; workers process async. Users do not wait 30s for SMTP.

Async survives peaks; trade-off is eventual consistency.

Kafka for event streams and replay; RabbitMQ for routing; BullMQ on Redis for Node teams. Workers must be idempotent — duplicate delivery must not send two emails. Monitor queue depth, oldest message age, dead-letter rate.

Working with databases

Most systems store state in a database.

Relational SQL: PostgreSQL, MySQL, SQL Server — tables, FKs, transactions (ACID), indexes. Orders, money, reporting.

NoSQL: MongoDB, Cassandra, Redis — when schema flexibility, write scale, or data shape fits — not as a fad SQL replacement.

ORMs generate SQL; raw SQL remains for reports and tuning.

Connection pools (PgBouncer, driver pool) cap open PostgreSQL connections — without them, 1000 HTTP requests open 1000 connections and kill the DB. Read replicas offload reporting SELECTs with small replication lag; writes stay on primary.

Inside the database

SELECT * FROM orders WHERE id = 100 flows through:

1. SQL parser
2. Optimizer — index seek vs full scan
3. Executor — disk pages or shared buffers
4. Rows back to the app

Same query: milliseconds with an index on id, minutes on a billion-row table scan. EXPLAIN ANALYZE is step one.

Transactions commit or rollback atomically; locks protect consistency and create wait queues when misused.

PostgreSQL stores 8 KB pages; shared buffers keep hot pages in RAM. VACUUM reclaims dead tuples after UPDATE/DELETE. B-tree on id makes lookup logarithmic; composite (user_id, created_at) speeds “user orders in range”. Isolation levels define what concurrent transactions see — use SELECT FOR UPDATE when reserving inventory.

Where performance problems come from

Bottlenecks by layer:

Slow SQL without indexes — year-long order reports scanning millions of rows. N+1 when ORM loads orders then one query per line item — fix with JOIN or Prisma include. Long transactions waiting on payment gateways hold row locks. Huge JSON catalogs without pagination. Too many HTTP calls — 40 tiny APIs vs one BFF aggregate. Multi-megabyte JS without code splitting blocks the main thread.

Start with where time is lost: DevTools (TTFB vs download), APM SQL spans, PostgreSQL slow query log, Redis hit rate. Optimize the slowest hop first — shaving 5ms off Nginx is pointless if SQL takes 3s.

How the response travels back

The app returns:

HTTP/1.1 200 OK
Content-Type: application/json

{"orderId":100,"status":"pending"}

Path: DB → app → web server → (CDN) → browser. Compression, HTTP/2, keep-alive reduce overhead. The browser parses JSON and updates UI (React state, rerender).

Errors are structured too: 4xx client, 5xx server.

Compression (gzip, brotli) shrinks JSON/HTML 3–5×. Chunked transfer streams before the full body is ready. Personal responses with Cache-Control: private bypass CDN edge caches.

Modern frontend

After HTML and JSON, JavaScript runs. React builds UI from components; virtual DOM (and modern alternatives) limit expensive real-DOM work.

SPA loads shell once, then APIs. SSR sends rendered HTML — better SEO and first paint. SSG builds pages at compile time (Astro, static Next). Hydration attaches event handlers to server HTML — fragile when markup mismatches.

Model choice trades interactivity, SEO, and deploy complexity; platform shifts in native CSS vs extra JS.

Astro ships minimal JS by default. Next.js mixes SSR, SSG, and client components. Core Web Vitals — LCP, INP, CLS — depend on TTFB and CDN, not only bundle size.

Observability and operations

You cannot fix a distributed system from one log file.

Logs — structured JSON in ELK, OpenSearch, Loki; correlation IDs across services.

Metrics — Prometheus + Grafana: RPS, p95/p99 latency, error rate.

Tracing — OpenTelemetry, Jaeger: one trace per request through Nginx → API → Redis → PostgreSQL.

SLO alerts beat user complaints. Without this, microservices become a black box.

Logs answer what happened for request id X. Metrics — how many errors per minute. Traces — which span consumed 90% of time. Propagate X-Request-Id from Nginx through all services.

What modern web architecture looks like

Typical production stack:

User → Browser → DNS → CDN (static) → Load balancer
    → Nginx → Backend API → Redis → Queue → PostgreSQL → Object storage (S3)

Not every project needs every box on day one. A startup may run monolith + managed Postgres; a marketplace with spikes adds CDN, Redis, queues, and horizontal API scaling.

Typical evolution: single VPS → CDN for static → Redis sessions → read replica → job queue → Kubernetes when deploys exceed ten per week. Files live in S3 / R2 with signed URLs, not PostgreSQL BYTEA at scale.

How the web changed in 20 years

2005: PHP on one server, MySQL, HTML templates, full reloads. Simple FTP deploy, narrow UX ceiling.

2015: SPA, REST APIs, Docker, early cloud. Rich UI, SEO pain, separate frontend/API teams.

2025: React / Next.js / Astro, Kubernetes at scale, Redis and Kafka as defaults, edge CDN, AI in product. Server Components shift logic back toward the server — see evolution of web architecture.

The web is faster for users and harder for teams. Copying Big Tech stacks without their constraints buys complexity without benefit.

FAQ

Further reading

Related posts on this blog: evolution of web architectures, hidden Node.js security risks, application caching, JWT mistakes, native CSS capabilities.

Conclusion

A link click starts a distributed system: DNS finds the host, TCP and TLS build a channel, HTTP carries intent through CDN and balancers to code that reads cache, enqueues work, and writes to the database. Bytes return, and the browser becomes interface.

Understanding the full route is not academic. It tells you where to cache, where to async, where to index — and where not to copy Netflix in a five-person team. Use this article as a map; on the next incident, walk layer by layer from the Network tab to EXPLAIN in the database.