Modular Edge Computing at Multi-Tenant SaaS Scale on Cloudflare Workers

The Monolith Problem at the Edge

The edge platform I helped build fronts the web traffic of hundreds of thousands of tenant accounts. This platform started the way most edge deployments do with one Cloudflare Worker, one fetch handler, and one route. A script that intercepts a request, rewrites a header, and forwards to origin is easy to reason about and trivial to deploy. But a worker in front of a large SaaS product does not stay small. It accumulates responsibilities.

Over time, the same entry point grows to own image optimization, failover pages, routing, header and cookie rewriting, and per-tenant config lookups. Each is a legitimate edge concern, mostly owned by a different team. What began as one file becomes a shared monolith that every team edits and no team owns. This is the same force that produced application monoliths a decade ago. At the edge it bites harder.

At a single-site scale you can live with it. At the SaaS scale the picture changes. Hundreds of thousands of tenant accounts, each is an independent customer with its own configuration and isolated from the rest, several featuring teams. The worker is in the synchronous path of every request for all of them, so a millisecond added there is a millisecond paid across the whole base.

This is a globally distributed, multi-tenant platform and it runs on more than one content delivery network (CDN). The same feature has to work on each. That constraint is why this article will keep comparing two of them, not to rank vendors, but because a platform team building the same capability twice sees exactly where an architecture is portable and where it is not.

Deployment Coupling

With one worker, there is one deployment. Any change redeploys the entire script, so every team’s cadence collapses into one shared cadence set by the slowest change in flight. A one-line header fix waits behind a half-finished experiment. The two ship together. The only unit of deployment is everything on main. A rollback of one feature is a rollback of all of them.

Blast Radius

A worker is not a backend service with replaceable instances behind a load balancer. It is the request path. An unhandled exception, a bad regular expression, or a hot loop in one feature does not degrade that feature, it degrades every feature and every tenant at once. Multi-tenancy turns one bad deployment into a simultaneous incident across the entire customer base.

Platform Limits

Workers run under hard ceilings such as a fixed per-request CPU budget and a cap on script size. Each feature's dependencies inflate the shared bundle and lengthen the parsing and startup work in the hot path, even for requests that touch none of that code. You ration one budget across teams that cannot see each other's consumption.

Ownership Mismatch

Failover logic should change rarely. An image pipeline wants to iterate fast. Force both through one file and you will have merge contention, unclear ownership, and fear-driven development, where nobody dares touch the file for fear of what else they will break.

The edge amplifies every downside of a monolith. Coupling that is a nuisance in an application server becomes an availability risk. The fix keeps the one thing the edge rewards, a lean entry point, while giving every team a unit it can own, test, and deploy on its own schedule.

Modular Workers Architecture

The obvious fix for a monolith is to split it, with the obvious objection being cost. Splitting a backend monolith into services buys isolation and pays for it with network hops. Each internal call becomes a DNS lookup, a TLS handshake, and a round trip you did not have before. At the edge, where the whole point is to shave milliseconds off the request path, that trade looks unaffordable. On Cloudflare, it is wrong.

The reason it is wrong is due to service bindings. A service binding is a worker-to-worker call that Cloudflare dispatches in the same isolate on the same machine, so it has none of that. The call has the shape of fetch() but the cost of a function call. That single property is what makes decomposition affordable at the edge.

The pattern is a thin gateway worker in front of a set of single-purpose feature workers.

Figure 1. Gateway plus feature workers, connected by service bindings. (Source: created by author.)

The gateway owns exactly two things: composition (i.e., which features apply to this request, and in what order) and cross-cutting concerns that belong everywhere (i.e., request preparation, header hygiene, and observability). It does not own feature logic. Every feature worker owns exactly one concern, its own dependency bundle, test suite, and deployment. That is the whole design: high cohesion inside each feature worker, low coupling between them, and a gateway that knows when each feature applies but not how it works.

The gateway's contract with each feature splits in two, so a feature that does not apply to a request never triggers a binding hop. Each feature exposes a cheap shouldApply() predicate the gateway imports and runs inline, and the worker itself, dispatched over a service binding only when the predicate says the work is needed. Decide inline, execute remote.

The gateway composes features in a fixed order, which the code below makes concrete. Gates like failover short-circuit before origin is touched. Request preparation runs next. Image optimization may take over the origin fetch. Finally, a response-phase gate catches conditions visible only in the response.

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    // Predicates run inline; only the heavy work is dispatched over a service binding.
    if ((await shouldServeFailover(request, env)).serve) return env.FAILOVER.fetch(request);
    const originRequest = prepareOriginRequest(request, env);
    const response = (await shouldOptimize(originRequest, env)).optimize
      ? await env.IMAGE_OPTIMIZER.fetch(originRequest)
      : await fetchFromOriginOrCache(originRequest, ctx);
    // A response-phase gate catches conditions visible only now (an origin 5xx, a rejected request).
    return shouldServeFailoverForResponse(request, response).serve
      ? env.FAILOVER.fetch(request)
      : sanitize(response);
  },
};

Two things follow from the gateway holding only a Fetcher handle for each feature, rather than the feature's code. A worker's dependencies and CPU budget stay inside it while the gateway bundle carries only the small predicates, which is what the platform-limits axis needed. A feature that fails must not fail the request. If a feature worker errors or times out, the gateway serves the untransformed origin response rather than an error page. Resilience lives at both layers; neither can take the whole request down.

Independence is not free, its price is coordination. If every feature worker lives in its own repository, the shared contract drifts, integration testing means gluing together artifacts from multiple repositories, and a change to the gateway interface becomes a multi-repo migration. We colocate the gateway and all feature workers in one monorepo. Package boundaries keep each worker an independent build and deploy target, and each feature's external facade is the only surface the gateway may import, so colocation buys shared conventions and one source of truth for the contract without reintroducing shared deployment. You receive the ownership boundaries of separate services and the coherence of one codebase. That combination, not the gateway pattern by itself, is what makes the architecture hold up as the feature teams multiply.

Coordination is not the only cost. The split also fragments observability. One worker sees the whole request, whereas a chain of workers each sees only its own slice. If you want to watch a request in flight without waiting on centralized logging, one mechanism is a special request header that makes the response carry diagnostics in its own headers. It works at the gateway, but diagnostics captured inside a feature worker downstream of a service binding are not propagated back up, so a request that crosses into the image worker loses the part that happened there. Tracing one request end to end is work the monolith gave you for free.

The Multi-CDN Reality

The architecture so far is a Cloudflare architecture. It is tempting to assume it ports to any CDN with serverless compute. It does not. I built the same image-optimization feature on both Akamai and Cloudflare. The gap between them is bigger than "just moving it to the edge" assumes. It starts with the primitive each platform gives you.

On Cloudflare, the unit of compute is the worker. The entrypoint is a single fetch handler that owns the request end to end: it inspects, it decides, it can call other workers over bindings, fetch origin, and rewrite the response. The gateway pattern falls naturally outside of that, because one piece of code holds the whole request and can compose the rest.

Akamai does not hand you that. The unit of configuration is the property, a rules engine that matches on request attributes and applies behaviors. Image optimization is one of those behaviors: a managed product (Image Manager) you enable with a rule, not with code you write. Compute (i.e., an Akamai EdgeWorker) is a guest inside that pipeline, invoked at named lifecycle events rather than owning the request. It does not call Akamai’s Image Manager and does not transform the image.

Here is what our opt-out EdgeWorker actually does, reduced to its shape:

// Akamai EdgeWorker: a lifecycle-event handler, not a fetch handler.
// It cannot call the image transform. It leaves a decision in request
// state and the PROPERTY, downstream, reads that decision.
export async function onClientRequest(request) {
  const key = deriveSiteKey(request);              // host + site prefix, normalized
  const optedOut = await lookupOptOut(request, key);

  // The entire output of this worker is one property variable.
  request.setVariable("PMUSER_SKIP_TRANSFORM", optedOut ? "true" : "false");
}

On Cloudflare, decision and execution live in the same code: if (shouldOptimize) return env.IMAGE_OPTIMIZER.fetch(...). On Akamai, the EdgeWorker writes a decision into the request state and a rule inside the property reads that variable and gates Image Manager. The two halves never share a call stack and there is no service-binding equivalent to connect your compute to the managed transform.

That difference cascades into the data plane. The opt-out state lives in a key-value store on both platforms, but the stores have different reach. Cloudflare Workers KV, its key-value store, is global. If you write a key, it is readable everywhere and the gateway does not think about geography. When I built the Akamai version, its EdgeKV namespaces, Akamai's equivalent key-value store, were provisioned per region and chosen at creation time, with no single namespace spanning all of them. So the Akamai worker carried code with no analog on Cloudflare. It mapped the request’s origin continent to a regional store before it could even issue the read.

// Akamai (as built): the store was regional, so before the read the worker
// had to map the request’s continent to a region and open that namespace -- no single store spanned all regions.
const region = CONTINENT_TO_REGION[continentOf(request)] || "amer";
const store = openNamespace(namespaceFor(region));

Porting the feature meant rewriting this layer, not copying it. The Cloudflare version deletes the continent map entirely. The same feature, on two platforms, shares the shape of its intent but not the shape of its code.

That geographic gap has since narrowed. Akamai later added a global namespace. The platforms move underneath you. A difference that you engineer around one year can close the next. Parity is not a state you reach and hold. It decays, in both directions.

Which model is the liability depends on the goal. This platform wanted global reach, so a store that is automatically global was the simpler fit and the regional routing was the tax. Flip the requirement, a data-residency rule that keeps a region's data in that region, and they trade places. Akamai's per-region namespace, documented as regionally scoped, is now the feature rather than the tax; Cloudflare KV is global by design with no region-pinning control, so residency there means reaching for a different primitive instead of a setting on KV.

So what carries across, if the code cannot? The intent and one hard-won invariant. Both platforms cap the memory edge compute may use, so on both we front the key-value store with a small, capacity-bounded local cache wiped on a short interval to stay fresh. The same structure, cap, and window, are now in different code. The same constraint produced the behavior on both platforms, a memory ceiling plus a hot path too latency-sensitive to hit the store on every request, because that constraint is a property of the edge, not the vendor.

So the multi-CDN reality is not "write once, deploy to both". It is a standing tax: A feature is done when it works on both CDNs. The two implementations diverge everywhere except the invariants the edge itself imposes.

Image Optimization at the Edge

Image optimization is one feature among the many a gateway composes, but is not the platform's purpose. On the two CDNs the build-or-buy decision behind this optimization broke in opposite directions, and that contrast is the lesson.

The two platforms sit at different levels. On Akamai, image optimization is a managed service. You configure a policy and it negotiates each request on its own, with no per-request code. On Cloudflare, it is a lower-level primitive: You specify the transform per request from inside a worker. Cloudflare does offer a managed tier of its own (e.g., Polish, a zone-level toggle), but it optimizes through the cache, matching URLs by file extension and skipping anything not publicly cacheable. This platform's image traffic is neither uniformly extensioned nor uniformly public, so the toggle was never a fit. Neither is more advanced. They ask for different things. The managed service takes configuration, while the primitive takes code. On this platform we consumed the first and built on the second. Cloudflare's own documentation is explicit that with the primitive, automatic format negotiation becomes the caller's job.

That is more work, but it buys the thing the multi-CDN section already argued for: a policy you own in code behaves identically regardless of what each CDN’s managed product does, lacks, or changes next quarter. It also forces you to be explicit about what "negotiate the image" means, which is three separate decisions, taken in order.

Not Every Image May Be Touched

The first decision is security. Some images on a multi-tenant platform are private, behind authentication that lives at the origin, although the edge does not reproduce it by default. The cache is what makes this fact dangerous. If you optimize a private image you have cached it at a shared edge keyed by URL, so the next request for that URL is served without ever reaching the origin that would have checked authorization. The worker therefore refuses to optimize anything it cannot prove is public. A request carrying any signal of authentication is passed straight through, untouched. This is not an image-quality rule. It is used to keep a performance feature from becoming a data leak.

Not Every Format Suits Every Client

AVIF and WebP are far smaller than the JPEG or PNG a site stores; independent benchmarks put AVIF at roughly half the size of an equivalent JPEG and WebP about a third smaller. But serving AVIF to a client that cannot decode it is worse than serving nothing. The client declares what it accepts in the Accept header, so the worker picks the best format listed and falls back to the original.

Not Every Device Needs Every Pixel

A phone on cellular and a desktop on fiber should not receive the same multi-megapixel asset, so the worker reads a device class from request signals and caps the width. Akamai selected breakpoints for us; on Cloudflare it is a few lines of our own code.

None of these are visual-quality settings. They are policy (i.e., security, compatibility, and cost) and policy is code. Here is the shape of the decision, with the checks in priority order:

function planOptimization(request: Request, config: TenantConfig): OptPlan | null {
  // Security gates first; any failure means "do not touch this image."
  if (request.method !== "GET") return null;
  if (hasSessionCookie(request)) return null;
  if (config.optOut) return null;
  if (isExcludedType(request.url)) return null;

  const accept = request.headers.get("Accept") ?? "";
  if (!accept.startsWith("image/")) return null;

  const format =
    accept.includes("image/avif") ? "avif" :
    accept.includes("image/webp") ? "webp" : "origin";

  return { format, maxWidth: widthForDevice(deviceClass(request)), fit: "scale-down", quality: DEFAULT_QUALITY };
}

Two design choices in that function separate a platform feature from a toy.

The first design choice is per-tenant opt-out (config.optOut), the direct answer to the granularity problem. Optimization is on by default, but any tenant can turn it off for their account without a deployment and without affecting anyone else, which is the per-account control a zone-level toggle cannot give you. That is why the decision reads tenant configuration at request time, and why the next problem is where that configuration lives and how it stays fast on a hot path.

The second design choice is what happens when optimization fails, because at scale it will. The transform can reject an image, time out, or hit a resource limit. The degrade-to-origin rule applies with no exceptions: A failed optimization must never become a failed image. The worker treats the transform's error as a branch, not an exception, either returning the original bytes or retrying the plain origin path.

Either way the user receives their image. For every optional feature on this hot path the rule is the same: The optimization is the enhancement and the original is the contract.

Deployment and Operations at Scale

For a single site, a deployment is a push. Replace the version, and if it breaks, push again. Across hundreds of thousands of tenant accounts, "replace it everywhere at once" becomes a blast radius chosen on purpose, because one bad version reaches every tenant in the time it takes to propagate. So here, a deployment is a controlled rollout, with three properties making it survivable.

A Release Is an Immutable Set of Versions, Not a Branch State

Each worker versions independently. A release pins one version of each into a single named tag. You never deploy "whatever is on main". Instead, you deploy a specific, reproducible set, which is what moves through every phase.

// A release is an immutable set of per-worker versions, pinned by a tag.
const release = {
  tag: "release/2024-11-14",
  workers: { gateway: "4.2.0", imageOptimizer: "2.7.1", failover: "3.0.0" },
};

Rollback Is Not a Special Operation

Because the previous release is still frozen under its own tag, rolling back is just deploying it, requiring neither a revert-commit scramble, nor a hotfix under pressure.

This design paid off early. A worker version switched to streaming request bodies for larger uploads, while a stream can be read only once. When an origin answered a POST with a redirect, the body was already consumed, so the worker returned a 500 instead of following it. The previous version had passed the whole request object through, so the body was still there when the redirect needed it, and recovery was a redeployment of the previous tag. Because the release moved in cohorts, the regression surfaced in the canary cohort (i.e., internal and pre-production accounts) before it reached the broad fleet.

Rollout Is Staggered Across Cohorts

A canary is sent first, then wider slices, verifying between phases with automated checks and a human gate. A transient error retries; a bad release halts the rollout instead of grinding through every account. It picks its blast radius in advance instead of discovering it afterward.

Figure 2. Staggered rollout across account cohorts, with verification gates and rollback to the previous tag. (Source: created by author.)

One more distinction should be noted: Not everything that varies per tenant travels through this pipeline. Worker code moves through the versioned, staggered, gated path above, because a code change is rare (releases land on the order of a few a quarter, not many a day) and its downside is an outage. Per-tenant configuration, such as the image opt-out, is data, not code. It lives in a separate plane, with the key-value store the opt-out needed read at request time and changeable with no deployment. Reading configuration on the hot path costs a store lookup per request, which is why each worker fronts the store with a small, bounded, short-lived cache. This is the same memory-bounded local cache the multi-CDN work arrived at for the same reason: The edge caps memory and the hot path cannot afford an extra round trip.

Testing Strategy for Edge Code

The temptation is to think edge code needs less testing than application code. It is a small function. It rewrites a header, picks a format, forwards to origin. What could possibly go wrong? The answer from the monolith problem is: everything, all at once, for everyone. Edge code runs in the synchronous path, while multi-tenant runs on a platform you do not control. That does not lower the testing bar. It raises it, and changes where the tests have to live.

The architecture was already built for this. The predicate-and-worker split arrived as a runtime economy, but it is also where tests attach. A predicate is a pure function of the request: given a request and some config, does this feature apply and how? That makes it the easiest thing in software to test, just inputs and expected outputs, no network and no platform to stand up. The Akamai EdgeWorker deliberately exports its internal helpers so they can be unit-tested in isolation. The cheap-decision layer is also the cheap-to-test layer.

// Predicates are pure functions of the request, so they test with no edge
// runtime at all: no network, no mocks, just inputs and expected outputs.
test("skips authenticated requests, even when the format is optimizable", () => {
  const req = new Request("https://site/img.jpg", {
    headers: { Accept: "image/avif", Cookie: "session=abc" },
  });
  expect(planOptimization(req, PUBLIC_CONFIG)).toBeNull();
});

So the first layer is dense, fast unit coverage of every feature’s decision logic, running in milliseconds with no edge runtime. If the predicates are right in isolation then you have closed off most of the ways a feature misbehaves.

The second layer is integration at the gateway. Its highest-value target is the one normal operation exercises least: failure. The architecture insists at two points that a feature which fails must degrade to origin rather than fail the request, a behavior that rarely occurs in a passing run, which is exactly why it has to be tested deliberately. The integration suite makes a feature worker throw, time out, and hand back garbage, then asserts the user still receives a valid response every time. The failure path is not an edge case here. It is the product.

The third layer separates edge testing from ordinary testing: You cannot unit-test the platform. No local test tells you whether the CDN’s transform honored your format request in a given region, whether a service binding dispatched in-isolate as promised, or whether a config read on the hot path returned fast enough under real load. That behavior exists only in production, so you test it there, with synthetic monitoring: scripted requests running continuously against live endpoints, per region and, because the platforms diverge, per CDN.

You do not validate that "the feature works", but that it works on each platform separately, because the platforms are not the same and never will be. The deployment-time health gate is the same instinct in a narrower window, a synthetic check with the authority to stop a release.

Key Lessons and Transferable Patterns

Strip away the CDN specifics and what remains is a way of working: Treat the constraints as the design input, not the inconvenience. Four of the patterns leave the edge intact.

Decompose along a cost boundary. Any system with a per-request cost gate can borrow "decide inline, execute remote", and use modularity without paying for it on the requests that do not need it.

Make the enhancement optional and the original the contract. Any optional enrichment on a hot path is judged less by its best case than by whether its worst case is invisible to the user.

Standardize the invariant when you cannot standardize the platform. For any multi-vendor system, own in code the part that must behave identically and expect the vendor-specific part to drift.

Separate planes by blast radius. Splitting changes by how often they happen and how much they can break is a general discipline, not an edge one.

The testing lesson compounds all of these lessons. Because each feature's decision is a pure function, its tests are mechanical enough to scaffold from a template or an assistant and gate before commit. Using the structure right, the mechanical work can move to a pipeline or an assistant. Deciding which features compose and in what order, still needs someone who knows the traffic.