Compute Fleet Strategy
A fleet strategy defines which instance types an organization uses, why, and how teams are steered toward preferred options. Without one, organizations end up with dozens of instance families, each with its own ODCR, Savings Plan coverage gap, and capacity planning headache.
Why fleet consolidation matters
Every distinct instance family in production creates:
- Commitment fragmentation — Savings Plans / CUDs are more effective when usage is concentrated in fewer families. Broad scatter means each family has thin coverage.
- Capacity planning overhead — each family needs its own ODCR analysis, pool-depth monitoring, and AZ distribution.
- Pricing negotiation weakness — cloud providers offer deeper discounts for volume on specific families. Scatter weakens negotiating position.
- Operational complexity — AMIs, container images, performance baselines, and monitoring differ per architecture (x86 vs ARM) and generation.
Fleet strategy doesn’t mean “everyone uses the same instance.” It means a curated preferred list with a clear process to deviate.
Blessed / preferred instance types
A “blessed list” is the organization’s shortlist of recommended EC2/VM types. Typically maintained by infrastructure governance or capacity engineering.
What makes a type blessed
- Volume discount — enough organizational usage to justify commitment
- Capacity depth — cloud provider has good availability in target regions/AZs
- ODCR alignment — existing reservations match the type
- Performance characteristics — validated for the org’s common workload profiles
- Support maturity — OS images, monitoring, security agents all tested
Steering mechanisms
| Mechanism | How it works |
|---|---|
| Documentation + recommendation | ”Use these unless you have a reason not to” |
| Capacity request friction | Non-blessed types require extra justification |
| Cost allocation incentives | Blessed types charged at blended rate; unblessed at on-demand |
| Platform defaults | Kubernetes node pools / managed platforms only offer blessed types |
| Automated recommendations | Rightsizing tools suggest blessed alternatives |
| Quota gates | Provisioning unblessed types requires approval |
The goal is not to block teams but to make the default path cheap and easy, and deviations intentional and justified.
Instance generation migration
Cloud providers release new instance generations regularly (m5 → m6i → m7i). Newer generations typically offer:
- 10–30% better price-performance
- Better power efficiency
- Access to newer features (larger networking, better EBS throughput)
Fleet strategy includes a generation migration plan:
- Validate new generation for org workloads (benchmark)
- Add to blessed list
- Make new deployments default to new generation
- Migrate existing workloads on a schedule
- Deprecate old generation from blessed list
- Coordinate with commitment strategy (don’t buy 3-year RIs on outgoing gen)
Graviton / ARM migration
ARM-based instances (AWS Graviton, GCP Tau T2A, Azure Cobalt) offer 20–40% better price-performance for compatible workloads. But migration requires:
Prerequisites
- Application compatibility with ARM (no x86-specific assembly, no x86-only dependencies)
- Container images built for ARM (multi-arch builds)
- All dependencies available on ARM (some native libraries lag)
- Performance testing with representative traffic
Migration economics
The correct metric is cost per unit of work, not instance hourly price:
Good: cost per successful request went from \$X to \$0.7X on Graviton
Bad: just compared hourly price without testing throughput
A workload that runs 20% slower on ARM but costs 40% less per hour still saves 28% net. But you must measure — never assume 1:1 vCPU equivalence.
Bad migration pattern
x86 service → same vCPU count on ARM → no performance test → hidden latency regression
Good migration pattern
- Identify eligible workloads (no x86 hard dependencies)
- Build ARM container images
- Benchmark with representative traffic (compare latency P50/P99, throughput)
- Compare cost per unit of work
- Check capacity availability and commitment alignment
- Canary rollout (ARM alongside x86)
- Measure unit economics after migration
- Full rollout with fast rollback path
How to steer teams toward the right instance
The actual challenge isn’t “which family fits which profile” (CPU-bound → c-family is obvious). The challenge is teams don’t profile their workloads at all — they copy what another service uses or pick big “to be safe.”
The fleet strategy fix is making the default path correct without requiring each team to be an instance expert:
Steering teams away from wrong choices
When a team requests a non-blessed or oversized instance, the conversation is:
| Team says | Response |
|---|---|
| ”I need c5.18xlarge" | "What’s your CPU utilization? Would 2× c5.9xlarge give you HA + fit blessed list?" |
| "I need p4d.24xlarge for inference" | "Have you benchmarked on g5.xlarge or inf2? P4d is for training at scale." |
| "I need r5.metal" | "What drives the memory need? Is it cache (could use ElastiCache)? Data (could use EBS-backed)?" |
| "I want the newest generation" | "Happy to support if it’s blessed. Let’s check ODCR/SP alignment first.” |
The key: never say “no” without offering an alternative that meets the actual need at lower cost or better alignment.
Fleet strategy metrics
| Metric | What it measures |
|---|---|
| Blessed type coverage | % of instances on blessed types |
| Generation currency | % on current vs previous generation |
| ARM adoption | % of eligible workloads on Graviton/ARM |
| Family concentration | number of distinct families in production |
| Commitment alignment | % of usage matching Savings Plan / RI / ODCR shape |
| Instance type sprawl | number of distinct instance type+size combinations |