Inference Public

Cloud foundations

The trouble with VPC peering at scale

A peering connection is the simplest way to connect two VPCs. A few dozen of them become a tangle that nobody can see in full.

Network cables passing through a cable guide beside a patch panel.

What a peering connection does

A VPC peering connection routes private traffic between two VPCs, which can belong to different accounts or sit in different Regions. AWS makes no hourly charge for the connection itself, so you pay only the relevant data transfer. Within a Region, security groups can refer to groups in the peered VPC. For two or three VPCs with a clear reason to talk, peering is often the right answer.

What it does not do

Peering is not transitive. If VPC A is peered with both B and C, traffic cannot pass from B to C through A, so B and C need their own connection. VPCs with overlapping address ranges cannot be peered at all. A VPC also cannot use its peer's internet gateway, NAT device, VPN, Direct Connect connection or S3 gateway endpoint, so each VPC needs its own route to the internet and to on-premises networks, or a design that works around the restriction.

The arithmetic of a full mesh

Connecting every VPC to every other needs n(n-1)/2 connections, which is 45 for ten VPCs and 190 for twenty. Each connection needs routes on both sides and security rules that match, and each VPC has a quota on how many peering connections it can hold.

Each individual connection was reasonable when it was made. Together they form a pattern that nobody can see whole, where adding a VPC touches many others, changing an address range is a small project and troubleshooting means walking route tables one pair at a time.

Ownership becomes unclear

A peering connection between accounts needs both sides to agree, which is sound in principle and means two owners for every link in practice. Nobody has a single place to say that production must never reach non-production, and connections created for a migration or a test are rarely removed when the reason has passed.

The alternatives

AWS Transit Gateway acts as a hub. Each VPC attaches once, route tables can separate production from non-production, and connections to on-premises networks can be shared centrally. Transit Gateway charges for each attachment by the hour and for each gigabyte of data sent through it, so a pair of VPCs exchanging large volumes may still justify a direct peering connection alongside the hub.

Where a consumer needs one service rather than a whole network, AWS PrivateLink can expose that service through an endpoint without joining the networks together. Whichever route you take, a central plan for IP addresses prevents the overlapping ranges that block future connections.

Untangling an existing mesh

Start with an inventory of every peering connection, its routes and its owners, and use flow logs to see which ones still carry traffic. Remove the unused connections first, plan a fix for any overlapping address ranges, then move VPCs to a hub in waves, keeping direct peering only where traffic volume or latency justifies it.

Working document: a connectivity register

A register of connections turns an invisible pattern into something that can be reviewed, costed and changed safely.

Illustrative template
FieldWhat to record
ConnectionIdentifier and type, such as peering, Transit Gateway attachment or PrivateLink endpoint.
EndsThe VPCs and accounts at each end.
OwnersThe accountable owner at each end.
PurposeWhich services need to communicate, and why.
Routes and rulesAddress ranges routed and the security rules that permit traffic.
Traffic and costTypical monthly volume and associated charges.
ReviewNext review date, and the removal date for temporary connections.

Read about infrastructure engineering or discuss your AWS foundations.

Further reading

Amazon VPC: VPC peering basics and limitations

AWS Transit Gateway pricing

AWS re:Post: choosing between VPC peering, Transit Gateway and Direct Connect

Related articles

Planning an OU strategy that outlasts the next reorganisation

Infrastructure engineering

All insights

Start a conversation

Let’s talk about what you need to build.

Bring the service, the constraint or the engineering question. We’ll work through a sensible starting point.

Discuss a project