BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News AWS PrivateLink Tunnel Endpoints Open a Whole CIDR Range to Another Account

AWS PrivateLink Tunnel Endpoints Open a Whole CIDR Range to Another Account

Listen to this article -  0:00

AWS has published a technical walkthrough of tunnel endpoints, a VPC endpoint type launched in September that lets an application in one VPC reach every address in a CIDR range shared from another account, instead of one resource at a time. The provider shares a CIDR-type resource configuration through AWS Resource Access Manager and the consumer creates a tunnel endpoint against it. From then on, anything inside the range is reachable, including resources that did not exist when the share was made.

Bruce Pinsky and Ashish Kumar of AWS frame it as a vendor-access problem, for "a vendor or service provider whose software must reach applications and resources in a customer's network." Until now, that meant a separate PrivateLink resource configuration per database or host, which breaks down when the resources are ephemeral, or the customer does not want to enumerate them. With a CIDR configuration, the customer defines a perimeter in address ranges, and "resources located outside the perimeter can't be reached by the vendor." Two further properties come with the design: the consumer and provider VPCs can use overlapping CIDR blocks, and connections "can start only from the consumer VPC, not from the provider network."

Consumer analytics service reaching all provider data sources, including new db-5, through one tunnel endpoint

(Source: AWS Networking & Content Delivery blog)

On the r/aws thread about the September launch, the first question was what this does that VPC peering does not, from a self-described veteran of AWS's advanced networking components. The answers that followed are the use cases: a peering connection exposes the whole VPC in both directions, while a tunnel endpoint shares a subset of the address space in one direction only; one commenter at a bank wrote that "peering with unregulated partners is a massive no-no, the auditors will rip you to shreds for it." Another pointed to clusters that do client-side load balancing, such as self-managed Kafka or Aerospike, which until now needed port-based routing on a Network Load Balancer and a separate port-mapping horizon for connections arriving over an interface endpoint. The same thread drew the line against VPC Lattice itself: Lattice services work at layer 7, tunnel endpoints at layer 4 without a resource configuration per target.

The unusual part is on the consumer side: the application's traffic has to arrive at the tunnel endpoint wrapped in GENEVE, the encapsulation protocol defined in RFC 8926, over UDP port 6081 and in the same Availability Zone. The endpoint strips the header, checks that the inner destination falls inside the shared range, rewrites the source to a resource gateway address, and forwards a plain packet into the provider VPC. Linux can do the wrapping in the kernel. The walkthrough creates an ip link add ... type geneve id 0 device inside its own network namespace with a default route through it, which sends the provider's ranges into the tunnel and leaves the instance's other traffic alone. On the far side, the resources see the resource gateway as the client, so the post advises security groups that reference the gateway's security group or subnet rather than a single address.

The documentation lists the constraints. Inner traffic must be TCP, with UDP permitted only for DNS, so ICMP and ping do not work, and IP fragments are dropped. Private DNS names are not supported on tunnel endpoints. The endpoint's subnets must sit in Availability Zones that overlap with the resource gateway's, or the association shows as inaccessible. A resource gateway's DNS setting cannot be changed after creation. And the access logs identify the consumer's endpoint but carry no field for the inner source address, so a provider can attribute traffic to a vendor and not to a workload behind that vendor's endpoint. The post also warns that sharing 0.0.0.0/0 grants access to everything the provider VPC can route to, including networks behind VPC peering, Direct Connect or a Site-to-Site VPN, and recommends sharing the narrowest ranges that work.

Both points have been tested in practice. Ryo Suzuki of the Japanese consultancy Classmethod connected two VPCs that both use 10.0.0.0/16 and got HTTP through, but only by running a GENEVE relay as root on the client, by routing a /32 per destination host, or by putting an HAProxy or NAT instance in front, each of which adds a hop or a single point of failure; his conclusion was that "route management according to the number of destinations is required." In a second experiment, he shared 0.0.0.0/0 from a VPC with a NAT gateway and reached the internet from a VPC with no gateway at all, which confirmed AWS's warning from the other side; he judged it workable as a temporary path and "difficult to use in a permanent production setup."

Pricing is an hourly charge per tunnel endpoint plus a per-GB charge for data processed, under PrivateLink pricing, with the provider side billed per GB through the resource gateway under VPC Lattice pricing, and the feature is available in 28 regions. Terraform support followed from the community: a contributor request opened two days after launch was implemented by another contributor and shipped in version 6.71.0 of the AWS provider, as a Tunnel endpoint type on aws_vpc_endpoint and a cidr_resource block on the Lattice resource configuration.

About the Author

Rate this Article

Adoption
Style

BT