AWS PrivateLink

This guide is for a team whose clusters run in AWS. By default, DevZero's agents reach DevZero over the internet, through your VPC's NAT gateway. With AWS PrivateLink, you create an interface VPC endpoint in the VPC your cluster runs in, and the agents' traffic goes to DevZero over the AWS network instead. It never leaves AWS, and it costs less than sending it through a NAT gateway.

With private DNS on, dakr.devzero.io, the hostname DevZero's agents use by default, resolves to the endpoint inside your VPC, so agents set to it need no change. Agents installed with the older URL https://api.devzero.io/dakr keep going over the internet until you point them at dakr.devzero.io. See Check which hostname your agents use.

Setting it up takes two steps. A team Admin allows your AWS account in the DevZero console, and you then create the endpoint in that account.

Who can do what

Other members see that AWS PrivateLink is managed by team admins.

Regions

Each DevZero region has one endpoint service. The console shows its exact service name, its region, and the regions you can connect from.

A VPC in the endpoint service's own region connects directly. A VPC in another listed region connects with cross-region PrivateLink. If your clusters run in a region that is not listed, contact support@devzero.io.

Allow your AWS account

Allow every account your clusters run in. A team can allow up to 20 accounts.

Create the endpoint

In each allowed account, create an interface VPC endpoint in the VPC your cluster runs in. The Create the endpoint section of the console page shows the Terraform below, filled in with your service name. In the example the service is in us-west-2.

resource "aws_security_group" "devzero" {
  name   = "devzero-privatelink"
  vpc_id = var.vpc_id

  ingress {
    description = "HTTPS from the cluster"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [var.vpc_cidr_block]
  }
}

resource "aws_vpc_endpoint" "devzero" {
  vpc_id              = var.vpc_id
  service_name        = "com.amazonaws.vpce.us-west-2.vpce-svc-..." # from the console
  vpc_endpoint_type   = "Interface"
  subnet_ids          = var.private_subnet_ids
  security_group_ids  = [aws_security_group.devzero.id]
  private_dns_enabled = true

  # Only when this VPC is outside the endpoint service's region:
  # service_region    = "us-west-2"
}

• service_name is the Service name from the console.
• subnet_ids are private subnets your nodes can reach, one per Availability Zone you want the endpoint in.
• The security group must let your nodes reach the endpoint on TCP 443.
• private_dns_enabled = true makes the DevZero hostname resolve to the endpoint inside this VPC. The VPC needs DNS resolution and DNS hostnames turned on, which AWS requires for private DNS.
• service_region is needed only when the VPC is in another region from the Connect from list. Set it to the endpoint service's Region. Creating a cross-region endpoint also needs the IAM permission vpce:AllowMultiRegion.

The endpoint is accepted on creation; there is nothing to approve on DevZero's side. In the AWS console you can do the same from VPC > Endpoints > Create endpoint: choose Endpoint services that use NLBs and GWLBs, enter the service name, and choose Verify service.

Check which hostname your agents use

Only traffic to the private DNS name goes through the endpoint. Read the URL your agents use from the cluster:

kubectl -n devzero-system get configmap devzero-zxporter-env-config \
  -o jsonpath='{.data.DAKR_URL}'

If it prints https://dakr.devzero.io (or https://dakr.in.devzero.io in the India region), there is nothing to change. If it prints https://api.devzero.io/dakr, upgrade your DevZero install with the URL set to https://dakr.devzero.io. With the dz-platform chart, set all three of these values:

• zxporter.zxporter.dakrUrl
• dakr-operator.operator.endpoint
• dakr-security.operator.dakrServerURL, if dakr-security is on

Check that it works

Back in the console, the account lists each endpoint it created, with its ID, region, and state. An endpoint that carries traffic is Available. It shows Pending for a few minutes after you create it.

From a pod in the cluster:

nslookup dakr.devzero.io   # private 10.x, 172.16-31.x or 192.168.x addresses
curl -sI https://dakr.devzero.io

Use dakr.in.devzero.io in the India region. If nslookup returns private addresses but the agents still send traffic over the internet, check which hostname they use. If nslookup still returns public addresses, private DNS is not on for the endpoint, or the VPC does not have DNS resolution and DNS hostnames turned on.

Remove an account

On the account, choose Remove and confirm. The account can no longer create endpoints to DevZero. Endpoints it already created stay connected until you delete them in AWS. To stop using PrivateLink, delete the endpoint. The agents go back to the internet once the hostname resolves to public addresses again.

Things to know

• Agents still authenticate. Allowing an account only lets it create an endpoint to DevZero. Agents behind the endpoint authenticate with their cluster tokens, as they do over the internet.
• Only DevZero's API goes through the endpoint. The endpoint carries traffic to dakr.devzero.io (or dakr.in.devzero.io), not to api.devzero.io. Pulling DevZero's container images and Helm charts still goes out through your NAT gateway.
• Everything in the VPC uses it. With private DNS on, every client in the VPC that resolves the DevZero hostname reaches it through the endpoint, including a browser or the CLI. It works the same way.
• Availability Zones. A same-region endpoint can use only the Availability Zones the endpoint service is offered in. Zone names such as us-west-2a differ between AWS accounts, so compare zone IDs such as usw2-az1. To see which zones the service offers, run aws ec2 describe-vpc-endpoint-services --service-names <service name>. If AWS refuses a subnet, choose one in another zone.
• An account shared with another team. DevZero cannot tell which team owns an AWS account. If another DevZero team also allows the same account, neither team sees that account's endpoints, and the console says so. Removing it from your team does not cut off the other team.
• Leaving a team. The last member cannot leave a team while it still allows AWS accounts. Remove them in Settings > Organization Settings > AWS PrivateLink first.
• Not available. If the page says AWS PrivateLink is not available in this region, your DevZero region offers no endpoint service yet, and your agents reach DevZero over the internet. Contact support@devzero.io if you need a private connection.
• Cost. AWS bills the endpoint to your account, by the hour for each Availability Zone it is in and by the GB it processes. Cross-region endpoints also pay AWS's inter-region transfer. See AWS PrivateLink pricing.