Service discovery and linksEstimated reading time: 11 minutes
Docker Cloud creates a per-user overlay network which connects all containers
across all of the user’s hosts. This network connects all of your containers on
10.7.0.0/16 subnet, and gives every container a local IP. This IP persists
on each container even if the container is redeployed and ends up on a different
host. Every container can reach any other container on any port within the
Docker Cloud gives your containers two ways find other services:
Using service and container names directly as hostnames
Using service links, which are based on Docker Compose links
Service and Container Hostnames update automatically when a service scales up or down or redeploys. As a user, you can configure service names, and Docker Cloud uses these names to find the IP of the services and containers for you. You can use hostnames in your code to provide abstraction that allows you to easily swap service containers or components.
Service links create environment variables which allow containers to communicate with each other within a stack, or with other services outside of a stack. You can specify service links explicitly when you create a new service or edit an existing one, or specify them in the stackfile for a service stack.
Hostnames vs service links
When a service is scaled up, a new hostname is created and automatically resolves to the new IP of the container, and the parent service hostname record also updates to include the new container’s IP. However, new service link environment variables are not created, and existing ones are not removed, when a service scales up or down.
Using service and container names as hostnames
You can use hostnames to connect any container in your Docker Cloud account to any other container on your account without having to create service links or manage environment variables. This is the recommended service discovery method.
Hostnames will always resolve to the correct IP for the service or container, and will update as the service scales up, scales down, or redeploys. The Docker Cloud automatic DNS service resolves the service name to the correct IP on the overlay network, even if the container has moved or is now on a different host.
Discovering containers on the same service or stack
A container can always discover other containers on the same stack using just the container name as hostname. This includes containers of the same service. Similarly, a container can always discover other services on the same stack using the service name.
For example, a container
webapp-1 in the service
webapp can connect to the
db-1 in the service
db by using
db-1 as the hostname. It can
also connect to a peer container,
webapp-2, by using
webapp-2 as the
proxy-1 on the same stack could discover all
by using the service name
webapp as hostname. Connecting to the service
name resolves as an
round-robin record, listing all
IPs of all containers on the service
Discovering services or containers on another stack
To find a service or a container on another stack, append
.<stack_name> to the
service or container name. For example, if
webapp-1 on the stack
needs to access container
db-1 on the stack
common, it could use the
db-1.common which Docker Cloud will resolve to the appropriate IP.
Discovering services or containers not included in a stack
To find a container or service that is not included in a stack, use the service or container name as the hostname.
If the container making the query is part of a stack, and there is a local match on the same stack, the local match will take precedence over the service or container that is outside the stack.
Tip: To work around this, you can rename the local match so that it has a more specific name. You might also put the external service or container in a dedicated stack so that you can specify the stack name as part of the namespace.
Using service links for service discovery
Docker Cloud’s service linking is modeled on Docker Compose links to provide a basic service discovery functionality using directional links recorded in environment variables.
When you link a “client” service to a “server” service, Docker Cloud performs the following actions on the “client” service:
Creates a group of environment variables that contain information about the exposed ports of the “server” service, including its IP address, port, and protocol.
Copies all of the “server” service environment variables to the “client” service with an
Adds a DNS hostname to the Docker Cloud DNS service that resolves to the “server” service IP address.
Some environment variables such as the API endpoint are updated when a service scales up or down. Service links are only updated when a service is deployed or redeployed, but are not updated during runtime. No new service link environment variables are created when a service scales up or down.
Tip: You can specify one of several container distribution strategies for applications deployed to multiple nodes. These strategies enable automatic deployments of containers to nodes, and sometimes auto-linking of containers. Note that if a service with EVERY_NODE strategy is linked to another service with EVERY_NODE strategy, containers will be linked one-to-one on each node.
Service link example
For the explanation of service linking, consider the following application diagram.
Imagine that you are running a web service (
my-web-app) with 2 containers
my-web-app-2). You want to add a proxy service
my-proxy) with one container (
my-proxy-1) that will balance HTTP traffic to
each of the containers in your
my-web-app application, with a link name of
Service link environment variables
Several environment variables are set on each container at startup to provide link details to other containers. The links created are directional. These are similar to those used by Docker Compose.
For our example app above, the following environment variables are set in the
proxy containers to provide service links. The example proxy application can use
these environment variables to configure itself on startup, and start balancing
traffic between the two containers of
To create these service links, you would specify the following in your stackfile:
my-proxy: links: - my-web-app:web
This example snippet creates a directional link from
my-web-app, and calls that link
DNS hostnames vs service links
Note: Hostnames are updated during runtime if the service scales up or down. Environment variables are only set or updated at deploy or redeploy. If your services will scale up or down frequently, you should use hostnames rather than service links.
In the example, the
my-proxy containers can access the service links using following hostnames:
The best way for the
my-proxy service to connect to the
containers is using the hostnames, because they are updated during runtime if
my-web-app scales up or down. If
my-web-app scales up, the new hostname
web-3 automatically resolves to the new IP of the container, and the hostname
web is updated to include the new IP in its round-robin record.
However, the service link environment variables are not added or updated until
the service is redeployed. If
my-web-app scales up, no new service link
environment variables (such as
WEB_3_PORT_80_TCP, etc) are added
to the “client” container. This means the client does not know how to contact
the new “server” container.
Service environment variables
Environment variables specified in the service definition are instantiated in each individual container. This ensures that each container has a copy of the service’s defined environment variables, and also allows other connecting containers to read them.
These environment variables are prefixed with the
HOSTNAME_ENV_ in each
In our example, if we launch our
my-web-app service with an environment
WEBROOT=/login, the following environment variables are set and
available in the proxy containers:
In our example, this enables the “client” service (
my-proxy-1) to read
configuration information such as usernames and passwords, or simple
configuration, from the “server” service containers (
Docker Cloud specific environment variables
In addition to the standard Docker environment variables, Docker Cloud also sets special environment variables that enable containers to self-configure. These environment variables are updated on redeploy.
In the example above, you will also find the following environment variables in the
WEB_DOCKERCLOUD_API_URLis the Docker Cloud API resource URL of the linked service. Because this is a link, the link name is the environment variable prefix.
DOCKERCLOUD_SERVICE_API_URLare the Docker Cloud API resource URI and URL of the service running in the container.
DOCKERCLOUD_CONTAINER_API_URLare the Docker Cloud API resource URI and URL of the container itself.
DOCKERCLOUD_NODE_API_URLare the Docker Cloud API resource URI and URL of the node where the container is running.
DOCKERCLOUD_CONTAINER_FQDNare the external hostname and Fully Qualified Domain Name (FQDN) of the container itself.
DOCKERCLOUD_SERVICE_FQDNare the external hostname and Fully Qualified Domain Name (FQDN) of the service to which the container belongs.
DOCKERCLOUD_NODE_FQDNare the external hostname and Fully Qualified Domain Name (FQDN) of the node where the container is running.
These environment variables are also copied to linked containers with the
If you provide API access to your service, you can use the generated token
DOCKERCLOUD_AUTH) to access these API URLs to gather information or
automate operations, such as scaling.