Docker Swarm Auto-Clustering and Scaling with PaaS Power Armor

So-called Swarm Cluster represents a set of Docker Engine nodes that are run in a swarm mode to constitute a fault-tolerant platform for application hosting with built-in cluster management features. It helps to keep your dockerized services constantly running and available through distributing the workloads across different servers and/or data centers. Moreover, by its original implementation, Docker swarm provides such extra benefits as automatic disaster recovery, zero-downtime updates, etc. But at the same time, running such cluster of Docker nodes could be fraught with some difficulties as this technology is rather new and thus still requires considerable system administrator’s efforts for its handling.

That’s why in today’s article we will describe how to avoid the complexity of manual Swarm building and maintenance with a dedicated Docker Swarm Cluster package for one-click installation. Being based on a bundle of recent Jelastic innovations – Cloud Scripting and native Docker containers support, – this solution allows to set up a ready-to-go dockerized cluster of the required size in a matter of minutes. So, let’s go on and dive into the specifics of its realization and usage.

Docker Swarm Cluster Topology Specifics

In accordance to the original swarm cluster implementation, structure of the environment you’ll get upon installation of the package is composed of two layers (node groups) with the following prescribed node roles:

• Manager – maintains the desired state of your swarm and all of the services that are run on it;
• Worker – receives and executes tasks, that are dispatched from Manager node(s).

During installation, you can declare the desired number of containers for each layer (so-called “replicas”) and choose one of the available installation types:

• Setting up a bare cluster to subsequently deploy the required Docker image manually;
• Create a swarm with the already deployed image basing on the provided compose file.

By default, each node has a pre-allocated room of resources for growth upon the necessity (up to 4 GB RAM per container), whilst you always pay only for the actually consumed capacities due to the pay for use Jelastic pricing approach.

Herewith, cluster memory consumption in idle mode is less than 128 MB (1 cloudlet) per container. This way, a reliable Docker swarm cluster of 8 nodes can be hosted for a price of just 8 cloudlets!

As an additional benefit of your swarm cluster hosting at Jelastic, all containers are equally automatically distributed across the available at a platform hardware nodes (i.e., physical servers) to ensure extra high availability of your application.

Swarm Cluster Auto-Scaling Settings

Another major advantage of the Docker Swarm Cluster package by Jelastic stands for a special mechanism of automatic new nodes’ inclusion into a cluster. Herewith, depending on a scaled layer type, each added container is automatically assigned the appropriate master or worker role.

This allows to pre-configure automatic horizontal scaling of server nodes (up to 10 instances per layer by default), which is executed basing on the current cluster load according to the following conditions:

• +1 node if RAM or CPU usage is > 70% for at least 5 minutes;
• -1 node if RAM or CPU usage is < 40% for at least 5 minutes.

By default, horizontal auto-scaling is applied to both manager and worker node types, whilst a user receives the appropriate email notification upon each such event (i.e., scaling trigger execution). Herewith, the minimal nodes count can’t fall below the value you’ve specified for each layer during the initial Docker swarm cluster creation.

In case you’d like to tune the predefined automatic scaling conditions, refer to the appropriate triggers’ settings within the Automatic Horizontal Scaling section.

Docker Swarm Installation

The Docker Swarm installation from Jelastic Marketplace is fully automated and requires just a few simple steps:

1. Click the Marketplace button at the top dashboard pane and search for the Docker Swarm Cluster solution within an opened window.

Hover over the appropriate plank and click Install to continue.

2. In a second, the installation frame will be opened.

Here, you need to tune some details to get the cluster you need:

• Manager – number of manager nodes to be included into a cluster (with Public IP being automatically attached to each container);
• Worker – number of worker nodes to be included into a cluster;
• Choose deployment type:
  • Clean Cluster – to create a bare cluster; as an extra option, you can simultaneously install the Portainer management UI for convenient operating with cluster nodes;
     
  • Deploy Stack YML – to perform docker stack deploy alongside with cluster creation and instantly deploy the required dockerized services by specifying link to the appropriate compose YML file;
     
• Environment – a name for your environment;
• Display Name – optional alias to be displayed for environment.

As an example, we’ll create a bare Docker swarm cluster with 3 manager and 5 worker nodes. So, once all the required configurations are stated, click Install and wait for a few minutes for Jelastic to automatically perform all required actions.

3. After your cluster is deployed, the corresponding environment will appear at the dashboard.

Next, in order to start working with your cluster, you need to access any of its manager nodes – below we’ll discover the ways this could be accomplished through.

Docker Swarm Connection

After the successful installation, your cluster can be accessed in the following ways for further management:

• Portainer UI
  In case you’ve installed a clean cluster with the Portainer UI, the appropriate management panel can be accessed by simply opening any of your manager nodes in a browser through HTTPS.
   

  Note: By default, connection to Portainer is secured with a self-signed SSL certificate, thus you might be shown the warning that it’s not trusted by your browser. This does not actually affect the Portainer functionality – you just need to confirm you do want to access the requested page.

  

  The required for that actions could differ depending on a used browser – for example, in Google Chrome you’ll need to expand the Advanced section and click on the Proceed to {env_URL} link.

  The appropriate authentication credentials to enter the Portainer itself can be found within the corresponding email notification you’ve received during the Docker swarm creation.

• SSH Gate
  Use Jelastic SSH Gate – just connect to your account and choose one of the Manager nodes within your Docker swarm environment to start managing the whole cluster.
   Also, starting with the 5.4 Jelastic release, you can establish an SSH connection to the required container directly via your browser by clicking on the Web SSH option next to it at the dashboard.
  
• Docker Machine
  Another way of container access is to establish a remote connection through Docker machine and generic driver. For that, execute the appropriate Create remote connection and Connect to the environment commands, provisioned within the solution installation success frame (also, you can find this data in your email box).
   

This concludes our article – creating and connecting Swarm cluster with Jelastic is that simple! Try it out right now and get your reliable and highly scalable cluster with access to native Docker functionality.