Friday, June 30, 2017

Using the Enterprise Version of MongoDB on GKE Kubernetes

[Part 3 in a series of posts about running MongoDB on Kubernetes, with the Google Kubernetes Engine (GKE). See the GitHub project gke-mongodb-demo for an example scripted deployment of MongoDB to GKE, that you can easily try yourself. The gke-mongodb-demo project combines the conclusions from all the posts in this series so far. Also see:]


In the previous two posts of my blog series (1, 2) about running MongoDB on GKE's Kubernetes environment, I showed how to ensure a MongoDB Replica Set is secure by default, resilient to system failures and how to ensure various best practice "production" environment settings are in place. In those examples, the community version of the MongoDB binaries were used. In this blog post I show how the enterprise version of MongoDB can be utilised, instead.

Referencing a Docker Image for Use in Kubernetes 

For the earlier two blog post examples, a pre-built "mongo" Docker image was "pulled" by Kubernetes, from Docker Hub's "official" MongoDB repository. Below is an excerpt of the Kubernetes StatefulSet definition, that shows how this image was referenced (highlighted in bold):

$ cat mongodb-service.yaml
        - name: mongod-container
          image: mongo
            - "mongod"

By default, if no additional metadata is provided, Kubernetes will look in the Docker Hub repository for the image with the given name. Other repositories such as Google's Container Registry, AWS's EC2 Container Registry, Azure's Container Registry, or any private repository, can be used instead.

It is worth clarifying what is meant by the "official" MongoDB repository. This is a set of images that are "official" from Docker Hub's perspective because Docker Hub manages how the images are composed and built. They are not, however, official releases from MongoDB Inc.'s perspective. When the Docker Hub project builds an image, in addition to sourcing the "mongod" binary from MongoDB Inc's website, other components like the underlying Debian OS, plus various custom scripts, are generated into the image too.

At the time of this blog post, Docker Hub currently provides images for MongoDB community versions 3.0, 3.2, 3.4 and 3.5 (unstable). The Docker Hub repository only contains images for the community version of MongoDB and not the enterprise version.

Building a Docker Image Using the MongoDB Enterprise binaries 

You can build a Docker image to run "mongod" in any way you want, using your own custom Dockerfile to define how the image should be generated. The Docker manual even provides a tutorial for creating a custom Docker image specifically for MongoDB, called Dockerize MongoDB. That process can be followed as the basis for building an image which pulls down and uses the enterprise version of MongoDB, rather than the community version.

However, to generate an image containing the enterprise MongoDB version, it isn't actually necessary to create your own custom Dockerfile. This is because, a few weeks ago, I created a pull request to add support for building MongoDB enterprise based images, as part of the normal "mongo" GitHub project that is used to generate the "official" Docker Hub "mongo" images. The GitHub project owner accepted and merged this enhancement a few days later. My enhancement essentially allows the project's Dockerfile, when used by Docker's build tool, to instruct that the enterprise MongoDB binaries should be downloaded into the image, rather than the community ones. This is achieved by providing some "build arguments" to the Docker build process, which I will detail further below. This doesn't mean the Docker Hub's "official repository" for MongoDB now contains pre-generated "enterprise mongod" images ready to use. It just means you can use the source Github project directly, without any changes to the project's Dockerfile, to generate your own image, containing the enterprise binary.

To build the Docker "mongo" image, that pulls in the enterprise version of MongoDB, follow these steps:

  1. Install Docker locally on your own workstation/laptop.

  2. Download the source files for the Docker Hub "mongo" project (on the project page, click the "Clone or download" green button, then click the "Download zip" link and once downloaded, unpack into a new local folder).

  3. From a command line shell, in the project folder, change directory to the sub-folder of the major version you want to build (e.g. "3.4"), and run:

  $ docker build -t pkdone/mongo-ent:3.4 --build-arg MONGO_PACKAGE=mongodb-enterprise --build-arg .

This tells Docker to build an image using the Dockerfile in the current directory (".") with the resulting image name being "pkdone/mongo-ent" and tag being ":3.4". By convention, the image name is prefixed by the author's username, which in my case is "pkdone" (obviously this should be replaced by a different prefix, for whoever follows these steps).

The two new "--build-arg" parameters, "MONGO_PACKAGE" and "MONGO_REP" are passed to the "mongo" Dockerfile.  The version of the Dockerfile with my enhancements uses these two parameters to locate where to download the specific type of MongoDB binary from. In this case, the values specified mean that the enterprise binary is pulled down into the generated image. If no "build args" are specified, the community version of MongoDB is used, by default.

Important note: When running the "docker build" command above, because the enterprise version of MongoDB will be downloaded, it will mean you are implicitly accepting MongoDB Inc's associated commercial licence terms.

Once the image is generated, you can also quickly test the image by running it in a Docker container on your local machine (as just a "mongod"  single instance):

$ docker run -d --name mongo -t pkdone/mongo-ent:3.4

To be sure this is running properly, connect to the container using a shell, check if the "mongod" process is running and check that the Mongo Shell can connect to the containerised "mongod" process.

$ docker exec -it mongo bash
$ ps -aux
$ mongo

The output of the Shell should include the prompt "MongoDB Enterprise >" which shows that the database is using the enterprise version of MongoDB. Exit out of the Mongo Shell, exit out of the container and then from the local machine, run the command to view the "mongod" container's logged output (with example results shown):

$ docker logs mongo | grep enterprise

2017-07-01T12:08:42.177+0000 I CONTROL  [initandlisten] modules: enterprise

Again this result should demonstrate that the enterprise version of MongoDB has been used.

Using the Generated Enterprise Mongod Image from the Kubernetes Project

The easiest way to use the "mongod" container image that has just been created, from GKE Kubernetes, is to first register it with Docker Hub, using the following steps:

  1. Create a new free account on Docker Hub.

  2. Run the following commands to associate your local workstation environment with your new Docker Hub account, to list the built image registered on your local machine, and to push this newly generated image to your remote Docker Hub account:

    $ docker login
    $ docker images
    $ docker push pkdone/mongo-ent:3.4   

  3. Once the image has finished uploading, in a browser, return to the Docker Hub site and log-in to see the list of your registered repository instances. The newly pushed image should now be listed there.

Now back in the GKE Kubernetes project, for the "mongod" Service/StatefulSet resource definition, change the image reference to be the newly uploaded Docker Hub image, as shown below (highlighted in bold):

$ cat mongodb-service.yaml
        - name: mongod-container
          image: pkdone/mongo-ent:3.4
            - "mongod"

Now re-perform all the normal steps to deploy the Kubernetes cluster and resources as outlined in the first blog post in the series. Once the MongoDB Replica Set is up and running, you can check the output logs of the first "mongod" container, to see if the enterprise version of MongoDB is being used:

$ kubectl logs mongod-0 | grep enterprise

2017-07-01T13:01:42.794+0000 I CONTROL  [initandlisten] modules: enterprise


In this blog post I’ve shown how to use the enterprise version of MongoDB, when running a MongoDB Replica Set, using Kubernetes StatefulSets, on the Google Kubernetes Engine. This builds on the work done in previous blog posts in the series, around ensuring MongoDB Replica Sets are resilient and better tuned for production workloads, when running on Kubernetes.

[Next post in series: Deploying a MongoDB Sharded Cluster using Kubernetes StatefulSets on GKE]

Song for today: Standing In The Way Of Control by Gossip


midn said...

replica Cartier is the renowned Frenchjeweller and watchmaker with a history of pioneering design and a drive for excellence. The first references to replica Ballon Blanc de Cartier watches wristwatches date back to 1888.

midn said...

Welcome to Cheap Ray-Ban Sunglasses uk here. Fake Ray-Ban Sunglasses and many more other brands sunglasses .

Benish said...

Really nice post. Thank you for sharing amazing information.
Python training in Chennai/Python training in OMR/Python training in Velachery/Python certification training in Chennai/Python training fees in Chennai/Python training with placement in Chennai/Python training in Chennai with Placement/Python course in Chennai/Python Certification course in Chennai/Python online training in Chennai/Python training in Chennai Quora/Best Python Training in Chennai/Best Python training in OMR/Best Python training in Velachery/Best Python course in Chennai

midn said...

Moncler have been developing their ski-inspired apparel since founded in 1952, moncler jackets for sale popular for their luxe quilted down coats originally produced for extreme cold-weather conditions. cheap moncler parka Sportswear-inspired clothing combined with street-style provides practical and versatile daily-wear pieces including Moncler t-shirts and polos perfect under a lightweight jacket.

midn said...

Short for Monestier-de-Clermont, an Alpine town near Grenoble, France, Moncler was founded in 1952 by René Ramillon as an outdoor-gear company, cheap moncler jackets moncler producing sleeping bags, tents, and other mountain-climbing necessities. Today, the French-Italian label is best known for its lightweight, super-warm quilted down jackets, cheap moncler trainers as well as ready-to-wear and accessories hardy enough to take on harsh winter weather.

Chandra Sekhar Reddy said...

Nice Information
Yaaron Studios is one of the rapidly growing editing studios in Hyderabad. We are the best Video Editing services in Hyderabad. We provides best graphic works like logo reveals, corporate presentation Etc. And also we gives the best Outdoor/Indoor shoots and Ad Making services.
video editing studios in hyderabad
short film editors in hyderabad
corporate video editing studio in hyderabad
ad making company in hyderabad

Chandra Sekhar Reddy said...

Nice Post
"Sanjary Academy provides excellent training for Piping design course. Best Piping Design Training Institute in Hyderabad,
Telangana. We have offer professional Engineering Course like Piping Design Course,QA / QC Course,document Controller
course,pressure Vessel Design Course, Welding Inspector Course, Quality Management Course, #Safety officer course."
Piping Design Course
Piping Design Course in India­
Piping Design Course in Hyderabad
QA / QC Course
QA / QC Course in india
QA / QC Course in Hyderabad
Document Controller course
Pressure Vessel Design Course
Welding Inspector Course
Quality Management Course
Quality Management Course in india
Safety officer course