The Plan

1. Docker by example: run a simple web app in docker.

2. A look at what's happening behind the scenes.

3. How we use docker at Overleaf, also by example.

Resources

These slides:

http://jdlm.info/ds-docker-demo

Source code for examples:

https://github.com/jdleesmiller/ds-docker-demo

Mental Model

Here's my (oversimplified) model for using docker.

1. You build a disk image.

2. You create a container with a (notional) copy of that image and then run your command inside it.

3. When the command finishes, you usually throw away the container.

Example Web App

Hello World! in Sinatra…

$ cd examples/web_app

$ cat hello_world.rb
require 'sinatra'

get '/' do
  "Hello World!"
end

Image from Dockerfile

Sort of like an ansible playbook or chef recipe.

FROM ubuntu:14.04                          # base image

MAINTAINER overleaf <team@overleaf.io>     # who are you?

RUN apt-get update && apt-get -y upgrade   # image setup commands
RUN apt-get install -y ruby
RUN gem install sinatra
RUN mkdir /app

ADD hello_world.rb /app/                   # copy files to image

ENV PORT 8000                              # set environment vars
ENV RACK_ENV production

Run docker build

To build the example web app:

$ docker build --tag hello_world .

--tag is how we'll refer to the image later.

. tells docker that the Dockerfile and ADDed files are in the current directory (examples/web_app).

Run the Web App in a Container

docker run -it --rm --publish 3000:3000 hello_world ruby /app/hello_world.rb
                                      # ^           ^ command in the container
                                      # ^ image tag

'--rm' removes the container once the command has exited

--publish forwards port 3000 on the host from port 3000 in the container

-it means interactive tty (terminal); this just lets us use Ctrl-C to interrupt the server.

Deploy the Image

You can save it to a tar file:

docker save hello_world | gzip > hello_world.tar.gz

and then import it with docker import.

Or you can run a private registry and push the image there.

Behind the Scenes…

Let's start with the build output.

Images are like git Commits

Each command in the Dockerfile makes a new image.

Step 0 : FROM ubuntu:14.04
...
Status: Downloaded newer image for ubuntu:14.04
 ---> 5506de2b643b

Step 1 : MAINTAINER overleaf 
...
 ---> 0bf3ae3662b4

Step 2 : RUN apt-get update && apt-get -y upgrade
...
 ---> 296a361125bd

Each image is identified by a hash of its content and its ancestor, like a git commit.

Only "Diffs" of Images are Stored

$ docker history hello_world
IMAGE               CREATED             CREATED BY                      SIZE
203c183acb03        2 minutes ago       ... #(nop) CMD [/bin/sh ...      0 B
d1df6bf66499        2 minutes ago       ... #(nop) ENV RACK_ENV=...      0 B
24a3db493063        2 minutes ago       ... #(nop) ENV PORT=3000...      0 B
e567e577d823        2 minutes ago       ... #(nop) ADD file:4164...     52 B
5695586ee7a3        2 minutes ago       ... mkdir /app          ...      0 B
a6447b00f7df        2 minutes ago       ... gem install sinatra ...   16.7 MB
60d6e9ae2318        3 minutes ago       ... apt-get install -y r...  17.67 MB
296a361125bd        4 minutes ago       ... apt-get update && ap...  33.49 MB
0bf3ae3662b4        5 minutes ago       ... #(nop) MAINTAINER ov...      0 B
5506de2b643b        5 days ago          ... #(nop) CMD [/bin/bas...      0 B
22093c35d77b        5 days ago          ... apt-get update && ap...  6.558 MB
3680052c0f5c        5 days ago          ... sed -i 's/^#\s*\(deb...  1.895 kB
e791be0477f2        5 days ago          ... rm -rf /var/lib/apt/...      0 B
ccb62158e970        5 days ago          ... echo '#!/bin/sh' > /...  194.8 kB
d497ad3926c8        8 days ago          ... #(nop) ADD file:3996...  192.5 MB
511136ea3c5a        16 months ago                                        0 B

The hello_world tag points to image 203c183acb03.

Incremental Rebuilds are Cheap

If we change examples/web_app/hello_world.rb to output a different message and rebuild the container, docker reuses cached images for the preceeding steps.

$ sed -i 's/World/Docker/' hello_world.rb
$ docker images -ta
...
Step 4 : RUN gem install sinatra
 ---> Using cache
 ---> a6447b00f7df
Step 5 : RUN mkdir /app
 ---> Using cache
 ---> 5695586ee7a3
Step 6 : ADD hello_world.rb /app/
 ---> 4c535a0a4a01 # now things have changed
...

Images form a Tree

$ docker images -ta
└─511136ea3c5a Virtual Size: 0 B
  ├─d497ad3926c8 Virtual Size: 192.5 MB
    ...
      └─5506de2b643b Virtual Size: 199.3 MB Tags: ubuntu:14.04
        ...
          └─5695586ee7a3 Virtual Size: 267.1 MB         # RUN mkdir /app
            ├─4c535a0a4a01 Virtual Size: 267.1 MB       # ADD hello_world...
            │ └─e7352ee9bedd Virtual Size: 267.1 MB
            │   └─fb17a307e866 Virtual Size: 267.1 MB Tags: hello_world:latest
            └─e567e577d823 Virtual Size: 267.1 MB       # ADD hello_world...
              └─24a3db493063 Virtual Size: 267.1 MB
                └─d1df6bf66499 Virtual Size: 267.1 MB
                  └─203c183acb03 Virtual Size: 267.1 MB # <-- the old hello_world

More About Containers

You can run a shell in a container to explore or use an image interactively:

docker run -it hello_world /bin/bash

If you make changes in the container and kill it (but don't remove it), you can restart it later, and your changes will persist. This can be useful in development, but docker has other solutions for data persistence that are better in production; see data volumes.

Web App Example Improvements

• The process runs as root inside our container, which is bad. Docker can run the process as an unprivileged user instead (exercise).

• We're using system ruby on Debian, which is far out of date. It's usually better to install with RVM (exercise) or find a docker image in the registry with ruby installed.

LaTeX in a Container

Because creating and destroying containers is very fast, we can use them for short-lived processes, too.

Example (examples/latex/Dockerfile):

FROM ubuntu:14.04

MAINTAINER overleaf <team@overleaf.io>

RUN apt-get update && apt-get -y upgrade

RUN apt-get install -y texlive-latex-base

WORKDIR /tmp # by default, start commands in the container in /tmp

LaTeX in a Container

To build and run:

cd examples/latex
docker build --tag texlive .
docker run -it --rm --volume `pwd`:/tmp texlive pdflatex main

--volume mounts the current working directory on the host as /tmp inside the container.

pdflatex main is the command that runs inside the container, to compile a file called main.tex.

LaTeX Example Improvements

• Again, we shouldn't be running the compile as root in the container, but that requires getting the permissions on the host to match up with those in the container.

• We could install a lot more software. A full install of TeX Live is over 4GB, and there are many related packages.

Docker at Overleaf

• We maintain a Dockerfile that builds an image with latex and lots of related tools.

• Each compile job gets its own short-lived container.

• It adds another layer of security.

• No real stability problems since ~0.9.

fin

Dr John Lees-Miller

@jdleesmiller @overleaf

Digital Science Tech Forum

30 Oct 2014