This post is dedicated to the project i had been embarked on recently. Basically, it’s a contribution to the open-source Docker project

It’s still a “work in progress” kind of thing, but even though, we have some rips to steak together 🙂

The Final code is in Docker’s GitHub Repo. Click here to get it!

What’s it all about?!

Rather than dwelling on what is Docker and what it’s good for(other websites made an excellent job on this!), I’m gonna go ahead and dive into the good stuff – work we had done.

Ready?!

Added Feature

A running container is equivalent to an operating system in action! Simple as that, really.

Saying that, it’s fair to assume that a container has its own volume of resources dedicated to it. For the sake of simplicity,  i’m considering a volume for a file-system(correct, a tree of files/directories..etc).

What’s the problem with that?!

Nothing. It’s all good and such, except the fact that you cannot add any external file-system(volume, remember?) to the container once you started it and kicked it running!

Our feature

We want to add a command to Docker, so that it’s possible to add an external file-system to the container.

By ‘external’ i mean that the new file-system is located on the host machine. However, you can generalize this statement and say: Located on a different namespace.

How it’s done?!

‘Can’t run before you learn to walk’ (The Forbidden Kingdom)

Thanks to Jet Li for the above insight. Now we need to learn how to contribute to Docker.

Fair enough..

Gathering the correct tools

  1. Get a Linux machine(VMware does the job).
  2. Create a Github account.
  3. Install Docker (yes, it’s funny that you need docker in-order to ‘compile’ Docker)
  4. Fork the Docker open-source repository and clone it to your computer
  5. learn to use git commit, push, pull,..
  6. punch line: follow the instructions here

Adding a new command to Docker’s API

  • You need to add a ‘.go’ file in the directory ‘/api/client’ along with all the API commands. The name of the file should be identical to the name of your command.
  • In that ‘.go’ file you should include whatever your command does in a function named ‘Cmd[YourCommandName]’
  • Commit and push the changes to your forked github branch.
  • Make the binary file

Example:

Say you want to add the command ‘talk’, so you do the following:

  • Add a new source file named ‘talk.go’
  • <contents of the file>
  • Update the project on git hub:
    $ git add talk.go #CWD should be /api/client
    $ git commit -s -m "Making a dry run test."
    $ git push --set-upstream origin dry-run-test
    - add your github account name + password
    
  • Now change directory to docker-master directory and make the project
  • Change directory to /bundles/1.10.0-dev/binary
  • Execute: ./docker-1-10-dev talk
  • DONE!

For the example above, I used the following code for the command talk:

package client
import (
"fmt"
Cli "github.com/docker/docker/cli"
flag "github.com/docker/docker/pkg/mflag"
)
// new command's function. Note the convention to naming the function!
func (cli *DockerCli) CmdTalk (args ...string) error {
// adding the new command to the commands data-structure,
// so that you would see it when running '--help'
cmd := Cli.Subcmd("talk", []string{"CONTAINER"},
Cli.DockerCommands["talk"].Description, true)
cmd.Require(flag.Exact, 0)
cmd.ParseFlags(args, true)
fmt.Println("Hi")
return nil
}

The results were promising as you see in the following screen shot:

ourNewCommand
Click to enlarge

 

Adding an external file system

So far so good!

Now we want to add a new API command that knows to add an external file system to a [running] container.

Two steps:

  1. Created a new container
  2. Followed the instructions found in this github page

Pretty straight forward, isn’t it?! The guy made a neat job explaining how to do such a thing in the above page. Note that you might not understand 100% of the things he wrote, but that’s OK in this stage.

We followed up, and came up with the following script:

#!/bin/sh

# input1: name of the container($1):
# input2: host-path($3):

set -e
CONTAINER=$1
HOSTPATH=$2
echo "script: container name is: " $1
echo "script: path to mount is: " $2
#directory in the container to add volume to
CONTPATH=/demoDummy

echo 'Script: installing nsenter: '
docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter

# get the real path(not symlink)-
#of file system which we need to attach to container.
REALPATH=$(readlink --canonicalize $HOSTPATH)
FILESYS=$(df -P $REALPATH | tail -n 1 | awk '{print $6}')

#find on which device the volume sits
while read DEV MOUNT JUNK
do [ $MOUNT = $FILESYS ] &amp;amp;amp;amp;amp;&amp;amp;amp;amp;amp; break
done &amp;amp;amp;amp;lt;/proc/mounts
[ $MOUNT = $FILESYS ] # Sanity check!

#get the real path for DEV:
DEV=$(readlink --canonicalize $DEV)
echo "Script: this is our dev : " $DEV

while read A B C SUBROOT MOUNT JUNK
do [ $MOUNT = $FILESYS ] &amp;amp;amp;amp;amp;&amp;amp;amp;amp;amp; break
done &amp;amp;amp;amp;lt; /proc/self/mountinfo
[ $MOUNT = $FILESYS ] # Moar sanity check!
echo "Script: SUBROOT = " $SUBROOT

SUBPATH=$(echo $REALPATH | sed s,^$FILESYS,,)
DEVDEC=$(printf "%d %d" $(stat --format "0x%t 0x%T" $DEV))

echo "Script: executing commands in the container..."
docker-enter $CONTAINER sh -c \
"[ -b $DEV ] || mknod --mode 0600 $DEV b $DEVDEC"
docker-enter $CONTAINER mkdir /tmpmnt
docker-enter $CONTAINER mount $DEV /tmpmnt
docker-enter $CONTAINER mkdir -p $CONTPATH
docker-enter $CONTAINER mount -o \
bind /tmpmnt/$SUBROOT/$SUBPATH $CONTPATH
docker-enter $CONTAINER umount /tmpmnt
docker-enter $CONTAINER rmdir /tmpmnt
echo "Script: end"

NOTE: A more detailed explanation of what’s going on in the script is going to be on the next post!! stay tuned, folks!

Now that we have the script in hand, we’ll go ahead and make a golang source code for the command we want to add.

For now, the golang code is going to run the script above. That’s why we learned first how to run bash scripts and bash commands using Go

the Go source file: add_volume.go

&amp;amp;amp;lt;pre&amp;amp;amp;gt;package client
import (
"bufio"
"fmt"
"log"
"os"
"path/filepath"
"os/exec"
Cli "github.com/docker/docker/cli"
flag "github.com/docker/docker/pkg/mflag"
)

// adds a file system to the runing container's file system.
//input(parameters): id of the runing container, root of the sub-file-system which to be added
func (cli *DockerCli) CmdAdd_volume (args ...string) error {
cmd := Cli.Subcmd("add_voulme", []string{"CONTAINER"}, Cli.DockerCommands["add_volume"].Description, true)
cmd.Require(flag.Exact, 2)
cmd.ParseFlags(args, true)

//print the CWD of this command
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println("cwd = %s", dir)

containerName := os.Args[2]
pathOnHost := os.Args[3]
bashScriptPath := "../../../api/client/script"

fmt.Println("docker add_volume: Container name to start: " , containerName)
fmt.Println("docker add_volume: path on host: " , pathOnHost)
fmt.Println("docker add_volume: Bash script's path to execuste: " , bashScriptPath)
fmt.Println("-------------------------------------------")
startContainer(containerName)
runBashScript(bashScriptPath, containerName, pathOnHost)

return nil
}

//***************************************************************
func runBashCommand(cmdName string, cmdArgs []string) {
cmd := exec.Command(cmdName, cmdArgs...)
cmdReader, err := cmd.StdoutPipe()
if err != nil {
fmt.Fprintln(os.Stderr, "Error creating StdoutPipe for Cmd", err)
os.Exit(1)
}

scanner := bufio.NewScanner(cmdReader)
go func() {
for scanner.Scan() {
fmt.Printf("%s\n", scanner.Text())
}
}()

err = cmd.Start()
if err != nil {
fmt.Fprintln(os.Stderr, "Error starting Cmd", err)
os.Exit(1)
}

err = cmd.Wait()
if err != nil {
fmt.Fprintln(os.Stderr, "Error waiting for Cmd", err)
os.Exit(1)
}

}
//*****************************************************************
// start a container: docker start &amp;amp;amp;amp;lt;name&amp;amp;amp;amp;gt;
func startContainer(containerName string) {
cmdName := "docker"
cmdArgs := []string{"start", containerName }
runBashCommand(cmdName, cmdArgs)
}
//*****************************************************************
// runs a bash script which found int path.
//TODO: check if the path actually leads to a bash script(aka chek for input sanity)
func runBashScript(path string, containerName string, pathOnHost string) {
out , err := exec.Command("/bin/sh", path, containerName, pathOnHost).Output()
if err != nil {
fmt.Println("Error: %s", err)
}
fmt.Printf("%s", out)
}&amp;amp;amp;lt;/pre&amp;amp;amp;gt;

~to  be continued! (10th December 2015)

Advertisements