The VolumeDriver was moved from config to HostConfig since Engine API v1.21

[mxylin]

Symptom: When creating containers on a Swarm (1.2.6) cluster with named volumes using Docker SDK for Python, the named volumes will always be created using local driver whether a volume driver is specified or not.

Cause: VolumeDriver config generated by current version (2.2.1) of Docker SDK for Python to create/run a container is ignored/filtered by Swarm manager.

Please check docker-archive/classicswarm#2669 for detailed description, thank you.

[shin-]

Thank you for the report. Does it work if you force the API version to 1.20?

[mxylin]

No, forcing API version to 1.20 doesn't make it work. (details later)

Currently, Docker Engine for Linux is not affected by this issue because it handles backward compatibility for VolumeDriver. (https://github.com/docker/docker/blob/master/runconfig/config_unix.go)

That is, when I want to create a container with a named volume and specify a volume driver using Docker SDK for Python:

• it works¹ if DOCKER_HOST points to a Docker Engine
• it doesn't work² if DOCKER_HOST points to a Docker Swarm (standalone) manager

To illustrate:

• This works¹:

                 create container
                with volume-driver
    docker-py  --------------------> Docker Engine   
  

• This doesn't work² because VolumeDriver got filtered/ignored at the Docker Swarm part:

                 create container
                with volume-driver
    docker-py  --------------------> Docker Swarm ------> Docker Engine   
  

1: works: a new named volume is created using the specified volume driver.

2: doesn't work: a new named volume is created using local driver.

Test details of enforcing API version to 1.20

Testing with docker-py/2.2.1, swarm/1.2.6

>>> import docker
>>> client = docker.from_env(version='1.20')
/usr/local/lib/python2.7/dist-packages/docker/api/client.py:155: UserWarning: The minimum API version supported is 1.21, but you are using version 1.20. It is recommended you either upgrade Docker Engine or use an older version of Docker SDK for Python.
  'Python.'.format(MINIMUM_DOCKER_API_VERSION, self._version)

>>> client.version()
{
 u'KernelVersion': u'4.4.0-66-generic',
 u'Os': u'linux',
 u'BuildTime': u'`date -u`',
 u'ApiVersion': u'1.22',
 u'Version': u'swarm/1.2.6',
 u'GitCommit': u'`git rev-parse --short HEAD`',
 u'Arch': u'amd64',
 u'GoVersion': u'go1.7.1'
}

>>> container = client.containers.create(image='ubuntu', command='uname', name='ubuntu-test', volume_driver='convoy', volumes={'testvol': {'bind': '/mnt/vol', 'mode': 'rw'}})

request dump of client.containers.create():

POST /v1.20/containers/create?name=ubuntu-test HTTP/1.1
Host: localhost:2375
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 314
Content-Type: application/json
User-Agent: docker-sdk-python/2.2.1

{
	"Tty": false,
	"NetworkDisabled": false,
	"Image": "ubuntu",
	"Cmd": [
		"uname"
	],
	"StdinOnce": false,
	"HostConfig": {
		"Binds": [
			"testvol:/mnt/vol:rw"
		],
		"NetworkMode": "default"
	},
	"AttachStdin": false,
	"Volumes": {
		"/mnt/vol": {}
	},
	"VolumeDriver": "convoy",
	"AttachStderr": true,
	"AttachStdout": true,
	"OpenStdin": false
}

result:

$ docker volume ls
DRIVER              VOLUME NAME
local               testvol

Testing with docker-py/1.10.6, swarm/1.2.6

>>> import docker
>>> client = docker.from_env(version='1.20')
>>> client.version()
{
 u'KernelVersion': u'4.4.0-66-generic',
 u'Os': u'linux',
 u'BuildTime': u'`date -u`',
 u'ApiVersion': u'1.22',
 u'Version': u'swarm/1.2.6',
 u'GitCommit': u'`git rev-parse --short HEAD`',
 u'Arch': u'amd64',
 u'GoVersion': u'go1.7.1'
}

>>> container = client.create_container(image='ubuntu', command='uname', name='ubuntu-test', volume_driver='convoy', volumes=['/mnt/vol'],host_config=client.create_host_config(binds=['testvol:/mnt/vol']))

request dump of client.create_container()

POST /v1.20/containers/create?name=ubuntu-test HTTP/1.1
Host: localhost:2375
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 311
Content-Type: application/json
User-Agent: docker-py/1.10.6

{
	"Tty": false,
	"NetworkDisabled": false,
	"Image": "ubuntu",
	"Cmd": [
		"uname"
	],
	"StdinOnce": false,
	"HostConfig": {
		"Binds": [
			"testvol:/mnt/vol"
		],
		"NetworkMode": "default"
	},
	"AttachStdin": false,
	"Volumes": {
		"/mnt/vol": {}
	},
	"VolumeDriver": "convoy",
	"AttachStderr": true,
	"AttachStdout": true,
	"OpenStdin": false
}

result:

$ docker volume ls
DRIVER              VOLUME NAME
local               testvol

Reference

The volume_driver support was added to docker-py in #628