Conv2DTranspose supports dilation

[yanboliang]

Summary

I saw the requirements for dilated transposed convolution at several places(#8159, tensorflow/tensorflow#21598). This PR makes Conv2DTranspose supports dilation for all backends.

How to test

Except the unit test I added, I also run the following test. As dilated transposed convolution is equivalent to standard transposed convolution with upsampled kernels with effective size kernel_size + (kernel_size - 1) * (dilation_rate - 1) , produced by inserting dilation_rate - 1 zeros along consecutive elements across the kernel' spatial dimensions. The following code verify these two scenarios that give the same output:

• kernel_size = 2, dilation_rate = 2
• kernel_size = 3, kernel is built from the above kernel and insert zeros along consecutive elements.

from keras.layers import Input, Conv2DTranspose
from keras.models import Model
from keras.initializers import Constant
import numpy as np

kernel = np.arange(24).reshape((2, 2, 2, 3))

inputs = Input(shape=(5, 6, 3))
m = Conv2DTranspose(2, kernel_size=2, padding='same', dilation_rate=2, kernel_initializer=Constant(kernel))(inputs)
model = Model(inputs=inputs, outputs=m)
model.summary()

x = np.arange(180).reshape((2, 5, 6, 3)).astype(np.float32)
y = model.predict(x)

print(np.squeeze(y))
print(y.shape)

print("--------------------------------------------------")

kernel = np.arange(24).reshape((2, 2, 2, 3))

# Insert zeros along consecutive elements across the kernel' spatial dimensions
k = np.zeros((3, 3, 2, 3))
k[0, 0, :, :] = kernel[0, 0, :, :]
k[0, 2, :, :] = kernel[0, 1, :, :]
k[2, 0, :, :] = kernel[1, 0, :, :]
k[2, 2, :, :] = kernel[1, 1, :, :]
kernel = k

inputs = Input(shape=(5, 6, 3))
m = Conv2DTranspose(2, kernel_size=3, padding='same', kernel_initializer=Constant(kernel))(inputs)
model = Model(inputs=inputs, outputs=m)
model.summary()

x = np.arange(180).reshape((2, 5, 6, 3)).astype(np.float32)
y = model.predict(x)

print(np.squeeze(y))
print(y.shape)

Related Issues

#8159
tensorflow/tensorflow#21598

PR Overview

• This PR requires new unit tests [y/n] (make sure tests are included)
• This PR requires to update the documentation [y/n] (make sure the docs are up-to-date)
• This PR is backwards compatible [y/n]
• This PR changes the current API [y/n] (all API changes need to be approved by fchollet)

[fchollet]

Thanks for the PR! Looks good to me at a high level.

[fchollet]

This must be added to the docstring.

[yanboliang]

The docstring has already exist at L671.

[fchollet]

Why do you need this argument?

[yanboliang]

This is not necessary, I just follow the test case above, will remove it.

[fchollet]

Is this unit test really sufficient?

[yanboliang]

We can specify the input, a constant kernel and check the output, but due to Theano backend need to rotate the kernel which makes the test complicated, and we need to hard code output matrix. The following code is I run for offline test, I will wrap them as a unit test.

from keras.utils.conv_utils import convert_kernel
kernel = np.arange(27).reshape((3, 3, 1, 3))
print(kernel.shape)
# kernel = convert_kernel(kernel) # Enable this line for Theano backend

inputs = Input(shape=(5, 6, 3))
m = Conv2DTranspose(1, kernel_size=3, padding='same', dilation_rate=2, kernel_initializer=Constant(kernel))(inputs)
model = Model(inputs=inputs, outputs=m)
model.summary()

x = np.arange(180).reshape((2, 5, 6, 3)).astype(np.float32)
y = model.predict(x)

print(np.squeeze(y))
print(y.shape)

[yanboliang]

The docstring has already exist at L671.

[yanboliang]

Add a test case here, kernel initialized with ones, then we don't have special handle for Theano backend, and check the output with the given input. The input data's batch size is only one, that we don't need to hard code too big output matrix.

[yanboliang]

The PEP-8 check failure is not relevant to this PR, it introduced by #8344 , and I sent a quick fix at #11040 .

[fchollet]

LGTM, thanks