Convolutional layers for 3D

[oeway]

This PR includes the following convolution layers:

• Convolution3D
• MaxPooling3D
• AveragePooling3D
• UpSampling3D
• ZeroPadding3D

And also the corresponding test functions for these layers.

You can use it like this:
Convolution3D(nb_filters, nb_time, nb_row, nb_col)
See the convolution 3D example in examples/shapes_3d_cnn.py by @MinhazPalasara.

The shape of the underlaying 5D tensor:
(samples, channels, time, rows, cols) if dim_ordering='th'
(samples, time, rows, cols, channels) if dim_ordering='tf'.

Here I used "time" for the case of movies, of course it can be depth of a 3D volume. I didn't use the notion "depth" because some people use "depth" to stand for filter or channel, the word "depth" sometimes appears in the docstring of Keras when it means channel.

The following functions are added into theano_backend.py:

• conv3d
• pool3d
• resize_volumes
• spatial_3d_padding

But notice that, currently, these layers won't work in tensorflow backend, due the fact that there is no convolution3d ops implemented in tensorflow. It would be great if anyone can implement it.

Details about convolutional 3D in theano:

There are two implementations available in theano, I used conv3d2d.conv3d for both GPU and CPU, because it's faster:

• on CPU:

        *  conv3d2d.conv3d:    160 ms per loop
output:
[  2.30778933   2.12729692  -3.06885958  -4.90572691  10.89875412
   3.7447443   -4.60463953   8.3501606    3.69148254  -0.93439424]

        *  nnet.conv3D: 1.19 s per loop
output:
[  2.30778956   2.12729645  -3.0688591   -4.90572643  10.89875507
   3.74474525  -4.60463953   8.35016346   3.69148254  -0.93439448]

• on GPU (k40):

        *  conv3d2d.conv3d:    36.2 ms per loop
output:
[  2.30778933   2.12729692  -3.06885958  -4.90572691  10.89875412
   3.7447443   -4.60463953   8.3501606    3.69148254  -0.93439424]

       *  nnet.conv3D: 941 ms per loop
output:
[  2.30778956   2.12729645  -3.0688591   -4.90572643  10.89875507
   3.74474525  -4.60463953   8.35016346   3.69148254  -0.93439448]

Notice that there are precision differences with conv3d2d.conv3d on GPU and CPU. The testing code can be found in this discussion: https://groups.google.com/d/msg/theano-users/1S9_bZgHxVw/0cQR9a4riFUJ.

Another PR(#718) was created by @MinhazPalasara but it's out dated. Many functions, the test example and test dataset in this PR are taken from #718, thanks @MinhazPalasara, @rbharath, @fchollet and others for improving that codes and giving suggestions.

Be careful that, these codes are not tested restrictedly, please report bugs if you encountered any.

[tboquet]

For the Travis test to pass, you could add:

@pytest.mark.skipif(K._BACKEND != 'theano', reason="Requires Theano backend")

[oeway]

Done, thanks.

[oeway]

By running the test code, I notice that, the loss value from each epoch is different when I run on GPU and CPU, I think it could possibly because the precision difference of conv3d2d.conv3d, but not sure if it's normal for a layer to give different result on GPU and CPU.

In my test, I tried to fix the random seed by np.random.seed(0), but maybe it's not working correctly because of theano. It still can't explain why CPU tends to have higher loss value with the same epoch number, I tried many times and it behaves similar.

• Run on GPU

Train on 4000 samples, validate on 1000 samples
Epoch 1/100
32s - loss: 0.7605 - acc: 0.7418 - val_loss: 0.1312 - val_acc: 1.0000
Epoch 2/100
32s - loss: 0.0711 - acc: 0.9910 - val_loss: 0.0062 - val_acc: 1.0000
Epoch 3/100
32s - loss: 0.1353 - acc: 0.9808 - val_loss: 0.0029 - val_acc: 1.0000
Epoch 4/100
32s - loss: 0.0045 - acc: 0.9995 - val_loss: 9.2182e-04 - val_acc: 1.0000
Epoch 5/100
32s - loss: 0.0019 - acc: 0.9995 - val_loss: 0.9107 - val_acc: 0.8010
Epoch 6/100
32s - loss: 0.0465 - acc: 0.9910 - val_loss: 1.4903e-04 - val_acc: 1.0000

• Run on CPU

Train on 4000 samples, validate on 1000 samples
Epoch 1/100
223s - loss: 0.9434 - acc: 0.5200 - val_loss: 0.6909 - val_acc: 0.6500
Epoch 2/100
245s - loss: 0.5292 - acc: 0.7597 - val_loss: 0.1592 - val_acc: 0.9680
Epoch 3/100
248s - loss: 0.1219 - acc: 0.9738 - val_loss: 0.0613 - val_acc: 1.0000
Epoch 4/100
249s - loss: 0.1347 - acc: 0.9698 - val_loss: 0.0566 - val_acc: 1.0000
Epoch 5/100
249s - loss: 0.0099 - acc: 0.9988 - val_loss: 0.0095 - val_acc: 1.0000
Epoch 6/100
249s - loss: 0.0039 - acc: 0.9990 - val_loss: 8.9441e-04 - val_acc: 1.0000

Any idea about this difference?

[fchollet]

I think 3D operators will be interesting to have in Keras. I will do a detailed code review in the near future. For now, please:

• drop the dataset and the associated example
• scrub references to "time", "rows" and "cols" from the added layers (including the docstrings). "rows" and "cols" (and channels) make sense for arbitrary 4D tensors, but "time" does not for 5D tensors. I'd replace them with "first_spatial_dim", "second_spatial_dim", "third_spatial_dim", "channels" (if you can think of something more concise while still abstract and explicit, that's fine too).
• Add exceptions in all added layers to output a helpful error message in case the layers are instantiated with the TensorFlow backend.

[oeway]

Thanks for the response.
For the references to "time", I have the similar concern as you, but it's not easy to find one which is abstract and explicit.

In real world, and in the context of CNN, 3D data usually means 3D volume or 2D movie. For 3D volume, it's spatial signal. And for movie, it's spatio-temporal signal. Names such as "first_spatial_dim", "second_spatial_dim" etc. would not apply in the case of a movie, because, time dimension for a movie is not the "first spatial dim", it's a temporal dim.

Maybe other people have better ideas about how to define it, but before we find a good one, perhaps, it would still be reasonable to use "time":

1. In the real world, movie-like data have a lot of applications simply because we have a lot of cameras which is capable of recording video, at least, it hard to say there is more application in 3D volume than movie, so it hard to make choice to define the first dim into time or z/depth.
2. In 2D, most application are working on static image, and it's very natural to treat 3D as movie by default.
3. For 1D signal, Keras say it's a temporal signal, and for 2D signal, it's spatial signal. And these assumptions are not always true, it's still useful because it matches the most common task in real-life and it's very intuitive to understand how layers work.
4. Theano should be more general than Keras, but it still uses "time", "movie" and "spatio-temporal" for 3D signal. For example, in conv3d2d.conv3d and nnet.conv3D, they both used "time" in the docstring.
5. When the user understand how Keras process a movie, and it would be very easy for them to extend to 3D volume data by replacing time into z dimension. Plus, we could just mention that in the docstring.

So, I would suggest we use a more specific word to make it easier to understand.

If other people have better idea on this, please discuss here.

[fchollet]

It's precise because "space" does not mean physical 3D space, it means
space. As in, a vector space in linear algebra. In the context of
convolution, "space" means that which on which you convolve (as opposed to
"channels").

On 2 February 2016 at 15:15, Will Ouyang [email protected] wrote:

Thanks for the response.
For the references to "time", I have the similar concern as you, but it's
not easy to find one which is abstract and explicit.

In real world, and in the context of CNN, 3D data usually means 3D volume
or 2D movie. For 3D volume, it's spatial signal. And for movie, it's
spatio-temporal signal. For "first_spatial_dim", "second_spatial_dim" etc.,
it's still not precise, because, time dimension for a movie is not the
first spatial dim, it's a temporal dim.

Maybe other people have better ideas about how to define it, but before we
find a good one, perhaps, it would still be reasonable to use "time":

1. In the real world, movie-like data have much more applications,
   simply because we have more cameras which can take videos, and much less
   device to capture 3D volume.
2. In 2D, most application are working on static image, and it's very
   natural to treat 3D as movie by default.
3. For 1D signal, Keras say it's a temporal signal, and for 2D signal,
   it's spatial signal. And these assumptions are not always true, it's still
   useful because it matches the most common task in real-life and it's very
   intuitive to understand how layers work.
4. Theano should be more general than Keras, but it still uses "time",
   "movie" and "spatio-temporal" for 3D signal. For example, in
   conv3d2d.conv3d and nnet.conv3D, they both used "time" in the docstring.
5. When the user understand how Keras process a movie, and it would be
   very easy for them to replace the time dimension into z dimension. Plus, we
   could just say it in the docstring about that.

So, I would suggest we use a more specific word to make it more easier to
understand.

If other people have better idea on this, please discuss here.

—
Reply to this email directly or view it on GitHub
#1623 (comment).

[rbharath]

+1 for @fchollet's notation. I use 3D CNNs on 3D renderings where the 3rd dimension isn't time.

[mmmikael]

+1 for the "vector space" notation.

This notation would be compatible with hypothetical N-d convolutions.
We also have 3 spatial dimensions in medical imaging btw.

[oeway]

OK, let's use the vector space notation then. Thanks for everyone.

[oeway]

Don't you guys think "first_spatial_dim" is too long?
Then I will have this in the docstring:

    # Input shape
        5D tensor with shape:
        `(samples, channels, first_spatial_dim, second_spatial_dim, third_spatial_dim)` if dim_ordering='th'
        or 5D tensor with shape:
        `(samples, first_spatial_dim, second_spatial_dim, third_spatial_dim, channels)` if dim_ordering='tf'.

In the arguments we have "nb_row", "nb_filter", and now we need to add "nb_first_spatial_dim" or "len_first_spatial_dim"?

Arguments
        nb_filter: Number of convolution filters to use.
        nb_first_spatial_dim: Number of elements in the first spatial dimension in the covolution kernel.

Alternatives are to use:

• "spatial_dim1", "nb_spatial_dim1", etc.
• "dim1", "nb_dim1/len_dim1"
• "x_dim", "y_dim", "z_dim", "nb_z_dim"

How about keep the "rows" and "cols", and add another notation for the extra dimension? That would be more consistent with 1D and 2D case. Examples are "z_dim" if we think "rows" and "cols" are from XY plane, then the extra dimension is Z.
What do you think?

[mmmikael]

How about keep the "rows" and "cols", and add another notation for the extra dimension? That would be more consistent with 1D and 2D case. Examples are "z_dim" if we think "rows" and "cols" are from XY plane, then the extra dimension is Z.
What do you think?

There is a reddit thread on exactly this topic: If a 2d matrix has rows and columns, what is the third dimension in a 3d matrix?.

I like slices.

[jocicmarko]

+1 for slices.

[jruales]

+1 for slices since they could mean time-slices or 2D slices of a 3D scan

[jruales]

What distinguishes the row, column, and new dimension from the batch and filter dimensions is that the convolution filter slides across these dimensions, so wouldn't it make sense to call these dimensions conv_dim1, conv_dim2, and conv_dim3?

[fchollet]

wouldn't it make sense to call these dimensions conv_dim1, conv_dim2, and conv_dim3?

+1

[oeway]

I have changed it to "conv_dim1" and "len_conv_dim1", if you guys have better idea, or more people vote for "slices, rows, cols", we could make another change to it.

@fchollet I think I am done, could you please do a code review if you have time, and merge it? Otherwise, I would need to follow your master repository all the time. Thanks.

[mdering]

I'm using this PR as a part of a project and it doesn't seem to be working with a fairly straightforward model design. Are you sure it's working as intended? specifically, it seems when I added a stride size for the 3D CNN layer it caused dimensions to be miscalculated. Removing subsamples=(2,2,2) seems to fix the problem.

[oeway]

Thanks for reporting the problem. I just checked, stride is not implemented in conv3d2d.conv3d currently. So, for now the only supported stride is (1,1,1), and you can remove those result yourself(maybe with a Lambda layer).

You can see the document here: http://deeplearning.net/software/theano/library/tensor/nnet/conv.html#theano.tensor.nnet.conv3d2d.conv3d

I will put this in the docstring then, sorry for the misleading.

[mdering]

Right that makes sense. You should probably note that this parameter is ignored and add in a check so that this doesn't happen to others.
Thanks!

[oeway]

@mdering I just added the support for strides by slicing the output of conv3d, it won't change the real computation, but you should get the result you want. Tell me if you have any problem, thank you.
Here is what I did:
conv_out = conv_out[:, :, ::strides[0], ::strides[1], ::strides[2]]

[jruales]

Will that new statement be optimized away when strides are (1, 1, 1)? Otherwise it might be good to add an if statement for that case.

conv3d2d uses 2d convs to create a 3d conv, so I wonder if it would be possible to implement strides efficiently inside conv3d2d by using strides in the conv2d calls. That would be a Theano PR though

[mdering]

@oeway I've tried it and it seems to be working correctly. Thanks. It would be nice to get a better fix for this upstream, since I'm running on some not so fantastic hardware, but in the meantime this is working great. Thanks again.

[mdering]

Hi, me again. These layers are not currently serializable to either json or yaml, because the relevant part of the get_config method is missing. Specifically you'll need to add

base_config = super(Convolution3D, self).get_config()
return dict(list(base_config.items()) + list(config.items()))

at the end of the method for this to work properly.

EDIT: theres actually an additional error with the deserializing process, that I wasn't really able to determine the cause of. When using a deserialized model arch, i got an error: NotImplementedError('height and width bordermodes must match')`
from https://github.com/Theano/Theano/blob/master/theano/tensor/nnet/conv3d2d.py#L236

Removing the border_mode from the parameters passed to the Theano backend seems to have done the trick. I don't anticipate this being a problem since this parameter is always 'valid' anyway, and that is the default value.

I should also add even after all this serialization and deserialization is still buggy. Maybe someone else can help more

[oeway]

@mdering Thanks for reporting the problem, I just fixed it according to your suggestion.

I think I found the problem, after deserialising, strings from the json code are becomes unicode string:

json_string = model.to_json()
model2 = model_from_json(json_string)
print model2.layers[0].border_mode, model.layers[0].border_mode

output: (u'valid', 'valid')
And in conv3d https://github.com/Theano/Theano/blob/master/theano/tensor/nnet/conv3d2d.py#L209

isinstance(u'valid', str), isinstance('valid', str)

output: (False, True)

However, I didn't see any problem with model_from_yaml(), do you have any problem yaml?

I have fixed the problem by passing a tuple as border_mode:
border_mode_3d = (border_mode, border_mode, border_mode)

Maybe we should fix model_from_json() function in Keras to prevent further problem, for now, using yaml can avoid that. In case you want to track this issue, follow #1702 .

Tell me if it works for you, thank you.

[ghost]

For verification reasons, I have run the exact same experiment without the use of the regularizer.
http://imgur.com/j4b0rNB

As you can see, the weights of the filter seem a bit random and are not identical slice-wise.

[oeway]

I did a quick check, I didn't find any reason for this, I used exactly the same implementation as Convolution2D in terms of regularization.

[jruales]

@MathiasPolfliet make sure that your input_shape is 5-dimensional

[ghost]

@jruales You mean here?

model_p1.add(Convolution3D(nb_filter=c_Filter[0], len_conv_dim1=c_Kernel[0], len_conv_dim2=c_Kernel[0], len_conv_dim3=c_Kernel[0], init='normal', W_regularizer=l2(0.4), border_mode='valid', input_shape=(1,c_Patch[0],c_Patch[0],c_Patch[0])))

According to the comments in the code it should only be 4-D:
provide the keyword argument input_shape
(tuple of integers, does not include the sample axis),
e.g. input_shape=(3, 10, 128, 128) for 10 frames of 128x128 RGB pictures.

[jruales]

Sorry, my bad, I was thinking of the actual shape of the input tensor to the network and forgot the sample dimension is excluded from the constructor argument

[jruales]

As a sanity check, could you please check the following:

• What is (W_conv1[0]).shape ?
• Are all the values along kernel_dim3 exactly the same, or just very close to each other?
• If you use W_regularizer=l2(0.0) do you obtain the same result as with W_regularizer=None?

Also please make sure you are using the latest version of the pull request's code

[oeway]

@fchollet I think now it's ready to merge. The ongoing discussion shouldn't be a problem of Convolution3D.

[fchollet]

Agreed. LGTM.

[ghost]

• 32x1x7x7x7
• Very close to each other after 100 iterations, exactly the same after 1000
• yes

Is someone able to reproduce these findings? If not, it'll be just a bug in my internal code...

[ghost]

Okay, when using two layers of convolution this problem arises in the second layer and not in the first layer. Can it have something to do with Flatten()?

[ghost]

I have determined what causes the behaviour. When convolving a nxnxn patch with an nxnxn filter the output is a 1x1x1, which leads to the strange behaviour. When using larger patches or smaller kernels 'fixes' it.

[jruales]

Convolving an nxnxn patch with an nxnxn kernel is equivalent to using a Dense layer with output size nb_filter. We could add a test to make sure these two layers output the same exact output (upon flattening) when under the same conditions.

I can't think of why the strange behavior you see would only appear when using regularization. I'm wondering if it has to do with the fact that you regularize your convolutional weights but not your dense weights? Or maybe somehow your data is repeated along the third convolutional dimension? I haven't had the chance to experiment with this in code, so I'm just throwing out ideas.

[DeeperCS]

I try to insert a 3D Conv layer to a Graph model but meet some errors. Is it because something not compatible with Graph?
The code I used is shown below:

graph = Graph()
graph.add_input(name='input1', input_shape=(1, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, nb_depth=3, nb_row=3, nb_col=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

the error occurs when compile the graph

ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

[oeway]

Hi @DeeperCS, you are using the wrong argument name, you should use:

Convolution3D(nb_filter=32, kernel_dim1 =3, kernel_dim2 =3,
kernel_dim3 =3, border_mode='same')

for simplicity, I would just write as:

Convolution3D(32, 3, 3, 3, border_mode='same')

On Wed, Mar 9, 2016 at 2:29 PM, DeeperCS [email protected] wrote:

I try to insert a 3D Conv layer to a Graph model but meet some errors. Is
it because something not compatible with Graph?
The code I used is shown below:

graph = Graph()
graph.add_input(name='input1', input_shape=(1, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, nb_depth=3, nb_row=3, nb_col=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

the error occurs when compile the graph

ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

—
Reply to this email directly or view it on GitHub
#1623 (comment).

[DeeperCS]

@oeway
Hioeway
Thank you for your reply~ I have tried your suggestions, but there are still some troubles.

---

For the code below:

graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

I got the response below:

<ipython-input-76-7951aa957c60> in <module>()
      5 
      6 #graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
----> 7 graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
      8 graph.add_node(Flatten(), name='flatten', input='conv1')
      9 graph.add_output(name='output', input='flatten')

TypeError: __init__() takes at least 5 arguments (3 given)

---

For the other code below:

graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

I got the response below(seems the same error as what I met few minutes ago):

<ipython-input-78-ba79d46c5110> in <module>()
      8 graph.add_node(Flatten(), name='flatten', input='conv1')
      9 graph.add_output(name='output', input='flatten')
---> 10 graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/models.pyc in compile(self, optimizer, loss, theano_mode)
    579             loss_fn = loss[output_name]
    580             output = self.outputs[output_name]
--> 581             y_train = output.get_output(True)
    582             y_test = output.get_output(False)
    583             y = T.zeros_like(y_test)

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_output(self, train)
    488 
    489     def get_output(self, train=False):
--> 490         X = self.get_input(train)
    491         size = theano.tensor.prod(X.shape) // X.shape[0]
    492         nshape = (X.shape[0], size)

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_input(self, train)
     87     def get_input(self, train=False):
     88         if hasattr(self, 'previous'):
---> 89             return self.previous.get_output(train=train)
     90         elif hasattr(self, 'input'):
     91             return self.input

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/convolutional.pyc in get_output(self, train)
    375                             input_shape[3] + pad_x,
    376                             input_shape[4] + pad_y)
--> 377             output = T.zeros(output_shape)
    378             indices = (slice(None), slice(None),
    379                        slice(pad_z//2, input_shape[2] + pad_z//2),

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in zeros(shape, dtype)
   2284     if dtype is None:
   2285         dtype = config.floatX
-> 2286     return alloc(numpy.array(0, dtype=dtype), *shape)
   2287 
   2288 

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in __call__(self, val, *shapes, **kwargs)
   2808 
   2809         """
-> 2810         ret = super(Alloc, self).__call__(val, *shapes, **kwargs)
   2811         try:
   2812             # It makes optimization difficult when useless allocs are thrown

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/gof/op.pyc in __call__(self, *inputs, **kwargs)
    609         """
    610         return_list = kwargs.pop('return_list', False)
--> 611         node = self.make_node(*inputs, **kwargs)
    612 
    613         if config.compute_test_value != 'off':

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in make_node(self, value, *shape)
   2689     def make_node(self, value, *shape):
   2690         v = as_tensor_variable(value)
-> 2691         sh, bcast = self.validate_shape(shape)
   2692         if v.ndim > len(sh):
   2693             raise TypeError("The Alloc value to use has more dimensions"

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in validate_shape(self, shape)
   2681             # if s is constant 1, then we're broadcastable in that dim
   2682             try:
-> 2683                 const_shp = get_scalar_constant_value(s)
   2684             except NotScalarConstantError:
   2685                 const_shp = None

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
    658                                 get_scalar_constant_value_elemwises):
    659                     const = [get_scalar_constant_value(i)
--> 660                              for i in v.owner.inputs]
    661                     ret = [[None]]
    662                     v.owner.op.perform(v.owner, const, ret)

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
    770                         else:
    771                             msg += ' x=%s' % str(v)
--> 772                         raise ValueError(msg)
    773 
    774                     if gp_broadcastable[idx]:

ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

---

Is there some other things that I did wrong?

[oeway]

Not sure what's the problem, but I just tried your code, for me it can get
through the compilation without any error.

Sorry I am quite busy at the moment. Good luck.

On Wed, Mar 9, 2016 at 3:09 PM, DeeperCS [email protected] wrote:

@oeway https://github.com/oeway

Hioeway
Thank you for your reply~ I have tried your suggestions, but there are

still some troubles.

For the code below:

graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

I got the response below:

in ()
5
6 #graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
----> 7 graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
8 graph.add_node(Flatten(), name='flatten', input='conv1')
9 graph.add_output(name='output', input='flatten')

TypeError: init() takes at least 5 arguments (3 given)

---

For the other code below:

graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

I got the response below(seems the same error as what I met few minutes
ago):

in ()
8 graph.add_node(Flatten(), name='flatten', input='conv1')
9 graph.add_output(name='output', input='flatten')
---> 10 graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/models.pyc in compile(self, optimizer, loss, theano_mode)
579 loss_fn = loss[output_name]
580 output = self.outputs[output_name]
--> 581 y_train = output.get_output(True)
582 y_test = output.get_output(False)
583 y = T.zeros_like(y_test)

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_output(self, train)
488
489 def get_output(self, train=False):
--> 490 X = self.get_input(train)
491 size = theano.tensor.prod(X.shape) // X.shape[0]
492 nshape = (X.shape[0], size)

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_input(self, train)
87 def get_input(self, train=False):
88 if hasattr(self, 'previous'):
---> 89 return self.previous.get_output(train=train)
90 elif hasattr(self, 'input'):
91 return self.input

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/convolutional.pyc in get_output(self, train)
375 input_shape[3] + pad_x,
376 input_shape[4] + pad_y)
--> 377 output = T.zeros(output_shape)
378 indices = (slice(None), slice(None),
379 slice(pad_z//2, input_shape[2] + pad_z//2),

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in zeros(shape, dtype)
2284 if dtype is None:
2285 dtype = config.floatX
-> 2286 return alloc(numpy.array(0, dtype=dtype), *shape)
2287
2288

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in call(self, val, _shapes, *_kwargs)
2808
2809 """
-> 2810 ret = super(Alloc, self).call(val, _shapes, *_kwargs)
2811 try:
2812 # It makes optimization difficult when useless allocs are thrown

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/gof/op.pyc in call(self, _inputs, *_kwargs)
609 """
610 return_list = kwargs.pop('return_list', False)
--> 611 node = self.make_node(_inputs, *_kwargs)
612
613 if config.compute_test_value != 'off':

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in make_node(self, value, *shape)
2689 def make_node(self, value, *shape):
2690 v = as_tensor_variable(value)
-> 2691 sh, bcast = self.validate_shape(shape)
2692 if v.ndim > len(sh):
2693 raise TypeError("The Alloc value to use has more dimensions"

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in validate_shape(self, shape)
2681 # if s is constant 1, then we're broadcastable in that dim
2682 try:
-> 2683 const_shp = get_scalar_constant_value(s)
2684 except NotScalarConstantError:
2685 const_shp = None

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
658 get_scalar_constant_value_elemwises):
659 const = [get_scalar_constant_value(i)
--> 660 for i in v.owner.inputs]
661 ret = [[None]]
662 v.owner.op.perform(v.owner, const, ret)

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
770 else:
771 msg += ' x=%s' % str(v)
--> 772 raise ValueError(msg)
773
774 if gp_broadcastable[idx]:

ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

---

Is there some other things that I did wrong?

—
Reply to this email directly or view it on GitHub
#1623 (comment).

[DeeperCS]

@oeway Thank you so much. I used the #MinhazPalasara/keras at the beginning, and I just checked that 3D convolution stuffs have merged to the master version, so I installed the latest keras 0.3.2 and tried again, unfortunately, these errors still around me - -！
Besides, if I changed the 3D conv layer to 2D conv and decrease the input dim, it can get through the compilation, so I doubted that there is some thing wrong between Graph and 3D conv layer.
Can you give me a example code that works well or list your platform information (e.g. versions of os, python, theano and keras) at your convenience ^ ^

[oeway]

Hi, I am using Ubuntu 14.04LTS, here some other information:

import sys
print sys.version
import keras, theano
print keras.version
print theano.version

2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2]
0.3.2
0.8.0.dev0.dev-4800a51cf2c1e5504bfeddb25c980c61bba90809

On Thu, Mar 10, 2016 at 6:07 AM, DeeperCS [email protected] wrote:

@oeway https://github.com/oeway Thank you so much. I used the
#MinhazPalasara/keras at the beginning, and I just checked that 3D
convolution stuffs have merged to the master version, so I installed the
latest keras 0.3.2 and tried again, unfortunately, these errors still
around me - -！
Besides, if I changed the 3D conv layer to 2D conv and decrease the input
dim, it can get through the compilation, so I doubted that there is some
thing wrong between Graph and 3D conv layer.
Can you give me a example code that works well or list your platform
information (e.g. versions of os, python, theano and keras) at your
convenience ^ ^

—
Reply to this email directly or view it on GitHub
#1623 (comment).

[DeeperCS]

@oeway Thanks a million~!!!~ I use 'conda create' to set a new environment and reinstalled theano-0.8.0 and keras-0.3.2 which make the graph with 3d conv layers works now, I thought the main reason is that I was using a wrong version of keras...

[bhack]

Convolutions are arriving in TF with tensorflow/tensorflow@784778e

[bhack]

3d convolution and pooling are available now in Tensorflow with tensorflow/tensorflow@6a187cc. Do you want update this?

[oeway]

Thanks for the reminder, I would look into that later.

On Thu, May 5, 2016 at 8:05 AM, bhack [email protected] wrote:

3d convolution and pooling are available now in Tensorflow with
tensorflow/tensorflow@6a187cc
tensorflow/tensorflow@6a187cc.
Do you want update this?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1623 (comment)