|
1 | 1 | # TensorFlow-ENet |
2 | 2 | TensorFlow implementation of [**ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation**](https://arxiv.org/pdf/1606.02147.pdf). |
3 | 3 |
|
4 | | -  |
| 4 | +This model was tested only on the CamVid dataset with street scenes taken from Cambridge, UK. For more information, please visit: http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/ |
| 5 | + |
| 6 | + |
| 7 | +### Visualizations |
| 8 | +Note that the gifs may be out of sync if the network doesn't load them together. You can refresh your page to see them in sync. |
| 9 | + |
| 10 | +##### Original Video Input |
| 11 | + |
| 12 | + |
| 13 | +##### Test Dataset Output |
| 14 | + |
| 15 | + |
| 16 | + |
5 | 17 | ### Contents |
6 | 18 |
|
7 | 19 |
|
8 | | -### Important Notes |
9 | | -As the Max Unpooling layer is not officially available from TensorFlow, a manual implementation was used to build the decoder portion of the network. This was based on the implementation suggested in this [TensorFlow github issue](https://github.com/tensorflow/tensorflow/issues/2169). |
10 | 20 |
|
11 | | -Batch normalization and 2D Spatial Dropout are still retained during testing for good performance. |
| 21 | + |
| 22 | + |
| 23 | + |
12 | 24 |
|
13 | 25 | **Note:** To use the checkpoint model, please set the `stage_two_repeat=3` as the checkpoint was trained on a slightly deeper version of ENet. |
14 | 26 |
|
| 27 | + |
| 28 | +### Training Arguments |
| 29 | + |
| 30 | + |
| 31 | + |
| 32 | +### Evaluation Arguments |
| 33 | + |
| 34 | +### Important Notes |
| 35 | +1. As the Max Unpooling layer is not officially available from TensorFlow, a manual implementation was used to build the decoder portion of the network. This was based on the implementation suggested in this [TensorFlow github issue](https://github.com/tensorflow/tensorflow/issues/2169). |
| 36 | + |
| 37 | +2. Batch normalization and 2D Spatial Dropout are still retained during testing for good performance. |
| 38 | + |
| 39 | +3. Class weights are used to tackle the problem of imbalanced classes, as certain classes appear more dominantly than others. More notably, the background class has weight of 0.0, in order to not reward the model for predicting background. |
| 40 | + |
| 41 | +4. The residual |
| 42 | + |
| 43 | + |
15 | 44 | ### Implementation and Architectural Changes |
16 | 45 | 1. By default, skip connections are added to connect the corresponding encoder and decoder portions for better performance. |
| 46 | + |
17 | 47 | 2. The number of initial blocks and the depth of stage 2 residual bottlenecks are tunable hyperparameters, to allow you to build a deeper network if required, since ENet is rather lightweight. |
| 48 | + |
18 | 49 | 3. Fused batch normalization is used over standard batch normalization for faster computations. See [TensorFlow's best practices](https://www.tensorflow.org/performance/performance_guide). |
19 | | -4. To obtain the class weights for computing the weighted loss, Median Frequency Balancing (MFB) is used instead of the custom ENet class weighting function. This is due to an observation that MFB gives a slightly better performance than the custom function, perhaps due to implementation differences. However, the option of obtaining the ENet custom class weights is still possible via the `get_class_weights.py` file. |
| 50 | + |
| 51 | +4. To obtain the class weights for computing the weighted loss, Median Frequency Balancing (MFB) is used instead of the custom ENet class weighting function. This is due to an observation that MFB gives a slightly better performance than the custom function, at least on my machine. However, the option of using the ENet custom class weights is still possible. |
20 | 52 |
|
21 | 53 |
|
22 | 54 | ### References |
23 | 55 | 1. [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://arxiv.org/pdf/1606.02147.pdf) |
24 | 56 | 2. [Implementation of Max Unpooling](https://github.com/tensorflow/tensorflow/issues/2169) |
25 | 57 | 3. [Implementation of PReLU](https://stackoverflow.com/questions/39975676/how-to-implement-prelu-activation-in-tensorflow) |
26 | 58 | 4. [Clarifications from ENet author](https://github.com/e-lab/ENet-training/issues/56) |
| 59 | +5. [Original Torch implementation of ENet](https://github.com/e-lab/ENet-training) |
| 60 | +6. [ResNet paper for clarification on residual bottlenecks](https://arxiv.org/pdf/1512.03385.pdf) |
0 commit comments