SPARK-2101: import unittest2 when using Python 2.6

[laserson]

The PySpark tests depend on unittest.skipIf, which was only added in Python 2.7. This should remedy the build failures on Python 2.6. The only remaining piece is that it may be necessary to install unittest2 with Python 2.6, if it's not provided automatically.

[laserson]

This may require modifying the Jenkins config to make sure unittest2 is available, if it's not by default.

[AmplabJenkins]

Can one of the admins verify this patch?

[JoshRosen]

Jenkins, add to whitelist and test this please.

[SparkQA]

QA tests have started for PR 1042. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17146/consoleFull

[SparkQA]

QA results for PR 1042:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17146/consoleFull

[JoshRosen]

It looks like unittest2 isn't the only change required for the unit tests to pass on Python 2.6.

In tests.py, createFileInZip() uses ZipFile as a context manager, which is only supported in Python 2.7+ (https://docs.python.org/2.7/library/zipfile.html?highlight=zipfile#zipfile.ZipFile):

        with zipfile.ZipFile(path, 'w') as zip:
            zip.writestr(name, content)
        return path

We should probably replace this with a try-finally block.

There's a neat tool called pyqver that aims to identify the minimum required Python version for a particular script. Unfortunately, it doesn't detect this ZipFile issue; I'll look into opening a pull request to add a check for it.

[JoshRosen]

Actually, scratch that: pyqver won't be able to check whether usage of an object as a context manager is supported because it won't be able to statically determine the type of the object.