bazel absence of ZIP64 support cause tensorflow build fail

[fo40225]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 1803
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): v1.11.0-rc1
• Python version: 3.6.5
• Bazel version (if compiling from source): 0.15.2
• GCC/Compiler version (if compiling from source): vs2017 15.8 / cl.exe 19.15.26726
• CUDA/cuDNN version: 9.2.148.1/7.2.1
• GPU model and memory: 1080ti 11GB
• Exact command to reproduce:

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
bazel-bin\tensorflow\tools\pip_package\build_pip_package C:/tmp/tensorflow_pkg

Describe the problem

If build artifact large than 4GB, it can't generate the python package.

A easy way to reproduce the issue

Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,7.0]: 3.0,3.2,3.5,5.0,5.2,5.3

related link:
#20332 (comment)
#22382
https://github.com/bazelbuild/bazel/blob/0.17.1/third_party/ijar/zip.cc#L74

Source code / logs

Uncompressed input jar has size ???, which exceeds the maximum supported output size 4294967295.
Assuming that ijar will be smaller and hoping for the best.

Unzipping simple_console_for_windows.zip to create runfiles tree...
[./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip or
        ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip.zip, and cannot find ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip.ZIP, period.

[gunan]

@meteorcloudy This is an interesting issue.
what would be the correct pacman command to install zip64?

[meteorcloudy]

The zip file is created by a custom zipper binary built from source in bazel, which doesn't have zip64 support. So installing zip64 from pacman is not going to help.

The root cause here is we are zipping redundant files in ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip because it depends on some other py_binary targets which are also built as a zip file. This results a zip file containing many other zip files, and all of them have //tensorflow:tensorflow_py as a dependency.

I'm trying to figure out a way to exclude redundant files.

[bstriner]

Probably good to clean up what is going into that zip, but could also try to fix the zipper. Zlib seems to have zip64 support just fine, so maybe there is an issue with the bazel zlib client or something? I don't see any related issues on bazelbuild.

[meteorcloudy]

Yes, ideally bazel's zip tool should support zip64. I filed bazelbuild/bazel#6211

But I prefer to refactor tensorflow's dependency to clean up the zip file, because it will also improve the performance of creating TF pip package a lot.

[meteorcloudy]

@fo40225 @bstriner Can you try #22483 to see if it fixes the problem?

[fo40225]

@meteorcloudy

v1.11.0-rc2 + cherry-pick 2a01b6a

It can build with most consumer compute capability.

Thank you for help.

[meteorcloudy]

@gunan You might want to cherry-pick #22483 into v1.11.0 ?

[gunan]

@angersson if we are doing another RC, yes.
If not, I would wait until 1.12