Facebook sync (2019-09-10)

[beauby]

No description provided.

[mdouze]

OK by me to merge this diff => Faiss 1.6.0.

• change in the code layout: many C++ sources are now in subdirectories impl/ and utils/. User-visible C++ includes changes are:

utils.h was split into:
   utils/distances.h: L2/IP distance functions 
   utils/random.h: random generation functions
   utils/utils.h: all the rest

distances.h -> utils/extra_distances.h: non-L2/IP distance functions

hamming.h -> utils/

Heap.h -> utils/

AuxIndexStructures.h: 
   impl/io.h: abstract I/O objects
   impl/AuxIndexStructures.h: the rest

ProductQuantizer.h -> utils/

IndexIVFPQ.h: split off
   IndexIVFPQR.h
   Index2Layer.h

AutoTune.h: split off index_factory.h

VectorTransform.h: split off IndexPreTransform.h

index_io.h: split off clone_index.h

gpu/GpuAutoTune.h: split off gpu/GpuCloner.h

• Faiss as a codec: We introduce a new API within Faiss to encode fixed-size vectors into fixed-size codes. The encoding is lossy and the tradeoff between compression and reconstruction accuracy can be adjusted. Read more at Faiss codecs

• ScalarQuantizer support for GPU, see gpu/GpuIndexIVFScalarQuantizer.h. This is particularly useful as GPU memory is often less abundant than CPU.

• added easy-to-use serialization functions for indexes to byte arrays in Python (faiss.serialize_index, faiss.deserialize_index)

• The Python KMeans object can be used to use the GPU directly, just add gpu=True to the constuctor see gpu/test/test_gpu_index.py test TestGPUKmeans

[QwertyJack]

Bravo!