In part 1 of this tutorial, I described the most basic form of a product quantizer. In this post, I’ll be explaining the IndexIVFPQ index from the FAISS library, which uses a product quantizer as well as a couple additional techniques introduced in their 2011 paper.
This post was written in my role as a researcher at Nearist, and will soon be on the Nearist website as well.
I recently created a project on GitHub called wiki-sim-search where I used gensim to perform concept searches on English Wikipedia.
I’ve recently needed to perform a benchmarking experiment with k-NN in C++, so I found mlpack as what appears to be a popular and high-performance machine learning library in C++.