To scale Solr indexes to hundreds of millions of documents (up to billions of documents) with as optimal search performance as possible, the general advice is to leverage multi-core parallelism as much as possible. In other words, more CPU processor cores/threads ensure better search throughput and latency of queries. With commodity GPUs capable of running thousands of parallel threads and having as high as 12GBs of fast device memory, the possibility of offloading certain workloads (querying or indexing) seems tempting. However, GPU memory and threads are very different in nature than system memory and CPU threads, and mixing them is not straightforward.
Query scoring seems like a very compelling candidate for GPU offloading, as complex function queries or complex re-ranking (LTR) of queries might be slow on CPU. Through our initial benchmarks on the geonames dataset (11M data points), scoring documents using the haversine distance from a given query point, we observed a 10X improvement when performed on a GPU as against a regular CPU based Solr query. Similarly, we saw sub-second performance on scoring of 100M results on a regular gaming GPU.
In this talk, we wish to present (a) the results of our initial experiments, (b) integration into Lucene, (c) licensing challenges, and (d) ideas for future explorations. (Reference: https://issues.apache.org/jira/browse/LUCENE-7745)
