Activate 2018 has ended
Back To Schedule
Thursday, October 18 • 2:55pm - 3:35pm
Who Moved My State? A Blob Storage Solr Story

Log in to save this to your schedule, view media, leave feedback and see who's attending!

Salesforce implemented and runs Solr clusters serving customers whose data volumes and query characteristics are hard to predict. These clusters automatically handle index growth and data rebalancing.

But to more efficiently use hardware, adapt to variable total and core-specific loads, and run in cloud infrastructures, changes were required. We introduced a new architecture in which cores are persisted in a Blob Store (S3) with Solr servers’ local SSD storage used only as a cache.

Cores are then loaded by servers as needed, and popular cores are replicated and maintained. Inactive cores are removed (but kept in Blob Store!). Servers can be shut down when overall load is low, because the remaining servers can serve any core. And performance is globally unaffected, because querying and indexing use a local core copy after the core is initially loaded from the Blob Store.

Core segment updates between Solr servers and the Blob Store build on Solr Replication logic, replacing the dialogue between two servers with Blob Store persisted metadata.

This architecture provides an elastic, highly available and scalable search cluster with a relatively simple implementation.

avatar for Ilan Ginzburg

Ilan Ginzburg

Architect, Salesforce
Ilan works on search infrastructure and integration problems from the Salesforce office in Grenoble, France. He holds business administration and computer science engineering degrees and a PhD in parallel computing.Prior to Salesforce, Ilan worked at Intel, HP Labs in Palo Alto, a... Read More →

Thursday October 18, 2018 2:55pm - 3:35pm EDT
Salon 6&7