ElasticSearch scaling and multi-tenancy considerations

Hello, A customer is growing their usage of Elastic and adding more customers, types of documents and indexes. Elastic is a main part of their multi-tenant SaaS offering.

TL;DR - They would like to transform their Elastic setup to be multi-tenant in order to create better isolation and accommodate the expected growth and have a couple of questions.

They consider a couple of points regarding going forward:

1. Currently each Elastic index contains the documents of all of their tenants. However, moving forward to new indexes, they consider creating a separate index per tenant. According to plan, they may have millions of documents per tenant. For example, an index for all emails, which is currently called ‘emailMessage’ will be split into many ‘emailMessage-TENANTID’.

• What does it mean from system resources point of view in case they expect to have a few thousand tenants? Since each index requires at least one separate shard and each shard means system resources they are not sure if they'll will not hit some system limit at some point which will prevent them from adding additional tenants.

Customer wording regarding two additional questions -

2. How bad / good does ES handle modifications – in one of our planned indices we expect to store from hundreds of thousands up to few millions of documents per tenant. We are also expecting that about 50% of them will be changed on a daily basis. Since ES basically deletes a document on each update we are worried that ES indices will and data will get fragmented, which will cause performance decrease. The question is if you have an experience with ES indices that take so many updates and how do they perform and / or is there any actions we should take when creating the indices as well as maintaining them to keep ES perform well as time goes by.

3. API – we are currently using ES’s RestHighLevelClient and we experience issues with keeping up with the latest versions of it – as it turns up the developers of this client does not value backward compatibility too much and some upgrades require a few days of development and testing for keeping the existing code work as it did before the upgrade. The question is if you have any recommendations for alternate (Java) client with which you have good experience.

*The customer use Amazon ElasticSearch service.