One of the most difficult issues to manage and resolve in OpenSearch is poor search performance. To optimize OpenSearch search performance, you need to find the heavy and slow searches in your system, which is no easy task.
Once you’ve succeeded at finding a “culprit” search that is degrading search performance, you need to know exactly how to configure your settings differently to resolve the issue and optimize future searches.
Aside from configuration, you also want to educate your users to stop running those searches. The best solution would be the block the culprit searches in advance so they can’t break your cluster and harm search performance.
The first thing you need to do to improve search performance in your system is see all of your searches in one place. Without that visibility, you won’t be able to get a clear picture of the state of all of your searches and you’ll have no way of determining which searches are slow.
Opster’s Search Gateway automatically produces detailed search logs, breaks them down into fields and displays them on OpenSearch Dashboards for easy viewing. The logs retrieved describe search times from end-to-end, meaning the complete amount of time that elapsed from when the search request was entered until the results were displayed.
This enables users to not only see all of their searches, but also to derive insights from the data efficiently and address the issues that are degrading performance. For instance, this would help users answer questions such as, “Is the search rate increasing or staying normal?”
The next step towards optimizing search performance is grouping search usage by users and applications. This grouping will help you analyze your searches because you’ll be able to know which users generated which searches and the specifications of each search, meaning you’ll be able to locate slow searches and identify the user/application running them.
The Search Gateway automatically accepts applicable headers with the user identifiers that are sent from the applications, and knows how to read them and associate each search with its user. This way queries can be assigned to their user and you can see usage patterns grouped by user.
Once you can see your searches and assign them to their source, you can begin to look for heavy searches.
The first challenge of detecting heavy searches is the fact that your system can be loaded for any number of reasons and searches will be slower as a result. Node disconnection, loaded indices, backups being stored in snapshots – all of these and more can slow down your searches, without heavy searches themselves having occurred.
In order to accurately pinpoint when searches are slow due to heavy searches, the Search Gateway scans the search and provides a score based on the search features to distinguish between slow searches and valid searches.
In addition, the Search Gateway keeps track of historic search execution statistics, grouping searches by their patterns. This allows the Gateway to detect when a search is slow due to its costly attributes, as opposed to when it runs slowly at a specific time, likely due to other factors such as those mentioned above.
Once the slow searches have been singled out, two questions arise: how to resolve added load occurring in ES due to those searches, and how to prevent those searches from executing again.
To handle the slow searches and optimize search performance in OpenSearch, Opster’s Search Gateway provides users with the unique ability to block, reject and prevent heavy searches from executing on the cluster.
The Gateway is configured to start detecting and preventing heavy searches straight out of the box, and specific rules can be set up in order to tell the system which searches should be allowed and which should not, according to different users.
To learn more about Opster’s Search Gateway and to book a demo, click here.
If you’re suffering from search latency issues or poor search performance, you should run Opster’s free Search Log Analyzer to optimize your searches.
With Opster’s Analyzer, you can easily locate slow searches and understand what led to them adding additional load to your system. You’ll receive customized recommendations for how to reduce search latency and improve your search performance. The tool is free and takes just 2 minutes to run.
