Overview
Sometimes you can observe that the CPU and load on some coordinating nodes (client nodes) is higher than others.
This can be caused by applications that are not load balancing correctly across the coordinating nodes, and are making all their HTTP calls to just one or some of the nodes.
Possible effects
A saturated coordinating node could cause an increase in search or indexing response latency, or an increase in write queue/search queue when the cluster is under load (despite there being processing capacity on data nodes). Eventually this could lead to queries timing out.
How to resolve it
You should fix this by putting a load balancer in front of your OpenSearch nodes, or by including ALL of the nodes in the client application.
os = OpenSearch(
['clientNode1', 'clientNode2','clientNode3'],
http_auth=('user', 'secret'),
scheme="https",
port=443,
)
The above example is how you can load balance across 3 nodes (you should include all the nodes) when using the python client without a load balancer. All of the official OpenSearch clients use similar arrays in their construction.
