Overview
Master nodes are responsible for actions such as creating or deleting indices, deciding which shards should be allocated on which nodes, and maintaining and updating the cluster state on all of the nodes. The cluster state includes information about which shards are on which node, index mappings, which nodes are in the cluster and other settings necessary for the cluster to operate.
If you have just one or two master nodes in your Elasticsearch cluster, then the loss of one of your master nodes due to mechanical failure, network or configuration issues will result in your entire Elasticsearch cluster becoming unavailable.
How to resolve it
For more information about what master nodes are, and how to create a robust dedicated master node configuration, see Dedicated Master Node in Elasticsearch – Important Tips
However, if you just want a quick fix without creating dedicated master nodes, then it is sufficient to ensure that 3 of your nodes are master eligible by setting in the elasticsearch.yml config file:
node.master: true
Then restart each node, waiting for the cluster to go green before restarting the next node.
Ideally these master nodes should be in different availability zones in your data centre so that a networking or power failure does not affect all 3 master nodes simultaneously.
