There have been non-stop failures of the Elasticsearch backend due to memory constraints. We need to get to the bottom of it. Is it because the site is getting hammered by bots at the same time that the VM is undersized? Without increasing the size of the VM, can we better protect and limit the resources for stability?
Aside from increasing site stability, there are additional platform tools we want to add like Metricbeat and/or Filebeat but we also want to convert to OpenSearch (issue 133) so we may well be duplicating the effort in short order.
What we did for Kibana
With VSCode running and port 5601 forwarded, we can browse Kibana on the local workstation e.g. http://localhost:5601
Configure security
- which means creating users and more. The whole left panel in this documentation probably needs doing. The good news is that we can set username and password in a local override for the yaml role
Configure metricbeat
For Elasticsearch observability, it's generally recommended to use Metricbeat rather than relying solely on Elasticsearch's self-monitoring capabilities. While self-monitoring is useful, using Metricbeat offers advantages in terms of resilience and comprehensive monitoring.
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/configuring-metricbeat.html
The sad thing is that there's no geerlingguy role for metricbeat. There's this one for reference or implementation.
Configure filebeat
Filebeat tracks log files. Modify the QB role requirements to setup these additional roles in /opt/meza/src/requirements.yml
roles:
# https://galaxy.ansible.com/ui/standalone/roles/geerlingguy/kibana/documentation/
# had to manually install with
# sudo ansible-galaxy role install --force https://github.com/geerlingguy/ansible-role-kibana/archive/refs/tags/4.0.2.tar.gz
- name: geerlingguy.kibana
version: 4.0.2
- name: geerlingguy.logstash
version: 4.0.0
- name: geerlingguy.filebeat
version: 3.0.0
