Build a Threat Hunting Lab. Installing and Configuring Elasticsearch and Kibana

Authors: Laiali Kazalbach and Mohamed Elsayed

Technical requirements

• VMware Workstation with at least 6 GB of RAM, 6 CPU cores, and a 70 GB SSD available to Virtual Machine (VM) guests.

• Build a CentOS VM (Will be called Elastic Machine)

• Use NAT as a Network Adapter

NOTE: Before moving any forward run ifconfig on the CentOS VM and write down the IP address.

Installing and Configuring Elasticsearch

On the CentOS VM Create the elastic.repo file under the /etc/yum.repos.d directory.

sudo nano /etc/yum.repos.d/elastic.repo

Copy and paste the following into the file;

[elastic]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Let’s validate the installations by importing the Elasticsearch signing key by running the following command;

sudo yum install elasticsearch

Before deploying the appropriate security settings, let’s check the system’s functionality once the installation is complete.

sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

Let’s make sure that cURL can communicate with the Elasticsearch API by making several requests:

curl localhost:9200

sudo nano /etc/elasticsearch/elasticsearch.yml

Most of the settings in this configuration file are default or commented out. It is primarily a guide (which is helpful). Once again, add lines to the configuration file using nano and Ctrl+X to save and exit.

Add the following lines;

xpack.security.enabled: true
discovery.type: single-node
xpack.security.authc.api_key.enabled: true

After you have added the above lines.

1. Scroll down to NETWORK

   1. Uncomment network.host and add the IP-Address of the CentOS machine

   2. Make sure http.port: is uncommented

2. Scroll down to DISCOVERY

   1. Uncomment discovery_seed_hosts: [X.X.X.X]

   2. Add the IP-Address CentOS VM between the brackets

3. After making this modification, restart Elasticsearch and confirm that the service has restarted by running the following commands;

sudo systemctl restart elasticsearch

systemctl status elasticsearch

Next, we must set the passwords for each account. Elastic offers a service that enables us to configure the passphrases for all our accounts. It is better to use the same password for simplicity and to avoid confusion.

Run the following command;

sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

You will be prompted to establish the passphrase for several accounts from this page. Remember them, since we will need them later.

Let’s test out the elastic username we just created with your IP-Address it should look like this ;

curl -u elastic You-IP-address:9200

After creating a CentOS virtual machine for the Elastic Stack and deploying and securing Elasticsearch, while this took a while, getting everything adequately completed at this point will make moving forward much simpler.

The Elastic Agent setup process comes next. Installing it now so that it may be configured later as the Fleet server makes the most sense.

To add host monitoring for logs, metrics, and other data types, Elastic agent provides a uniform, streamlined solution. In addition to these capabilities, it can shield hosts from attacks, query operating system information, relay information from distant services or hardware, and more. Users can roll out infrastructure monitoring with only one agent more quickly and with less effort. Each agent has a policy that may be modified to include new types of data and additional layers of security.

Still on the CentOS VM. Run the following commands to install the Elastic Agent;

sudo yum install elastic-agent

sudo systemctl enable kibana

We can now proceed with integrating Kibana and Elasticsearch.

Connecting Kibana to Elasticsearch

A Java KeyStore is used by both Kibana and Beats to store and manage credentials. We’ll add elasticsearch.username and elasticsearch.password to the KeyStore.

Kibana requires these credentials to gain access to Elasticsearch. elasticsearch.username is kibana_system and elasticsearch.password is something you set:

sudo /usr/share/kibana/bin/kibana-keystore add elasticsearch.username

Enter value for elasticsearch.username: kibana_system

sudo /usr/share/kibana/bin/kibana-keystore add elasticsearch.password

Enter a value for the elasticsearch password: your own password.

We can set up an encryption key for our stored objects in Kibana and enable remote connections. To do this, we’ll need to make two adjustments to the Kibana configuration file.

Go to Kibana’s configuration file;

sudo nano /etc/kibana/kibana.yml

Uncomment #server.host: “localhost” and change localhost to 0.0.0.0 to allow external access.

Add the following 2 line in the configuration file;

xpack.encryptedSavedObjects.encryptionKey: "thirty-two-or-more-random-characters"

xpack.fleet.agents.tlsCheckDisabled: true

Go down to ;

Uncomment #elasticsearch.hosts: and enter the IP address of the CentOs machine between the brackets and the port number of elasticsearch 9200.

It should be similar to the below line , but with the IP address of your machine.

elasticsearch.hosts: ["http://123.456.789.345:9200"]

After making these two little adjustments, let’s restart Kibana and connect from our browser:

sudo systemctl restart kibana

We still need to make a few port adjustments on our Elastic VM to enable remote access. Using the firewall-cmd tool run the following command.

sudo firewall-cmd --add-port=5601/tcp --add-port=9200/tcp --add-port=8220/tcp --permanent

sudo firewall-cmd --reload

Back on our host machine, open a web browser and browse to http://123.456.789.345:5601

replace the IP-Address with the IP address of your CentOs machine, and you should be presented with a Kibana web interface asking for a username and password: