Connect UiPath Orchestrator to Elasticsearch with Kibana in Azure

UiPath Orchestrator can send logs to Elasticsearch to be analyzed and visualized if needed.

Elasticsearch can be installed in Azure manually, via Kubernetes or easiest way is to deploy it as ARM template form Azure marketplace.

There are several configuration screens that needs to be filled in, for the purpose of this implementation and per customer needs we implemented internal solution only without certificates.

Deployment consisted of 3 data/master nodes in a cluster, each with 3, 128 GB premium SSD disks configured as RAID 0, and machines configured as back end pool in internal load balancer. Additional virtual machine was deployed to use as Kibana server.

Since UiPath Orchestrator is deployed as App service, it had to be integrated vitth existing vNet so it can access Elasticsearch nodes residing in different subnet of the same vNet.

I have tested template deployment several times and every time although deployment passed OK, services where not running as expected. Customers environment is security restricted in some way and I suspected that it may be issue. I confirmed it by deploying the same template in fresh Azure environment where everything worked as expected.

Since these restrictions caused template to not deploy correctly, I took these steps to resolve the issue.

Reinstall Kibana

sudo apt-get remove --purge kibana
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install kibana

Modify kibana.yml file

File is located in /etc/kibana directory. File contains much more settings, but these are needed for Kibana to connect to Elasticsearch service.

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "IP Address Of Kibana Server"

# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://IP Address of Internal Load Balancer:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "Elasticsearch user"
elasticsearch.password: "Elasticsearch password"

These next steps have to be done on every Elasticsearch node.

Elasticsearch user

Create Elasticsearch user and add him to superuser group.

./elasticsearch-users useradd UserName -p Password -r superuser

Modify elasticsearch.yml file

Instead of node names in config file, put their IP addresses. Yes, this could be resolved by adding GlobalNames DNS zone in DNS server, but it is easier this way and less work. 🙂

cluster.name: "yourclustername"
node.name: "your node name"
path.logs: /var/log/elasticsearch
path.data: /datadisks/disk1/elasticsearch/data
discovery.seed_hosts: ["IP address node 1","IP address node 2","IP address node 3"]
cluster.initial_master_nodes: ["IP address node 1","IP address node 2","IP address node 3"]
node.master: true
node.data: true
network.host: [_site_, _local_]
node.max_local_storage_nodes: 1
node.attr.fault_domain: 1
node.attr.update_domain: 1
cluster.routing.allocation.awareness.attributes: fault_domain,update_domain
xpack.license.self_generated.type: trial
xpack.security.enabled: true
bootstrap.memory_lock: true

Restart services

service elasticsearch restart
service kibana restart

Modify web.config file of UiPath App service in Azure

Go to your App service, select Advanced Tools under Development Tools, select Go.

In Kudu console select Debug cosnsole, CMD

Go to /site/wwwroot and scrool to Web.config. Select small pencil on the left side

Modify lines 35 and 38 to include URI to your internal load balancer and username and password for Elasticsearch deployment.

Restart App service just in case.

You should be able to access Kibana via http://YOUR-KIBANA-IP:5601