badtrash@badbook:/data/badtrash/work/es_spark/sandbox$ ./cluster_setup.sh 
Delegating to setup/setup.sh

0. HOST SYSTEM SANITY CHECKS
===============================================================================\n
 - This is debian based distro
- Checking is virtualbox is installed
- Checking is vagrant is installed

1. DELETING PREVIOUS VMS (if exist)
===============================================================================\n

2. BUILDING ELK-MS CLUSTER VMs
===============================================================================\n

1.1 BUILDING mes_master VM
Bringing machine 'mes_master' up with 'virtualbox' provider...
==> mes_master: Box 'niceideas-debian-stretch64.box' could not be found. Attempting to find and install...
    mes_master: Box Provider: virtualbox
    mes_master: Box Version: >= 0
==> mes_master: Box file was not detected as metadata. Adding it directly...
==> mes_master: Adding box 'niceideas-debian-stretch64.box' (v0) for provider: virtualbox
    mes_master: Unpacking necessary files from: file:///data/badtrash/work/es_spark/sandbox/setup/vagrant/niceideas-debian-stretch64.box
==> mes_master: Successfully added box 'niceideas-debian-stretch64.box' (v0) for 'virtualbox'!
==> mes_master: Importing base box 'niceideas-debian-stretch64.box'...
==> mes_master: Matching MAC address for NAT networking...
==> mes_master: Setting the name of the VM: mes_master
==> mes_master: Clearing any previously set network interfaces...
==> mes_master: Preparing network interfaces based on configuration...
    mes_master: Adapter 1: nat
    mes_master: Adapter 2: hostonly
==> mes_master: Forwarding ports...
    mes_master: 22 (guest) => 2222 (host) (adapter 1)
==> mes_master: Running 'pre-boot' VM customizations...
==> mes_master: Booting VM...
==> mes_master: Waiting for machine to boot. This may take a few minutes...
    mes_master: SSH address: 127.0.0.1:2222
    mes_master: SSH username: vagrant
    mes_master: SSH auth method: private key
==> mes_master: Machine booted and ready!
==> mes_master: Checking for guest additions in VM...
==> mes_master: Configuring and enabling network interfaces...
==> mes_master: Mounting shared folders...
    mes_master: /setup => /data/badtrash/work/es_spark/sandbox/setup/provisionning
    mes_master: /vagrant => /data/badtrash/work/es_spark/sandbox/setup/vagrant
==> mes_master: Running provisioner: shell...
    mes_master: Running: /tmp/vagrant-shell20170809-17563-1d2bvvw.sh
==> mes_master: mesg: 
==> mes_master: ttyname failed
==> mes_master: : 
==> mes_master: Inappropriate ioctl for device
==> mes_master: -- PROVISIONNING INFORMATION ---------------------------------------------------
==> mes_master: VM name : mes_master
==> mes_master: VM is master
==> mes_master: IP address is 192.168.10.10
==> mes_master: -- BASIC SETUP -----------------------------------------------------------------
==> mes_master:  - Updating the packages
==> mes_master:  - Upgrading the appliance
==> mes_master:  - Installing a few utility tools
==> mes_master:  - Installing the latest OpenJDK
==> mes_master:  - Installing build tools
==> mes_master:  - Installing swapspace
==> mes_master:  - Installing python
==> mes_master:  - Installing other Mesos dependencies
==> mes_master:  - Installing Spark Dependency : Scala
==> mes_master:  - Installing Zookeeper with apt (using plain debian package) 
==> mes_master:  - Installing ntp
==> mes_master:  - Disabling IPv6
==> mes_master:  - Creating test user: mes
==> mes_master:  - Adding user mes to sudoers
==> mes_master: -- INSTALLING ELASTICSEARCH ----------------------------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading elasticsearch-5.5.0
==> mes_master:  - Extracting elasticsearch-5.5.0
==> mes_master:  - Installing ElasticSearch
==> mes_master:  - Registering test cleaning traps
==> mes_master:  - Starting ElasticSearch
==> mes_master:  - Checking Elasticsearch startup
==> mes_master:  - Stopping Elasticsearch
==> mes_master: -- INSTALLING LOGSTASH ---------------------------------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading logstash-5.5.0
==> mes_master:  - Extracting logstash-5.5.0
==> mes_master:  - Installing LogStash
==> mes_master: -- INSTALLING KIBANA -----------------------------------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading kibana-5.5.0
==> mes_master:  - Extracting kibana-5.5.0
==> mes_master:  - Installing Kibana
==> mes_master:  - Registering test cleaning traps
==> mes_master:  - Starting ElasticSearch
==> mes_master:  - Checking Elasticsearch startup
==> mes_master:  - Starting Kibana
==> mes_master:  - Checking Kibana startup
==> mes_master:  - Stopping Kibana
==> mes_master:  - Stopping Elasticsearch
==> mes_master: -- INSTALLING CEREBRO ---------------------------------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading cerebro-0.6.5
==> mes_master:  - Extracting cerebro-0.6.5
==> mes_master:  - Installing Cerebro
==> mes_master:  - Registering test cleaning traps
==> mes_master:  - Starting ElasticSearch
==> mes_master:  - Checking Elasticsearch startup
==> mes_master:  - Starting Cerebro
==> mes_master:  - Checking Cerebro startup
==> mes_master:  - Stopping Cerebro
==> mes_master:  - Stopping Elasticsearch
==> mes_master: -- INSTALLING MESOS ------------------------------------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading mesos-1.3.0 (pre-built version for Debian Stretch from niceideas.ch)
==> mes_master:  - Extracting mesos-1.3.0
==> mes_master:  - Installing mesos
==> mes_master:  - Basic environment setup 
==> mes_master:  - Registering test cleaning traps
==> mes_master:  - Starting Mesos Master
==> mes_master:  - Checking Mesos master startup
==> mes_master:  - Starting Mesos Agent
==> mes_master:  - Checking Mesos Agent startup
==> mes_master:  - Stopping Mesos Agent
==> mes_master:  - Stopping Mesos Master
==> mes_master:  - Cleaning build folder
==> mes_master: -- INSTALLING SPARK -----------------------------------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading spark-2.2.0
==> mes_master:  - Extracting spark-2.2.0
==> mes_master:  - Installing spark
==> mes_master:  - Checking Spark Installation
==> mes_master: -- INSTALLING ES-HADOOP (Spark to ES connector) -------------------------------
==> mes_master:  - Changing to temp directory
==> mes_master:  - Downloading elasticsearch-hadoop-5.5.1
==> mes_master:  - Extracting elasticsearch-hadoop-5.5.1
==> mes_master:  - Installing eshadoop spark (to spark jar folder)
==> mes_master: -- SETTING UP NTP --------------------------------------------------------------
==> mes_master:  - Configuring master NTP server
==> mes_master:  - restarting ntp
==> mes_master: -- SETTING UP ZOOKEEPER --------------------------------------------------------
==> mes_master:  - Adapting configuration in file zoo.cfg
==> mes_master:  - Generating myid
==> mes_master:  - restarting zookeeper
==> mes_master: -- SETTING UP ELASTICSEARCH ----------------------------------------------------
==> mes_master:  - Increasing system vm.max_map_count setting
==> mes_master:  - Creating elasticsearch user (if not exist)
==> mes_master:  - Creating elasticsearch data directory in /var/lib/elasticsearch
==> mes_master:  - Creating elasticsearch PID file directory
==> mes_master:  - Opening rights to logs folder to user elasticsearch
==> mes_master:  - Linking elasticsearch logs to /var/log/
==> mes_master:  - Simlinking ES config to /usr/local/etc/elasticsearch
==> mes_master:  - Simlinking ES binaries and scripts to /usr/local/sbin
==> mes_master:  - Copying elasticsearch systemd file
==> mes_master:  - Checking Systemd file
==> mes_master:  - Adapting configuration in file elasticsearch.yml
==> mes_master:  - Addressing issue with multiple interfaces but only one global
==> mes_master:  - Adapting configuration in file jvm.options
==> mes_master:  - Testing systemd startup - starting Elasticsearch
==> mes_master:  - Testing systemd startup - Checking startup
==> mes_master:  - Testing systemd startup - stopping elasticsearch
==> mes_master:  - Enabling elasticsearch on startup
==> mes_master: -- SETUPING LOGSTASH -----------------------------------------------------------
==> mes_master:  - Simlinking logstash binaries and scripts to /usr/local/bin
==> mes_master:  !! TODO !! 
==> mes_master:  - Changing permissions of logstash working directories
==> mes_master: -- SETTING UP KIBANA -----------------------------------------------------------
==> mes_master:  - Creating elasticsearch user (if not exist)
==> mes_master:  - Simlinking Kibana config to /usr/local/etc/kibana
==> mes_master:  - Simlinking Kibana binaries and scripts to /usr/local/sbin
==> mes_master:  - Adapting kibana config
==> mes_master:  - Copying kibana systemd file
==> mes_master:  - Checking Systemd file
==> mes_master:  - Testing systemd startup - starting Elasticsearch (required as dependency)
==> mes_master:  - Testing systemd startup - starting Kibana
==> mes_master:  - Testing systemd startup - Checking startup
==> mes_master:  - Testing systemd startup - stopping Kibana
==> mes_master:  - Enabling Kibana on startup
==> mes_master: -- SETTING UP CEREBRO ----------------------------------------------------------
==> mes_master:  - Creating elasticsearch user (if not exist)
==> mes_master:  - Simlinking cerebro config to /usr/local/etc/kibana
==> mes_master:  - Simlinking Kibana binaries and scripts to /usr/local/sbin
==> mes_master:  - Copying cerebro systemd file
==> mes_master:  - Checking Systemd file
==> mes_master:  - Testing systemd startup - starting Elasticsearch (required as dependency)
==> mes_master:  - Testing systemd startup - starting cerebro
==> mes_master:  - Testing systemd startup - Checking startup
==> mes_master:  - Testing systemd startup - stopping cerebro
==> mes_master:  - Enabling cerebro on startup
==> mes_master: -- SETTING UP MESOS ------------------------------------------------------------
==> mes_master:  - Simlinking mesos config to /usr/local/etc/mesos
==> mes_master:  - Simlinking mesos binaries to /usr/local/bin
==> mes_master:  - Simlinking mesos system binaries to /usr/local/sbin
==> mes_master:  - Creating mesos master working directory in /var/lib/mesos/master
==> mes_master:  - Creating mesos slave working directory in /var/lib/mesos/slave
==> mes_master:  - Creating mesos log directory in /var/log/mesos
==> mes_master:  - Copying mesos-init-wrapper script
==> mes_master:  - Copying mesos-master systemd file
==> mes_master:  - Copying mesos-slave systemd file
==> mes_master:  - Creating common mesos environment file
==> mes_master:  - Creating mesos master environment file
==> mes_master:  - Creating mesos ACLs file to forbid root from submitting tasks.
==> mes_master:  - Creating mesos slave environment file
==> mes_master:  - Checking mesos slave Systemd file
==> mes_master:  - Checking mesos master Systemd file
==> mes_master:  - Testing systemd startup - starting mesos-master
==> mes_master:  - Testing systemd startup - Checking startup of master
==> mes_master:  - Testing systemd startup - stopping mesos-master
==> mes_master:  - Enabling mesos-master on startup
==> mes_master:  - Testing systemd startup - starting mesos-master
==> mes_master:  - Testing systemd startup - starting mesos-slave
==> mes_master:  - Testing systemd startup - Checking startup of slave
==> mes_master:  - Testing systemd startup - stopping mesos-slave
==> mes_master:  - Enabling mesos-slave on startup
==> mes_master: -- SETTING UP SPARK ------------------------------------------------------------
==> mes_master:  - Creating spark user (if not exist)
==> mes_master:  - Simlinking spark config to /usr/local/etc/spark
==> mes_master:  - Simlinking spark binaries to /usr/local/bin
==> mes_master:  - Simlinking spark system binaries to /usr/local/sbin
==> mes_master:  - Creating spark event log directory in /var/lib/spark/eventlog
==> mes_master:  - Opening rights to logs folder to user spark
==> mes_master:  - Linking spark logs to /var/log/
==> mes_master:  - Copying spark-mesos-dispatcher systemd file
==> mes_master:  - Checking Systemd file
==> mes_master:  - Defining SPARK_HOME
==> mes_master:  - Creating spark environment file
==> mes_master:  - Creating spark default configuration file
==> mes_master:  - Copying log4j.properties from template
==> mes_master:  - Creating start-mesos-dispatcher.sh wrapper
==> mes_master:  - Testing systemd startup - starting spark-mesos-dispatcher
==> mes_master:  - Testing systemd startup - Checking startup
==> mes_master:  - Testing systemd startup - stopping spark-mesos-dispatcher
==> mes_master:  - Enabling spark-mesos-dispatcher on startup
==> mes_master:  - Generating /etc/hosts for proper DNS resolution
==> mes_master:  - !! TODO : remove hardcoding of IPs and make them configurable at Provisionning level !!
==> mes_master:  - Changing hostname - THIS REQUIRES REBOOT
==> mes_master:  -> Hostname changed successfully => PLEASE REBOOT VM ASAP !
==> mes_master: Running provisioner: reload...
==> mes_master: Attempting graceful shutdown of VM...
==> mes_master: Clearing any previously set forwarded ports...
==> mes_master: Clearing any previously set network interfaces...
==> mes_master: Preparing network interfaces based on configuration...
    mes_master: Adapter 1: nat
    mes_master: Adapter 2: hostonly
==> mes_master: Forwarding ports...
    mes_master: 22 (guest) => 2222 (host) (adapter 1)
==> mes_master: Running 'pre-boot' VM customizations...
==> mes_master: Booting VM...
==> mes_master: Waiting for machine to boot. This may take a few minutes...
    mes_master: SSH address: 127.0.0.1:2222
    mes_master: SSH username: vagrant
    mes_master: SSH auth method: private key
==> mes_master: Machine booted and ready!
==> mes_master: Checking for guest additions in VM...
==> mes_master: Configuring and enabling network interfaces...
==> mes_master: Mounting shared folders...
    mes_master: /setup => /data/badtrash/work/es_spark/sandbox/setup/provisionning
    mes_master: /vagrant => /data/badtrash/work/es_spark/sandbox/setup/vagrant
==> mes_master: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> mes_master: flag to force provisioning. Provisioners marked to run always will still run.

1.2 BUILDING mes_node1 VM
Bringing machine 'mes_node1' up with 'virtualbox' provider...
==> mes_node1: Importing base box 'niceideas-debian-stretch64.box'...
==> mes_node1: Matching MAC address for NAT networking...
==> mes_node1: Setting the name of the VM: mes_node1
==> mes_node1: Fixed port collision for 22 => 2222. Now on port 2200.
==> mes_node1: Clearing any previously set network interfaces...
==> mes_node1: Preparing network interfaces based on configuration...
    mes_node1: Adapter 1: nat
    mes_node1: Adapter 2: hostonly
==> mes_node1: Forwarding ports...
    mes_node1: 22 (guest) => 2200 (host) (adapter 1)
==> mes_node1: Running 'pre-boot' VM customizations...
==> mes_node1: Booting VM...
==> mes_node1: Waiting for machine to boot. This may take a few minutes...
    mes_node1: SSH address: 127.0.0.1:2200
    mes_node1: SSH username: vagrant
    mes_node1: SSH auth method: private key
==> mes_node1: Machine booted and ready!
==> mes_node1: Checking for guest additions in VM...
==> mes_node1: Configuring and enabling network interfaces...
==> mes_node1: Mounting shared folders...
    mes_node1: /setup => /data/badtrash/work/es_spark/sandbox/setup/provisionning
    mes_node1: /vagrant => /data/badtrash/work/es_spark/sandbox/setup/vagrant
==> mes_node1: Running provisioner: shell...
    mes_node1: Running: /tmp/vagrant-shell20170809-22033-18hbkum.sh
==> mes_node1: mesg: ttyname failed: Inappropriate ioctl for device
==> mes_node1: -- PROVISIONNING INFORMATION ---------------------------------------------------
==> mes_node1: VM name : mes_node1
==> mes_node1: VM is slave 
==> mes_node1: Master IP is 192.168.10.10
==> mes_node1: IP address is 192.168.10.11
==> mes_node1: -- BASIC SETUP -----------------------------------------------------------------
==> mes_node1:  - Updating the packages
==> mes_node1:  - Upgrading the appliance
==> mes_node1:  - Installing a few utility tools
==> mes_node1:  - Installing the latest OpenJDK
==> mes_node1:  - Installing build tools
==> mes_node1:  - Installing swapspace
==> mes_node1:  - Installing python
==> mes_node1:  - Installing other Mesos dependencies
==> mes_node1:  - Installing Spark Dependency : Scala
==> mes_node1:  - Installing ntp
==> mes_node1:  - Disabling IPv6
==> mes_node1:  - Creating test user: mes
==> mes_node1:  - Adding user mes to sudoers
==> mes_node1: -- INSTALLING ELASTICSEARCH ----------------------------------------------------
==> mes_node1:  - Changing to temp directory
==> mes_node1:  - Downloading elasticsearch-5.5.0
==> mes_node1:  - Extracting elasticsearch-5.5.0
==> mes_node1:  - Installing ElasticSearch
==> mes_node1:  - Registering test cleaning traps
==> mes_node1:  - Starting ElasticSearch
==> mes_node1:  - Checking Elasticsearch startup
==> mes_node1:  - Stopping Elasticsearch
==> mes_node1: -- INSTALLING MESOS ------------------------------------------------------------
==> mes_node1:  - Changing to temp directory
==> mes_node1:  - Downloading mesos-1.3.0 (pre-built version for Debian Stretch from niceideas.ch)
==> mes_node1:  - Extracting mesos-1.3.0
==> mes_node1:  - Installing mesos
==> mes_node1:  - Basic environment setup 
==> mes_node1:  - Registering test cleaning traps
==> mes_node1:  - Starting Mesos Master
==> mes_node1:  - Checking Mesos master startup
==> mes_node1:  - Starting Mesos Agent
==> mes_node1:  - Checking Mesos Agent startup
==> mes_node1:  - Stopping Mesos Agent
==> mes_node1:  - Stopping Mesos Master
==> mes_node1:  - Cleaning build folder
==> mes_node1: -- INSTALLING SPARK -----------------------------------------------------------
==> mes_node1:  - Changing to temp directory
==> mes_node1:  - Downloading spark-2.2.0
==> mes_node1:  - Extracting spark-2.2.0
==> mes_node1:  - Installing spark
==> mes_node1:  - Checking Spark Installation
==> mes_node1: -- INSTALLING ES-HADOOP (Spark to ES connector) -------------------------------
==> mes_node1:  - Changing to temp directory
==> mes_node1:  - Downloading elasticsearch-hadoop-5.5.1
==> mes_node1:  - Extracting elasticsearch-hadoop-5.5.1
==> mes_node1:  - Installing eshadoop spark (to spark jar folder)
==> mes_node1: -- SETTING UP NTP --------------------------------------------------------------
==> mes_node1:  - Configuring slave NTP server
==> mes_node1:  - restarting ntp
==> mes_node1: -- SETTING UP ELASTICSEARCH ----------------------------------------------------
==> mes_node1:  - Increasing system vm.max_map_count setting
==> mes_node1:  - Creating elasticsearch user (if not exist)
==> mes_node1:  - Creating elasticsearch data directory in /var/lib/elasticsearch
==> mes_node1:  - Creating elasticsearch PID file directory
==> mes_node1:  - Opening rights to logs folder to user elasticsearch
==> mes_node1:  - Linking elasticsearch logs to /var/log/
==> mes_node1:  - Simlinking ES config to /usr/local/etc/elasticsearch
==> mes_node1:  - Simlinking ES binaries and scripts to /usr/local/sbin
==> mes_node1:  - Copying elasticsearch systemd file
==> mes_node1:  - Checking Systemd file
==> mes_node1:  - Adapting configuration in file elasticsearch.yml
==> mes_node1:  - Adapting configuration in file elasticsearch.yml - enabling discovery of master
==> mes_node1:  - Addressing issue with multiple interfaces but only one global
==> mes_node1:  - Adapting configuration in file jvm.options
==> mes_node1:  - Testing systemd startup - starting Elasticsearch
==> mes_node1:  - Testing systemd startup - Checking startup
==> mes_node1:  - Testing systemd startup - stopping elasticsearch
==> mes_node1:  - Enabling elasticsearch on startup
==> mes_node1: -- SETTING UP MESOS ------------------------------------------------------------
==> mes_node1:  - Simlinking mesos config to /usr/local/etc/mesos
==> mes_node1:  - Simlinking mesos binaries to /usr/local/bin
==> mes_node1:  - Simlinking mesos system binaries to /usr/local/sbin
==> mes_node1:  - Creating mesos slave working directory in /var/lib/mesos/slave
==> mes_node1:  - Creating mesos log directory in /var/log/mesos
==> mes_node1:  - Copying mesos-init-wrapper script
==> mes_node1:  - Copying mesos-slave systemd file
==> mes_node1:  - Creating common mesos environment file
==> mes_node1:  - Creating mesos slave environment file
==> mes_node1:  - Checking mesos slave Systemd file
==> mes_node1:  - Testing systemd startup - starting mesos-slave
==> mes_node1:  - Testing systemd startup - Checking startup of slave
==> mes_node1:  - Testing systemd startup - stopping mesos-slave
==> mes_node1:  - Enabling mesos-slave on startup
==> mes_node1: -- SETTING UP SPARK ------------------------------------------------------------
==> mes_node1:  - Creating spark user (if not exist)
==> mes_node1:  - Simlinking spark config to /usr/local/etc/spark
==> mes_node1:  - Simlinking spark binaries to /usr/local/bin
==> mes_node1:  - Simlinking spark system binaries to /usr/local/sbin
==> mes_node1:  - Creating spark event log directory in /var/lib/spark/eventlog
==> mes_node1:  - Opening rights to logs folder to user spark
==> mes_node1:  - Linking spark logs to /var/log/
==> mes_node1:  - Defining SPARK_HOME
==> mes_node1:  - Creating spark environment file
==> mes_node1:  - Creating spark default configuration file
==> mes_node1:  - Copying log4j.properties from template
==> mes_node1:  - Generating /etc/hosts for proper DNS resolution
==> mes_node1:  - !! TODO : remove hardcoding of IPs and make them configurable at Provisionning level !!
==> mes_node1:  - Changing hostname - THIS REQUIRES REBOOT
==> mes_node1:  -> Hostname changed successfully => PLEASE REBOOT VM ASAP !
==> mes_node1: Running provisioner: reload...
==> mes_node1: Attempting graceful shutdown of VM...
==> mes_node1: Clearing any previously set forwarded ports...
==> mes_node1: Clearing any previously set network interfaces...
==> mes_node1: Preparing network interfaces based on configuration...
    mes_node1: Adapter 1: nat
    mes_node1: Adapter 2: hostonly
==> mes_node1: Forwarding ports...
    mes_node1: 22 (guest) => 2200 (host) (adapter 1)
==> mes_node1: Running 'pre-boot' VM customizations...
==> mes_node1: Booting VM...
==> mes_node1: Waiting for machine to boot. This may take a few minutes...
    mes_node1: SSH address: 127.0.0.1:2200
    mes_node1: SSH username: vagrant
    mes_node1: SSH auth method: private key
==> mes_node1: Machine booted and ready!
==> mes_node1: Checking for guest additions in VM...
==> mes_node1: Configuring and enabling network interfaces...
==> mes_node1: Mounting shared folders...
    mes_node1: /setup => /data/badtrash/work/es_spark/sandbox/setup/provisionning
    mes_node1: /vagrant => /data/badtrash/work/es_spark/sandbox/setup/vagrant
==> mes_node1: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> mes_node1: flag to force provisioning. Provisioners marked to run always will still run.

1.3 BUILDING mes_node2 VM
Bringing machine 'mes_node2' up with 'virtualbox' provider...
==> mes_node2: Importing base box 'niceideas-debian-stretch64.box'...
==> mes_node2: Matching MAC address for NAT networking...
==> mes_node2: Setting the name of the VM: mes_node2
==> mes_node2: Fixed port collision for 22 => 2222. Now on port 2201.
==> mes_node2: Clearing any previously set network interfaces...
==> mes_node2: Preparing network interfaces based on configuration...
    mes_node2: Adapter 1: nat
    mes_node2: Adapter 2: hostonly
==> mes_node2: Forwarding ports...
    mes_node2: 22 (guest) => 2201 (host) (adapter 1)
==> mes_node2: Running 'pre-boot' VM customizations...
==> mes_node2: Booting VM...
==> mes_node2: Waiting for machine to boot. This may take a few minutes...
    mes_node2: SSH address: 127.0.0.1:2201
    mes_node2: SSH username: vagrant
    mes_node2: SSH auth method: private key
==> mes_node2: Machine booted and ready!
==> mes_node2: Checking for guest additions in VM...
==> mes_node2: Configuring and enabling network interfaces...
==> mes_node2: Mounting shared folders...
    mes_node2: /setup => /data/badtrash/work/es_spark/sandbox/setup/provisionning
    mes_node2: /vagrant => /data/badtrash/work/es_spark/sandbox/setup/vagrant
==> mes_node2: Running provisioner: shell...
    mes_node2: Running: /tmp/vagrant-shell20170809-26262-1njec1.sh
==> mes_node2: mesg: 
==> mes_node2: ttyname failed
==> mes_node2: : 
==> mes_node2: Inappropriate ioctl for device
==> mes_node2: -- PROVISIONNING INFORMATION ---------------------------------------------------
==> mes_node2: VM name : mes_node2
==> mes_node2: VM is slave 
==> mes_node2: Master IP is 192.168.10.10
==> mes_node2: IP address is 192.168.10.12
==> mes_node2: -- BASIC SETUP -----------------------------------------------------------------
==> mes_node2:  - Updating the packages
==> mes_node2:  - Upgrading the appliance
==> mes_node2:  - Installing a few utility tools
==> mes_node2:  - Installing the latest OpenJDK
==> mes_node2:  - Installing build tools
==> mes_node2:  - Installing swapspace
==> mes_node2:  - Installing python
==> mes_node2:  - Installing other Mesos dependencies
==> mes_node2:  - Installing Spark Dependency : Scala
==> mes_node2:  - Installing ntp
==> mes_node2:  - Disabling IPv6
==> mes_node2:  - Creating test user: mes
==> mes_node2:  - Adding user mes to sudoers
==> mes_node2: -- INSTALLING ELASTICSEARCH ----------------------------------------------------
==> mes_node2:  - Changing to temp directory
==> mes_node2:  - Downloading elasticsearch-5.5.0
==> mes_node2:  - Extracting elasticsearch-5.5.0
==> mes_node2:  - Installing ElasticSearch
==> mes_node2:  - Registering test cleaning traps
==> mes_node2:  - Starting ElasticSearch
==> mes_node2:  - Checking Elasticsearch startup
==> mes_node2:  - Stopping Elasticsearch
==> mes_node2: -- INSTALLING MESOS ------------------------------------------------------------
==> mes_node2:  - Changing to temp directory
==> mes_node2:  - Downloading mesos-1.3.0 (pre-built version for Debian Stretch from niceideas.ch)
==> mes_node2:  - Extracting mesos-1.3.0
==> mes_node2:  - Installing mesos
==> mes_node2:  - Basic environment setup 
==> mes_node2:  - Registering test cleaning traps
==> mes_node2:  - Starting Mesos Master
==> mes_node2:  - Checking Mesos master startup
==> mes_node2:  - Starting Mesos Agent
==> mes_node2:  - Checking Mesos Agent startup
==> mes_node2:  - Stopping Mesos Agent
==> mes_node2:  - Stopping Mesos Master
==> mes_node2:  - Cleaning build folder
==> mes_node2: -- INSTALLING SPARK -----------------------------------------------------------
==> mes_node2:  - Changing to temp directory
==> mes_node2:  - Downloading spark-2.2.0
==> mes_node2:  - Extracting spark-2.2.0
==> mes_node2:  - Installing spark
==> mes_node2:  - Checking Spark Installation
==> mes_node2: -- INSTALLING ES-HADOOP (Spark to ES connector) -------------------------------
==> mes_node2:  - Changing to temp directory
==> mes_node2:  - Downloading elasticsearch-hadoop-5.5.1
==> mes_node2:  - Extracting elasticsearch-hadoop-5.5.1
==> mes_node2:  - Installing eshadoop spark (to spark jar folder)
==> mes_node2: -- SETTING UP NTP --------------------------------------------------------------
==> mes_node2:  - Configuring slave NTP server
==> mes_node2:  - restarting ntp
==> mes_node2: -- SETTING UP ELASTICSEARCH ----------------------------------------------------
==> mes_node2:  - Increasing system vm.max_map_count setting
==> mes_node2:  - Creating elasticsearch user (if not exist)
==> mes_node2:  - Creating elasticsearch data directory in /var/lib/elasticsearch
==> mes_node2:  - Creating elasticsearch PID file directory
==> mes_node2:  - Opening rights to logs folder to user elasticsearch
==> mes_node2:  - Linking elasticsearch logs to /var/log/
==> mes_node2:  - Simlinking ES config to /usr/local/etc/elasticsearch
==> mes_node2:  - Simlinking ES binaries and scripts to /usr/local/sbin
==> mes_node2:  - Copying elasticsearch systemd file
==> mes_node2:  - Checking Systemd file
==> mes_node2:  - Adapting configuration in file elasticsearch.yml
==> mes_node2:  - Adapting configuration in file elasticsearch.yml - enabling discovery of master
==> mes_node2:  - Addressing issue with multiple interfaces but only one global
==> mes_node2:  - Adapting configuration in file jvm.options
==> mes_node2:  - Testing systemd startup - starting Elasticsearch
==> mes_node2:  - Testing systemd startup - Checking startup
==> mes_node2:  - Testing systemd startup - stopping elasticsearch
==> mes_node2:  - Enabling elasticsearch on startup
==> mes_node2: -- SETTING UP MESOS ------------------------------------------------------------
==> mes_node2:  - Simlinking mesos config to /usr/local/etc/mesos
==> mes_node2:  - Simlinking mesos binaries to /usr/local/bin
==> mes_node2:  - Simlinking mesos system binaries to /usr/local/sbin
==> mes_node2:  - Creating mesos slave working directory in /var/lib/mesos/slave
==> mes_node2:  - Creating mesos log directory in /var/log/mesos
==> mes_node2:  - Copying mesos-init-wrapper script
==> mes_node2:  - Copying mesos-slave systemd file
==> mes_node2:  - Creating common mesos environment file
==> mes_node2:  - Creating mesos slave environment file
==> mes_node2:  - Checking mesos slave Systemd file
==> mes_node2:  - Testing systemd startup - starting mesos-slave
==> mes_node2:  - Testing systemd startup - Checking startup of slave
==> mes_node2:  - Testing systemd startup - stopping mesos-slave
==> mes_node2:  - Enabling mesos-slave on startup
==> mes_node2: -- SETTING UP SPARK ------------------------------------------------------------
==> mes_node2:  - Creating spark user (if not exist)
==> mes_node2:  - Simlinking spark config to /usr/local/etc/mesos
==> mes_node2:  - Simlinking spark binaries to /usr/local/bin
==> mes_node2:  - Simlinking spark system binaries to /usr/local/sbin
==> mes_node2:  - Creating spark event log directory in /var/lib/spark/eventlog
==> mes_node2:  - Opening rights to logs folder to user spark
==> mes_node2:  - Linking spark logs to /var/log/
==> mes_node2:  - Defining SPARK_HOME
==> mes_node2:  - Creating spark environment file
==> mes_node2:  - Creating spark default configuration file
==> mes_node2:  - Copying log4j.properties from template
==> mes_node2:  - Generating /etc/hosts for proper DNS resolution
==> mes_node2:  - !! TODO : remove hardcoding of IPs and make them configurable at Provisionning level !!
==> mes_node2:  - Changing hostname - THIS REQUIRES REBOOT
==> mes_node2:  -> Hostname changed successfully => PLEASE REBOOT VM ASAP !
==> mes_node2: Running provisioner: reload...
==> mes_node2: Attempting graceful shutdown of VM...
==> mes_node2: Clearing any previously set forwarded ports...
==> mes_node2: Clearing any previously set network interfaces...
==> mes_node2: Preparing network interfaces based on configuration...
    mes_node2: Adapter 1: nat
    mes_node2: Adapter 2: hostonly
==> mes_node2: Forwarding ports...
    mes_node2: 22 (guest) => 2201 (host) (adapter 1)
==> mes_node2: Running 'pre-boot' VM customizations...
==> mes_node2: Booting VM...
==> mes_node2: Waiting for machine to boot. This may take a few minutes...
    mes_node2: SSH address: 127.0.0.1:2201
    mes_node2: SSH username: vagrant
    mes_node2: SSH auth method: private key
==> mes_node2: Machine booted and ready!
==> mes_node2: Checking for guest additions in VM...
==> mes_node2: Configuring and enabling network interfaces...
==> mes_node2: Mounting shared folders...
    mes_node2: /setup => /data/badtrash/work/es_spark/sandbox/setup/provisionning
    mes_node2: /vagrant => /data/badtrash/work/es_spark/sandbox/setup/vagrant
==> mes_node2: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> mes_node2: flag to force provisioning. Provisioners marked to run always will still run.

3. ELK-MS CLUSTER SANITY CHECKS
===============================================================================

(TODO To Be Implemented)