Hadoop - Connecting Alpine to a cluster with Resource Manager High Availability enabled

Hadoop - Connecting Alpine to a cluster with Resource Manager High Availability enabled

Connecting Alpine to a cluster with Resource Manager High Availability enabled

Connecting Alpine to a cluster with Resource Manager High Availability enabled

If Resource Manager High Availability is configured on the cluster, login to the command line of one of the resource manager hosts, find the yarn-site.xml file (e.g. /etc/alternatives/hadoop-conf/yarn-site.xml), and from that file (similar to the below example) copy the parameters with their values into your Alpine data connection - on the Additional Parameters page (after configuring all the other settings - Integrating Spotfire Data Science with Kerberos):

yarn.resourcemanager.ha.rm-ids=rm60,rm70
yarn.resourcemanager.webapp.https.address.rm70=nn2.alpinenow.local:8090
yarn.resourcemanager.webapp.address.rm70=nn2.alpinenow.local:8088
yarn.resourcemanager.admin.address.rm70=nn2.alpinenow.local:8033
yarn.resourcemanager.resource-tracker.address.rm70=nn2.alpinenow.local:8031
yarn.resourcemanager.scheduler.address.rm70=nn2.alpinenow.local:8030
yarn.resourcemanager.address.rm70=nn2.alpinenow.local:8032
yarn.resourcemanager.webapp.https.address.rm60=nn1.alpinenow.local:8090
yarn.resourcemanager.webapp.address.rm60=nn1.alpinenow.local:8088
yarn.resourcemanager.admin.address.rm60=nn1.alpinenow.local:8033
yarn.resourcemanager.resource-tracker.address.rm60=nn1.alpinenow.local:8031
yarn.resourcemanager.scheduler.address.rm60=nn1.alpinenow.local:8030
yarn.resourcemanager.address.rm60=nn1.alpinenow.local:8032
yarn.resourcemanager.zk-address=cm.alpinenow.local:2181,nn1.alpinenow.local:2181,nn2.alpinenow.local:2181
yarn.resourcemanager.recovery.enabled=true
yarn.resourcemanager.ha.automatic-failover.embedded=true
yarn.resourcemanager.ha.automatic-failover.enabled=true
yarn.resourcemanager.ha.enabled=true

Also, add this extra parameter with a coma-separated list of all resource manager hostnames:

failover_resource_manager_hosts=nn1.alpinenow.local,nn2.alpinenow.local

Note: If there is no TLS/SSL enabled for any of the cluster services, adding only 'failover_resource_manager_hosts' parameter should be good enough.