Shop

Your Notifications (0)

There are currently no notifications. You're all set!

Demos Trials
Knowledge Base
ChemDraw and ChemOffice+ Products Signals Notebook E-Notebook and ChemBioOffice Enterprise Spotfire and Signals Spotfire Products Columbus Signals Image Artist
Downloads
Downloads
Community
Support Forums Ideas Portal Product Basics
About Support
Legal Resources Support and Maintenance Plans Product Life Cycle Support News
Contact Support

Hadoop - Connecting Alpine to a cluster with Resource Manager High Availability enabled

Rich Talbot
  • Updated
Date Posted:
Product: TIBCO Spotfire®

Problem:

Hadoop - Connecting Alpine to a cluster with Resource Manager High Availability enabled


Solution:

Connecting Alpine to a cluster with Resource Manager High Availability enabled

Connecting Alpine to a cluster with Resource Manager High Availability enabled

If Resource Manager High Availability is configured on the cluster, login to the command line of one of the resource manager hosts, find the yarn-site.xml file (e.g. /etc/alternatives/hadoop-conf/yarn-site.xml), and from that file (similar to the below example) copy the parameters with their values into your Alpine data connection - on the Additional Parameters page (after configuring all the other settings - Integrating Spotfire Data Science with Kerberos):

yarn.resourcemanager.ha.rm-ids=rm60,rm70
yarn.resourcemanager.webapp.https.address.rm70=nn2.alpinenow.local:8090
yarn.resourcemanager.webapp.address.rm70=nn2.alpinenow.local:8088
yarn.resourcemanager.admin.address.rm70=nn2.alpinenow.local:8033
yarn.resourcemanager.resource-tracker.address.rm70=nn2.alpinenow.local:8031
yarn.resourcemanager.scheduler.address.rm70=nn2.alpinenow.local:8030
yarn.resourcemanager.address.rm70=nn2.alpinenow.local:8032
yarn.resourcemanager.webapp.https.address.rm60=nn1.alpinenow.local:8090
yarn.resourcemanager.webapp.address.rm60=nn1.alpinenow.local:8088
yarn.resourcemanager.admin.address.rm60=nn1.alpinenow.local:8033
yarn.resourcemanager.resource-tracker.address.rm60=nn1.alpinenow.local:8031
yarn.resourcemanager.scheduler.address.rm60=nn1.alpinenow.local:8030
yarn.resourcemanager.address.rm60=nn1.alpinenow.local:8032
yarn.resourcemanager.zk-address=cm.alpinenow.local:2181,nn1.alpinenow.local:2181,nn2.alpinenow.local:2181
yarn.resourcemanager.recovery.enabled=true
yarn.resourcemanager.ha.automatic-failover.embedded=true
yarn.resourcemanager.ha.automatic-failover.enabled=true
yarn.resourcemanager.ha.enabled=true
 

Also, add this extra parameter with a coma-separated list of all resource manager hostnames:

failover_resource_manager_hosts=nn1.alpinenow.local,nn2.alpinenow.local
  
Note: If there is no TLS/SSL enabled for any of the cluster services, adding only 'failover_resource_manager_hosts' parameter should be good enough.