Skip to main content

How does Pyspark work in TIBCO Data Science Team Studio?

Rich Talbot
  • Created
Date Posted:
Product: TIBCO Spotfire®

Problem:

How does Pyspark work in TIBCO Data Science Team Studio?


Solution:

How does Pyspark work in TIBCO Data Science Team Studio?  Pyspark can be initialized in the python notebooks by clicking the button "Initialize Pyspark for Cluster". A code is automatically generated by Team Studio to initialize spark context. This code will run on the Hadoop cluster as the python notebooks server doesn't understand spark code.


NOTE: Make sure that there are no connection and name to Ip address resolution issues between the Python Notebooks server and the Hadoop cluster. 

Comments

0 comments

Article is closed for comments.