:23: error: value load is not a member of org.apache.spark.sql.SparkSession

Issue:

While I was loading Data from an RDBMS Database to Spark, I happened to see the error below,

val details=spark.load(“jdbc”,Map(“url” -> “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”, “dbtable” -> “scott.employees”))
<console>:23: error: value load is not a member of org.apache.spark.sql.SparkSession
val details=spark.load(“jdbc”,Map(“url” -> “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”, “dbtable” -> “scott.employees”))
^
val details=sqlContext.load(“jdbc”,Map(“url” -> “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”, “dbtable” -> “scott.employees”))
<console>:23: error: not found: value sqlContext
val details=sqlContext.load(“jdbc”,Map(“url” -> “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”, “dbtable” -> “scott.employees”))

Resolution:

The commands for SPARK 1 and SPARK 2 are totally different. I was trying to use SPARK 1 commands on SPARK 2 setup. Below is the command to address the above error.

val oracletable=spark.read.format(“jdbc”).option(“url”, “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”).option(“driver”, “oracle.jdbc.OracleDriver”).option(“dbtable”, “employees”).load()
oracletable: org.apache.spark.sql.DataFrame = [CUSTOMER_ID: string, ADDRESS_TITLE: string … 36 more fields]

Advertisements
This entry was posted in Spark. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s