Category Archives: Spark

Load Oracle Table to Spark and Create an External Table using Scala (#Hadoop #Oracle #Scala #Spark)

In this blog post, I would like to demonstrate the steps to load RDBMS table (Oracle in this case) into Spark and Create an External table. Before we start the actual steps, lets understand few of the definitions which are … Continue reading

Posted in Hadoop, Scala, Spark | Leave a comment

:23: error: value load is not a member of org.apache.spark.sql.SparkSession

Issue: While I was loading Data from an RDBMS Database to Spark, I happened to see the error below, val details=spark.load(“jdbc”,Map(“url” -> “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”, “dbtable” -> “scott.employees”)) <console>:23: error: value load is not a member of org.apache.spark.sql.SparkSession val details=spark.load(“jdbc”,Map(“url” -> “jdbc:oracle:thin:scott/tiger@//oracle:1521:sample”, … Continue reading

Posted in Spark | Leave a comment

Exception in thread “main” java.lang.UnsupportedClassVersionError:scala/tools/nsc/MainGenericRunner

Issue: [hadoop@node1 ~]$ scala -version Exception in thread “main” java.lang.UnsupportedClassVersionError: scala/tools/nsc/MainGenericRunner : Unsupported major.minor version 52.0 at java.lang.ClassLoader.findBootstrapClass(Native Method) at java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1070)   Cause:  Unsupported Version of Scala and Java   Resolution: For Scala version scala-2.12.1, use Java 1.8 +. For Scala … Continue reading

Posted in Scala, Spark | Leave a comment

Spark Setup on RHEL 7.2 and Apache Hadoop 2.7

Spark has multiple APIs (Spark Scala, Spark Java, Spark Python, Spark R) on which it can be setup. Lets see Spark setup on Scala API. Download the SPARK from the link http://spark.apache.org/downloads.html Un-compress the downloaded file [hadoop@node1 hadoop]$ tar -xvf … Continue reading

Posted in Spark | Leave a comment