1.参考网址
根据官网上的文档进行操作,出现了各种不同的问题,将 其记录一下
2.问题及解决方案
(1)java版本编译错误
maven默认是java1.5,但1.5并不支持lambda语句,故需要更改jdk,需要修改三个地方:
file:setting
file:project structure
(2)JVM申请内存不足
解决方案:http://blog.csdn.net/yizheyouye/article/details/50676022
(3)其他问题
未设置master URL:修改SparkSession,设置master为local
read处报错:加上双引号
sparkSession识别不出来:导入相应的包
3.修改后代码
/* SimpleApp.java */import org.apache.spark.sql.*;public class WordCount { public static void main(String[] args) { String logFile = "/home/quickly/spark-2.2.0-bin-hadoop2.7/README.md"; // Should be some file on your system SparkSession spark = SparkSession.builder().master("local").appName("Simple Application").getOrCreate(); DatasetlogData = spark.read().textFile(logFile).cache(); long numAs = logData.filter(s -> s.contains("a")).count(); long numBs = logData.filter(s -> s.contains("b")).count(); System.out.println("Lines with a: " + numAs + ", lines with b: " + numBs); spark.stop(); }}