Spark SQL程序设计（2）

读取JSON数据

{"age":"45","gender":"M","occupation":"7","userID":"4","zipcode":"02460"}

{"age":"1","gender":"F","occupation":"10","userID":"1","zipcode":"48067"}

scala> val jsondf=spark.read.textfile("//user.json") 

生成另外一个json文件

scala> josndf.write.mode("overwrite").json("//user2.json")

查看DF

scala>userDF.show(4) scala> userDF.printSchema root |-- age: string (nullable = true) |-- gender: string (nullable = true) |-- occupation: string (nullable = true) |-- userID: string (nullable = true) |-- zipcode: string (nullable = true) 

修改DF/DS元信息