Big data processing tools – Spark SQL and DataFrames (08:25)

This video lesson will give you an understanding of:

• How structured data manipulation is used in Spark to allow for optimisation.

• The two main interfaces for structured data manipulation: SQL (structured query language) and DataFrames

• What operations are provided by Spark to interact with structured, large datasets.

By the end of this video lesson, you will:

• Understand why structured data manipulation allows Spark to ensure query optimisation and efficient resource utilisation

• Understand how SparkSQL and DataFrames work

• See examples of transformations and actions, as well as SQL queries that can be applied over structured data in Spark.