May 17, 2016: Elastic Cloud Storage (ECS) is Now Certified with Apache Spark and Supports HAWQ 1.3.1

May 17, 2016: Elastic Cloud Storage (ECS) is Now Certified with Apache Spark and Supports HAWQ 1.3.1
Spark Leverages ECS Single, Shared, Global, HDFS Storage Cloud

ECS 2.2 SP1 HDFS has been certified to support Apache Spark 1.4.1 using the Hortonworks Certification Test Suite. In addition, ECS 2.2 SP1 has introduced support for Pivotal HAWQ 1.3.1 (HDB 1.3.1.0/1.3.1.1) with PivotalHD 3.0.1 . With ECS 2.2.1, HAWQ support is qualified for operation with ECS HDFS in a Kerberos environment.

Spark is a high speed data processing engine which enables the execution of streaming, machine learning, or SQL workloads that require access to large data sets for high speed analytics. Hortonworks HDP provides a YARN-based architecture that enables Spark and other Hadoop services to share a common cluster and dataset. ECS benefits this architecture by providing a global, shared, HCFS (Hadoop Compatible File System) which eliminate silos of Hadoop DAS and simplifies and modernizes the workflow with benefits including; “data in-place” analytics, “enterprise-grade” Hadoop storage, and HDFS compatibility.