Apache Ignite Adds Spark DataFrames Support – iProgrammer

Apache Ignite, a distributed database for high-performance computing with in-memory speed, has been updated with support for Spark DataFrames and machine learning.

Ignite can be used as a traditional SQL database via JDBC drivers, ODBC drivers, or its own native SQL APIs. By default, it runs purely in-memory, but clusters can be configured to run on a mix of disk and memory.


It supports ‘co–located compute in Java, Scala, Kotlin, C#, and C++, meaning that you can create the equivalent of stored procedures in modern JVM languages, C# or C++ to develop and execute custom tasks across a distributed database. Ignite can operate in a strongly consistent mode that provides full support for distributed ACID transactions across multiple cluster nodes, caches, tables, and partitions.

Ignite supports continuous queries, meaning that rather than using a trigger to react to specific events, in Ignite you can write and run continuous queries written in languages such as Java or C# that process streams of changes on the database and application side.

Highlights of the latest version of the distributed database are general availability of its machine learning feature, and support for Spark DataFrames.

Ignite machine learning has TensorFlow integration as well as built-in algorithms and tools so it can be used to build scalable machine learning models without having to transfer data out of Ignite. You can train, deploy, evaluate, and update your ML and DL models continuously and at scale.

Perhaps even more strikingly, the latest version adds support for Spark DataFrames. The announcement