SELECT ApacheKafka WITH StreamingSQL FROM RealTimeData

In another life, I taught the Book of Genesis to high school students, including The Tower of Babel excerpt. It struck me ironic that God’s wrath strikes down the tower, cofounds the universal language and scatters humans around the globe to teach King Nimrod a lesson in hubris; meanwhile, the boys in my class were texting their girlfriends across the country and playing video games with friends in Europe and Asia. 

Technology allows us to form a new Tower; in particular, the ability to stream real-time events. For this technology to be built and managed, a common language is necessary, and more often than not, SQL is the common tongue of developers, architects and analysts. Recently, Matteo De Martino, Senior Scala Engineer at Lenses.io  presented on the benefits of streaming SQL to react to real-time data for business critical decisions. 

Anyone competent enough to build a pivot table on Excel understands how to act on data. You take a snapshot or data table and make an active query to parse out distribution and trends. While this is an important retrospective task, it does not allow for you to make business critical decisions in real-time. 

The streaming SQL processors that Matteo explored in his Kafka Office Hours allow for users to process real-time data. As infinite amounts of data stream through an Apache Kafka cluster, users can model the transformations of data and write back to Kafka.

Lenses SQL Engine takes it a step further with SQL Processors. Not only can you model transformations, you can execute them as well. Users are able to scale, manage and govern their data with DataOps.

This activates data, allowing Lenses.io customers to process data as it arrives and update the running state automatically.

Streaming SQL focuses on future data, so businesses can make time critical decisions. Matteo’s presentation outlines the advantages of Lenses Streaming SQL & SQL processors. Watch his video below and then try SQL for yourself using the Lenses Box, a self contained Apache Kafka environment.