Event-Driven Architecture is unblocking data-driven decisions in shipping

In March 2021, a 200,000 tonne ship got stuck in the Suez Canal, and the global shipping industry suddenly caught the world’s attention. It made us realize ships play an important role in our daily lives. Really important in fact; 90% of the things we consume arrive by ship.

Take a look at this map. By visualizing vessel routes over time, the pattern creates a map of the earth. Note the lack of vessels travelling close to the coast of Somalia where piracy is common.

Source: https://www.vox.com/2016/4/25/11503152/shipping-routes-map

Curious to learn more about data-driven decisions in the maritime industry, we talked to several of our customers who operate in this sector, particularly in relation to our favourite topic - how data and Event-Driven Architecture (EDA) are transforming the way businesses operate. Whilst the maritime industry is unique in so many ways, the trends we encountered are common to so many industries right now. Progress in maritime can be summarized as follows.

Higher availability of sensor data across the high seas

Where previously real-time data was difficult to generate and access at sea - now everyone agrees that much more data is becoming available: Data from the ships, the crew movement, weather, satellite and monitoring every square meter of the world. The factors driving this deluge of data are multiple:

• The availability of internet satellite connectivity, even in the most remote parts of our oceans

• The costs of internet connections are becoming lower, mean the economic viability of the internet on vessels is becoming much more realistic

• IoT sensors on ships can collect a host of data such as propeller speed, engine data and cargo condition

• Contextual data is becoming available “off vessel”. This means more accurate weather data and satellite monitoring of every square metre of the oceans.

Event-Driven Architecture: Opportunities in the shipping industry

Event-Driven Architecture (EDA) is a modern application design paradigm, centering on the processing, communication and persistence of events. An event is simply when something happens - an occurrence, or a change in state that’s significant for the system. In maritime this could be an increase in ship speed, a change in vessel position, or the appearance of an anomaly in ship engine performance.

The ability to process and act on events such as these presents opportunities ranging from broad environmental impact, to ironing out all kinds of operational inefficiencies.

Reducing greenhouse gas emissions

The biggest prize to be gained from processing events is the reduction of greenhouse gas emissions and fuel savings. This can be achieved with route optimization, power delivery from the engines and cargo efficiencies. The International Maritime Organization is targeting GHG reductions of 40% by 2030 and the hopes for 70% reduction by 2050.

Crew safety and welfare

We heard from Marlow Navigation how Event-Driven Architecture can make it easier for vessel owners to draft and change crew. They help ensure crew travel less and reduce time spent waiting around in hotels due to weather delayed arrivals: “During Covid, the Marlow Navigation crew app, connected by an EDA, helped us to prevent potential spreading of the virus by keeping a real-time track-and-trace of cases.” - Kyriakos Poyias, Solution Architect - Marlow Navigation.

Route optimization

For outsiders to the maritime industry, we're blissfully unaware of the pressures captains and their crews are under to get goods to port on time. They have to balance prompt arrival at ports, fuel consumption and crew safety by avoiding stormy conditions. Real-time data driven route optimization, based on many data sources, can help captains make informed decisions based on traffic, port traffic density, and weather.

Live vessel tracking such as MarineTraffic provides real-time information on the movements of ships, with uses ranging from predictive modelling for fuel consumption to changing shipping routes.

DevOps

From a tech perspective, event-driven systems enable more agile development processes leading to greater innovation and new business opportunities. Marlow Navigation told us about how they had worked with a monolithic system for 20 years, meaning change was slow:

“We introduced an EDA enabling a faster feedback to improvement loop between our end users and development teams.

End users were inspired to offer feedback on how to improve their user experience as they saw their changes being quickly implemented. Offering this level of flexibility is a game changer.” - Kyriakos Poyias, Solution Architect - Marlow Navigation.

Meanwhile, 90POE helps ship owners and operators be more efficient and effective with their OpenOcean Studio platform. “We see our customers are able to make decisions based on real-time data to help drive all fleet-related activities. For example, better understanding of relevant info can help with the forecast of arrival, routes, fuel consumption, which in turn helps crew planning.

As the quality of data increases the potential to run predictive models grows, helping captains and their crew to improve their journeys in a multitude of ways.” - Gareth Lloyd, Chief of Architecture & Strategy - 90POE

Barriers to realizing the potential of EDA in shipping

We heard from all our shipping customers there is little or no standardization of data formats. This presents challenges for those wanting to consume and process the data.

Entirely new data sources are being created through IoT sensors installed on vessels, providing data such as engine temperature, condition of the hull, whereabouts of cargo and propeller speeds. With the price for a sensor varying from under $10 to $1000s to collect the same information, the quality is variable.

This lack of trust in the data cements the mindset that it's better to go by opinion rather than embrace the benefits of event-driven business.

Everyone we talked to agreed there is a deluge of data coming in the future, and this presents scalability challenges for anyone trying to consume and create value from the data. Huge volumes of highly varied quality and varied structure mean we need technologies such as Apache Kafka.

MarineTraffic shows three active tropical storms swirling the world's oceans.

Our customers have one thing in common besides leading in the event-driven maritime space: They have used Lenses with Apache Kafka to provide self-service access to events.

This makes it possible to speed up time-to-market of streaming applications through reducing reliance on engineering talent to operate event-driven applications, but it also enables compliance through governance and privacy controls for event data. 

Lenses for Kafka helps Vortexa with Machine Learning insights so valuable traders will pay for the data. Marlow Navigation and others are using Lenses to break down Monolithic legacy systems to create a real-time Event-Driven Architecture, and open up new business opportunities.

Whilst the maritime industry is unique in so many ways, the trends we encountered are common to so many industries right now.

As we see in every other industry we serve, Event-Driven Architectures are opening up new possibilities. The challenges are human as much as technical: traditional ways of working with batch data or databases don’t apply and there’s lack of expertise to make these transformations.

The question is how quickly will you get to the outcomes you hope for?

Useful links & further reading

• Drive through Suez yourself: https://edition.cnn.com/interactive/2021/03/cnnix-steership/

• How industrial IoT will disrupt the shipping industry: https://www.iotevolutionworld.com/smart-transport/articles/442702-how-industrial-iot-will-disrupt-shipping-industry.htm 

• A visualization of the world’s shipping routes: https://www.vox.com/2016/4/25/11503152/shipping-routes-map