Ray marine MFD apps three ways: from drone control to sat comms to video streaming

MIBS 2018 saw the manufacturers putting the multi in MFD, with Garmin and Raymarine each introducing multiple new multifunction app and app-like display integrations interfacing to all sorts of third party boat systems, plus consumer electronics gadgets and online services — actually every single brand seen below, with lots more likely coming. The results are exciting, but what the heck are MFD apps? Fortunately, Raymarine is illustrating all three ways they can be created and why it’s useful to know how it’s done…

What’s an MFD app?

At the recent Miami International Boat Show there were three major categories of integrations on display. All three can end up looking pretty much the same when they’re displayed on your MFD but how they got there can be important. MFD manufacturers can use another company’s SDK (Software Development Kit) to control systems on the boat. Or they can publish an API (application programming interface) allowing third party developed integrations to be displayed on the MFD. Lastly, we also saw an MFD show off its ability to run off-the-shelf Android apps.

The first integration, an MFD reaching out to a third party via their SDK, takes the most work for the MFD developers but can result in the most customized and tightest integrations. An early example was MFD developers each coding to Fusion’s FusionLink audio control API; the interfaces are each different and often include difficult integrations like an onscreen sound bar. In Miami, Raymarine introduced the impressive SDK-based UAV (unmanned aerial vehicle, or drone) control detailed below.

The second type of integration, where others code to the MFD’s API, is actually a series of dynamic HTML5 screens being rendered on the MFD. The plotters able to display these integrations are running either Linux or Android, both of which have native HTML5 rendering (Android’s core is a modified version of Linux). These integrations require third party hardware on the boat that serves up the HTML5 via Ethernet or WiFi, like this Lumishore Lumi-Link Bridge.

HTML5 integrations — Garmin calls them OneHelm, while Raymarine groups them as part of their new Lighthouse Apps — allow the MFD manufacturers to easily offer integrations with willing third party marine companies without having to do the development themselves. So, wow, the Mazu all-in-one satellite communications system will suddenly be accessible on a bright Axiom screen — when Lighthouse 3.4 releases, hopefully this spring — and the Seakeeper monitoring app is coming to both Garmin and Raymarine. But there are limitations to HTML5 integration we’ll discuss below.

The last option is native Android apps and is currently unique to Raymarine (though Furuno TZT2 and Si-Tex NavStar MFDs also run Android). While these apps will be Raymarine approved and only available from their Lighthouse app store, a product manager told us that the Netflix and Spotify apps they demonstrated at MIBS were running without modification. That means nearly zero effort on Raymarine’s part while the Axiom user gets access to apps from big time services that would never build apps for special marine hardware. Again there are limitations — like the MFD must be online for music or video streaming — and there’s also the question “Is this taking marine navigation electronics too far?” But please consider the analysis below before making a judgement.

SDK “App”: deep drone control

First let’s deal with some possible confusion. When Lighthouse 3.4 arrives and you tap that new Apps button on the Home screen, you will not see Ray’s new drone interface on the new Installed Apps screen. These are the optionally installed “Lighthouse Apps” discussed below. But then again Raymarine has long said that the “LightHouse 3 home screen offers bold and simple app icons.”

In fact, Raymarine has used, perhaps overused, the “app” term for even completely internal MFD functions like chart plotting since LightHouse 2 days, though in some cases the system icon represents an SDK-based integration with, say, a pan and tilt FLIR thermal camera. Other SDK examples are the Theyr Weather app built into LH2 r17 — especially interesting as it will appear in LH3 as a straight-up Android app (seen below) — and the impressive new drone app.

The two of us got to see a beta version of an Axiom Pro controlling that tiny DJI Spark drone, and then posed on the bow as the drone streamed HD video to the Axiom over WiFi. That would be pretty cool in itself, but note how the chart window is showing the Spark’s location relative to the boat, as well as its “heading” (camera direction), and that’s just scratching the integration surface.

The context menu above suggests the many levels of drone control possible. Free Flight mode, for instance, overlays the streaming video screen with controls similar to the Spark’s optional joystick controller. But because DJI has developed an ActiveTrack mode able to follow a moving object, it’s also possible to have the drone auto fly in tandem with your boat capturing that underway footage you always wanted. In fact, the planned Fish On mode will purportedly combine several Spark features so you can keep fishing as the drone takes off and films the action while circling your tracked boat.

Combining a camera drone with an MFD is also useful for finding fish, as Raymarine was able to demonstrate in their Miami demo video, and future UAV app possibilities include flying to a charted route. Couldn’t that be useful before entering an unfamiliar gunkhole? And if you’ve ever tried to fly a drone using a small, dim display in bright daylight, you know how much easier it will be using a big bright Axiom MFD.

The bigger picture here is that Raymarine is working hard to integrate Spark (and even more feature rich Mavic) drone control to this level, but that’s what it takes sometimes to integrate with a complex product from an industry working at an entirely different scale. DJI alone has 2.7 billion dollars in annual sales — roughly double the entire global recreational marine electronics industry — but the Ray developers could go to the DJI developer site and get to work. Having Android in the Axiom helps, but this SDK-based app development is still in sharp contrast to the easy way HTML5 browser apps can add system integration.