Monday, June 7, 2010

App drag-and-drop between devices: cross-device continuity is the future of the UI

The next "killer app" across all forms of end-user devices (smart-phones, tablets, HDTVs, PCs, etc) is the ability to "drag-and-drop" any application between any device, giving a seamless and continuous UI experience.  In effect, this mobilizes mobile and web apps, and makes all devices which a given user interfaces with, part of an overall continuous UI tapestry.

A quick look at the the direction in HDTVs, sets the stage for this next wave of UI innovation.  An important trend in newer model HDTVs is to offer apps/widgets, which can be downloaded/upgraded from an online "app store" as needed.  You name it: video Skype, Hulu, Netflix, Twitter, whatever.  Whether these be native apps or "web apps", they ought to evoke parallels to every other modern type of mobile or desktop platform.  What's occurring is none other than a convergence of end-user platforms, each with their own strengths.  But the key is that many of the same apps are offered across all possible platforms.

What is now needed, before we get too deep in, is an API for seamless "live" migration of applications across platforms.  You can think of this as real-time sync of applications with the application closing on one device and starting at the same context (and similar settings) on the next device.  And there's really nothing preventing this from occurring across various types of platforms.  It could of course be platform dependent, but following is a high level way to look at "UI continuity" which has the widest reach -- and thus the most value to the user.

First, the user would initialize (once) a sync API for any given device.  Thereafter, a mechanism (e.g. a desktop icon per-device, in-app menu item, etc.) would allow the user to request to mobilize a given app between devices.  Following this user request, the device OS would request context information be exchanged between similar apps (helps if the app is already installed on each device, but this could also evoke an auto-install).  One could think of this exchange as an XML file describing things such as, in the case of a video, a URL to the video content, a place-mark relating to where in the content the user left off, app settings, etc.  Once the exchange completed, the app on the destination device could take over, giving the feel of continuity to the user.  This kind of "migration" does not require a full environment sync.  Nor does it mandate app/OS version compatibility.  In fact, it allows the app on the destination device to adapt the app context to the new environment, and perhaps even be a different (version of an) app.

To the user, it means that at one moment they're surfing the web on their tablet, and at the next they migrate some apps to their HDTV.  Then bump them to the graphical console in their automobile.  Then to their work PC.  Or even to a projector for a demo.  For apps which are desired to run continuously, why not bump them to the "cloud" while a mobile device goes offline, for example?

I believe this level of continuity would revolutionize the user experience and give whoever implements it, a strong advantage.  Google, with its motivations to put Android and ChromeOS on every type of popular user device, and strong back-end infrastructure, would be a likely 1st adopter.  Intel & ARM would also have a vested interest in selling their processor designs into next-generation devices which support this.  Only time will tell whether Apple, which generally exerts more control over its ecosystem would be quick to adopt.  It would be cool to be able to do this, even if only across iPhone/iPad/Apple TV.  In any case, it's important that this API gets baked in early, because app developers need to hook into it now.

And fwiw, perhaps there's room for a 3rd party app which creates a near-term nexus (API framework with authentication etc) to intermediate this level of inter-device app mobility.  Would make a good M&A target...

Disclosure: no positions

1 comment:

Digital Spaghetti said...

While it's not as advanced as the API you are suggesting, you should check out the application called 'Bump' available for the iPhone and Android platforms.

It allows you to bump phones together to transfer image, music and contact information and might be the right route you are looking for.