Status Update, Jan 2008

Happy 2008, everyone! It’s been a while since I posted here, so perhaps a status update is a good way to start the new year.

Over the past weeks, I’ve been doing a lot of reading, brainstorming, whiteboarding, experimenting, diagramming, idea bouncing, mocking-up and proof-of-concept hacking, and now I feel I’ve got a much better idea for how to build Candyspark sync, and a more concrete list of the parts and components that will be part of it.

So here’s a rundown of the current plan, citing the technologies and techniques that will be included. All of this is, of course, subject to change as we make progress.

Overview: How Candyspark Works

I put together this diagram back in November 2007, showing the main pieces of the Candyspark system as I envisioned them. At the time, I hadn’t dug too deeply into how to actually build this, so a few things have changed and a lot more details have been filled in, but it’s still basically the same idea.

Decisions and Updates
Here are some of the things I’ve been working on, decisions made, and other updates to my plan for Candyspark:

1. XMPP for connectivity to mobile devices — We’re going to use XMPP (aka, the Jabber protocol) to communicate between mobile devices and the rest of the system. If you’re interested in Internet technology and haven’t investigated XMPP, I’d really suggest taking a look — it’s pretty cool. XMPP is to real-time communication what HTTP is to document access, and it’s got the potential to explode over the next few years as it starts to catch on in more niches.

There will be XMPP client code on the phone, which connects to an XMPP server either on desktop or internet. This brings us a well-designed and well-tested protocol with powerful extension capabilities, and will ease integration with other technologies. Plus, it should work over any TCP/IP connection (WiFi and cellular natively, BT and USB w/ a helper) as long as ports are open, and it makes it clear that there’s minimal difference between desktop and internet syncing.

2. XULRunner as a desktop dev platform — That’s the open source Rich Internet Applications environment that’s been built by the Mozilla Foundation. It’s the basis of a lot of popular apps, including Mozilla’s own like Firefox and Thunderbird, but also third-party efforts like Joost, Songbird, and Miro. It’s open source, free to use and redistribute and cross-platform on Windows/Mac/Linux.

XULRunner should make it easy(ish) to build out the Candyspark desktop UI using HTML/CSS/Javascript without the need for heavy native development in C++/Java/whatever. On the other hand, it can be extended with native code using XPCOM; thus can integrate native libraries like ffmpeg, opensync, Bluetooth/USB helpers, etc.

For prototyping, we’ll build out some simple XUL user interface screens, either on the standalone XULRunner executable, or as Firefox plugins. Unfortunately, there doesn’t seem to be a very well-trodden path into XUL development, though there are some efforts to improve that, like Mozpad and OpenKomodo.

However, we’re still keeping an eye on Adobe AIR as an alternative. Adobe’s got a lot of experience building good developer tools and making development a more pleasant experience, and they’re making it clear that they plan to push hard to support development on the AIR platform. AIR also has a much better installer story, making it easy for new users to download, install and configure in just a few clicks from the web.

3. Candyspark Central: Server-Based Sync — With the use of XMPP, it’s clear how the same mobile client can sync with both the desktop and the Internet. It’s also clear, from talking to potential users and investigating competing products, that server-based syncing is appealing to a lot of people. Thus, we’ll design and build a server version alongside the desktop version. (This isn’t in the diagram above.)

From a business angle, the idea of Candyspark being both desktop- and server-based complicates the story somewhat, and leaves some issues to be sorted through regarding market positioning and business model. On one hand, having a server-based component makes for a more comprehensive solution and can lower the hurdle for new users to begin using the product.

From a technology angle, however, the server-based solution is an excellent fit with the other parts of the system. We’ll use server-side Javascript (running in SpiderMonkey or Rhino) to maintain a common code base with the desktop client/RIA. I’ve also spared a bit of thought for how this would eventually scale (horizontally, simply, and cheaply, of course…).

4. Mobile Client — A client will be installed on the mobile device to enable communication with the rest of the system (via XMPP as discussed above), and access to the device’s files, data and services. To begin, this will need to be a user-installed app, which means Candyspark will only work with smartphones that make that possible; eventually, we hope to get pre-installed and/or integrated with the device operating systems, so that won’t be a long-term requirement.

For prototype purposes, I’ve been hacking using Python on S60 (PyS60) on Nokia smartphones. When we get to the beta, we’ll probably launch using the Python version compiled into a SIS file; we may eventually port to C++ to improve size and performance, but only after we get the basic structure down. Symbian Platform Security will definitely be an issue for some Sparklet applications, so it’s good to get some exposure to those issues early. Also, S60 has the largest installed base among smartphone OSes, so is a very important platform for finding beta users.

We’re planning versions for Windows Mobile and Android as well. Android, in particular, should be interesting, as it already encourages peer-to-peer communications with other Android devices using built-in XMPP client code. So far, they only expose a few API methods and assume connection to the GTalk server, but it will be interesting to watch that evolve as they flesh out the platform details. [edit: look like people are figuring out how to get around Google's artificial limitations] It could be a very interesting platform for Candyspark.

We’ll probably open source the mobile client code, primarily to make it possible for device builders (including open-source device hackers like those from OpenMoko) to integrate with the Candyspark system.

5. The Candyspark Platform: Sparklets — One big area of the Candyspark plan is to expose a developer platform for third parties to build sync applications, which we call ’sparklets’. These are client applications which run in the desktop client or on the Candyspark server, which manage information flow between the mobile device, the desktop, and the Internet. Within the Candyspark framework, all user data syncing is handled by sparklets, and each individual sparklet is responsible for one kind of syncing.

The Candyspark product will come with a set of pre-installed sparklets for handling various tasks: contact sync, desktop file sync, podcast subscriptions, etc. Third parties will be able to build their own sparklets, using tools we provide, and make them available to users via the Candyspark Catalog (for free, or perhaps for sale; imagine the iTunes Music Store, but for sync plugins). Power users will be able to build their own sparklets, for private use or for publication.

This is shaping up to be the aspect of Candyspark with the most open questions. In Marc Andreessen’s terms, we’re building a Level 3 Internet platform. Issues of security and privacy will be very challenging, since we’re dealing with users’ most important data. Developers can be highly demanding users to build tools for; as Marc writes: “lots and lots of people have opinions about platforms, but the people whose opinions matter are programmers, and people who can make decisions about what programmers program”; we need to make a platform worth building on.

This is probably the toughest part of the plan to pull off, but I think it will be worth it.

6. Business Issues — The basic plan is to start small and cheap, and get a rough prototype built as soon as possible while self-funded. With that, we’ll pursue two avenues forward in parallel: building awareness with companies who could license our technology (like handset manufacturers, carriers, specialty service providers, etc.), and launching a public beta to get users to try out our product. I’m not going to go into budgets, revenue model, funding and all the rest here, because, a) none of your business, and b) largely speculative at this point. So best to leave that out.

Onwards

From here, the next big step is to build out these components enough to get the system as a whole operational. There are still plenty of open questions, but there are also several well-defined components ready to be built, at least in prototype form.

If you’re interested in hearing more about what I’m thinking, if you have questions or suggestions, if you want to be a beta (or alpha) tester, or if you think you might want to be part of this project in some way, please do let me know.