Productivity Highs with RIA Frameworks

After traveling out and back, I see everything from a different perspective. Web applications, today, are incredibly different from the way they were in the 90s. The biggest difference, from my perspective, is the way we develop them. Coming from the RIA perspective, using Silverlight and Flex, I am accustomed to a certain productivity that I pine for now that I am developing for the web again.

If you know me, you know that I was a big fan of Silverlight and Flex. They were frameworks which let me develop amazing desktop experiences in the browser. I was doing things in 2007 that to this day is still impossible to do with HTML5 today. Those frameworks had one flaw, however. They required plug-ins to run.

Death to the Plug-In

I’ve never been much of an apologist for plug-ins. I saw them as a necessary evil. An evil that became unsustainable in this brave new world of mobile computing. When Apple, Microsoft and Adobe killed Silverlight and Flex, I wasn’t sad because I loved those platforms. I was sad because I knew I couldn’t be as productive writing the same kinds of applications in HTML/JavaScript/CSS.

First Generation SPA Frameworks

This is where the Single-Paged Application (SPA) frameworks came to my aid. There are too many to count, but from my vantage point, it has felt like Backbone.js and Knockout.js have been the primary options in the SPA market. I landed on Backbone.js as my go-to choice because it was a more complete solution than Knockout.js with its client-side route handling and model synchronization. I found myself at home with building SPAs because they were so similar to RIAs.

Unfortunately, my productivity never came close to that of the plug-in frameworks. Backbone.js and Knockout.js simply aren’t high enough of an abstraction for them to provide the same abilities to me. They are the first generation of SPA tools and with that brings the pains of a first-generation technology.

Filling in the Gaps

We’re starting to see some pretty nice add-ons to our first-generation tools. These add-ons fill the gaps that the frameworks leave by attempting to make you more productive. Specifically, Backbone.js has a poor view abstraction, making things like collection views and deeply nested views difficult. Marionette.js builds upon Backbone.js to provide more structure, guidance and capabilities. Using Marionette.js boosted my productivity significantly by taking care of much of the boiler-plate code that I found myself writing with Backbone.js alone. It is still not ideal, mostly because the abstractions are not extremely strong. They are strongER, but nothing compared to the capabilities I pine for. Knockout.js has a similar packaging called Hot Towel. Hot Towel fills in the holes that Knockout.js left by providing model synchronization, route handling and child view handling.

Unfortunately, these approaches only take us as far as the base technologies allow us to. Without a strong abstraction to begin with, these add-ons can only ever takes us to generation 1.5.

Second Generation SPA Frameworks

At the same time, the second generation of the SPA frameworks are coming forward. They come from a higher level of abstraction and the code you write demonstrates it by being significantly less (half or less). These frameworks (Angular.js and Ember.js) provide everything we wanted from our first generation tools but they also bring more capabilities like convention-based development paradigms and dependency injection. What’s more, we are starting to see tooling (such as Batarang) built around these abstractions which let our development lifecycle be that much smoother.

I am focusing my efforts on these second-generation tool sets for now. I am not abandoning my Backbone.js/Marionette.js tools completely, but I am not planning to invest much more time in them. Instead of patching the holes that the first generation tools created, I am choosing to spend my time using (and recommending) the next generation of tools. Specifically, I am focusing my efforts on Angular.js. It fits my development style better and I feel like the abstractions make more sense to me. I’m not knocking on Ember.js in any way. It is a fine choice, in my opinion.

The Future of SPA Development

But this isn’t where we are settling. I think it will require a third generation of tools before the SPA space really takes off. I have no idea what these frameworks will be, but I expect they will approach the productivity that I grew accustomed to with Silverlight and Flex. The levels of abstractions will be higher and the tooling will be richer. I expect that we will start to approach similar developer/designer workflows that were so amazing in the SPA days. I expect we will see multiple language integration, including debugging and profiling tools built around these frameworks. They will feel much less like a web framework and more like a desktop development framework. The user experiences we create will be comparable to the experiences we created with Flex and Silverlight.

Paying Back the Debt

The funny thing is that we will have taken 5 years to get there. It will be 5 years of stepping out of the plug-ins and into the browser so that we can achieve the same quality of application that we were already producing. In some ways, it feels like a regression, but we have to realize that the plug-ins were really just a disguised form of technical debt. They provided a one-off environment so that we were able to do things that weren’t possible in the browser, but they did so by not investing in the browser. Ultimately, the browser is the platform we were headed towards so we incurred debt. We just have to pay it off before we can really innovate again.

Conclusion

I’m excited to see where the future of front-end development goes. I’m excited to see that great strides are being made and I am happy to be at the front of the wave. The community is starting to get excited too. Here in Ann Arbor, we will be kicking off a JavaScript user group very soon which will give us a place to discuss these technologies and approaches. I look forward to chatting it up!

On a personal note

This is my last blog post written for SRT Solutions. I am moving on to something new. More on that soon.

Dennis Burton and I worked on this with several other SRT colleagues and we are very proud of it. We presented it as a half-day CodeMash session last month and the room was overflowing with over 100 participants, all heads-down and hacking out code. Since then, we’ve picked up another 300 or so users who have gone through the 22 different possible paths in the site.

<< An Example Backbone/Parse.com App

The Todos example was extremely easy to get up and running with Heroku.  Even though this is a CoffeeScript based server running with Node.js, Heroku can support the application.  I won’t go over the install process for the pre-requisites (Git and Heroku) because it is easy to find on the web.  Assuming they are installed, deploy the app like this:

Pull down the Todos example

> git clone git@github.com:BrianGenisio/todos-parseback.git

Create the Heroku app

> heroku create -s cedar yourappname
Creating yourappname… done, stack is cedar
http://yourappname.herokuapp.com/ | git@heroku.com:yourappname.git
Git remote heroku added

Create file: package.json

{
  "name": "yourappname",
  "version": "0.0.1",
  "dependencies": {
    "express": "2.5.0",
    "coffee-script": "1.1.3",
    "restler": "0.2.1"
  }
}

Install the packages locally (for testing)

> npm install

Configure and test the app

> cp config.coffee.example config.coffee

Edit config.coffee per the steps in the previous article and run it:

> coffee proxy.coffee

Point your browser to http://localhost:3001 and play.  Everything should be working right now locally.  Next, lets get it up on Heroku.

Create file: run.js

# Necessary because Heroku can’t run CoffeeScript directly
require("coffee-script");
require("./proxy.coffee");

Create file: Procfile

web: node run.js

Deploy to Heroku (via git)

> git add .

> git commit -m "Adding Heroku files"

create mode 100644 run.js

> git push heroku master

Counting objects: 23, done.

Delta compression using up to 4 threads.

Total 23 (delta 1), reused 17 (delta 0)

—–> Heroku receiving push

—–> Node.js app detected

       http://yourappname.herokuapp.com deployed to Heroku

To git@heroku.com:yourappname.git

* [new branch]      master –> master

Try it out

> heroku open

Opening http://yourappname.herokuapp.com/

That’s it!

Seriously.  That was easy.  Now the app is hosted.  Heroku and Parse.com do all the dirty work for you.

<< Working with Backbone.js and the Parse.com Proxy    Hosting the Todos example on Heroku >>

This is the post where I put it all together.  I showed how to create a proxy to use Parse.com as your data store.  I also showed how to modify Backbone.js to use Parse.com.  Now, I will show how easy it is to use Backbone.js models and collections with a service like Parse.com.  Note that Parse.com is not the only service out there that does this.  StackMob is another example of a “place to put your stuff” service.  There are others.

In this app, I will borrow from the default Backbone.js example: Todos.  Todos is a simple to-do list, which uses the browser’s local storage to save data. 

I will adapt it to be hosted by the Parse.com proxy that I showed in my previous post.  Instead of storing locally, the app will put push to Parse.com.  Want to play with it?  I have hosted everything on Heroku: http://todos-parsepack.herokuapp.com/

There are only a few tiny steps to make this work:

1. Replace localstorage.js with parse.com.js
2. Modify the model and collection to use Parse.com instead of local storage
3. Configure the proxy
4. Run the proxy

1. Modify index.html

The example given with Backbone.js uses the “localstorage” override for Backbone.sync.  We want to use the default REST version Backbone.sync so we remove it.  We also want to use the base prototypes to adapt Parse.com to Backbone.

<script src="backbone-localstorage.js"></script>

<script src=”parse.com.js"></script>

2a. Modify the model

The example uses a vanilla Backbone.Model.  Instead, change it to Backbone.ParseModel and give it the location of the data.

window.Todo = Backbone.Model.extend({

window.Todo = Backbone.ParseModel.extend({

    urlRoot: "/data/Todos",

2b. Modify the Collection

The example uses a Backbone.Collection.  Instead, change it to Backbone.ParseCollection and give it the location of the data.

window.TodoList = Backbone.Collection.extend({

window.TodoList = Backbone.ParseCollection.extend({

    url: "/data/Todos",

Also, since we aren’t using local storage, remove the localStore details.

localStorage: new Store("todos"),

3. Configure the App

That is all you need to do in order to modify the app.  Everything else is exactly the same.  You do, however, need to create an App at Parse.com.

Use the Application ID and Master Key and edit the config.coffee file:

applicationID: 'APPLICATION ID'
masterKey:    'MASTER KEY'
apiPath:  '/data/'
port:     3001

4. Run the proxy:

> coffee proxy.coffee

Test the app

Add items to your list.

See them show up in your data store.

Conclusion

That was insanely easy.  You just need to tell you models and collections to go to the server instead of local storage, configure the proxy and run it.  This solution installs well on Heroku as you can see.  This app wasn’t built with any type of user models, so everyone in the world gets to share Todos in this example… that can also be done with Parse.com, but I that is a different post ☺.

If anyone finds this useful, I would be curious to know if it helps.  If you want the source code for this project, take a look at the GitHub project

As a bonus, I wrote up the modifications I needed to make in order to host the Parse-backed Todos example on Heroku. It was really easy and worth the writeup.

<< Making Backbone.js Work with Parse.com     An Example Backbone/Parse.com App >>

In the last post, I presented some base prototypes for Backbone.js models and collections that play nicely with Parse.com

Next, I will put all the pieces together and show how to use Backbone.js with Parse.com.  All of the code is available on GitHub. https://github.com/BrianGenisio/Parseback/tree/parse.com-proxy-with-backbone.js

Create an “App” on Parse.com called “MyPeeps”

Once you crate the App, you will be given an Application Id and a Master Key which are essentially your user/password authentication for the site.

Set the ApplicationID and MasterKey in the config.coffee file

Create the model and collections (people.coffee)

The model and collection extends from ParseModel and ParseCollection described in the last post.

window.app =
  models: {}
  collections: {}

class app.models.Person extends Backbone.ParseModel
  urlRoot: "/data/People"

class app.collections.People extends Backbone.ParseCollection
  model: app.models.Person
  url: "/data/People"

Create index.html

<html>
<head>
  <script type="text/javascript" src="/vendor/jquery-1.6.2.js"></script>
  <script type="text/javascript" src="/vendor/underscore-1.1.7.js"></script>
  <script type="text/javascript" src="/vendor/backbone-0.5.3.js"></script>

  <script type="text/javascript" src="app.js"></script>
</head>
<body>
  Use the developer tools in your browser to play
</body>
</html>

Run the proxy server

Point your browser to http://localhost:3001. Open your console and start playing

Use your developer tools to create and work with your backbone models (created in people.coffee).

Create

Create a new Person model with the attributes you want and save it.  The “People” class will get created on Parse.com and the attributes will get populated. Everything is schema-less so it gets created on demand.

List

Create a new People collection and fetch the items.  When the server responds, you will see your new item in the collection with the attributes you set.

Update

When you update the attributes on your Person model and save it, the updates will go back to Parse.com.

Show

If you create a new Person model with a known id, you can fetch it from Parse.com

Delete

If you don’t want your object anymore, you can destroy it.

Querying

A very useful feature of the Parse.com API is the ability to query data.  To illustrate this, let’s populate the database with three Person models: Brian Genisio, Eli Genisio and Jon Doe.

I can pass a query parameter into the fetch function because Person derives from ParseModel.  Here, I am querying for any object in the database with a last attribute of “Genisio”.  Note that I only receive the two and Jon Doe is not retrieved.

Queries can be more complex.  Here is a similar query asking for all objects who’s last name attribute is not “Doe”.

Conclusion

I’ve illustrated how easy it is to use Backbone.js to work with the Parse.com backend.  Everything available in the REST API is abstracted with the Model and Collection paradigm that Backbone provides.  By placing a small proxy between your HTML client and Parse.com, you can quickly build single-paged applications and forget about the back-end. (For data-centric apps)

In my next post, I will give an example of a single-paged application that uses Parse.com as the back-end.

In the previous post, I showed how you can create a minimal server that proxies requests on to Parse.com for back-end data storage.

Next, I need to consider the Backbone.js side of things.  Backbone gives a really great way of describing models and collections as long as the data conforms to its REST flavor.  Unfortunately, Parse.com does things a little bit differently.  I have encapsulated this code into some base prototypes that can be used later. 

parse.com.coffee

class Backbone.ParseCollection extends Backbone.Collection
  parse: (resp, xhr) ->
    data = super
    data.results

  fetch: (options) ->
    if options?.query?
      options.data = "where=" + JSON.stringify options.query
      delete options.query
    super

class Backbone.ParseModel extends Backbone.Model
  setId: (data) ->
    data.id = data.objectId unless data.id
    data

  constructor: (model) ->
    @setId model
    super

  parse: (resp, xhr) ->
    @setId super

  toJSON: () ->
    result = super
    delete result.createdAt
    delete result.updatedAt
    result

These base classes help me to adapt the data and behavior of Parse.com:

Collection Fetching

Backbone wants a JSON array from the server.  Parse.com gives you a JSON object with an array called “results” inside the root object.  The code overrides Backbone.Collection.parse to return the array property from the Parse.com response.

Collection Queries

You can make server-side queries via the query parameters in the Parse.com API.  The code overrides Backbone.Collection.fetch to take a “query” option.  Mutate it to the format that Parse.com wants in the API.  I’ll discuss this more in the next post.

Model IDs

Parse.com IDs have the name of “objectId” but Backbone expects the IDs to be named “id”.  The code overrides the Backbone.Model.parse and Backbone.Model.constructor functions to set the ID based on the “objectID” that comes from Parse.com.  This will be used later when updating or deleting the object.

Model Metadata

Parse.com includes two metadata fields: “createdAt” and “updatedAt”.  While they may be useful, Parse.com fails if you send these fields via a PUT request.  The code overrides Backbone.Model.toJSON to remove these items from the serialization, thus keeping them from being sent to Parse.com as well.

We now have base prototypes that allow us to communicate with the server.  Instead of extending Backbone.Model and Backbone.Collection, you should make your models extend Backbone.ParseModel and Backbone.ParseCollection. 

In the next post, I will show how to do this and we will play in the browser a bit in order to show how Backbone.js models and collections interact with the proxy server.

In my previous post, I asked the question: “Is there any reason why can’t we use a service like [Parse.com] in an HTML5/Javascript single-paged app?”  The answer is “yes”.

The reason: the same origin policy.  The browser won’t let you request Parse.com data from a different domain.  There are certainly tricks you can play if you just want to GET the data (like JSONP for example), but there is no good way to make POST/PUT/DELETE requests.  You have to stay on your own domain if you want to use all that a REST API has to offer.

Using my newfound CoffeeScript skills, I came up with this proxy server based on Node.js with Express and Restler.  While I’m at it, the server can return my index.html and all other assets (images, Javascript, CSS and HTML templates).

Here is the code (full code for this post is available on GitHub https://github.com/BrianGenisio/Parseback/tree/parse.com-proxy-with-readme):

proxy.coffee

express = require 'express'
restler = require 'restler'
config = require './config'

app = express.createServer()
app.use express.bodyParser()

app.get "/", (req, res) =>
  res.sendfile "#{__dirname}/index.html"

app.all "#{config.apiPath}*", (req, res) =>
  console.log "REQUEST: ", req.url, req.body || "no body"

  data = JSON.stringify(req.body)

  restOptions =
    username:  config.applicationID
    password:  config.masterKey
    data:      data
    method:    req.method.toLowerCase()
    headers:
      'Content-Type':  'application/json'
      'Content-Length': data?.length || 0

  complete = (data) =>
    console.log "COMPLETE: ", data
    res.json JSON.parse(data)

  error = (data, res) =>
    console.log "FAILURE: ", data, res

  restler.request("https://api.parse.com/1/classes/#{req.url.replace(config.apiPath, '')}", restOptions)
    .on("complete", complete)
    .on("error", error)

app.get "/*", (req, res) =>
  res.sendfile "#{__dirname}#{req.url}"

app.listen config.port

console.log "Server started on #{config.port}"

This light-weight proxy will open a port and listen to requests.  If you request “/” or any other file, it will dump it back.  If you request “/data/*”, it will proxy your request off to Parse.com and return the result back.  It will handle all of the GET/POST/PUT/DELETE requests that Parse.com supports.

This proxy server loads a config file:

config.coffee

module.exports =
  applicationID: 'Your Parse.com Application ID'
  masterKey:     'Your Parse.com Master Key'
  apiPath:  '/data/'
  port:     3001

The first two configuration options are provided to you by Parse.com when you create a new “App” with them.  All of your entities for that “App” will have the same Application ID and Master Key.

Next is the path you want the proxy to use when passing data requests on.  In this example, http://mydomain.com/data/People would proxy off to http://api.parse.com/1/classes/People.

Finally you can specify the port to run the proxy on.  This is self explanatory.

Now all I can run my server: coffee proxy.coffee

Using curl, I can test all of the actions that my new server supports:

Create

> curl -X POST -H "Content-Type: application/json" \

> -d ‘{"first": "Brian", "last": "Genisio"}’ \

>

http://localhost:3001/data/People

{"createdAt":"2011-11-05T19:11:25.873Z","objectId":"fiIMd53m0j"} 

List 

> curl -X GET http://localhost:3001/data/People

{"results":[{"updatedAt":"2011-11-05T19:11:25.873Z","last":"Genisio","createdAt":"2011-11-05T19:11:25.873Z","first":"Brian","objectId":"fiIMd53m0j"}]} 

Update 

> curl -X PUT -H "Content-Type: application/json" \ 
> -d ‘{"middle": "Michael"}’ \ 
> http://localhost:3001/data/People/fiIMd53m0j

{"updatedAt":"2011-11-05T19:15:25.122Z"} 

Show 

> curl -X GET http://localhost:3001/data/People/fiIMd53m0j                         
{"middle":"Michael","updatedAt":"2011-11-05T19:15:25.122Z","last":"Genisio","createdAt":"2011-11-05T19:11:25.873Z","first":"Brian","objectId":"fiIMd53m0j"}  

Delete 

> curl -X DELETE http://localhost:3001/data/People/fiIMd53m0j

{} 

Index.html

> curl http://localhost:3001

I AM AN HTML FILE

Other assets

> curl http://localhost:3001/foo.bar

I AM A TEXT FILE

Considering my entire server is less than 50 lines of code, I am happy with this compromise.  My technology (the browser) does not allow me to go completely server-less but this gets me pretty close.

In the next post, I will show how I got Backbone.js to talk to my proxy server.

Making Backbone.js work with Parse.com >>

For my data store, I have been playing with a new (currently in Beta) service called Parse.com.  Parse.com gives you a schema-less data store — a place to put your stuff with a REST API.  They are focusing on being the persistence layer for mobile apps when you don’t want to mess with a server.  Is there any reason why we can’t use a service like this in an HTML5/Javascript single-paged app?  It turns out that the answer is more complicated than “no” but it is something that we can certainly get around. 

This series will be broken into several parts:

1. Introduction (this page)
2. A Proxy Server for Parse.com
3. Making Backbone.js Work with Parse.com
4. Working with Backbone.js and the Parse.com Proxy
5. Building an Application with Parse.com Models
6. Hosting the Todos example on Heroku (Bonus)

All of the code (front-end and back-end) in these examples will be written in CoffeeScript but there is no reason you can’t do the same thing with Javascript.  Let me know if you find this short series useful. 

Disclaimer:  I am really just hacking and documenting my work.  I have no idea how this will perform in a real-world scenario.  It is a thought experiment.  If anything, this is really just a place for me to document my work.

Some History

Until recently, Flex has been criticized for requiring a separate runtime install. The criticism is valid – users were often confused by this and found the need to upgrade something other than the app they wanted to install. The user experience became one where the user was left wondering, “what just happened?”

In one case, my customer required that nothing be allowed installed on the users’ machine (network access was not available either). This meant no Air runtime and no app. We had a workaround which included an app embedded inside of a PDF but that left the user feeling like they were watching Inception (an app within an app…).

Although this was the case in OSX, Windows and Android, it was not the case in iOS (iPhone and iPad). Apple doesn’t allow third-party runtimes on their devices so Adobe created a native compiler and packager that bundles the Air libraries directly into the app.

Captive Runtime

In Air 3.0, however, the requirement for a separate Air runtime has been lifted. With the “Captive Runtime” feature, developers can bundle the native Air libraries with the app. When apps are bundled this way, they can be deployed any way you want. They won’t require an installation of Air OR the application itself (on the desktop; mobile apps still need to be installed but can be without the separate runtime). This doesn’t mean you are restricted from installing your app but it means that it is no longer required. In addition to iOS, Captive Runtime will work in Android, OSX and Windows.

Let’s see how easy it is to use. I will create a “Hello World” app for illustrative purposes but significantly more complicated applications work just as easily.

Flash Builder 4.6 has integration built-in integration for captive runtime.  When you “Export Release Build” from Flash Builder, you simply create a “Signed application with AIR runtime bundled”.

Everything else is the same. You add a signed certificate to the app as you normally would and any other packaging settings.  When you are complete, you will get a native application that is ready to execute but doesn’t require Air to be installed.

This app can be copied to a thumb drive, shared from network storage or embedded into an installer. It is an application just like any other.

Note that this does bloat the size of your application. This may or may not be an issue for you depending on your audience and deployment mechanism. You can always package your app as a traditional Air app if you prefer.

The Windows story for building your app is exactly the same as the OSX side. The only difference is that the app is packaged as an EXE with the Air support alongside it.

The story for Android is similar. You have a different dialog for packaging, but the idea is the same. The APK that is produced will have Air bundled inside.

Of course, you don’t need to use Flash Builder to package your app. You can do it from the SDK and the command line. Instead of repeating what you can already find out in other places, I will just refer you to Andrew Trice’s blog for more info on that.

If you would like to get a sneak peek of Flash Builder 4.6, you can apply for the Flex and Flash Builder pre-release.

• Presentation
• Code Demos
• ScoreKeeper (Shared screens) Demo