Motivation

There's many good reasons you'd want to use JSON-RPC as a transport in your web applications. First of all, instead of feeling like back door access to your webserver, AJAX can be used like most any other remote procedure call system. Your JavaScript starts feeling like a small but real client application rather than just a hidden trick up the html sleeve. Secondly, JSON is simple and makes your JavaScript simpler to write and read. This is especially true when coupled with some of the newer JavaScript libraries, like JQuery.

This is what I'm going to do in this slice: Use JQuery and the json-xml-rpc js library to connect to Web2Py's actions using its native JSON-RPC support.

It's not a perfect approach but it supplies a level of decoupling between server and client that makes me want to look past its small defects. This enlightening article by Luke Kenneth Casson Leighton goes into more details of the approach (see the section "Full-blown Javascript-led Development"). This is also the method used by frameworks such as GWT and PyJamas.

Basics

I'm creating 1 view and 2 controllers. The first controller is simply for calling the startup view, the second is for the JSON-RPC methods. There's no real reason not to use a single controller for both purposes but I want as much separation as I can between MVC generated code and rpc functions. It would be quite trivial, and possibly better practice, to use a static html document for the startup view.

The view includes JQuery, the json-xml-rpc library (see notes at end) and my own external js file.

<!-- JS Libs -->
<script language="javascript" src="{{=URL(r=request,c='static',f='js/jquery-1.3.2.min.js')}}" > </script>                       
<script language="javascript" src="{{=URL(r=request,c='static',f='js/rpc.js')}}" > </script>                    

<!-- Page JS -->
<script language="javascript" src="{{=URL(r=request,c='static',f='js/BasicJSONRPC.js')}}" > </script>

The BasicJSONRPC.py controller contains nothing more than the reference to the view.

def index():
    response.view = "BasicJSONRPC.html"
    return dict()

def BasicJSONRPC():
    response.view = "BasicJSONRPC.html"
    return dict()

The BasicJSONRPCData.py controller is where the real meat lives. We'll start simple.

import math
from gluon.tools import Service
service = Service(globals())

def call():
    return service()

@service.jsonrpc
def systemListMethods():
    #Could probably be rendered dynamically
    return ["SmallTest"];

@service.jsonrpc    
def SmallTest(a, b):
    return a + b

The systemListMethods action is required by the json-xml-rpc library. By default the library actually calls "system.ListMethods" which can't be supported by Python. We thus remove the period in the call inside the rpc library. The python function just needs to return an array of strings of all the possible methods to call.

Now that we have the controller ready, we can move on to the client portion. The URL to access the rpc methods is something like "http://localhost/Application/Controller/call/jsonrpc". Using this URL and the json-xml-rpc library, we create a JavaScript DataController object which we'll use for all future procedure calls.

var ConnectionCreationTime = null;
var DataController = null;
var Connected = false;

function InitDataConnection() {
    Connected = false;
    var url = GetConnectionURL();   // http://localhost/Application/Controller/call/jsonrpc 
    try {
        //Here we connect to the server and build the service object (important)
        DataController = new rpc.ServiceProxy(url);
        Connected = true;
    } catch(err) {
        Log("Connection Error: " + err.message);
        Connected = false;
    }
    var now = new Date();
    ConnectionCreated = now;    
}

By default, the json-xml-rpc lib creates the DataController for asynchronous calls. Since you don't want your JavaScript to be blocked during your requests, asynchronous is the desired behavior. If you'd however like to run a quick test of of your remote methods, you can run the following lines of JavaScript in the Firebug console.

InitDataConnection();
rpc.setAsynchronous(DataController,false);
DataController.SmallTest(1,2);

The json-xml-rpc documentation gives the details of how to run asynchronous calls.

function RunSmallTest() {    
    if(Connected == false)
        Log("Cannot RunSmallTest unless connected");
    else {                                            
        var a = GetAValue();
        var b = GetBValue();
        Log("Calling remote method SmallTest using values a=" + a + " and b=" + b);
        DataController.SmallTest({params:[a,b],
            onSuccess:function(sum){                
                Log("SmallTest returned " + sum);                
            },
            onException:function(errorObj){
                Log("SmallTest failed: " + errorObj.message);
            },
            onComplete:function(responseObj){
                Log("Call to SmallTest Complete");
            }
        });
        Log("Asynchronous call sent");            
    }
}

Dictionaries and arrays can be returned by your python functions as demonstrated by our BiggerTest function:

@service.jsonrpc    
def BiggerTest(a, b):
    results = dict()
    results["originalValues"] = [a,b]
    results["sum"] = a + b
    results["difference"] = a - b
    results["product"] = a * b
    results["quotient"] = float(a)/b
    results["power"] = math.pow(a,b)
    return results

Don't forget to update the systemListMethods function to include any new functions.

The test results in Javascript (called synchronously in Firebug console):

>>> InitDataConnection();
POST http://127.0.0.1:8000/BasicJSONRPC/BasicJSONRPCData/call/jsonrpc    200 OK    20ms    rpc.js    (line 368)
>>> rpc.setAsynchronous(DataController,false);
>>> var results = DataController.BiggerTest(17,25);
POST http://127.0.0.1:8000/BasicJSONRPC/BasicJSONRPCData/call/jsonrpc    200 OK    20ms    rpc.js    (line 368)
>>> results.originalValues
[17, 25]
>>> results.originalValues[1]
25
>>> results.sum
42
>>> results.difference
-8
>>> results.quotient
0.68

Authentication

It works!...kinda. Cookies are posted with every request and web2py is thus able to parse the session id cookie for JSON-RPC calls. Security can be added to your remote functions by securing the call function:

@auth.requires_login()
def call():
    return service()

If you were to also set @auth.requires_login on the main BasicJSONRPC.py controller, your users would login when they first load the page and all subsequent RPC calls will be correctly authenticated. The problem with this comes with timeouts. If a user lets the page idle until timeout occurs, she or he can still trigger rpc calls to the server. Authentication will then fail and the default web2py value of auth.settings.login_url, "/default/user/login", will be called as a view. The problem is that since a view is not a valid json-rpc message, the json-xml-rpc lib will discard it and fail. You can catch the error but it's not easy to identify it. The simplest solution I've found, and I'm hoping that others will find a better one, is to set the value of auth.settings.login_url to an action in the rpc controller which returns nothing but a simple string.

In db.py:

auth.settings.login_url = URL(r=request, c="BasicJSONRPC",f='Login')

"Login" will be a non JSON-RPC action (since we don't want it to require authentication) which returns an easily recognizable string:

def Login():
    return "Not logged in";

We can then detect authentication failure from the client side by running a check whenever an rpc call fails. In the onException handler of the asynchronous call:

onException:function(errorObj){
    if(errorObj.message.toLowerCase().indexOf("badly formed json string: not logged in") >= 0)
        PromptForAuthentication();
    else
        Log("SmallTest failed: " + errorObj.message);
}

The obvious flaw in this approach is that we've lost the very practical login view for regular html views. Therefore, while authentication works for RPC calls, it breaks it for html views.

Simplifying the calls

I don't think that it's possible to really simplify the syntax used by the json-xml-rpc library to make asynchronous call. It is however possible to somewhat automate many parts of it for calls that simply get or update client side data objects. This is especially useful if you're trying to handle errors and authentication in a consistent way. I therefore often use the following client wrapper function to make my asynchronous calls:

function LoadDataObject(objectName,params,responseObject,errorObject) {
    Log("Loading data object \"" + objectName + "\"")
    eval("" + objectName + " = \"Loading\"");
    eval(objectName +"Ready = false");
    if(responseObject === undefined) {
        if(Connected != true) {
            Log("Not connected, connecting...");
            InitDataConnection();
        }
        var listUndefined = eval("DataController." + objectName + " !== undefined")
        if(Connected == true && listUndefined == true) {
            var paramsString = "";
            for(var i in params) {
                paramsString += "params[" + i + "],";
            }
            //Removing trailing coma
            paramsString = paramsString.substring(0,(paramsString.length - 1));            
            eval("DataController." + objectName + "({params:[" + paramsString + "], \n\
                onSuccess:function(response){            \n\
                    LoadDataObject(\"" + objectName + "\",[" + paramsString + "],response)    \n\
                },                        \n\
                onException:function(error){            \n\
                    Log(\"Error detected\");    \n\
                    LoadDataObject(\"" + objectName + "\",[" + paramsString + "],null, error);    \n\
                },                        \n\
                onComplete:function(responseObj){        \n\
                    Log(\"Finished loading " + objectName + "\");\n\
                }                        \n\
            });");
        } else {
            eval(objectName + " = \"Could not connect\"");
            eval(objectName + "Ready = false");
            Log("Could not connect.  Either server error or calling non existing method ("  + objectName + ")");
        }        
    } else {        
        if(errorObject === undefined) {            
            eval(objectName + " = responseObject");
            eval(objectName +"Ready = true");    
        }
        else {            
            Log("Failed to Load Data Object " + objectName + ": " + errorObject.message)
            eval(objectName + " = errorObject");            
            eval(objectName + "Ready = false");
        }
    }
}

The function can be reused for any number of data objects. The requirements are:

• define a data object variable that has the same name as the rpc function (e.g. UserList)
• define another variable with named as follows "*<data object name>*Ready" (e.g. UserListReady)
• call the wrapper function by passing the name of the rpc action as a string and an array containing any required parameter values (e.g. LoadDataObject("UserList", ["admins",false])

During the call, the ready variable will be set to false and the data object variable will be set to the string "Loading". If an error occurs, the ready variable will remain false and the data object variable will be set to the error object. You can poll the two variables if need be. The attached code contains a full example (BiggerTest).

Notes about the json-xml-rpc library

The json-xml-rpc lib is a single JavaScript file which can obtained by downloading the rpc-client-javascript zip file from their google hosted code site. It has excellent documentation. There is however a bug in their code. In revision 36, I had to change lines 422 to 424:

//Handle errors returned by the server
if(response.error !== undefined){
    var err = new Error(response.error.message);

to

//Handle errors returned by the server
if(response.error && response.error !== undefined){                             
    var err = new Error(response.error.message);

I also had to remove the periods in the calls to system.ListMethods on lines 151 and 154 so that a systemListMethods function could be supported by Python

Sample Code

You can download my sample code here. Feedback is most welcome.