rpc4django

Updates March 2010 Edition

Posted on by David

This post is mainly going to be an update on what I am thinking and what I’ve been working on the past few weeks.

Work

At the beginning of this year, I took a new position (same company) in a security group. Our primary focus is to ensure that the company is shipping secure, OSS compliant, legally compliant code. However, my specific role in that is to develop tools (with Django) to help in making sure that happens. This is an exceptionally interesting project and involves pulling in vast amounts of data (terabytes) from many sources (multiple VCS, multiple databases) and presenting it in a comprehensive manner. This project and my work has led to some good problems:

Some of our databases are MSSql databases. This is a problem since we’re a Linux shop. Pyodbc works great for connecting to MSSql from Linux, but unfortunately, there are some incompatibilities with django-pyodbc. In addition, the project doesn’t seem to be that widely used so it isn’t supported or documented as well as it could be. We are considering sqlalchemy/elixir as well, but I’ve been able to patch up django-pyodbc to get it (mostly) working with the Django trunk. I also have some concerns about the django-pyodbc project as a whole. I’m considering working on this project pretty heavily.

Also, as part of my work, a coworker and I detailed a security flaw we found with urllib2. It resulted in basic authentication credentials being sent to sites that did not request it (and weren’t running SSL).

Future of RPC4Django

I have been considering moving RPC4Django from my personal subversion repository to Google Code or Github. I feel that there are a few advantages of this:

  • It is easier for others to contribute and get involved.
  • A public bug tracker that would let other people easily raise issues instead of emailing me directly. This way we have public archives and the information can be found by anyone interested in RPC4Django.
  • If I were hit by a bus, some one could easily take it over

I might make a mailing list as well. Are there any strong opinions on this?

RPC4Django v0.1.6 is Available

Posted on by David

Go get it!

Thanks to everyone who tested out this release before it went live.

Changes
  • Changed the XMLRPC dispatcher to allow sending nil which translates to the Python None. This was already allowed with JSONRPC
  • Cross site request forgery framework support
  • Added cross domain access control
  • Added unit tests for base64 encoded binary
  • Added access to HttpRequest object form inside a called RPC method and associated unit tests

RPC4Django v0.1.6 is Almost Done!

Posted on by David

I have checked the latest changes into subversion. Here is a list of the features supported:

  • Changed the XMLRPC dispatcher to allow sending nil which translates to the Python None. This was already allowed with JSONRPC
  • Cross site request forgery framework support
  • Added cross domain access control
  • Added unit tests for base64 encoded binary
  • Added access to HttpRequest object form inside a called RPC method and associated unit tests

Some of these features were fairly major and I am lucky that I have pretty good unit tests in place. However, since most of these features were user requested, I really hope that I can get some of you guys to run this version in your environment and let me know how it works.

In addition, I’m sorry that I am so late in making this release. As of a week ago, I began a new job (although with the same company). The transition was somewhat taxing, but I should have a little more time freed up for open source work now that everything has settled down.

RPC4Django v0.1.6 Feature Requests

Posted on by David

A number of feature requests have come in and I am going to outline what I’m going to try to get done. I hope to put out a release next week (I’ve said that before), but that greatly depends on what features go into it.

Access to HttpRequest

One user submitted feature that seems particularly useful is the ability to access variables and data in the HttpRequest that dispatches the RPC method. For example:

This RPC method will be authenticated, but it cannot tell who the user is for example. It does not have the ability to read the REMOTE_USER variable. This would be particularly useful if the RPC method was submitting data and needed to know who the submitter was. To rectify this situation, I am proposing sending the HttpRequest object as a keyword argument:

Methods that need access can accept all the additional keyword arguments and methods that don’t care can be written as normal and do not need to change.

Work with the cross site request forgery framework

Django has added a new module to protect against cross site request forgery (CSRF). Unfortunately, if a user adds the django.middleware.csrf.CsrfViewMiddleware to their project, it will break RPC4Django. CSRF protection works by ensuring that all GET requests are side-effect free and then adding a little bit of random data (the CSRF token) to every form that submits via POST. The server can then verify that the correct random data was POSTed and this ensures that the form was submitted from your site and not some remote site.

Unfortunately, RPC is usually intended to be used in a cross site fashion. The fix is relatively painless, but requires some adversarial testing to make it work under Django 1.0, Django 1.1 and Django 1.2. The code (views.py) will look something like this:

I need to test it out very thoroughly and on a number of setups, but I think this code should do it.

HTTP access control

Usually the Javascript XmlHttpRequest object is subject to the same origin policy. This is a security measure that ensures that Javascript will only access data from the same domain, protocol and port as the initial request. There have always been hacks to get around this and the browser vendors are finally providing a uniform, secure way around.

Basically, RPC4Django will need to respond to an HTTP OPTIONS request with the header Access-Control-Allow-Origin specifying what domains are allowed to send requests. Both allowing HTTP access control and which domains are allowed will be options in settings.py.

Patches and feature requests

As always, I am happy to accept patches, but this is what I’ll be working on next week. Please let me know if there are any other important features that you would like to see in RPC4Django.