FOSDEM 2013 Lineup

Posted on by David

If you’re going to be at FOSDEM, don’t forget to say hi! Here is a list of talks that struck my fancy at first glance.

Saturday

Sunday

Quick Note on Pelican & Github Pages

Posted on by David

Originally, I built the San Diego Python users group website as static HTML hosted on Github pages. However, as time progressed, the group wanted to have posts for our events with links to presentations and whatnot. I looked at Jekyll but what would a Python users group be generating its website with Ruby. Normally, I’m all about using the best tool for the job, but all of the group’s leads/members know Python. I settled on Pelican which seems to be the most fully featured of the Python static website generators out there.

One repository

Under ideal conditions I would have one repository that hosted all our markdown/reST files and also contained the static HTML output that Github serves. Project pages are a great way to do that and Pelican already has some integration. Your markdown/reST goes in the master branch and your HTML output goes in the gh-pages branch. ghp-import facilitates this quite nicely. There is already a make target for it in Pelican!

Custom domain

I already had pythonsd.org and normally on Github pages, pointing to a custom domain is as simple as adding a CNAME file. One tricky part that I didn’t realize is the CNAME file needs to be in the gh-pages branch. I added an extra line to the Makefile to copy it to the output which automatically gets put into the gh-pages branch by ghp-import. If you don’t do this, your CNAME file will be overwritten by the next “make github”.

I considered having two repositories because I had some doubts about how clean a one repo solution would be. In the end, I just needed to figure out exactly what ghp-import was doing. It worked fine and accomplishes exactly what I want.

Versioning Software Packages

Posted on by David

There are lots of different versioning schemes and versioning is definitely not a solved problem. Some projects use dates, some use an ever increasing number from (perhaps generated by their version control system), some adopt the de facto standard of major.minor.patch and some converge to pi. Fundamentally though, all of these versioning schemes convey some additional information. With any of these schemes, it is easy to compare two versions and say which was released later. Some have even more semantic meaning. Without sounding too pedantic, there is some formal discussion of this as configuration management.

Versioning conveys information

Semantic versioning is a fairly documented versioning scheme that conveys information about API stability and compatibility especially in relation to dependency management. Without repeating too many of semantic versioning’s details, essentially it allows for easy identification of backwards compatible vs. incompatible changes and changes that do not modify the API. It is definitely a step forward, but as evidenced by the fact that a 2.0.0 version of it is still a release candidate, versioning in the abstract cannot be considered complete.

A number of systems developers use every day rely on the semantics of versioning. Take Python as a perfect example. Code from Python 2.6 will almost always run without modification on 2.7. The reverse is sometimes true but certainly less likely to be true. When I use Travis-CI, I do not specify that my program must be tested under Python 2.7.0, 2.7.1, etc. Travis assumes that if it works in Python 2.7, it will work in any of those versions. These patch versions are rarely necessary for dependency management and certainly not with semantic versioning. Semantic versioning really shines when testing a package with multiple dependencies or recursive dependencies. Instead of having to test X versions of Package1 against all Y versions of Package2, basics of API compatibility can be assumed and testing becomes easier. With that said, verifying a subset of that is always nice to do too.

Additional versioning metadata

API compatibility and version comparison lend themselves well to being discovered from the version numbering scheme, but what about other types of things developers might want to know about a particular package. One piece of metadata that would be interesting to know is a classification about what changes a new version contains. Did any security vulnerabilities get fixed? Was this a bugfix release? Was this a re-release because something went wrong in the release process in the previous release? You can imagine a versioning scheme that looks something like this: 1.0.5-sb where s stands for security and b stands for bug. It would be nice for that metadata to be machine discoverable, and it would be much easier to identify dependencies that require updating.

Perhaps a better solution is to attach this metadata to version control tags. Newer version control systems have annotated tags (I’m using the git term, but Mercurial has something similar). I can imagine a tag 1.0.5 with the annotation [security fix]. Like raising awareness for semantic versioning, it requires changing the software world by getting everyone to adopt your methods which is a tall order. In addition, some things do not lend themselves very well to version control tagging such as deprecating releases. It would be nice to discover when a version is deprecated and therefore no longer taking security fixes, but that happens long after the time a release is tagged.

Conclusion

I’m not ready to make any sort of concrete proposal. However, I think this is a really interesting space and I think that information such as which versions have security vulnerabilities becomes much more valuable now that many software products capture their dependencies in requirement files or gemfiles. There are definitely projects that have taken this on parts of this such as the open source vulnerability database. Over the next few years, this is going to become somewhat solved rather than upgrading dependencies in the ad hoc fashion that that developers do now. There will be something better than subscribing to a bunch of mailing lists and feeds to keep up with security fixes for dependencies.

Additional notes
  • distutils.version.LooseVersion allows for version comparison pretty close to semantic versioning
  • There are marketing reasons to version things too.
  • I got ideas for this post when I was reading the 2.0.0-rc1 version of the semantic versioning docs and I noticed that the tagging specification section was removed since I read it last time.

Intro to Django Workshop November 2012

Posted on by David

Next weekend the San Diego Python Users Group (we) will be hosting an Introduction to Django workshop. This post is going to try to document the planning process for having a workshop like this one in your area.

Sponsorship

One thing that cannot be stressed enough is getting a sponsor. Brightscope is sponsoring this Intro to Django workshop. Our Intro to Python workshop was sponsored by the PSF and that worked extremely well. This doesn’t seem important on the surface but without planned meals it is tough to get people to stay in a room together for seven hours. Otherwise people would be constantly leaving to get drinks or coffee and the workshop wouldn’t flow as well. Free food is also a pretty good motivator and it gets people to show up.

Planning

We sent out a little survey last week to find out about people’s skill levels and what they already know about Python, HTML, and Javascript — pretty much the prerequisites of Django. Just about everybody knows HTML and some Javascript but that’s where things diverge. We have a pretty good mix of operating systems and the Python experience runs the full gamut. However, this step is key to planning out the presentation and schedule. We are having a brief newbie session on the Friday before the workshop for people to install Python and get their development environments setup. You can’t start from Django tutorial #1 if you don’t have Python installed! The goal is to have everyone on the same page so that we can have a simple working web app by noonish on Saturday. The afternoon will be spent somewhat independently adding a feature or two to that app.

Workshop materials

Preparing for the Intro to Python workshop earlier this month was easy since we liberally used the Boston Python group’s workshop materials. However, for this one we did not have as much to go off of. We are leaning heavily on the Django tutorials, the Django book (to a lesser extent) and Chander Ganesan’s PyCon talk from this year. At Djangocon this year, Russell Keith-Magee mentioned that the DSF was going to put together some training/workshop materials for just this sort of thing, but it is not ready yet. Hopefully what we create is useful and can bootstrap that process. Our materials are very early stage right now, but feel free to make suggestions or fork them for your own purposes.

Ok. Enough procrastinating. Time to work on the actual workshop materials.