Opensuse build service, access through ssh tunnel

For a few weeks now, I am working on providing binary packages for numpy and scipy for major linux distributions using the build service from opensuse. You can find a brief description of the build service on Miguel de Icaza blog here : http://tirania.org/blog/archive/2007/Jan-26.html. As he put himself, "For the last couple of years the folks at SUSE have been building a new build system that could help us make packages for multiple distributions from the same source code."

The system was a bit hard to set up because it is really rpm centric, even if it can also build debian packages. But it basically works: it provides a build farm (x86 and amd64 only) based on xen images, it provides command line tools to access the web api to build/modify packages. The one I am using is osc: it works a bit like subversion (you checkout, commit, etc...), and can use it to build locally, also (using a local minimal install of distributions in chroot jails).

Now, it is still a bit rough on the edges. The osc client does not work well behind proxy (this is really a python issue, actually, as the http handler in python does not handle https proxy well, AFAIK), and I had to use some strange hack suggested by one of the build service developer to connect through a SSH tunnel instead. Here is how to do it by using a tunnel from the port 9999 on the ssh machine to the public opensuse build system server.

1. First, in $HOME/.oscrc, set apisrc to api.opensuse.org:9999
2. add api.opensuse.org as an alias to localhost in /etc/host, eg : 127.0.0.0 localhost api.opensuse.org. This is the tricky part, because otherwise, the SSL certificates do not work: http://lists.opensuse.org/opensuse-buildservice/2007-04/msg00018.html
   
3. run your ssh tunnel as ssh -L 9999:api.opensuse.org:443 sshserver

Note that some commands of osc need to access the web, and thus you still need a http_proxy set if you access through the internet with a proxy. As I understand it, python urllib2 function does not handle https proxy, but http proxy work.

Creating a image disk (dmg) from command lineI

The docs for packaging an application for mac os X say to use the disk util application to make a .dmg file, but I would rather be able to do it from command line. It looks like the following work to make a dmg from a folder:

hdiutil create -fs HFS+ -srcfolder SRCFOLDER -volname VOLNAME IMGNAME

where SRCFOLDER is the folder where you have the files to put in the dmg, VOLNAME the volume name and IMGNAME the name of the .dmg file. As I know nothing yet on ressource forks, I do not know whether it keeps them or not (some scripts I found on the internet had this problem: http://www.macosxhints.com/article.php?story=20020311215452999).

First steps with Mac OS X

I recently bought a macbook, as I needed a laptop (my old toshiba was holding with tape for more than one year, and the hardrive finally failed a few days ago). I bought a macbook because they are cheap in Japan for a student (less than 1000 euros including apple care), and as Apple has a few, mostly non customizable models, it should be easier to get information for running linux on it.

This post will be the first one of a serie describing how to use traditional unix tools, and maybe also mac os X specific tools to produce usable softwares related to numpy/scipy/pymachine on mac os X platforms: compilation and packaging issues, etc...

First, the necessary bits for building numpy + scipy

1. Install the developers tools (xcode and co):
   
2. Install subversion: UB are here

Informations for compiling UB: here and here. Packaging information can be found here.

Creating a dmg from a folder:

the command line seems to be hdiutil (you can do it with the disk util, but I would rather be able to do it from command line). It looks like the following work to make a dmg from a folder:

hdiutil create -fs HFS+ -srcfolder SRCFOLDER -volname VOLNAME IMGNAME where SRCFOLDER is the folder where you have the files to put in the dmg, VOLNAME the volume name and IMGNAME the name of the .dmg file.