Tuesday, June 12, 2007

Opensuse build service, access through ssh tunnel

For a few weeks now, I am working on providing binary packages for numpy and scipy for major linux distributions using the build service from opensuse. You can find a brief description of the build service on Miguel de Icaza blog here : http://tirania.org/blog/archive/2007/Jan-26.html. As he put himself, "For the last couple of years the folks at SUSE have been building a new build system that could help us make packages for multiple distributions from the same source code."

The system was a bit hard to set up because it is really rpm centric, even if it can also build debian packages. But it basically works: it provides a build farm (x86 and amd64 only) based on xen images, it provides command line tools to access the web api to build/modify packages. The one I am using is osc: it works a bit like subversion (you checkout, commit, etc...), and can use it to build locally, also (using a local minimal install of distributions in chroot jails).

Now, it is still a bit rough on the edges. The osc client does not work well behind proxy (this is really a python issue, actually, as the http handler in python does not handle https proxy well, AFAIK), and I had to use some strange hack suggested by one of the build service developer to connect through a SSH tunnel instead. Here is how to do it by using a tunnel from the port 9999 on the ssh machine to the public opensuse build system server.
  1. First, in $HOME/.oscrc, set apisrc to api.opensuse.org:9999
  2. add api.opensuse.org as an alias to localhost in /etc/host, eg : 127.0.0.0 localhost api.opensuse.org. This is the tricky part, because otherwise, the SSL certificates do not work: http://lists.opensuse.org/opensuse-buildservice/2007-04/msg00018.html
  3. run your ssh tunnel as ssh -L 9999:api.opensuse.org:443 sshserver
Note that some commands of osc need to access the web, and thus you still need a http_proxy set if you access through the internet with a proxy. As I understand it, python urllib2 function does not handle https proxy, but http proxy work.

No comments: