tag:blogger.com,1999:blog-46890156272518539862024-02-08T12:10:34.967-08:00pymachine's blogdavidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-4689015627251853986.post-28714006230493168432007-08-15T22:54:00.000-07:002007-08-15T22:56:56.960-07:00Convex optimization: a primerFor quite some time now, I've been looking to getting some <span class="blsp-spelling-corrected" id="SPELLING_ERROR_0">introductory text on convex optimization. I found a course with a book available online there: http://www.stanford.edu/class/ee364/. Will take a look at it once I finished my poster for ICSLP07.</span>davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com1tag:blogger.com,1999:blog-4689015627251853986.post-74441596600018660712007-06-12T19:38:00.000-07:002007-06-12T20:04:21.980-07:00Opensuse build service, access through ssh tunnelFor a few weeks now, I am working on providing binary packages for numpy and scipy for major linux distributions using the build service from opensuse. You can find a brief description of the build service on Miguel de Icaza blog here : <a href="http://tirania.org/blog/archive/2007/Jan-26.html">http://tirania.org/blog/archive/2007/Jan-26.html</a>. As he put himself, "For the last couple of years the folks at SUSE have been building a new build system that could help us make packages for multiple distributions from the same source code."<br /><br />The system was a bit hard to set up because it is really rpm centric, even if it can also build debian packages. But it basically works: it provides a build farm (x86 and amd64 only) based on xen images, it provides command line tools to access the web api to build/modify packages. The one I am using is osc: it works a bit like subversion (you checkout, commit, etc...), and can use it to build locally, also (using a local minimal install of distributions in chroot jails).<br /><br />Now, it is still a bit rough on the edges. The osc client does not work well behind proxy (this is really a python issue, actually, as the http handler in python does not handle https proxy well, AFAIK), and I had to use some strange hack suggested by one of the build service developer to connect through a SSH tunnel instead. Here is how to do it by using a tunnel from the port 9999 on the ssh machine to the public opensuse build system server.<br /><ol><li>First, in $HOME/.oscrc, set apisrc to api.opensuse.org:9999</li><li>add api.opensuse.org as an alias to localhost in /etc/host, eg : 127.0.0.0 localhost api.opensuse.org. This is the tricky part, because otherwise, the SSL certificates do not work: <a href="http://lists.opensuse.org/opensuse-buildservice/2007-04/msg00018.html">http://lists.opensuse.org/opensuse-buildservice/2007-04/msg00018.html</a><br /></li><li>run your ssh tunnel as ssh -L 9999:api.opensuse.org:443 sshserver</li></ol>Note that some commands of osc need to access the web, and thus you still need a http_proxy set if you access through the internet with a proxy. As I understand it, python urllib2 function does not handle https proxy, but http proxy work.davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com0tag:blogger.com,1999:blog-4689015627251853986.post-81999357764586905862007-06-03T21:03:00.000-07:002007-06-12T20:03:55.138-07:00Install a R source package in local directory<span style="font-size:100%;">If you don't have root access to your machine, it seems that it is not possible to install a package for R using install. Let's say you want to install some packages in $HOME/local/lib/R/library. Two things are necessary:<br /></span><ol><li><span style="font-size:100%;">Install the package with R at the correct location</span></li><li><span style="font-size:100%;">Tell R to look for the package at this location</span></li></ol><span style="font-size:100%;">The first is a simple matter of doing the following at the root of the package to install sources:<br /><br /></span><span style="font-size:100%;"><span style="font-size:85%;">R CMD INSTALL -l $HOME/local/lib/R/library .</span><br /><br />Then, you just have to launch R with the path set in R_LIBS:<br /><br /><span style="font-size:85%;">R_LIBS=$HOME/local/lib/R/library/ R</span><br /><br />You can then use library("name") to import the package from its name<br /><br /><br /></span>davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com1tag:blogger.com,1999:blog-4689015627251853986.post-48182861843472377852007-05-31T00:12:00.000-07:002007-05-31T00:20:06.939-07:00Creating a image disk (dmg) from command lineI<span>The docs for packaging an application for mac os X say to use the disk util application to make a .dmg file,</span> but I would rather be able to do it from command line. It looks like the following work to make a dmg from a folder:<br /><br /><span style="font-size:85%;">hdiutil create -fs HFS+ -srcfolder SRCFOLDER -volname VOLNAME IMGNAME</span><br /><br />where SRCFOLDER is the folder where you have the files to put in the dmg, VOLNAME the volume name and IMGNAME the name of the .dmg file. As I know nothing yet on ressource forks, I do not know whether it keeps them or not (some scripts I found on the internet had this problem: <a href="http://www.macosxhints.com/article.php?story=20020311215452999">http://www.macosxhints.com/article.php?story=20020311215452999</a>).<a href="http://www.macosxhints.com/article.php?story=20020311215452999"><span style="font-style: italic;"></span></a>davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com0tag:blogger.com,1999:blog-4689015627251853986.post-11438926073605490892007-05-30T23:07:00.000-07:002007-05-31T00:11:27.476-07:00First steps with Mac OS XI recently bought a macbook, as I needed a laptop (my old toshiba was holding with tape for more than one year, and the hardrive finally failed a few days ago). I bought a macbook because they are cheap in Japan for a student (less than 1000 euros including apple care), and as Apple has a few, mostly non customizable models, it should be easier to get information for running linux on it.<br /><br />This post will be the first one of a serie describing how to use traditional unix tools, and maybe also mac os X specific tools to produce usable softwares related to numpy/scipy/pymachine on mac os X platforms: compilation and packaging issues, etc...<br /><br />First, the necessary bits for building numpy + scipy<br /><ol><li>Install the developers tools (xcode and co):<br /></li><li>Install subversion: UB are <a href="http://downloads.open.collab.net/binaries.html">here</a> </li></ol>Informations for compiling UB: <a href="http://developer.apple.com/technotes/tn2005/tn2137.html">here</a> and <a href="http://developer.apple.com/opensource/buildingopensourceuniversal.html">here</a>. Packaging information can be found <a href="http://developer.apple.com/documentation/DeveloperTools/Conceptual/SoftwareDistribution/index.html">here</a>.<br /><br /><span style="font-weight: bold;">Creating a dmg from a folder:<br /><br /></span>the command line seems to be hdiutil (you can do it with the disk util, but I would rather be able to do it from command line). It looks like the following work to make a dmg from a folder:<br /><br />hdiutil create -fs HFS+ -srcfolder SRCFOLDER -volname VOLNAME IMGNAME where SRCFOLDER is the folder where you have the files to put in the dmg, VOLNAME the volume name and IMGNAME the name of the .dmg file.<br /><span style="font-weight: bold;"></span>davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com0tag:blogger.com,1999:blog-4689015627251853986.post-13920573168389583632007-05-24T21:40:00.000-07:002007-05-24T21:55:56.643-07:00Committing a project under bzr to a svn subtreeUsing subversion is kind of painful for my personal projects, mainly because :<br /><ol><li>I do not have a root account on my workstation, and I cannot run a subversion sever on it</li><li>I have only a pure http available for my university webpage, and as such, I cannot serve any subversion non locally.</li></ol>I am using for some time bzr, which is a decentralized source version control. I started using it a few months ago because I found bzr really easy to use, and as it does not require anything else than http, I can serve my projects from the university.<br /><br />Now, when I want to submit some of those projects to subversion repositories, it is a problem. There is bzr-svn, which is a plugin able to understand svn repositories, and put its metadata under bzr control. Unfortunately, I could not use it to submit one of my projects as a module of an existing subversion server (maybe because I do not know much about source control systems). This is where tailor comes in: " Tailor is a tool to migrate changesets between <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/ArxRepository">ArX</a>, <a class="missing wiki" href="http://progetti.arstecnica.it/tailor/wiki/BrzRepository" rel="nofollow">Bazaar?</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/BzrngRepository">Bazaar-NG</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/CvsRepository">CVS</a>, <a class="missing wiki" href="http://progetti.arstecnica.it/tailor/wiki/CodevilleRepository" rel="nofollow">Codeville?</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/DarcsRepository">Darcs</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/GitRepository">Git</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/MercurialRepository">Mercurial</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/MonotoneRepository">Monotone</a>, <a class="wiki" href="http://progetti.arstecnica.it/tailor/wiki/P4Repository">Perforce</a>, <a class="missing wiki" href="http://progetti.arstecnica.it/tailor/wiki/SubversionRepository" rel="nofollow">Subversion?</a> and <a class="missing wiki" href="http://progetti.arstecnica.it/tailor/wiki/TlaRepository" rel="nofollow">Tla?</a> repositories." (the main page of the project, as far as I can tell, is <a href="http://progetti.arstecnica.it/tailor">there</a><span style="text-decoration: underline;"></span>).<br /><br />I first used the last release of tailor, but it did not work quite well, and after some time, I finally understood it was because of desync between the tailor version and my version of bzr. Anyway, after fetching the last published sources, I could tranlate my bzr project to subversion. Here are the steps:<br /><ol><li>I assume there is a subversion repo http://svnrep, to which I want to commit my project.</li><li>I want to put it in http://svnrep/mainproject/trunk/foo.</li><li>My bzr project is in $HOME/foo.bzr</li></ol>I first generate the tailor config file for this convertion as the following:<br /><br />tailor --verbose --source-kind bzr --target-kind svn \<br /> --repository $HOME/foo.bzr \<br /> --target-repository svn://svnrep/ \<br /> --subdir tmp \<br /> --target-module trunk/foo bzr2svnproj > svn2bzrproj.tailor<br /><br />subdir tmp means that tmp will be the working directory. Then, the actual convertion is done using the commands:<br /><br />tailor -D -v -c floupi.tailor<br /><br />It works for 2 of my projects which are not big (a few tens of source files and revisions), so I don't know if this can be used for bigger projects easily.davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com0tag:blogger.com,1999:blog-4689015627251853986.post-1318061151272269632007-05-06T20:09:00.000-07:002007-05-06T20:24:21.880-07:00Get things done, task management and open source softwareI've been looking this week end for some kind of task manager software, and remembered seeing some posts on the <a href="http://trichech.us/">pymc's blog</a>. Problem is, all those software are either mac specific or web-based only. What I am looking for is<br /><ol><li>a software which is able to handle projets as a a set of tasks</li><li>there should be time management (eg set the time a task should take, and a timeline to get a view of all the projects at the same time)</li><li>should work on linux, and ideally should be open source.</li></ol>I found two softwares which seem to fit this description:<br /><ol><li><a href="http://www.rousette.org.uk/projects/">tracks</a>: web-based (use ruby on rails)<br /></li><li><a href="http://www.thinkingrock.com.au/">thinking rock</a>: java-based.</li></ol>I took the time to set-up tracks, and found some review <a href="http://www.linux.com/article.pl?sid=07/04/19/2019213">there</a>. Thinking rock also got its review recently <a href="http://www.linux.com/article.pl?sid=07/04/13/1441252">here</a>. I will take time to use them a bit the next few days, to see which one fits me the best.davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com0tag:blogger.com,1999:blog-4689015627251853986.post-87690301467029305692007-04-26T20:38:00.000-07:002007-05-27T17:18:41.296-07:00Introduction to pymachineOk, time to write something on this blog. This blog will mostly follow my progress on pymachine, a machine learning toolbox for the <a href="http://www.scipy.org/">numpy/scipy</a> environment. This project is supported by the Google Sumer of Code 2007, whose basic description can be seen <a href="http://code.google.com/soc/psf/about.html">here</a>, and the full proposal can be seen<span style="text-decoration: underline;"></span> <a href="http://projects.scipy.org/scipy/scipy/wiki/MachineLearning">there</a>. All development will happen within the scipy community (mailing list, source code repository and project management).<br /><br />The project can be split into two parts:<br /><ol><li>first, cleaning existing machine learning related algorithms in scipy (eg scipy.clusters, several toolboxes in scipy.sandbox like pyem and svm). By cleaning, I mean adding tests, adding proper docstrings, and of course correcting bugs of the existing code.<br /></li><li>wrapping those cleaned toolboxes into a higher level package, in the spirit of data mining softwares ala <a href="http://www.ailab.si/orange">orange</a>, <a href="http://www.cs.waikato.ac.nz/ml/weka/">weka</a> and co. This will include some graphical tools for data representation and manipulation, as well as basic storage facilities.<br /></li></ol>davidhttp://www.blogger.com/profile/05147314872569597713noreply@blogger.com1