Wednesday, October 25, 2006

The Fun of Functional Infrastructure

Say the word "infrastructure" and you conjure up the images of paved roads, hot water, and that series of tubes that makes up the Internet.
In other words, infrastructure is that necessary stuff that lets you get your job done, but it doesn't seem sexy. Every open source project needs a high-quality infrastructure, though, and it turns out that building infrastructure is extremely fun, even though it's not as sexy as building "the next killer app".
I've spent a lot of time building software infrastructure. The Debian project is (mostly) not about building new applications. It's about brining together projects with a solid infrastructure so that real people can use it. The Haskell Cabal is a piece of infrastructure that helps other people to compile and distribute applications.
So what makes building software infrastructure seem less fun, even though it's really great? It's the idea of measuring the results of your work indirectly. It's easy to say that we wrote n lines of code this year, or that "my application" has n users. Depending on the size of n, that might make us feel good, or feel bad. But measuring results indirectly is much harder. You have to be content to say things like, "I helped 10 programmers build 15 applications for 5000 users, but I only helped each one a little bit." That's an indirect result.
But if you are more interested in the result than in who creates it, then you're in a good position to let yourself have fun building infrastructure. If you are more interested in fun than in glory, then you're in a good position to get fulfillment in life :)
In fact, infrastructure is really fun to build, at least in computing. You get to take a series of tubes and put them together in a new way. You get to lay the groundwork for many projects that come behind you. There's a bit of systems administration, a bit of hacking, and a lot of contact with the your peers in the community. People will help you figure out what to build since they're chasing right behind you using your stuff (at least in the open source world). People will respect and even occasionally thank you for helping them to do their job.
Of course, some pieces of infrastructure become sexy. Debian's apt-get is considered pretty darn cool among those in the know. Writing awesome compilers that real people use (like GHC or Cryptol) is the dream of a lot of computing science academics. The Internet itself is actually cool these days, or so I hear.
The Haskell language and community is badly in need of infrastructure right now. We need people to flesh out and improve the existing debuggers, code coverage tools, and IDE support. We need a robust package database, sysadmin support, and project repositories. We need folks to make Debian and Gentoo packages from Haskell packages.
If you like people, and want people to get to know you and to appreciate your work, then go out there and discover an important piece of infrastructure that people need. Ask them what exactly they want, how they would build it, but be sure to bring your own vision to the project as well. Most of all, have fun building something with your friends.  

Tuesday, October 17, 2006

Haskell Packages 6.6?

I'm excited because we're pretty well along the way to getting cabal-install and friends working nicely. We've got almost 30 packages in the database.
Let's imagine something that would be awesome for the Haskell community. A set of Haskell packages which are all known to work together with a particular version of cabal (the one that GHC comes with), and a particular version of GHC.
GHC version 6.6 was just released, and I think we should try to make this happen.
Currently, we have a set of 27 "unstable" packages. They may or may not work with each-other.
I just created an empty directory "testing". I propose that we start testing packages, starting with the Cabal release that'll went into GHC 6.6, and make sure they work nicely together. Once they're known to work, we can migrate them from the "unstable" directory to the "testing" directory.
Once we have a sufficient collection of packages, and once ghc 6.6 is released, we can make a snapshot of this directory, call it "stable-6.6" or something. Then if you have ghc 6.6 and cabal-install, you should be able to "cabal-install p" for any package, and it'll definitely work.
So what will we need for this to happen?
  • An installed version of ghc 6.6 on the hackage/darcs server. Maybe in a chroot or something. Maybe from the nightly build tree or the previous snapshot?
  • Some initial set of packages (maybe just cabal-install) to start off.
  • Some script that goes through and builds all of the "unstable" packages in dependency order. I think cabal-install can do this already. In fact, it would be ideal if we used cabal-install for this.
  • The script should also run ./setup haddock and ./setup test. If the packages seem to work w/ 6.6 and the other packages in "testing", it should get migrated from "unstable" to "testing".
  • A web interface (lemmih is working on it)
  • A script to upload packages to "unstable" (Paolo is working on it).
  • Someone to spearhead all of this! We need volunteers. Please email me if you think you can head this up.