" /> Ezra's Research: January 2007 Archives

« December 2006 | Main | February 2007 »

January 23, 2007

Programming Graphics and Getting Eyeballs

The language called "Processing" is a Java-like language, designed for coding up lovely interactive graphics. It compiles to Java and it's easy to deploy your apps on the web for people to enjoy them. Many of the demos on the site are mouth-wateringly impressive—check them out before you read on.

What the Processing people have done really well is to think about all the primitives that you'll want when coding up graphics demos—so they have good ways of playing with color, controlling the opacity of a shape, getting nice bezier curves, rotating and skewing, plus fairly good type-drawing and 3D operations. They've also thought through the programmer startup cost, making it easy to sit down and start using the thing without much knowledge.

The area where Processing is weaker is in the ergonomics of development. It uses an imperative style where you have to build up values and images in sequential steps. It is clumsy, for example, to create a list of dots and then render each one to the screen; clumsier still to express how they change or move over time, since you need to write out the frame-by-frame changes, rather than writing it as a function of time or a system of equations. Many facets of the API are controlled using peculiar beginFoo() and endFoo() calls, which affect how the statements in between behave. Heaven help you if you call beginCamera() and leave off the endCamera() call. These drawbacks make errors likely and have a cost in debugging time and code-reading time.

There is an alternative way to make such demos, in the name of "functional reactive animation" (FRP). In the FRP style, you write the behavior of each thing and each shape as a function expressing how it changes over time. You have at your disposal the raw parameters of time, and various input values, such as mouse position and key presses, so you can make the shapes react to these things.

I'll describe briefly what FRP programs look like (if you already know, skip the next few paragraphs). In the imperative style, you forcibly update persistent variables and so to understand the behavior you need to look at all the ways those variables can be updated over time, which can be quite complex. By contrast, in the FRP style, the behavior of an object is completely defined by an expression, so you can understand that behavior just by studying that expression—a more compact way of appreciating the behavior.

For example, to define a dot that moves at a constant rate in a circle around a point (centerX, centerY), I might write it something like this:

dotPosn = (centerX, centerY) + (radius * (sin time), radius * (cos time))

(I'm positing an operation of summing two vectors—a simple thing to define in many languages, though a bit cumbersome in Java.) The above defines the moving center of the dot I want to draw; drawing it at each animation frame would require just an expression as simple as this:

myDot = circle(dotPosn, 1, redColor)

That is, draw a circle centered at the value determined by dotPosn, with radius 1, in the color red (defined elsewhere). Objects can also be defined in a mutually-recursive way, so that their behaviors depend on one another.

So FRP is an appealing paradigm for writing interactive graphical demos. But none of the existing implementations are as appealing as Processing. The original implementation, Fran, is a closed-source Windows-only application (How's that for a conversation killer?) and is no longer being actively developed. The newer FrTime (part of DrScheme) is a much more serious contender, as it is open-source and runs on all the major platforms. Yampa is another, which I haven't had the chance to dip into.

These are good efforts; but there are still some pieces missing. Processing apps can be deployed on the web very easily—which makes the pay-off for writing them so much higher. Writing a graphics demo and showing it to your friends at your next pizza party might be fun. Writing a graphics demo and slapping it on the web is a whole lot more fun. See Linerider and the great popularity it has had over the last few months.

Another problem is that most FRP implementations are very conservative: they tend to offer the minimum set of tools that makes it theoretically possible to implement anything in the space. Processing takes a different approach: it tries to offer the minimal toolbox that you're likely to reach for in the course of making a demo. So FrTime gives you accum-e, which can probably be used to express any discrete time-changing behavior imaginable; but Processing gives you shininess (for setting a solid object's surface properties) and strokeCap (for adjusting the shape of the ends of lines).

A great Summer of Code project, or Master's project, would be to compile FRP code from one of the above languages down to the JVM, so you can deploy it over the web. Another one: take FrTime or Yampa and beef it up with all these nice graphics operations, to add the kind of vocabulary that graphic designers and animators like to use. The latter project would be straightforward from a CS point of view, but it would require attention to detail and a good sense of visual aesthetics. The former is more of a CS research project, and might require some clever thinking because of the apparent mismatch between functional-style code and the JVM's computational model. But there are plenty of other languages that compile to the JVM, so it's not beyond the pale.

If I were just starting now on my PhD, I'd find a way to include this in my work, or I'd put off the PhD until I could get this done! Tinkering with graphics demos and slapping them up on the web is the kind of hacking I'd really like to be doing. As it is, I'll have to leave it to some young whippersnapper, alas.

January 12, 2007

A Productive Week

This was a productive week. Since Wednesday, I've spent a significant amount of time revising some shoddy proofs I'd written back last summer, and managed to double-check them, fix some flaws, and even streamline them a bit. They feel like they might almost be correct!

For those who don't work with me directly, the proofs are concerning a calculus that clarifies the semantics of the client and server annotations that Links supports, which allow you to move your code from one to the other effortlessly, and ensure that it only runs in the appropriate place (if you care). I'm not sure whether this system has any broader impact than for explicating Links, but it's been a very educational experience for me trying to build it. Keep an eye out for it to be published sometime in the 2007–2010 timeframe.

January 10, 2007

Think in Closures?

I did a search for "plt scheme" and I got this terrific ad from Jane St Capital:

January 9, 2007

More web gimmicks

I don't intend to keep doing daily status updates, but this week so far I've been somewhat productive. Today, I:

  • Implemented sendSuspend in Links,
  • Checked in a Links routine freshResource, which redirects the user to a new URL for the continuation of the execution; this can achieve the old trick of redirecting the user to a safe URL after doing a destructive action (a step forward in web-savvy abstractions, but a small one),
  • Demonstrated a tidy web-login library that uses sendSuspend and freshResource,
  • Met with Phil & resolved to do some theory,
  • Checked a bunch of proofs in a sort of paper I'm working on.

January 8, 2007

Work Today

Spent some time hacking on Links:

  • Attempted to convert our manual (in POD format) into LaTeX. In the end I felt that it's easier to work with POD format as long as we don't need any fancy stuff like formulae.
  • Added a naive dead-code-elimination optimization.
  • Spent some time experimenting with converting the CPS translation (for the JS target) into a one-pass conversion. Although this was a good exercise and revealed some bugs and opportunities for optimization, I think a true one-pass translation is still beyond me.