Twoville: Programming with Time and Direct Manipulation

Following are the notes from a talk I gave in February 2024 at the PLATEAU workshop on HCI and programming languages at the University of California, Berkeley.

Here's the setting: we live in a community full of kids who wouldn't make it in the computer science degree program like the one I teach in just a few blocks away from their homes. The public schools don't offer much in the way of computer science courses, so we take computing to them through after-school programs and summer camps. Many of these kids don't have a lot of resources, and they might have spent the first years of their lives in a refugee camp. We don't use the usual tools like Scratch and robotics because we want output that has a life apart from a computer. Instead we use a programming language for making things parametrically. It's called Twoville, and it's a work in progress.

Twoville

In Twoville we write code to produce 2D designs, which sounds like a lot of educational programming environments. Twoville is different from many of them in that the shapes can be modified using either the mouse or code. This feature is best understood in a live demonstration. We usually have students start with a circle, like this:

Many graphics libraries would have you draw a circle with code like this:

circle([50, 50], 45, :cornflower)

Positional parameters are like unlabeled buttons on an elevator. Their meaning is obscure if you're new to the programming scene. An alternative is keyword parameters:

circle(center = [50, 50], radius = 45, color = :cornflower)

But this makes for wide code. It doesn't fit on tablets and budget laptops issued by schools, leading to horizontal overflow or awkward wrapping. Instead, the parameters are set on the circle after it is made. In Twoville we're aiming for an ergonomic but explicit syntax, so we take a with-block approach.

Bidirectional Editing

Another very important feature of Twoville is bidirectional editing. When the user places the cursor in a shape's code, handles appear. Each handle corresponds to one of the properties that was set in code. If I drag on a handle, it changes both the design and the code. Let's see this at work by assembling a cloud “frankenshape”:

When we first implemented bidirectional editing, we replaced the existing property \(p\) with a value \(p'\) determined by the mouse position. But then we ran into occasions where the property was built out of a complex expression. It felt wrong to simply overwrite what the programmer had in place. So, we tried to figure out ways to preserve the structure of the original expression. We'll show what we have currently on this circle:

The expression preservation algorithm walks this gauntlet to decide how to update the source code:

If the expression is an invertible binary operator, we favor updating a numeric literal operand, if it has one. Failing that, we update its rightmost operand. If this operand is precious, the programmer may lock it with parentheses.

To implement this selective updating, we must maintain a representation of the expression structure and the values of its subexpressions to help figure out what the new value should be. A lot of work needs to be done to make this system consistent and stable. It doesn't handle properties generated by loops very well currently and perhaps it never will given my grading and email backlog.

Besides rectangles and circles, we can make several other kinds of shapes. We can for instance make a mountain range using the polygon command:

Sharp points don't hold up very long in the physical world. On mountains, the sharp points erode. On vinyl stickers, the sharp points catch and start to peel. We round off our polygon's sharp corners by trading out vertex for chamfer or fillet.

Fabrication

The output of a Twoville program is a scalable vector graphics (SVG) file. We feed the SVG file into a fabrication tool like a vinyl cutter, laser cutter, pen plotter, or embroidery machine to turn our design into a physical object that has a life independent of the computer used to make it. I'll share a few examples of the objects that have been made with Twoville.

Time

Physical fabrication has been our primary goal, but the longer you “reside” in a language, the more you want to use it for other things, right? We wanted to program some animations, so we added time as a concept to the language. Property assignments can be made at particular keyframes, as with this circle:

The syntax is meant to be a text-based alternative to the graphical timeline widget that one might find in an animation tool. As the program executes, an animation curve is assembled keyframe by keyframe out of these time-blocks.

Direct manipulation works with time too. Suppose we want to fold three line segments up into an equilateral triangle. Let's use some turtle geometry to walk the three segments:

At frame 0, the bend is 0 degrees. At frame 100, we might guess the bend is 60 degrees. We find that's wrong and use the mouse to pull it closed with 120-degree bends:

Conclusion

That's Twoville, our language for making T-shirts and stickers and animations. It has a lot of bugs that I didn't show you, but we're on our second prototype, and we think it has fewer than the first. While we have very much enjoyed developing it, its status a piece of software is secondary to its role as our vehicle to support the kids in our community as they learn math and computer science.

Questions

In case you have no questions of your own, I've got a few prepared for you.

What ages of students have been using Twoville? We've done many hour-long workshops with middle schoolers. An hour isn't enough time to go much farther than frankenshapes. We've also done a couple of week-long summer camps for high schoolers. And my own elementary and middle school children have had a number of opportunities to use it, but they get a level of support that others don't.

Why invent a new language? To support bidirectional editing, I need to have intimate knowledge of the program. In particular, I need to know which expressions produce the spatial properties so that when the mouse changes those properties, I can update the expression. I need to write my own interpreter to achieve this intimacy. Supporting all of Python sounded like too much work. Smaller, domain-specific languages have fewer commands and fewer alternatives, which makes them easier to learn and easier to implement.

Why not use blocks? Someday I might support both modes, allowing one form to be translated into the other. I have written blocks languages in the past, and I recognize that they fix several problems. They make the commands of the language discoverable, and they reduce the typing burden that many young people are not ready to take on. But blocks also have ergonomic issues. They consume a lot of space, they are hard to share outside the IDE, and they are slower to assemble for experienced users. Having a small language makes a blocks interface feel less important, so it's a low priority right now.

Is there a 3D version? I started with a 3D language named Madeup and spent a year taking it to an elementary school classroom. But 3D printing is far too slow. Students fought over the scraps of plastic and the misprints. The vinyl cutter finishes most of the jobs I send it in under a minute. Everyone can go home with something, even in hour-long workshops.