Dynamicland & me

The work on this page requires a little context. These projects were all made at Dynamicland. The space uses cameras, projectors, and a custom language to facilitate turning the environment (walls, tables, anything you can stick a piece of paper onto) into a programmable space. Really, though, the research is more focused on the space, and the people: what kinds of interactions can you have when computing is easy, accessible, and ubiquitous? It's about democratized control & access, through the lens of computation. For more detail I'd encourage you to read The "Next Big Thing" is a Room or Omar Rizwan's project-based explanation of the space. Omar's piece goes a little into Realtalk, the superset of Lua that Dynamicland is built with. It's a research lab started by Bret Victor, an intentional community, a gathering space, a learning space. Here are some things I've made there in the context of research, education, art, and exploration.

Table of Contents:

Spatially editing programs with my fingers

The video above demonstrates how the inherently spatial design of the programming system paired with live editing makes for a powerful combination. First, some context:

So, in the video linked above I used my nails as dot input to a program that can live edit the numbers in a Realtalk program (specifically, whichever number on the page of the program is closest to a given dot). With this powerful chain of simple tools in Realtalk I alter the color, radius & stroke width of the circle (the output of the patched program) using only my fingers.

Live patching numbers in Realtalk programs to generate random variations

The prototype I created takes a page it's pointing at & randomizes all the numbers in the program. I'm working toward being able to chain or show various branches of randomly chosen numbers for the program, with a hope this this could be:

  1. an accessible way to explain genetic programming
  2. as well as present an interface for people familiar with genetic programming to embody the process & be able to literally point to what variation bias they want to reproduce between generations.

randomized circle program running


Iambicland screenshot

3D red/cyan stereoscopic projections!

Originally I was inspired to work with anaglyph projection in Dynamicland to get a slight 3D effect on the 2D programs. Then, I got distracted by a much more novel concept: can you use just the red & blue glasses to hide UI in Dynamicland? This is interesting because everything in Dynamicland is, on principle, public & thus shared information. This could be a model for a kind of opt-in privacy! First, I started by swapping the blue/red lenses in two pairs of glasses:

Preparing the glasses. (left) Completed single-channel glasses. (right)

When you put on the single-channel color glasses the opposing color light effectively disappears:

Original, red-only, & blue-only versions of dots with their corresponding colors highlighted.
Original, red-only, & blue-only versions of a pair of necker cubes.

Note that the necker cubes above disappear almost completely while the blue/red dots, because they're opaque objects & not just projected light, are less well hidden. Designing experiences for a game or private messaging application in this way could be an interesting way to keep all Dynamicland programs public by default but facilitate privacy & privileged information when necessary. Future work in this direction would include using gray-light frequency-shift keying (FSK) to track & encode messages using publicly available projectors. This would build on the technique outlined in Moveable Interactive Projected Displays Using Projector Based Tracking (Johnny C. Lee, et al.).

I wrapped up this experiment & left it to the community with this little illustration:

Dials as interfaces in Dynamicland

One of the most powerful consequences of Realtalk having first class concepts of space in the language is that a lot of things we'd normally have to simulate come "for free" in the physical world. A good example of this is rotation: instead of calculating translations to simulate rotation we can stick pages on turntables ("lazy susans") & rotate them for real. Ricardo Cabello used atan2() to calculate the rotation of the paper using a corner of the program's quadrilateral against the center. With this method we get dials that control everything from text spacing, to animations, to music. Some examples I've developed:

3D Tracking In Dynamicland

All the other experiments above save the height-base dial, and indeed most programs in Dynamicland, are spatial but naturally limited to 2D because it's an easier starting point than 3D (nearly trivial projection mapping, easy spatial input tracking with an RGB camera, etc.) Along with Toby Schachman, one of the researchers, I've been exploring using ultra-high frequency (UHF) RFID to get low-cost 3D tracking to be accessible in Dynamicland. There are no visuals for now but we can get a stream of UHF RFID signals into Realtalk as JSON, the next natural step is to transform that into Lua tables with which to trigger/animate programs.