Visual First Tooling

Recently, I’ve been more optimistic about visualizations over existing general purpose text-based programming languages than purely visual languages. With Houdini, Blender, vvvv, TouchDesigner, etc. you can get started comparatively quickly. There’s no need to worry about setting up an environment or learning a new language’s syntax. But picking one of these is kind of like choosing an operating system. You get significant speed boosts inside, but very if what you make are able to exist outside the tool. Even if your tool, like Blender, is free and open-source the requirements for building a nicely wrapped software package mean the dynamic, living system you make on the inside can generally only leave as a baked .obj file or a video. All of this increases production and maintenance costs, pushing each environment toward proprietary structures.

One barrier with visual coding tools is that they tend toward lock-in and centralizing forces (visual design, UX, target platform)—very few can afford to be sustained long term. Whereas textual languages are more like ‘protocols’; scrappier and interoperable.

– Matt DesLauriers (Jun 13, 2023)

Well, fine maybe they can’t make it outside without a runtime. But how about tool to tool, node graph to node graph? Not without a lot of work. OpenMFX is an attempt to unite the nodes in Blender, Houdini, and other tools, but it seems to be a single guy taking it all on. And his challenges are more legal than technical; one group’s license for Gaussian blur is proprietary and the other’s implementation of subdivision is GPL 3.0. You can find the same story with every A/V editing tool repackaging ffmpeg commands into a subscription. Kenneth Perlin’s original Simplex Noise implementation was patented till early 2022 so maybe it shouldn’t be so surprising to have incompatible math functions due to licensing.

The largest benefits I get from visual tools are liveness, feedback, and an expansive “standard library”. Write the standard library in any language of your choice. bonus points if it has good performance or there’s a large community of people to pitch in. And for the rest? We don’t think of them that way, yet GUIs like nodes.io and Theatre.js act just like IDEs. Right now there’s a whole minefield picking tools now because you don’t want to get work stuck an abandoned format, but imagine if Cinema 4D was like Sublime Text, Houdini was JetBrains, Blender was neovim. Imagine, if you will, that portable data was the bottom layer of your stack