The Strange Agency

We can use Stable Diffusion to explore industrial design, and the images it generates are quite fanciful.

Unfortunately we have little control over the form factor of the generated images. This can be an advantage, as the model gets quite original with its designs, but what if we want to constrain ourselves to a format, like Eurorack?

Starting with v1-5-pruned-emaonly [cc6cb27103] using a fairly simple prompt, “modular synth eurorack mixer compressor equalizer mastering studio”, we get this quite slick synthesizer.

Nice design, but can we control it? Let’s start with a line drawing of some Eurorack modules.

The Canny mode of ControlNet might be a good candidate, so we begin there. The drawing needs to be inverted to get white lines on a black background. Here we process it with control_v11p_sd15_canny [d14c016b], using the Stable Diffusion web UI with the ControlNet extension.

The result is nothing like the original, but we have gained control of knob and button placement. This was with the default settings, we can also play with a little less control, turning down the Control Weight parameter to 0.5.

A far more interesting synthesizer panel! We can pimp out the prompt with some more studio jargon, for a somewhat glitzier result. Not much of an improvement, at least not with the current seed. The prompt below is, “modular synth eurorack mixer compressor equalizer mastering studio - bakelite knobs buttons dials switches jacks eighth inch stereo mono - UV LED USB 48V dB LCD OLED”.

We can try adding some Trademarks of Their Respective Holders™ to the pompt, “braun dieter rams apple ive industrial design teenage engineering”. This seems to clean up the panel a little.

The above Control Weight of 0.5 seems like a pretty good option, but we can try lowering it. In this situation, 0.4 seems to be the threshold below which our synth drifts into an entirely different form factor.

Next we look at using segmentation, control_v11p_sd15_seg [e1f51eb9], for which a rendering with filled areas might be better.

In actuality, neither work well with the Control Weight at 0.5.

We have to head back up toward around 0.7 to get the segmentation ControlNet to maintain our panel layout. It is not clear which is better, using lines or solids.

We can try a couple more options, keeping other parameters the same. Below we have a grey version and one with a bit more detail added.

The results follow.

The difference is looks pretty arbitrary. Perhaps increasing the Control Weight to the default of 1.0 will have a more pronounced effect?

The detailed version results in some interesting details where the faders would go, but here too it is not necessarily an improvement.

Finally, we can try a depth model, control_v11f1p_sd15_depth [cfd03158], using the more-detailed schematic.

And we can relax the Control Weight, as above.

As can be seen above, we can go down to about 0.5, below which the model starts to lose the plot. Around 0.8 looks like a nice balance of creativity and adhesion to our layout. To wrap up, let’s borrow some artsy prompts from Lexica and see what that does for us.

As per this Lexica prompt, we can add “edward hopper and james gilleard, zdzislaw beksinski, mark ryden, wolfgang lettl highly detailed, hints of yayoi kasuma” to our original “modular synth eurorack mixer compressor equalizer mastering studio - bakelite knobs buttons dials switches jacks eighth inch stereo mono - UV LED USB 48V dB LCD OLED - braun dieter rams apple ive industrial design teenage engineering”.

And if we get rid of all the studio prompting, using only the art prompt above? We get some sort of Fantastic Planet modular. Quite nice!