Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Squarepusher - A Tool To Convert Images To Audio For Oscilloscope X/Y Mode Displays

by bpoag (Monk)
on Feb 01, 2013 at 17:01 UTC ( #1016588=CUFP: print w/ replies, xml ) Need Help??

I have a side hobby that involves resurrecting ancient electronics, and retrofitting them for modern purposes. One such tinkering project was to make an old early 60's oscilloscope and turn it into a device for displaying bitmap images.

To do this, I needed something that would convert a bitmapped image into audio. The resulting audio can then be played back with the Left and Right audio channels hooked up to the Horizontal and Vertical inputs on the oscilloscope.

Given the constraints of the hardware (an always-on single beam of electrons being redirected and shot against the backend of phosphor tube) this poses a bit of a challenge. First of all, youre constrained to 48 KHz of bandwidth, the upper limit of what most audio cards are capable of. Secondly, there's a limit to how quickly the oscilloscope can move the beam around. It's very easy to tell the oscilloscope to draw something too intricate, which results in terrible image flicker. Comprimises must be made to ensure that the image is both legible and as close to devoid of flicker as possible.

Simply put, Squarepusher takes a 1-bit PBM image, and converts it into a 48KHz 2-channel (stereo) .WAV audio file. Why 1-bit? Because oscilloscopes are naturally monochrome display devices, read: no color. Secondly, the brightness of any given spot on the phosphor is a function of how long the beam stays in that location. There simply isn't enough time to modulate the beam at the microsecond level to emulate different greyscale values. You're effectively constrained to a small resolution (256x256) and the equivalent of 1-bit depth... a pixel is either on, or off. PBM happens to be a good format for such purposes, since ASCII PBM is nothing but a file of zeroes and ones. As a side note, to convert an image to 1-bit PBM, you can use pnmtools, GIMP, or Photoshop. The resulting image must be 256x256, and only 256x256.

Why 256x256? Think of it like this. There's an upper limit to the number of points you can continually re-plot on an oscilloscope display before it becomes insanely flickery. There simply isn't enough time. At 256x256 resolution, in a worst case scenario, all 65536 pixels will need to be plotted, but it will usually be less. Even when the pixel count is far less than 65536, you'll still see noticable flicker..So, even at 256x256, that's already kind of above the functional threshold of the hardware. 256x256 just happens to be the best comprimise between image resolution and the draw speed of the beam. In theory, my code can be modded to do much higher resolutions, but you're going to need a wicked fast oscilloscope that might require violating every known law of physics to make it happen. :) I suppose you could change the resolution to 512x128, and it would look just as good, but, I figure most people would prefer having a nice square to work within, versus a rectangle. 256x256 is the best comprimise between image resolution the need to move the beam quickly. You're not likely to ever see a script such as this operate at any larger resolution.

Squarepusher can render an image using any one of three different modes; "Standard", "turbo", and "plow".

In Standard mode, the beam is directed across the display in a fashion similar to how you're reading this text; from left to right, and repositioning to a zero location once the end of a line is reached. This makes for a nice drawing, but some time is wasted in the horizontal sync (the time required to move the beam back to begin painting the next line.)

In Turbo mode, Squarepusher will only bother drawing an outline of the image you provided, skipping any region that contails a solid-filled area of pixels. This helps reduce flicker at the cost of image clarity, but it can also produce some interesting effects.

In Plow mode, Squarepusher will paint the image on the phosphor boustrophedonically. In English, like a farmer with an ox plows a field. The first scanline will be drawn left-to-right, and the following will be drawn right-to-left, in order to minimize the amount of time wasted respositioning the beam for the next pixel.

Each mode takes three values. Frames, Lossyness, and Skip.

Frames: Squarepusher will render multiple frames, which you will probably want to do most of the time. Multiple frames are also a must-have for images that require some persistence-of-vision trickery to look good. More on that in a moment.

Lossyness: Often times, it simply takes too long for the beam to paint every single pixel of an image. The results will look terribly flickery. To get around this, we can specify a "lossyness" value. With this value, the painting algorithm can be instructed to randomly jump ahead, painting only a portion of the pixels on a given scanline. This is fine, provided you are rendering multiple frames, because eventually, the entire scanline will be painted, and thus the entire image. Specifying a lossyness value of "10", will tell the algorithm to randomly jump anywhere from 1 to 10 pixels ahead, checking to see if that pixel needs to be drawn, jumping another 1-10 pixels ahead, seeing if that pixel needs to be drawn, and so on. The net result will be a legible image via persistence-of-vision that seems a little bit glittery or grainy, but complete, and mostly flicker-free. Again, comprimise is the name of the game--We only have 48 KHz to work with, compared to something like NTSC television, which has something in the neighborhood of 4 MHz.

Skip: Additional time can be saved by specifying a skip value. This value dictates the painting algorithm to skip a number of scanlines, sort of like the horizontal equivalent of the lossyness value. For example, specifying a skip value of 5 will tell the painting algorithm to paint a scanline, randomly skip ahead 1-5 scanlines, and continue painting. Used in conjunction with the lossyness value, the best possible combination of image clarity and refresh rate can be obtained.

Squarepusher dumps its output to stdout, so, be sure to redirect it into a file. You'll see the command template, usage, and example usage below.

Watch the video of this script beating the crap out of a 52 year old oscilloscope here. (Edit: Updated 2/2/13)

#!/usr/bin/perl ## ## Squarepusher v0.1 written by Bowie J. Poag, 1/29/13 <bpoag@comcast. +net> ## ## Squarepusher converts a 1-bit 256x256 PBM image to a WAV audio bits +tream ## suitable for playing on an XY-mode oscilloscope display. ## ## Usage: ./squarepusher.pl <mode> <#frames> <lossyness> <skip> <filen +ame> ## Example: ./squarepusher.pl turbo 50 20 3 foo.pbm ## English: Using turbo mode, spit out 50 frames with a lossyness valu +e of 20, ## rendering every 3rd line of the image foo.pbm. ## ## Valid mode names are standard, turbo, and plow. ## ## Standard: Straightforward left-to-right traversal of the image, Thi +s will ## render the image on the screen in a sawtooth sort of fashion, like +one might ## read words a sentence in a paragraph. ## ## Turbo: This method attempts to optimize beam time by only rendering + the outline ## of a filled area, rather than the area itself. Useful if you have a + high-detail ## image that flickers a lot, and doesn't need precise detail. ## ## Plow: Like "Standard", but boustrophedonic traversal. The picture i +s drawn ## on the screen in a zig-zag fashion, alternating left-to-right and r +ight-to- ## left. This saves time in that the beam does not need to return to a + zero position ## to render the next scanline. ## ## Image conversion is beyond the scope of this script. If you want to + use some ## other image format with this script, you'll need to first convert i +t using ## netpbm tools, GIMP, Photoshop, or some other image processing tool +capable ## of exporting 1-bit images to ASCII PBM $mode=$ARGV[0]; $frames=$ARGV[1]; $lossy=$ARGV[2]; $skip=$ARGV[3]; @bitmap=`cat $ARGV[4]`; ## This is a 48 KHz 2-channel (stereo) 8-bit WAV file header. ## Normally, we would want to have correct subchunk values, but, ## i'm a lazy bastard, so we're just going to tell whatever's ## going to play this audio sample that it's 0xFFFFFFFF bytes ## in size, and have them deal with the underrun. ## ## I do this because I'd rather be waterboarded than have to ## deal with a file format that uses mixed big- and little-endian ## data. ## $header="\x52\x49\x46\x46\xFF\xFF\xFF\xFF\x57\x41\x56\x45\x66\x6D\x74\ +x20\x10\x00\x00\x00\x01\x00\x02\x00\x80\xBB\x00\x00\x00\x77\x01\x00\x +02\x00\x08\x00\x64\x61\x74\x61\xFF\xFF\xFF\xFF"; if ($lossy==1) { $frames=1; } if ($lossy<=0 || $lossy >=255) { $lossy=1; } # Strip the header information off.. shift(@bitmap); shift(@bitmap); shift(@bitmap); # A little cleanup.. foreach $item (@bitmap) { chomp($item); $total.=$item; } @image=split(//,"$total"); # The weird conditional block below is a hack. # It's basically a quick convolve filter, that will # produce a vector outline of any solid white area. # this saves beam traversal time, and significantly # reduces image flicker on images with a large ratio # of white to black pixels. # # In English, it will take any solid white shape, # and draw it as simply the outline of that shape. # # If you don't want to use this optimization, just # comment out the whole if statement. for ($q=0;$q<$frames;$q++) { if ($mode=~/turbo/) { for($x=0;$x<(256*256);$x+=rand($lossy)) { if(($image[$x]=="0" && $image[$x+1]!="0") || ($image[$x]=="1" && $image[$x+1]!="1") || ($image[$x]=="0" && $image[$x+257]!="0") || ($image[$x]=="1" && $image[$x+257]!="1")) { $col=$x%256; $row=int($x/256); if ($x<(256*256)-256) # Don't try to optimize the last line. { if ($row<256 && $col<256 && $row%((rand($skip))+1)==0) { $sample.=chr($row); $sample.=chr($col); } } } } } if ($mode=~/standard/) { for($x=0;$x<(256*256);$x+=rand($lossy)) { if ($image[$x]=="0") { $col=$x%256; $row=int($x/256); if ($row<256 && $col<256 && $row%((rand($skip))+1)==0) { $sample.=chr($row); $sample.=chr($col); } } } } if ($mode=~/plow/) #Boustrophedontastic! :) { for ($x=(256*256);$x>0;$x-=256) { $thisRow=int($x/256); if ($thisRow%2==0) { for ($c=0;$c<256;$c+=rand($lossy)) { if ($image[$x+$c]=="0") { $col=$c; $row=$thisRow; if ($row<256 && $col<256 && $row%((rand($skip))+1)==0) { $sample.=chr($row); $sample.=chr($col); } } } } if ($thisRow%2==1) { for ($c=256;$c>0;$c-=rand($lossy)) { if ($image[$x+$c]=="0") { $col=$c; $row=$thisRow; if ($row<256 && $col<256 && $row%((rand($skip))+1)==0) { $sample.=chr($row); $sample.=chr($col); } } } } } } } # We now have a rather large scalar who's contents are identical to an + 8-bit two-channel WAV audio stream, suitable for framing.. # $dump=$header.$sample; print $dump;

Comment on Squarepusher - A Tool To Convert Images To Audio For Oscilloscope X/Y Mode Displays
Download Code
Re: Squarepusher - A Tool To Convert Images To Audio For Oscilloscope X/Y Mode Displays
by Paladin (Priest) on Feb 01, 2013 at 23:17 UTC
    I watched the video, that's pretty awesome. I regret I have only 1 ++ to give this. :)
Re: Squarepusher - A Tool To Convert Images To Audio For Oscilloscope X/Y Mode Displays
by jdporter (Canon) on Feb 02, 2013 at 01:17 UTC

    That's freakin' phenomenal. My first thought was: "And people think Perl's no longer useful!"

    Which is why it pains me to have to tell you .... your Perl is ... um ... not great.

      To my old eyes it looks like FORTRAN. I assume you're talking about the loops. What could/should be done to make it better? Actually curious, I'm not good at this yet.

        Can’t speak for jdporter, but to my way of thinking the loops (C-style instead of foreach-style) are the least of the problems. To my old eyes the major issues are:

        1. $mode never changes, but it’s re-tested each time through the main loop.

        2. The code is monolithic and should be refactored into subroutines (one for each mode).

        3. use warnings; and use strict; are missing, and all the variables are global.

        Hope that helps,

        Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Re: Squarepusher - A Tool To Convert Images To Audio For Oscilloscope X/Y Mode Displays
by exixx (Acolyte) on Feb 04, 2013 at 14:20 UTC
    This is the coolest thing I've seen on a scope in a long long while. Lissajous figures have nothing on this.
Re: Squarepusher - A Tool To Convert Images To Audio For Oscilloscope X/Y Mode Displays
by Anonymous Monk on Apr 21, 2013 at 06:27 UTC
    Hi, i'm running the code but my audio output has no 'data chunk' in it. Is there only certain types of sound editors that can play these files? Is there anything in particular that needs to be done in order too output the wav file. (i'm running this on windows if that makes a difference)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://1016588]
Approved by moritz
Front-paged by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (6)
As of 2014-07-11 10:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (224 votes), past polls