Sound-Viz

Blog, ICM, ITP , 0 Comments

How does one visualize sound? Is it through color? Through shape? Through line? What about the sound is being visualized? Quality? Pitch? Amplitude? Volume? These are all good questions, and I will answer none of them. But what I will do is tell you how I took sound, in this case a song, and visualized the fft.analyze() which returns an array of amplitude values. These values range between 0 and 255; I decided to use HSB color space making it a bit easier to have a varied range of colors represented. 

However, let me start by saying working with sound and trying to visualize it is extremely frustrating. I could never achieve what I was looking to do. I was not successfully able to make each square a different color representing a different index position in the fft.analyze() array. I’m sure it has something to do with the way this block of code is written.

for (y_pos = 0; y_pos <= height; y_pos += a_size * 1.5) {
   for (x_pos = 0; x_pos <= width; x_pos += a_size * 1.5) {

     var spectrum = fft.analyze();
     for (var i = 0; i < spectrum.length; i++) {
       // var spec2 = spectrum[i];
       // var col = map(spec2, 0, 255, 0, 360)
       var col = map(spectrum[3], 0, 255, 0, 360)

       noStroke();
       fill(col, 100, 100);

       rect(x_pos, y_pos, a_size, a_size);
     }
   }

But hey, this is what learning is all about; fail often and fail early so to set yourself up for success in the future. All in all I was able to produce something that is interesting and visualizes the sound in a unique way. I was also able to work on adding doms through the use of a secondary .js file as well as making the whole sketch responsive to window size.

This video uses standard RGB colorspace, from the colors generated it seems as though the first three indexes of the array are being used to get the color however, I have not had a chance to confirm my suspicions.
In this example I used HSB colorspace in order to have a broader spectrum of colors. I also change the background to black to make the more saturated colors pop.

The single color working sketch can be found here

https://editor.p5js.org/phil-in-a-can/sketches/XYvmTBJPv

As always I welcome feedback in the comments section and if you figure out how to make each square a different color, please let me know!!

UPDATE

Well what can I say, “I get by with a little help from some friends”. After reaching out to my cohort and having a few discussions a solution to my issue came to the surface.

The updated and working sketch can be found here

https://editor.p5js.org/phil-in-a-can/sketches/aqFqyHpnz

The function draw() was updated with the following code.

function draw() {
  background(0);
  var spectrum = fft.analyze();
  for (let x = 0; x < 13; x++) {
    for (let y = 0; y < 13; y++) {
      if (spectrum[x + y *13]) {
        var col = map(spectrum[x + y * 13], 0, 255, 0, 360)
        noStroke();
        fill(col, 100, 100);
      }
      rect(x*width/19*2, y*height/19*2, width / 19, width / 19);
    }
  }
}

Thanks to David Currie for the help with this.

This shows the same song as the above videos with the adjusted code.
And this video is a different song with the adjusted code.