E-Unconscious
I wanted to use data from Spotify for this project however, Spotify for some reason takes “up to 30 days” for the data top be assembled. Which is pretty absurd considering it only took me an hour to retrieve ALL of my data from Google, I’m mean all of it. All 20 GB of it. So Spotify was out and Google was in, now to decide which of the datasets to use.
I started looking at location data, which is a massive 12mb JSON file, so large Visual Code would not load it correctly. I eventually got it top load by changing some setting. P5.js web editor was out as it only has a 5mb upload limit. I tried to set up Visual Code but was spending too much time trying to get it to work. And all of this before I even started to try and figure out a way to visualize that data in p5. So I moved on to search history, I didn’t really want to use this dataset but there are a lot of examples of how use it somehow. In the end I used a modified version of the code from DanO found here https://github.com/dano1234/Rest-of-You/blob/master/WordCountTakeOutJSON/sketch.js to generate a word usage list. My next step is to take the words and visualize them into some graphic. I want to do this computationally using p5, but I’m a bit rusty so it might take me a few to do it, haha. Below is the code I ended up using, it didn’t workout of the box. Some changes need to be made, most notably the var = i was defined on line 20, so the for loop in line 401 needed to have a different variable. ON line 17 I tried to use dot notation as per p5’s request, but as the comment says, history.myActivity does not work to return length however, history['myActivity'] does work to return length. The name of the JSON file needed to match the sketch so I changed that to “MyActivity” and IN the JSON I renamed it to "myActivity" otherwise p5 didn’t know how to interoperate the file.
let history = [];
let index = {};
let keys = []
function preload(){
console.log("loading");
//history = loadJSON("BrowserHistory.json");
// you need to put your file in the same folder and put name here
history = loadJSON("MyActivity.json");
}
function setup(){
createCanvas(400,1200);
//history = history['Browser History']; //browser history is one level in
history = history['myActivity']; //if you surround your json with {"myActivity": ...... millions of lines ... } you will get length to work
//otherwise you have to hard code the length in like 100000 in for loop, go figure.
console.log(history.length);
for(var i = 0; i < history.length; i++){
// var date = new Date(history[i].time_usec/1000); // create Date object if time is given in milliseconds or even usecs
//var date = history[i].time;
let words = history[i].title.split(" ");
// let words = history[i].title;
// let words = history[i].title;
for(var j = 0; j < words.length; j++){
let thisWord = words[j];
if ( index[thisWord ] == null ) {
index[thisWord ] = 1;
keys.push(thisWord);
}else{
index[thisWord ] += 1 ;
}
}
}
keys.sort(function(a,b) {
return index[b] - index[a];
});
for(var e = 0 ; e < 100; e++){
let key = keys[e];
console.log(key , index[key]);
text(index[key] + ":" + key, 10, 12 * i);
}
}
function draw(){
}The working p5 sketch can be found here