Final

As mentioned in the previous post, for my ICM final I chose to continue my exploration of spring dynamics combined with ml5's handpose model.

‍

Ordered connections (full screen)

Random connections (full screen)

Random connections (webcam size — mouse click to activate)

‍

‍

Had to redo a lot of the code from last week — I went back and forth a lot on how to structure my classes, especially because this is something I would like to keep exploring so I wanted it to be as robust and efficient as possible so that I could possibly utilize the same class for other projects down the road.

The biggest issue with both my code from several weeks ago as well as last week's was that I was constantly checking which point my pinch is close enough to and attaching that point to the pinch's location. This caused several issues: for one, it made the pinch catch a different point if it was near it even if it was already pinching another one. It was also sometimes catching several points at once, and "welding" them together.

While testing my options, at some point I was printing the hand objects to the console and found out it had a "handedness" property! I realized I could use that to differentiate between the hands and use that to assign the pinched points. I just had to account for all of the possibilities — made a little comment-table to help me make sure I'm going through all of them with my if statements.

‍

Another persisting issue is that of the scale of the whole sketch. I tried Ellen's solution (thank you Ellen!!). Took some time to figure out because I structured things differently — but left the activation of the model on mousePressed() as it seems convenient enough. Ideally I'd initiate this after the webcam has loaded. I looked for a javascript function that delays execution (tried setTimeout but apparently its asynchronous so it not suitable for this). I tried putting all the code inside mousePressed() in a custom function and calling it at the end of getVideo() — that seems to work, but there's an issue that happens regardless of where it's called where it actually creates a much smaller canvas(?). It's not consistent as sometimes it just works, and other times another click or two of the mouse seems to fix it somehow... I left it at that for now.

As the scale became larger I had to change some of the parameters as it really changes the interaction. For example, the distance required for pinching changed drastically as well as the number of particles I thought was suitable for this. I'm still not sure it works as reliably in the larger scale (doesn't feel like it).

‍

Another option of solving the scale issue was calculating the proportions and basically adjusting everything accordingly. A lot of trial and error, and also it stretches the video, so this wouldn't work well if I wanted to show the live webcam footage, but because I don't this actually works pretty nicely.

‍

Initially I wanted to add a "leg" on each particle to make it look like some sort of centipede, and I couldn't really figure it out. I then tried to just create a line that's perpendicular to each spring but that didn't work as well. While trying other things I got to the current aesthetic which I really liked, and felt like it was a much better match to the way this thing moves and behaves. It really wasn't that much like a centipede anyway :-)

‍

‍

Eventually I decided to attach the particles randomly and not by order as it created more interesting structures, some of which had springs sort of "hanging" off of some of the anchors, which made for a nice secondary-animation effect when moved.

‍

I'd like to thank Lior Ben-Gai for helping me with several parts of the code for this project. 🤍