Heartbeat Harmonies


ITP Physical Computing
Arduino / p5.js / AIGC
Fall 2021, New York.


Role:
Physical Computing, p5.js Code

Collaborator:
Jingjing Wu - Music Visualization, Fabrication

OVERVIEW


Heartbeat Harmonies is an interactive project that utilizes AI to generate music based on the user's heartbeat. The music seamlessly syncs with the user's real-time heart tempo, resulting in a immersive musical experience that resonates with their emotions.





RESEARCH AND PROCESS


Generating Melody Cased on BPM


In my initial attempt, I used the tone.js library to generate music. However, I encountered some instability issues with tone generation when restarting the sketch. Additionally, due to my limited experience in composing, the melody didn't quite meet the standards of what could be considered as "music."



We then found SoundRaw, an AI composer that provides the flexibility to customize the tempo of each music piece. 

Change Tempo of Music Real-time Based on Heartbeat


However, since the heart rate is constantly changing, we continuously adjust the tempo of the music in real-time after generating a music piece based on the initial heartbeats. This dynamic adaptation ensures that the music remains synchronized with the user's heartbeat throughout the experience.


Make Sound Visualization


Created by Jingjing


Make a Container




Tools Used:
  • Arduino NANO 33 IoT
  • Pulse Sensor

Skills Used:
  • Coding (p5.js, Arduino C++)
  • 3D Modeling and 3D Printing