Leaf Dance


My light project was inspired by the reflection of plants reflected on the curtains on sunny days. They are swaying with the wind according to the wind speed, and they seem to be swaying with the melody. The color of the reflection will change according to the plant itself and the light.

shadow .jpg

I choose to use the Demucs module to train my music files. Demucs is based on U-Net convolutional architecture, it can separate drums, bass and vocals from the rest with state-of-the-art results, surpassing previous waveform or spectrogram based methods . I successfully decomposed my wav file (music named Piano Form II by Jesse Woolston) in Codelab into separate audios files (bass.wav, other.wav, vocal.wav, drums.wav)


Then I import these audio files into P5 Sketch and analyze and process some of the data. For example, I map the amplitude of the drum file to light ring 2, and the amplitude of the other file to light ring 1, and the amplitude of the piano file controls the motor's speed.

​Then I use Serial communication to transfer the data in P5 to Arduino.
Selection of physical components:
L298N motor driver
JGB37-3530 12V 300RPM gear motor
24-Bit WS2812 5050 RGB LED Ring

​Arduino UNO


Import weather data

After using the melody control of music, I tried to add real weather data. I obtained the wind direction and wind speed data of New York from the ITP's Weather Band database , I tried to map the wind direction to the steering of motors, and processed the wind speed to control whether the motor is rotating.

I use P5.js to get the data and visualize the process of weather data.

Later, I tried to introduce illuminance data to control the color change of LED Rings, but I found that although I successfully transmitted the three data of wind speed, wind direction and illuminance to Arduino at the same time in P5, if I use the illuminance data from P5 sketch and combine with the FastLED library to control the RGB value of the  light, it will affect other transmission data, resulting in the failure to receive normal data.

But I haven’t found a better solution. At present, I separate the control of the light from the control of the motor. The change of the light has nothing to do with the weather data temporarily.

The followings are my demo videos:

IMG_2836 2.heic