Exploring interspecies interaction via a heart beat sensor triggering water movement, mapped to sound and visuals
Software apps and online services
- TouchDesigner
- Arduino IDE
Hand tools and fabrication machines
- Biomaterials - Laser cutter (generic)
Story
A microchallenge is a week long workshop inside the Master in Design for Emergent Futures at IAAC Barcelona. During this week we have the time, space and support of professionals to create a functioning prototype, which can also be a speculative artifact, that contributes to our personal research and practice. This prototype should be based on the iteration and usage of: digital fabrication tools, biofabrication, AI, electronics, robotics, coding. For this particular microchallenge the topic was about "Intelligences", because of our shared interests in our personal research we grouped:
Belen Comotto
Maria Vittoria Colombo
Ziming Shang
Kevin Giovanni Enriquez Ambrocio
During this week, we wanted to decided to address "more than human" intelligences and ways into making visible even the tiniest life forms that conform our reality. We explore more-than-human interactions in a micro scale, diving into the concept of interspecies collaboration. Specifically, we’re interested in the speculative exchange between human input and micro organisms living in water systems.
The interaction unfolds in a loop, beginning with a human input via the heart rate sensor. This triggers a vibrating sensor attached to a Petri dish filled with water, causing the water to move in patterns based on the heart beat rhythm.
These movements are captured through a microscope connected to a computer, where the images are analyzed in TouchDesigner. The system identifies key features and maps them out, creating a matrix of coordinates, which is then translated into sound.
Fabrication and iteration process:
Along the way of the development and testing of the working of the prototype we tested several different zooms from the microscope, setups of the Petri dish, different vibration intensities and different mediums of water with microorganisms. In the end we concluded that the best outcome resulted from using a mixture of different origins of the water from different ponds, medium intensity vibration (150 / 255), and 10x magnification from the microscope. These parameters provided a clear enough visual from the camera sensor that would be usable in the TouchDesigner file.
Sensors:
The first sensor we needed to approach for this prototype was the one measuring our heart beats which translated into vibrations for the Petri dish. We tested various setups, adjusting values and coding repeatedly for 3 different sensors: a heart monitor sensor, a finger heart rate sensor and finally a heart beat sensor. However, the results were often imprecise and unreliable with the first two, so we decided to stick to the heart beat sensor for better accuracy (v1706 heart beat sensor).
As can output of the heart beat we needed to achieve a vibration to create the ripples in the water of the Petri dish. We experimented with vibrating motor, speakers, and piezo parts. The vibrating motors, placed on the sides of the Petri dish, provided the most consistent performance. To optimize vibration, we chose a plastic Petri dish and placed it on a silicone base.
TouchDesigner:
We used Python code with the OpenCV plugin in TouchDesigner to apply computer vision, detecting edge points in the video, as well as blob tracking. From these points, we extracted a dataset of x and y coordinates, which we translated into sound by mapping them across four layers of audio oscillators, each using different waveforms and frequencies. This process generated the final audio output.
Water:
We analyzed three different types: stagnant puddle water, seawater, and water from a public park. Each sample revealed distinct elements, which we identified and visualized using TouchDesigner’s features.
Wearable:
We also decided to create a wearable, gelatin-based biomaterial infused with spirulina for coloring. We crafted a rectangular sheet of this material to integrate with the sensor.
Recipe: 100g gelatin, 60g alginate, 550ml water — bring to a boil, then pour into a mold. Add spirulina for pigmentation.
The concept was to design a garment around the neck and chest that appeals to a fish like/ algae like concept allowing the biomaterial to hold the sensor against the neck while proposing a hybridization of the human with the living organisms that live in the water.
FUTURE DEVELOPMENT OPPORTUNITIES
For the future we would like to:
Wearable > focus on developing a more ergonomic and practical design, experimenting with various biomaterials, fabrics, and forms that allow a more easy reading and wear.
Electronics > right now the structure with the arduino in the breadboard and the cables around is not very compact and gives space for error, we would like to minimize as much as we can the circuit and create a seamless integration.
Scale > once the design is solidified, it could be intriguing to scale the concept into a larger installation — a multisensory experience where sound and visuals merge, allowing the audience to engage with the piece in a more immersive and impactful way.
Tweaking > while presenting the project on the final expo of the workshop we realized that the idea was quite complex and not properly understood, as a further iteration we would like that through the experience and design the concept would be clearly understood. Also the fact that the design is very delicate makes it not that approachable and limits the play and freedom, this is a major aspect that would make the experiment more engaging.
Finally as being a large group each of us had different interests that wanted to bring up in this process and thus it added several layers of complexity that could be further explored: the design of functioning wearables, the visual processing, the audio generator and seamless circuits. In the future I believe we had opened several pathways to continue further down these rabbit holes in each individual research.
This article was first published on Hackster on March 7, 2025
cr: https://www.hackster.io/mariavittoriacolombo/human-microenvironment-interaction-99b73f
Author: Maria Vittoria Colombo, belll, ziming shang and Kevin Enriquez Ambrocio
