Can a machine teach you what your body knows?
I’m writing this article both as a musician with embodied and practical knowledge — and as a coder who has built music software.
I want to bring your attention to the following example. Say you are making an app that enables the user to fill in a “shaker” part — something that makes dense, repetitive rhythms. If you’ve ever actually held any kind of shaker, you know that they are generally light and high-frequency (compared to a large drum). You understand that playing them takes much less energy than playing something huge that you hit with a stick. It’s the kind of understanding you can only get from the physical world — it takes more time and effort to make a loud sound on a big thing than to shake a small rattle.
Going back to the app example — if I abstract the “shaker” function, enabling the user to assign any instrument they like to something that is designed to be dense like a real-life shaker, then I am making something where the space of possibilities is not filtered by what the body of an acoustic musician knows about the music. From a coding perspective, a retrigger is a retrigger, and you can assign two subsequent notes any dynamics you like. From a playability perspective, the dynamics will be informed by context — a “da-DUM” pick-up has a specific meaning within the music. An acoustic musician has the option of projecting their embodied cognition onto the electronic instrument — knowledge about gesture, gravity, tuning, being in sync with others, being part of the ensemble.
I notice that most music software made by tech people disregards the bodily dimension of music-making. There is an interesting paper called “Of Epistemic Tools: musical instruments as cognitive extensions” by Thor Magnusson. In it, he writes: “whereas the body of the acoustic instrument is physical, the body of the digital musical instrument is intrinsically theoretical.” The electronic-music lineage doesn’t have much experience with the gestural dimension of playing an instrument; the focus is in a different place — perhaps on texture, on the spectrum, on form. As a musician I understand that as an aesthetic. But as somebody who has experience with dance and theatre, I am always mindful of the role of the physical expression in music-making — especially in the participative, collective dimension: circle dances, choral singing, percussion bands — and I feel that the seated, thinking body produces music that is very different from the moving, dancing body. Whatever our body does or doesn’t do, we cannot escape taking our embodied experiences to music-making.
To me, the joy of music is intimately connected to the embodied experience of rhythmic tension and resolution interplaying with the internal senses of your own body. Here is a quote from Shakira talking about “Hips Don’t Lie”: “The reason why I named that song that is because, when I’m in the studio, I know when a song is ready and it can be taken out of the oven, and it’s exactly when my hips start moving — when my body reacts physically to a song, I know — if it’s a dance song — that song is done. So I used to say to my musicians, ‘my hips don’t lie! Are they moving? They’re not moving! So this is not ready.’ And that’s how I came up with it, the idea of the song.”
The question that interests me as an educator is whether we can pass this embodied knowledge on through technology. Here is one concrete example. From my teaching experience, it is absolutely impossible for someone to develop a sense of internal timekeeping just from staring at a screen. This is a reference we carry in our bodies. My years of teaching singing and musicianship tell me it is extremely individual — some of us were allowed to “bounce around” as children, some of us were told to be still and/or quiet. But the gist is that internal timekeeping only develops with motion; it can’t be shown to someone. Someone has to do something. Magnusson quotes Winograd and Flores: “my ability to act comes from my familiarity with hammering, not my knowledge of a hammer.” You cannot hand someone the hammer and expect the hammering. We learn through imitation, and technology that models movement and gets us developing our body awareness can give us skills that no amount of tapping can. Perhaps if you had an app that actually shows you your weight transfer — from one foot into another — especially combined with some good music, then it would have the potential to bring your attention back to your own body: to develop a skill you then take with you when you switch off the app. That is a prerequisite for timekeeping, and other things — like clapping off the beat — can be taught on top of it.
Why does this matter? Because practical musicianship knowledge is currently either reserved to those who have it in their community or in their family, or to those who have access to thorough musical education, which can be pricey and exclusive. And institutionalised music education is often very specialist. It originates in a European educational philosophy which separates it from other performing arts and brings a certain reductionism — studying music separately from movement, from storytelling. It implies a particular vision of embodiment, and enforces a canon of music-theoretical thinking that ennobles certain elements (like harmony in an analytic sense) and glosses over others (like swing or articulation). The beauty of getting into electronic music, by contrast, is ease of access: learning from the community via tutorials and direct exchange is much simpler, more affordable, and easier to do in your own time.
Given all this, I think it is extremely important to be conscious of what it is exactly that the embodied knowledge of playing an acoustic instrument brings to an electronic musician — while at the same time keeping in mind issues of access and exclusivity. I dream of electronic tools for teaching musicianship which complement, rather than contradict, in-person education. Tools which acknowledge the player’s body, which encourage them to make unmediated sound in their own surroundings, to play with others, which inspire participation and movement. Tools which don’t try to be something they can’t be — instead, they facilitate the flow of embodied knowledge between experienced and inexperienced players, they model and create opportunities for imitation, they teach the player to rely on their internal sensing, they develop skills which persist when you put the tool down.
None of this is simple. Any app is mass-produced, and a one-to-many kind of transmission comes with its own inherent biases. Technology always bears the mark of whoever made it, and whatever a tool makes easiest, it makes normal, positioning the other options as “inconvenient” or “exotic”. Magnusson writes, “… the piano keyboard tells us that microtonality is of little importance (and much Western music theory has wholly subscribed to that script); the drum-sequencer that 4/4 rhythms and semiquavers are more natural than other types; and the digital audio workstation, through its affordances of copying, pasting and looping, assures us that it is perfectly normal to repeat the same short performance over and over in the same track.” No one-to-many tool can teach what a private music teacher can. There are obvious limits to what any technology can carry across: you can’t learn swing from sheet-music notation. But I don’t think we need to throw the baby out with the bathwater. I believe that when we examine our assumptions thoroughly, we can create tools that facilitate musical exchanges between players of various levels of experience, rather than intercept them.
In the end, I think it is essential for coders to acknowledge the value of embodied cognition. Maybe the biggest barrier is the shared preconception that music, like maths, is “innate talent” rather than a learnable ability. What electronic musicians can learn from acoustic musicians is exactly what the acoustic tradition holds in the body and has never quite been able to write down. And the interesting question is not how to replace that knowledge with a machine — it is how to build a machine that gives more people a way in.