← All projects
2022NIME 2022

Interactive Musical Score

A touch-sensitive paper score for exploring the relationship between musical notation and sound. Published as a demo at NIME 2022.

The Interactive Musical Score: An Electronic Score To Hear The Notes From A Paper Score

Adrien Lefevre, Xiao Xiao, and Clément Duhart

International Conference on New Interfaces for Musical Expression · Demo · June 22, 2022

DOI: 10.21428/92fbeb44.5a2b68f0

TapTheScore combines a replaceable paper score, printed conductive traces and capacitive sensing. Touching a note triggers its pitch, making the printed score an interface for musical exploration.

  • NIME 2022
  • Tangible Interaction
  • Music Technology
The assembled interactive paper score and its sound electronics.

Research objective

The project explores how a paper score can help children connect written notation with sound. Pressing a note triggers its pitch, and moving between notes produces a melody.

The design retains the physical format of a score while adding immediate auditory feedback. It was presented as a demonstration at NIME 2022.

Project videointeractive-musical-score__video_01

Interaction rationale

Reading notation requires a learner to associate symbols with pitch. TapTheScore makes this association directly accessible through touch, allowing a learner to select and compare notes without first operating a separate instrument.

The prototype explores an interaction technique for music education. Its educational objective should be distinguished from a demonstrated improvement in learning or retention.

Tangible interface

A cardboard score is placed over conductive traces printed on a flexible substrate. Holes at the note positions expose those traces to the user’s finger. The connected electronics detect contact through capacitive sensing and generate sound.

The paper layer can be changed independently of the electronics, allowing different scores to use the same sensing surface.

System overview

The diagram below shows the relationship between the printed score, sensing substrate and sound-generating electronics.

System overview: paper score, conductive lines and electronic circuit.

Sensing and electrical connections

The upper layer contains the printed notation and note cutouts. Beneath it, conductive lines carry the touch signal to the electronics. A PCB connection using conductive z-tape was explored to transfer signals between the flexible substrate and circuit.

Touching a note through the cardboard changes the capacitive measurement. The microcontroller maps the detected contact to a pitch.

Score fabrication

Notation, staves and cutouts were designed together in Photoshop so that the printed and cut layers aligned. Separate exports supplied the printable score and the cutting pattern.

Printed score on the left and the corresponding note cutouts on the right.

Layer alignment and materials

A score of “J’ai du bon tabac” was used for the prototype. Note cutouts were prepared in Cricut Design Space and cut with a Cricut Maker 3.

The sensing layer used conductive lines on a Kapton polyimide sheet. The flexible substrate supported the printed traces and could tolerate the fabrication process used for the conductive ink.

The flexible polyimide substrate with printed conductive lines.
Conductive lines connected to the Circuit Playground board.

Contact prototype

An early connection method used a binder clip containing six exposed wires positioned against the conductive lines. This provided a removable electrical contact between the sensing sheet and the sound electronics.

Early prototype of the wired binder-clip connection.

Sound generation and outcome

An Adafruit Circuit Playground Classic detects capacitive input. A CircuitPython program maps that input to generated notes and supports changes in timbre.

The outcome is a functional tangible music interface and a NIME demo publication. Further work identified in the project includes smaller electronics, additional controls and evaluation in a learning setting.