A Joystick-Controlled Granular Pitch-Shift Effect (written in Pure Data)

This video is a demonstration of a pitch-shifting granular effect with some delay properties. It serves as a proof-of concept on how a joystick could serve as a musical interface to engage with sound in a creative and meaningful way.

As the concept of joysticks is familiar to many, and intimately so for avid flight simulator enthusiasts, it could serve as an intuitive interface to interact with parameters of musical expression.

However, to include meaning into movement, one should carefully consider the mapping of parameters, thus it makes sense to start with an overview of the available features.

The joystick used here is the Logitech Extreme 3D Pro. It comes with a range of different control elements:
6 buttons on the bottom
A throttle / rotary knob
A trigger + another trigger at the thumb position
A hat switch on the top with 4 additional buttons
The actual joystick with detection for the pitch, pan and yaw axes.

The underlying general idea used for this iteration of the project is to sort the control elements by supposed expressivity, from allowing a wide range of values to simple On/Off messages:
Pitch / Pan / Yaw movement
Throttle
Hatswitch, Triggers, Buttons

Pitch , Pan and Yaw are treated as individual dimensions routed to single musical parameters for this iteration rather than a combined 2D/3D mapping of multiple output synthesis parameters. Future side projects will focus on mapping the 3D space to multiple synthesis parameters using machine learning.

Due to the close proximity of the resting position of the thumb and index finger, the trigger and thumb trigger can be used while engaging with the 3 dimensions. This led to the inspiration for two different concepts: Using the thumb trigger as a value gate for the pitch axis, sending only pitch values while engaged, and the index trigger as a value snapshot for the other axes, only letting through the value at the moment of interaction.

This allows the pitch axis to be treated similarly to a whammy bar found as a vibrato device in many electrical guitars, controlling a main sound parameter directly. Within the granular effect at hand, it will control the pitch (=frequency) of the grains.

The snapshot mechanism here allows for a fast dial-in of an additional effect, inspired by how audio effects are sometimes introduced only for a certain section within a song as a means of emphasis/deemphasis. The axes will be used here to each control the volume of additional granular "voices", one pitched one octave up and one down.

The throttle presents itself as a mixture between a classic rotary knob and a slider, which lends itself to be used as a fader-like control element. Here, it is used to fade between the dry signal and the wet granular effect.

The hatswitch here is used to increase or decrease the size of the grains used within the synthesis process. However it should be noted, that during testing this proved to be undesirable, as fast, expressive changing of the grain size becomes hardly possible.

The remainder of the buttons are to be used as activation switches, increasing the range of the pitch values as well as panning the secondary voices into fixed positions.

This setup can be used in a multitude of possibilities, some of which are presented in this video:
Voice alteration
Percussive / transient effects,
As a live effect for acoustic guitar

A setup which will be explored in the future, possibly with a slightly altered synthesis setup or parameter mapping is having the joystick mounted on top of an electric guitar to use the right hand for plucking the strings and controlling the effect.

As always, if you have any questions or remarks, please write them in the comment section!

Thanks!

PS.: The patch includes the hid-object by Hans-Christoph Steiner, as well as a heavily modified version of the live granular synthesis patch from pd-tutorial.com (3.7.2.1) by Johannes Kreidler.