Jordan Rudess and the MIT jam_bot

A brief introduction to Jordan Rudess’s music career, sample works, and his collaboration with MIT researchers Perry Naseck and Lancelot Blanchard on jam_bot.

Jordan Rudess with MIT jam_bot (image)

Jordan Rudess Brief Biography

Jordan Charles Rudess (born November 4, 1956) is an American keyboardist and composer best known as the longtime keyboard player for Dream Theater, one of the most prominent bands in progressive metal. Widely regarded for his virtuosic technique and innovative use of music technology, Rudess has blended classical training with rock and electronic experimentation throughout his career.

A child prodigy, Rudess entered the Juilliard School Pre-College Division at age nine, where he studied classical piano. Rudess was a member of the progressive rock supergroup Liquid Tension Experiment alongside bandmates who would later recruit him into Dream Theater in 1999. Rudess has contributed to numerous acclaimed albums, and has regularly released solo albums.

Rudess was named “Best New Talent” by Keyboard magazine (1994) and has repeatedly been voted among the top keyboardists in progressive rock and metal polls. Beyond performance, he is also known for pioneering expressive music apps and interactive instruments (e.g. MorphWiz, GeoShred, and SampleWiz) reflecting his ongoing interest in the intersection of music and technology. His interest in the intersection of music with AI technology is reflected in his work with MIT on jam_bot.

Sample of Keyboard Works

Work with MIT on jam_bot

About the jam_bot project:

Jam_bot is a live-performance AI model developed through a collaboration between Jordan Rudess and the MIT Media Lab that allows real-time improvisation performance between a musician and a generative AI system that utilizes music Language Models. The collaboration was supported through the MIT Center for Art, Science & Technology via Rudess’s visiting-artist residency. Rudess’ main collaborators in the project are Media Lab graduate students Lancelot Blanchard and Perry Naseck, with Professor Joseph Paradiso, head of the Responsive Environments group, overseeing the project.

About the system:

This jam_bot system is based on the concept of "symbiotic virtuosity" [Blanchard et al, p.10], a form of virtuosity that emerges when human and machine duet in real time, influencing each other musically rather than having AI generate a finished track offline by copying a human’s musical style.

Jam_bot is developed based on music transformer - a neural network architecture that predicts what to play next, much like language models predict the next word, except here it generates musical phrases in real time. The system was fine-tuned on Rudess’s own recordings, so it can reflect his improvisational style while still creating new ideas simultaneously. Acting as an AI-augmented instrument, the system is built for live performance rather than an offline song generator, where artists cannot really control the output. With jam_bot, Rudess stays in control, and the model reacts to what’s happening moment by moment.

Human-AI Interactive Live Music Performance:

On September 21st, 2024, the MIT Media Lab hosted a live concert featuring GRAMMY-winning Jordan Rudess improvising on 6 distinct pieces alongside the jam-bot system. To manifest and visualize the generated outputs of the AI system, they constructed a 16-foot-tall kinetic sculpture integrated into the live stage performance.

To Rudess, jam_bot is far more than just a Generative AI system. It can not only mimic Rudess’s improvisational style but also acts as a co-performer that can inspire him during the performance. The piece is completed through a live, back-and-forth exchange in which both Rudess and the system respond to each other’s lines, and the performance only comes together through that mutual feedback.

Useful Links: