EMERGENT SYSTEM is a body of work including 1001 drawings on 60 sheets of paper—and an experimental animated film, synchronized to music by Missy Mazzoli, with choreography by Faye Driscoll.

Themes of systems, patterns, rules—and our human inclination to push against them—weave throughout the work: in Mazzoli’s music, the techniques through which the animation is created, and the systematic nature of the drawings.

The approach to narrative in the film EMERGENT SYSTEM evolves as the piece progresses—it begins abstract and concludes with a story, but dwells mostly in the liminal space between the narrative and non-narrative filmmaking. The animation techniques reach back in time in the history of the form to early 20th century "visual music" works by artists like Mary Ellen Bute and Oskar Fischinger, and all the way forward to modern day machine learning and AI. The threads it weave form a parable about the nature of collaboration itself, the human mind, and the point at which logic and rules can become poetry.

This work was commissioned by PEAK Performances at Montclair State University, and debuted there with music performed live on six grand pianos by Grand Band. In conjunction, an exhibition of drawings from the work, organized by the Office of Arts + Cultural Programming at Montclair State University in collaboration with the George Segal Gallery, was mounted in the Alexander Kasser Theater.

Accessibility
The film includes some flickering and striped patterns including the color red, in the first movement and at the end.

Below you'll find—
1. Drawings
2. Themes and Process
3. The Three Movements
4. Machine Learning (Finale)
4. Context
5. Exhibition
6. Book
7. Credits

Drawings

For each shot in the film, all the animation "frames" are drawn on a single piece of paper, in a diptych, or, in one case, a triptych. Filmmaking itself is one of the subjects here. The project is, in part, a consideration of a shot, or a moment of animation, as an object with form.

Carefully edited fragments of bodies in motion are rendered into works on paper. These works enable us to look through the eyes of an animator and consider the start and end points of an action, as well as the subtle changes that occur in our bodies across a period of time that create motion, meaning and beauty.

To stage the bodies in motion, I worked with Driscoll and four dancers to devise and shoot footage of a series of relationships and actions. The ideas in Mazzoli’s music, my initial concept for the film, and Driscoll’s unique perspective and practice served as starting points.

Themes and Process

The film is a series of layered collaborations and transformations. Forms and colors move in systems that arise from Missy's music. Bodies appear, manipulating the forms (and each other). I shot footage of the choreography, and made 1001 drawings based on the footage. The rotoscoped drawings repeat and become patterns themselves. A woman tries to change the patterns around her. Finally, frames from the film itself are reinterpreted by a "machine learning" computer algorithm (more on that below). At each step, a pattern emerges and a rebellion is staged against that pattern.

The act of recognizing patterns, and patterns of patterns, is a model of how brains work used in both neuroscience and artificial intelligence. When we imagine our minds at work, somewhere in the stack of discrete patterns the mechanics become too complex to contemplate and give way to emotion.

The film is divided into three movements.

I
The first movement begins with music driving patterns of form, color and rhythm. The forms are inspired by the American quilt tradition and the punch-cards used in looms and early computers—historical modes of using code for creative work, tied deeply to female creators. In each scene a set of rules is established, and a form pushes against that system. I created “rules” by writing computer code connecting MIDI data from Mazzoli’s music to Adobe After Effects animation software; I then “broke” these rules by disconnecting that code and introducing traditional keyframe animation. These scenarios point towards early 20th century "visual music" works by artists like Mary Ellen Bute and Oskar Fischinger. By the end of the first movement bodies begin to appear, most notably Mazzoli’s own hand.

II
The second movement is concerned with relationships between bodies and relationships between the pianos. The pianos and the people are trying to get in sync (or perhaps pull apart).

III
In the third movement, a narrative begins to emerge. A woman awakens in a world constructed from the elements introduced in the previous movements. She attempts to affect the world—to make it better.

Machine Learning (Finale)

The work's final 20 seconds was generated by machine learning — a primitive form of artificial intelligence — trained on a dataset consisting only of frames from this same film.

The machine learning then rendered animations that it “believed” would look like the sequences it was trained on.

I worked closely with Professor Andrea Danyluk and Chan Woo Kim at Williams College to create the machine learning program. We used the generative adversarial network (GAN) model.

Collaboration between artists working in different media is at the heart of EMERGENT SYSTEM; this final “collaboration” with a computer algorithm reflects that theme, and opens up new ways of considering our relationships with technology—in the past, present and future.

This finale points towards the notion of "The Singularity", popularized by Ray Kurzweil—the moment when artificial intelligence will surpass human intelligence, beyond which the nature of human civilization is difficult to fathom. In the parable of this film, the world beyond the Singularity is made from pieces of our world, remixed by machines. To make that future world better, we must repair our own world now, before our time runs out. Our world is the stuff the future will be made from.

Context

This project is part of a larger artistic body of work of mine: experiences that combine animation with live music, created in collaboration with composers. The combination of visuals and music, in dialogue with each other, immerse audiences deeply in the ideas and aesthetics at the core of each work. In a time when cinema can feel disposable—something we can order up or discard on our phone on a whim—these kinds of live experiences feel particularly resonant.

Simultaneously, I am also exploring ways to capture animation in physical works-on-paper.

My other projects in this body of work include PLAN OF THE CITY, MANNAHATTA, and A MARVELOUS ORDER.

Exhibition

A selection of drawings, GIFs and videos from this body of work were presented in a solo exhibition by Standard Space, a boundary pushing gallery in Sharon, CT, in 2021. At the opening composer Missy Mazzoli performed and the film was screened—both with quadraphonic sound, designed by sound artist Ben Manley.

Book

Drawings from this project have been included in a limited edition book, published by the Wassaic Project, titled Now, More Than Ever. The drawings are given their own spread as well as included as flipbooks in the corners of the pages.

The way these flipbooks render animation in a physical form that the viewer can hold and touch fits wonderfully with the goals of this project.

The video below was shot by Small Editions, the studio that produced the book. Buy a book here.

Credits

Emergent System
Directed and animated by Joshua Frankel
Music Three Fragile Systems by Missy Mazzoli
Choreography by Faye Driscoll
Music performed on six grand pianos by Grand Band
Pianists Erika Dohi, David Friend, Paul Kerekes, Blair Mcmillen, Lisa Moore, Isabelle O'Connell
Dancers Lyric Danae, Sean Donovan, Paul Singh, Eliza Tappan
Director of Photography Clayton Combe
Deep Learning Engineer Chan Woo Kim
Assistant Animator Yupu Ding
Thank you Mana Contemporary, Andy Hamingson, Dr. Andrea Danyluk, David Van Brink, Jason Baruch, Carter McGowan, Penina Biddle-Gottesman, Justin Edward Call, Ryan Brack and Eve Biddle

Commissioned by PEAK Performances @ Montclair State University (USA)