Video Games and Jazz Composition

Cinema vs. Music

There's been a fair amount of noise in the last decade and a half suggesting that the starting point in the evaluation of the expressive capacity for videogames should be cinema.  From the mini-explosion of FMV games that accompanied the rise of CD-ROM through Quantic Dreams's gorgeous interactive drama "Heavy Rain", we've seen the emulation of cinema (cinematic devices, cinematic effects and cinematic emotion) regarded as a key touchstone of artistic merit.  While there are plenty of dissenters, many of them take the view that videogaming can be "more" than cinema; its interactive elements rendering the fledgling medium potentially superior, but still of a kind with the relatively recent but artistically "proven" medium of film.

Of course there are many nuanced positions on the subject, but I believe continual use of the language of cinema will ultimately restrict games, presupposing design solutions by virtue of having internalized the problems of film in our vocabulary.  The argument could be made that games need their own, unique vocabulary far removed from the language of other media, but I believe there is merit in analyzing and incorporating elements, concepts and solutions from other media.  I'm not disputing the usefulness of the cinema metaphor, only its predominance.

Therefore I'd like to suggest a different metaphor for informing the creation and evaluation of games: Jazz Composition. 


I'd be remiss here not to link to Chris Dahlen's Edge article, "You Can Keep The Popcorn" which in some ways renders this article redundant (and is probably way more entertaining), but this has been kicking around in my head for a long time, and I don't have any readers yet, so here I go anyway.  As above, my descriptions are tremendously simplified for clarity.  Since recent polling on the subject suggests that something like 90% of adults have never attended a jazz concert, I'll also assume that the reader doesn't know much about jazz, and summarize its breadth in a few reductionist paragraphs.

Jazz Values

Jazz is a performance art.  While the advent of recording technology allows us to indefinitely archive individual performances, jazz as an art form is about the moment, the performers' ability to spontaneously create, arguably more than any other form of Western music.  While improvisation is enshrined and lauded in plenty of other musics (blues, rock, bluegrass, classical, etc.), in jazz the focus is almost entirely on the spontaneous invention of the soloist and his interaction with the (also-improvising) musicians with whom he is performing.  A jazz performer is expected to improvise, in the moment, and not to repeat himself from performance to performance.  Stories are legion about smartass rhythm players or horn sections playing along with or reharmonizing the solos of any performer who dared to write out his solo in advance.  Invention during performance is a CORE VALUE of jazz.

A musical composition, on the other hand, is a relatively static document.  It usually consists of a written melody with a set of accompanying written harmonies, to a variable degree of specificity. 

The art of jazz composition, then, is a serious balancing act.  Tunes become popular jazz tunes (jazz "standards") only when they provide a rich platform and context for individual expression.  Traditionally, tunes don't need to be in the jazz idiom in order to become jazz standards (in fact the early decades of jazz pulled many standards from Broadway show tunes and Tin Pan Alley), but the compositions which persevere in the jazz canon absolutely need to provide sufficient material to be compelling on their own and feed the imagination of the performers.  However, they also need to stay out of the way enough to allow the voice of the performer to be heard.  People didn't visit the clubs to hear "My Favorite Things," they went to hear John Coltrane performing "My Favorite Things." 

The jazz composition proper, then, provides a melodic and harmonic framework for improvisation within an individual performance.

Jazz Performance Conventions - Small Group

Imagine a group of four men in sunglasses taking the stage in a small, smoky club. Bass, drums, piano, sax.  The sax blares out a familiar melody for about a minute and a half, then the four men take turns leading the combo through ever more complex solos until you hear the group return to the original melody and everyone fades out.

In traditional small jazz combo performance, the composed theme (the "head", or the tune's melody and harmony) is played by the group before the soloists improvise over the harmonic framework using the melodic material as a thematic resource.  During the solos, the rhythm section improvises over the written harmonies, sometimes outlining the harmonies precisely, sometimes "substituting" other harmonies, sometimes responding to the soloist's improvisation by departing from the written structure.  Generally, following several repetitions of the song structure, the group restates the composed theme and fades out or ends the tune.   A lot of musicians diverge from these practices (or else it wouldn't be an especially interesting model with respect to games), but as often as not, when a small group performs jazz compositions, even in "free jazz," the shape of the performance follows the above guidelines.

It tends to be a somewhat loose, flexible affair, with a fluid relationship to pre-composed material and focusing mostly on entended individual statements.

Jazz Performance Conventions - Large Group

Now imagine a 16 piece group like you've seen in old black & white pictures taking a bandstand in a ballroom or concert hall.  There are risers, with groups of trumpeters, sax players, and trombonists all sitting together.  The bandleader is on piano and counts off the tune. The group launches into it and varies between blaring power, quiet complexity, and dramatic crescendos, returning to rearranged bits of melody throughout, generating an ever-shifting canvas on which each soloist leaves his statement.

In traditional large group, or "big band" jazz performance, a fair amount more attention is paid to planning the context in which individual expressions are made than in traditional small group performance.  Each section of the large groups tends to have its accompanimens planned out from moment to moment.  Big band arrangements tend to be elaborate, with each non-solo, non-rhythm section part written out from start to finish, including dynamics and figures to be played behind soloists.   Even the rhythm section, which tends to be given more leeway than the melody instruments, is expected to stay precisely within the harmonic guidelines of the composition to avoid clashing with the elaborate arrangements.  Even the soloist is generally expected to follow the contour of the accompaniment, and meet the overall dramatic arc of the dynamics and rhythm the rest of the large group os playing behind his solo.

Part of the reason for this lack of freedom is the sheer number of musicians involved in a large group; while playing music with any harmonic or rhythmic complexity, significantly more control is required in order to make the music coherent.

What the large group affords, however, is a broader range of dynamics, texture and drama.  In particular, an individual soloist's statement is given a much richer context, with many more harmonic and rhythmic elements at play than are possible in a smaller group.  Many jazz musicians speak of soloing over a big band as akin to riding an enormous wave; the soloist is buoyed and swept along by the sheer strength, size, and directionality of the larger group.  Instead of, say, the meditation on motivic development generated in a small group context, a big band performance often allows each solo to be a dramatic setpiece, with the complicated arrangements lushly setting the stage for an individual's statement.  Though the individual statement is more constrained, lesser soloists are elevated by the presence of the larger group, and master performers are afforded a more powerful setting for the soloist than would otherwise be possible in even the most telepathic small groups.

Gamers as Performers and Audience

Videogames are in a fairly unique position among expressive media in that the gamer represents consumer, audience, and performer.  The gamer is an active participant in a game, the co-author of the action occurring on the screen.  The gamer is entertained by the spectacle unfolding on the screen before him, but is also challenged to rise to the occasion, to co-create the work of art being consumed in the moment.

Much has been made of giving the player choice in order to increase his sense of buy-in, of increasing the feeling that he is an active participant rather than a simple consumer.  A depressing amount of that literature, however, seems to be devoted to how to deceive the player into believing that is true rather than making it true.  Discussions of lighting or scene setting often have to do with providing "cues" to the players based on visual conventions everyone is familiar with in order to draw attention to the ideal, most dramatic path prescribed by the game designer.  You'll often see the phrase "the illusion of choice," the designer's job being to trick the player into feeling like a participant rather than being a participant.

I believe that the player's role should be considered an artistic role, akin to a jazz performer.  For the player, playing the game becomes a personal expression.  And viewed in this way, the role of the designer becomes akin to the role of the jazz composer.

Designer as Jazz Composer

If the function of the designer is to enable the player to co-author a personal expression, a game can be seen as a jazz composition.  A game becomes the player's context, a framework responsible for detailing the rules and possibilities for player performance.  It should provide sufficient material to give the player a compelling set of choices, and stay far enough out of the way to allow the player the leverage to make a meaningful statement.  With in-the-moment improvisation as a CORE VALUE, the designer's job is to enable that improvisation; to give the player enough material to make improvisation meaningful, but not to crowd him with an ideal path through.


Note that I'm making no mention of narrative here.  Because so much game literature is so heavily focused on drama and film, "meaningful" choices are often described solely in terms of narrative.  But a game need not even have a narrative to provide meaningful gameplay choices, especially provided that the player is empowered to make choices that are purely expressive.

If we view genre, scale, or relative freedom as performance contexts, we can make sense of the designers' problems as akin to those of the jazz composer.  Jazz Compositions are often designed for a specific context.  While jazz compositions can be "arranged" for small group, large group, duet, etc, many jazz composers keep the size and character of the intended performance in mind while writing the music.

As such, you could view something like a simulation rpg or true sandbox game as analogous to the freedom and flexibility afforded in a small group context.  The direction and thrust of the game are determined by the player's interpretation of where the rules should take him.  The pace belongs to the player, the high points are generated by the player.  The game world is rich enough to react to the player's cue, like a solid backing band in a small combo.  The moment-to-moment choices choices not fully prescribed, because in this context, flexibility is the driver behind the core value of enabling individual expression. The core of the game can be judged on whether thetoolset it provides players is compelling and provides significant possibilities for personal expression, and that toolset is only meaningful in the prescribed setting.  The realization of the game can be judged on how well/satisfyingly the game world responds the the player's use of that toolset.  If the player can derive the same satisfaction from the game's reaction to his statements that a good player can get from playing with a sympathetic and responsive band, I'd suggest that the game's context is successful; the game is well realized.  By pushing the boundaries of responsiveness to player actions AND the content of the player toolset  (the improvisatory vocabulary) AND the setting for the game, and by viewing them as related-but-separate, I think games in this context and as a whole can advance in new directions. 


A game like Uncharted, however, with its scripted dramatics and crescendos would be analogous to an epic big band piece.  The player has room for improvisation, but the context is VERY specific, and the thrust and direction are determined for him.  A poor player can ride the wave of each setpiece, and may or may not DO anything especially memorable, but the context in which he is allowed to take action imbues even fairly lackluster performances with expressive power.  An excellent game of this type should elevate the poor player, but should also allow the excellent player to create incredible moments, in which the fairly limited toolset can be fully realized to add drama, hilarity, or spectacle to a player's active role in the greater narrative.  As above, we can separate the player's toolset from the responsiveness of the world from the setting.  And we notice that in a game with this focus, setting takes the most prominent role, followed by the player's toolset.  Game world responsiveness is restricted because the toolset is restricted, but the relationship between them can be evaluated. The core of the game is its toolset and its possibilities plus the highly promninent context/setting.  As above, evaluating the realization of the game relies on judging the connection between player action and game world response.  This is important, because while it might be narratively satisfying to reward completion of a setpiece with, say, a cutscene or important piece of plot, in this context it has nothing to do with the connection between action and response I've identified as important to game realization.  Going from point a to point b isn't usually a choice, so the sensory rewards dangled in front of the player for doing so are unrelated to the question of the realization of the game rules.  Things like enemy response, effectiveness of unusual tactics, paths through platforming sections, and so on are germane to the connection between player choice and response, and are this germane to this realization.  These actions are given relevance and weight by their narrative or visual framework, and it would be dishonest to evaluate the work as a whole in the absence of those things, but by abstracting in this way we open the door to new types of experimentation without discarding elements essential to gaming as a medium rather than as a platform for competition.

In this view then, a solid game, at its core, should provide significant expressive possibilities in any context, though one context may be viewed as richer or as a better fit.  The choices and contexts given to players must be evaluated in terms of both the richness of their inherent ideas, and the expressive possibilities they afford to the player.  The player is both performer and audience, and both of these aspects need to be satisfied.  To that end, we can begin to leverage the types of experiments jazz composers have pursued over the years, both in hybridizing the performance conventions of different jazz contexts and experimenting with the core of what kinds of rules enable meaningful individual expression.