<nettime> DEFINING MULTIMEDIA (3/4)

[Ken Jordan <ken@kenjordan.tv>]

Ken Jordan
DEFINING MULTIMEDIA
(3 of 4)

[Note: This is part 3 of a paper-in-progress that grew out of my
collaboration with Randall Packer, Multimedia: From Wagner to Virtual
Reality (W.W. Norton, 2001, and on ArtMusuem.Net). Part 1 proposed a
definition of digital multimedia based on five core characteristics. Part
2 compared our definition to the one proposed in Lev Manovich's The
Language of New Media. Part 4 will be posted soon. Comments are welcome.]


3. The Modernist Thread

For the purpose of our project, Randall and I felt that the term "digital
multimedia" seemed to be the most appropriate -- rather than "new media,"
"digital media," etc. -- because it emphasizes the form's continuity with
efforts in the arts that came before. The word "multimedia" was coined by
artists in the 1960s to describe avant-garde practices that not only mix
diverse media, but also emphasize audience participation, non-linear
narrative structures, and indeterminacy. There is a line in the
development of computer-based media that runs parallel to an important
trajectory in modernism. We want to make that connection explicit.

This is not to say that digital multimedia grew out of a cohesive,
carefully coordinated strategy. But looking back, you can identify a few
consistent themes that drove the medium's development over a half century.
These themes were pursued concurrently with other, at times conflicting,
objectives. But in retrospect the extent of a consistent vision shared by
the scientists and artists who pioneered multimedia is quite profound --
as is the mutual influence between science and art (with conceptual and
technological breakthroughs feeding one another) that led to the
computer-based media we know today. Eventually these diverse efforts
coalesced into a meta-medium, to borrow a phrase from Alan Kay. [1] Kay is
the man who tied the loose threads of digital multimedia together in the
late 1960s, by designing the prototype for the first true multimedia
computer, the Dynabook.

Vannevar Bush began it all by proposing a mechanical device that operated
literally "as we may think." [2] The challenge, as he discussed it in his
famous article of 1945, was to create a machine that supported the mind's
process of free association in the act of creation. This aspect of Bush's
hypothetical machine, which he dubbed the memex, tends to get overlooked
today. What gets attention instead are the many ways the memex foreshadows
the personal computer -- particularly its ability to call up media objects
from a database. Bush did not use the word "database," because the memex,
as he described it, was not a digital device. It was analog: a desktop and
storage space that gave access to microfilm, audio recordings,
photographs, and movies. It was, in a way, a kind of library -- but with a
crucial difference.  Libraries arrange information linearly. Bush,
however, was interested in rearranging information according to the
idiosyncratic paths of personal association that each individual invents
during the creative process. He wanted a machine that encouraged
spontaneous, associative, stream-of-consciousness thinking, and then left
a trail of that thought process behind so that it could be retrieved, not
only by the individual who created it, but by others as well. In this way,
the memex would allow people to share their private, unconsidered thoughts
as they leap between ideas moment by moment.

Bush was interested in identifying a central aspect of consciousness, and
making a device that effectively expanded consciousness through mechanical
means. If you look at the history of the personal computer from this
perspective -- as an ongoing project to create a media machine that
enhances the intuitive, associative tendencies of consciousness -- it
connects digital media inextricably to important currents that run through
modernism.

Bush had taken, essentially, an esthetic position -- an esthetic position
that shares remarkable qualities with some unexpected bedfellows. These
are contemporaries with whom Bush is never associated, particularly as he
was FDR's chief science advisor and the architect of the military
industrial complex. Still, as the person who proposed that information
should be organized and saved mechanically in a way that captures the
spontaneous movement of the mind, it is inevitable that he should be
linked to others who shared similar interests in mid-century.

For example, during the 1940s Charlie Parker was pioneering a new musical
vocabulary based on spontaneous improvisation -- one that went far beyond
the method established by Louis Armstrong. Parker's radical approach to
improvisation, the charts be damned, placed non-linear associative
thinking above all else in jazz, and led to the free jazz of John
Coltrane, Ornette Coleman, and others in the 1960s and 70s. In painting,
Jackson Pollock was taking a similar approach at the time, dripping paint
in loops following the dictates of his spirit, never following a plan or a
sketch. The privileging of spontaneous action was central to Pollock's
practice. In literature, during these same years, Jack Kerouac pursued a
method of "spontaneous bop prosody" -- as he called it -- that led him to
write novels that captured the movement of his mind moment-by-moment in
the act of creation; a steady stream of honest personal observation that
used associative thinking as its central organizing principle.

The prim bureaucrat Vannevar Bush might have been surprised to find
himself in such unkempt, but august, company. However, looking back the
similarities between Bush and the mid-century American avant-garde are
obvious. They shared an esthetic that treats the individual's private
impulse as primary, and that gives people permission to act in a
non-linear, irrational way, as society would define it. Bush's interest
was to enable each of us to shape data into the form that serves us best,
rather than to conform our private thought process to an organization set
by others. This opposition between self and society is not absolute, of
course (though in mid-century the tension between private impulse and
social conformity was an intellectual flash point, especially because of
the threats of Fascism and Stalinism, on the one hand, and the theories of
Freud, on the other). That digital media can trace its birth to the intent
to mine this opposition, however, is significant.

Bush's vision inspired a generation of computer pioneers in the 1960s, and
led directly to the personal computer. Douglas Engelbart, for one, was
famously inspired by "As We May Think," and dedicated himself to building
a working model of Bush's association machine -- this during the same
years that Coltrane, Pollock, and Kerouac (not to mention their many
cohorts, and the legions of young artists they inspired) had broken
through to the mainstream. The assumption that "great art" was made
through the formal arrangement of spontaneous impulses was not only the
mantra of cultural bohemians; it was a notion hotly debated in the popular
press. The birth of the personal computer belongs to this moment.

Engelbart expanded on Bush's premise by designing an oNLine System that
would "augment human intellect," as he put it, [3] based on the insight
that the open flow of ideas and information (as represented by texts and
pictures) between collaborators was as important to creativity as private
free association. At the same time, J.C.R. Licklider envisioned universal
networked access to the full "library" of human knowledge. This idea led
him to spearhead the early development of the Internet while he ran a
research program for the Defense Department, ARPA. Soon after, Ted Nelson
followed with a proposal for a "hypermedia" system (he coined the term)
that would fulfill Bush's objective to arrange materials from this
"library" in a manner that reflects how the mind moves freely from one
thought to another. [4]

Central to all these efforts was the notion that the user should not only
have access to media objects, so she can organize them as she pleases, but
that the computer user should also be able to interact with media objects,
and change them to suit the needs of the moment. Editing and recombining
digital media was seen as essential to the utility of the computer.
Licklider, in his seminal article "Man-Computer Symbiosis," [5] proposed
that the computer should act as an extension of the human capabilities for
cognition and communication -- which includes, of course, the manipulation
of media. Engelbart's oNLine System was designed specifically for the
collaborative manipulation of digital media over a wired network. In
keeping with Bush's vision of the memex as a way to enhance creativity,
these pioneers insisted that the computer user's ability to interact with
and change media should be as great as possible. Tim Berners-Lee has often
said that he considered the edit function in the first Web browser to be
just as important as the ability to link between Web pages; for the Web to
be successful, he felt it essential that each reader could also be an
author, able to annotate Web pages by adding "private links." [6]

This approach to interactivity paralleled currents in the avant-garde,
particularly in performance. In 1948, John Cage introduced the idea of
live performance as unscripted event, in which the audience encounters
people, objects, and activities within a defined space, in surprising
juxtaposition to one another. The audience is encouraged to become
creative participants in the work of art as it occurs. [7] This type of
performance, which Allan Kaprow later named Happenings [8], shared many
concerns with the way engineers were shaping online interactive
environments. Both engineers and artists were addressing the question: how
do you encourage the appropriate dynamic encounter between people within a
framed situation? And they reached a similar conclusion: give the
user/participant as much freedom to act as possible.

Implicit in Bush's memex is the suggestion that a mechanical device can
replicate the intimate movement of the mind at play, by representing media
objects of all kinds in any order, as the user desires. From this, it
follows that a computer might one day effectively mimic the encounter of
consciousness with the world through the senses, by arranging media
objects in a way that mimics reality. Though Bush himself did not make
this leap, engineers influenced by his vision in the early 1960s did, and
none more profoundly than Ivan Sutherland.

Sutherland was the first person to propose that bits and bytes could be
represented as three-dimensional virtual environments. In his article from
1965, "The Ultimate Display," [9] he began with the idea that by
digitizing information -- transforming it into ones and zeros -- all data
became subject to the graceful manipulations made possible by mathematics.
This, in turn, invites the computer programmer to shape data into a
three-dimensional form that mimics the way we encounter information in the
physical world. Like Bush, Sutherland's approach to the formal arrangement
of information is essentially an esthetic stance. This particular esthetic
stance can be traced back to the mid-19th century writings of Richard
Wagner, which declared that art should do its best to recreate the full,
multi-sensory engagement between the self and the world. To facilitate his
vision, Wagner reinvented the conventions of the opera house, and in 1876
opened the Festpielhaus Theater in Bayreuth, Germany. It was the first
modern theater to employ Greek amphitheatrical seating, surround-sound
accoustics, the darkening of the house, and the placement of musicians in
an orchestra pit -- all to focus the audience's attention on the dramatic
action, and transport them into an illusionary world staged within the
proscenium arch. Wagner's call for an immersive "collective artwork" that
fuses all the arts into a single expression [10] -- his "Gesamtkunstwerk"
-- is echoed in the last paragraph of Sutherland's 1965 paper:

"The ultimate display would, of course, be a room within which the
computer can control the existence of matter. A chair displayed in such a
room would be good enough to sit in. Handcuffs displayed in such a room
would be confining, and a bullet displayed in such a room would be fatal.
With appropriate programming such a display could literally be the
Wonderland into which Alice walked." [11]

Sutherland presented this paper at an engineering conference, and it was
first published in a technical journal. But it is hard to ignore how much
it reads like a manifesto written by an Italian Futurist. There is, in
fact, a remarkable similarity between the tone and intention of articles
by certain computer media engineers and fiery artistic manifestos. The
modernist imperative to "make it new" (in Pound's famous phrase), and the
belief that society will be transformed as a result, is very much present
in writing by computer scientists. Digital multimedia may well force us to
reconsider the entire historic arc of modernism, including its supposed
end, since the esthetic stance of modernism has become increasingly
relevant in response to digital media.

When Alan Kay designed the prototype for the Dynabook, in the late 1960s,
the intellectual foundation was in place for a digital multimedia that
synthesized all existing art forms, and presented them in an environment
that enabled meaningful interactivity and hyperlinks. With the requisite
processing power, it would eventually incorporate Sutherland's experiments
with three dimensional representations. This meta-medium, to use Kay's
term, carried with it specific, idealistic attitudes and intentions about
human creativity and communications. It reflected a commitment to media
forms that are nonhierarchical, open, collaborative, and emulate the free
movement of the mind at play. It is, in sum, an extraordinary vision.

-----

Notes:

[1] Alan Kay and Adele Goldberg, "Personal Dynamic Media," in Multimedia:
>From Wagner to Virtual Reality, Randall Packer and Ken Jordan, eds. (New
York: W.W. Norton, 2001), p. 167

[2] Vannevar Bush, "As We May Think," ibid, p. 135

[3] Douglas Engelbart, "Augmenting Human Intellect: A Conceptual Framework,"
ibid, p. 64

[4] Ted Nelson, "excerpt from Computer Lib/Dream Machines," ibid, p. 154

[5] J.C.R. Licklider, "Man-Computer Symbiosis," ibid, p. 55

[6] Tim Berners-Lee, "Information Management: A Proposal," ibid, p. 189

[7] John Cage, "Diary: Audience 1966," ibid, p. 91

[8] Allan Kaprow, "Untitled Guidelines for Happenings," ibid, p. 279

[9] Ivan Sutherland, "The Ultimate Display," ibid, p. 232

[10] Richard Wagner, "Outlines of the Artwork of the Future," ibid, p. 3

[11] Ivan Sutherland, ibid, p. 236


------------
Ken Jordan
ken@kenjordan.tv
212-741-6173

"Be as if." - Andrew Boyd 


#  distributed via <nettime>: no commercial use without permission
#  <nettime> is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: majordomo@bbs.thing.net and "info nettime-l" in the msg body
#  archive: http://www.nettime.org contact: nettime@bbs.thing.net