PROCEEDINGS
28th International Symposium
on Electronic Art
MAY 16-21,
PARIS

Organizers

Co-organizers

Supported by

     quebc   

ISEA2023 stands with Ukraine 

28th
International
Symposium on
Electronic Art
ISEA2023

Publishing Director: Emmanuel Tibloux, Director of École des Arts Décoratifs – PSL
Editor: Emmanuel Mahé, ISEA2023 Academic Chair, Head of Research of École des Arts Décoratifs – PSL
Copy-editing: Pauline Barthe, Edith Buser, Elsa Carnielli 
Proofreading: Pauline Barthe, Elsa Carnielli, Manuelle Freire 
Publicization Special Adviser: Lucile Haute 
Graphic Design: Quentin Juhel 
Types: Syne by Bonjour Monde, Open Runde by Laurids Kern 

ISEA2023 proceedings has been designed with frugality in mind. The pdf version features black typography on a white background in standard A4 format, which saves ink and paper for (full or partial) office printing. The web version is static and resource-efficient.
It has been made with the editorial platform BOOKOLAB, a collaborative tool for editing and formatting publications for different reading media (printable version as PDF & screen version as web page) designed by Lucile Haute, and developed by Arman Mohtadji & Benjamin Dumond with GRAV and Paged.js, with the support of Université de Nîmes and École des Arts Décoratifs  PSL. Projekt source code: gitlab.com/armansansd/bookolab

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without prior written permission of the individual authors.

Individual authors of papers and presentations are solely responsible for all materials submitted for the publication. The publisher and the editors do not warrant or assume any legal responsibilities for the publication’s content. All opinions expressed in the book are of the authors and do not reflect those of the publisher and the editors.  

Despite intensive research, it was not possible to identify all copyright holders. If proper acknowledgment has not been made, we ask copyright holders for notice.

isea2023.isea-international.org
 @ISEAInternational 
 @iseainternational 
 @ISEASymposium 

ISEA2023 Paris, International Symposium on Electronic Art

ISBN 978-2-905710-71-0 (PDF)
ISBN 978-2-905710-69-7 (web)
DOI 10.69564/ISEA2023-PROCEEDINGS

École nationale supérieure des arts décoratifs
31, rue d’Ulm, 75005 Paris – France
www.ensad.fr
© École des Arts Décoratifs  PSL, 2024

 

Summary

Presentation

The 28th edition of ISEA, co-organised by Le Cube Garges and the École des Arts Décoratifs  PSL, took place in Paris at the Forum des images from 16 to 21 May 2023, as well as through an artistic programme in some 50 partner cultural venues throughout France in spring-summer 2023. The videos of which are online here.

Following an international call for proposals, more than 1400 submissions from 70 countries were evaluated by an artistic and scientific committee of 200 international experts. The theme of this edition was Symbiosis. In this time of global health, ecological, economic and democratic crisis, symbiosis is a polysemous notion that allows us to explore in a transversal and interdisciplinary way the mutations and transformations underway in the digital age, to question the meaning supposedly given to progress, especially in the current environmental and health context, and to imagine possible and viable futures for our planet and our ecosystems. 

The symposium catalog, including the symposium program and the artistic program, is available as a downloadable pdf here.

Figures

+ 1 500 participants at Symposium conferences and exhibitions at the Forum des images 
+ 700 participants in online conferences during Symposium week
+ 7 000 visitors to the Augmented Catalogue in the first 15 days
+ 1400 proposals 
+ 660 academic proposals
+740 artistic proposals
+ 70 applicant countries
+ 50 cultural partner venues

Co-organizers

ISEA2023 was co-organised by Le Cube Garges, executive producer, École des Arts Décoratifs  PSL, academic programming director, and the Forum des images, Symposium venue. École des Arts Décoratifs  PSL is a public institution of higher education under the authority of the Ministry of Culture, whose mission is the artistic, scientific and technical training of artists and designers, as well as the development of research creation activities. pioneer in thie field of collaborations between art, design and sciences, it created a research laboratory, EnsadLab in 2007. Within the University Paris Sciences & Lettres (PSL), towhich it is affiliated, it set up the PSL SACRe (Science Art Creation Research) doctorate program in 2012. It is the first practice-based doctorate to be set up within an art and design school in France.

Le Cube Garges is an interdisciplinary and digital cultural innovation center with six facilities covering 8,000 m². Focused on creative renewal, it combines the discovery of new artistic forms, inclusive practices, digital training and interdisciplinary reflection on societal issues.

SYMBIOSIS,
THE MAIN THEME 

Symbiosis is essential to life. No living species can survive without cooperation, let alone humans with each other and with their environments. More than a simple coexistence, symbiosis induces interdependence. It can be positive, neutral and sometimes negative in certain cases of parasitism. From these interrelations can emerge new hybrid entities, like the lichen which is a compound of algae and fungi. The first providing photosynthesis and the second moisture, one benefits from the contribution of the other. Our becoming lichen would be in a way the cross-cutting question underlying the symposium. Understood in a broad acceptance, the symbiosis is also played out at the scale of the symposium: it is a transdisciplinary event between creation and research (visual arts, theater, music, design, cinema, human and social sciences, engineering sciences , natural sciences) and intersectoral (art, crafts, industry, research, education, cultural innovation).

THE SUB-THEMES

The call for proposals for the symposium was divided into three main sub-themes: symbiotic individuations, organizations and imaginaries. It was a starting point to structure the call without limiting the scientific concept of symbiosis, and to encourage cross-cutting proposals outside disciplinary and technical classifications.

The selected interventions thus made it possible to discover works that call on symbiosis as a scientific concept, but above all projects that used it as a metaphor or more simply as an inspiration. It was an invitation to discover in a new way, both serious and curious, projects closely associating creation, research and technologies.

Symbiotic individuations

Which type of “hybrid beings” are coming into existence today, and thereby associating the human and the non-human? What new forms will they take, and will they be the next inhabitants of our planet? Experiments on trans-humanism, the emergence of new artificial autonomies, and new representations of natural elements were examples of the exploratory perspectives that were to be presented and discussed.

Symbiotic organizations

Human societies have founded their equilibrium upon multitudinous and interdependent forms of symbiosis across a wide swathe of domains: ecology, economy, culture, technology, and more. What happens when a given event destroys this fragile equilibrium? Researchers, artists, designers, and leading experts offered their perspectives on these resolutely transdisciplinary approaches, with the shared ambition of contributing to a world that is more resilient and at ease in its alterity.

Symbiotic imaginaries

In order to exist and prosper, symbiotic relationships—relationships in which each partner derives reciprocal benefits indispensable to their survival—are essential. Symbiosis is a vital necessity at all levels of all known ecosystems. But “necessary” does not imply that it should be reduced to its pragmatism or instrumentality. Symbiosis also opens us to new imaginaries, be they novel or redundant in the context of their corresponding era. If modes of collective imagination vary according to perspectives, they are always multiple and in dialogue with reality—what, therefore, constitutes the symbiotic imaginations of today and how and why invent new ones? 

THE “CLUSTERS”

Given the large number of conferences, roundtables and various oral presentations, the academic chairs proposed clusters to enable the symposium audience to compose its own cross-cutting program to complement the Symbiosis themes. This gave meaning to the general programme. 

The papers grouped together in each of these clusters could be classified differently and often appeared in several other clusters. These groupings are therefore retained here, with each cluster listed in alphabetical order (authors). Each paper is colour-coded to indicate the sub-theme to which the author(s) responded (sub-theme reminder), together with the cluster in which it was scheduled for the symposium audience and any other clusters to which it might have been attached. A final glossary by sub-theme and cluster is provided to offer readers several entry points into this important volume. These classifications in no way restrict the understanding of the papers, they are simply reading aids.

Abstract of all interventions are available on the ISEA2023 website and each of them is accessible to the widest possible audience: https://isea2023.isea-international.org/en/symposium-program


“L’étrange labo microcosmique des Oumpalous” exhibition at Le Cube Garges, artwork from Anne Marie Maes, Installation Exoskeleton: Topography of a Second Skin, for ISEA2023 © Nadia Rabhi

Academic Chairs and International Programming Committee

Academic Chairs

The Academic Chairs were responsible for the final programming of the symposium based on the evaluations of the IPC. The names and biopics of the Academic chair and of the three co-chairs are published below.

The International Programming Committee (IPC) was composed of 140 internationally renowned experts. The Committee evaluated the 1400 proposals in a double-blind process for full & short papers, panels and roundtable discussions. The names of the IPC are listed below.

Dr Emmanuel Mahé — ISEA2023 Academic Chair

Dr (HDR) Emmanuel Mahé (France), Director of Research at École nationale supérieure des Arts Décoratifs (PSL), researcher in information and communication sciences dealing with issues related to organisations and innovation processes in the fields of arts, sciences and design; scientific officer at the High Council for the Evaluation of Research and Higher Education.
Website of Emmanuel Mahé

Emmanuel Mahé © DR

Dr Elena Papadaki — ISEA2023 sub-topics co-chair

Dr Elena Papadaki (UK/Belgium) is a researcher, educator and curator; her research interests lie in the intersection of technology-reliant artworks, curatorial discourse, interdisciplinary collaboration and audience reception.

She leads the MA Design at University of Greenwich and currently serves as a Pathway Councillor for the Royal Society of Arts (RSA), acting as a key advisor and expert to the delivery group “Design for Life–Students for Change”.
Website of Elena Papadaki

Elena Papadaki ©DR

Dr Maria Ptqk — ISEA2023 sub-topics co-chair

Dr Maria Ptqk (Spain) is a curator, researcher and cultural producer. With an academic background in law and economy, she holds a PhD in artistic research with a doctoral thesis on the cyberfeminist collective subRosa. Her work investigates the ecological and technoscientific realm from a postcultural perspective, in the crossroads of experimental art practices and collective in/trans/disciplinary knowledge.
Website of Maria Ptqk

Maria Ptqk © DR

Pr François-Joseph Lapointe — ISEA2023 sub-topics co-chair

Prof. François-Joseph Lapointe (Quebec) is an artist and scientist at the University of Montreal, holding two doctorates (in evolutionary biology and in dance and performance). He has published over 120 articles in the field of molecular systematics, population genetics and metagenomics. As part of his artistic practice, he applies biotechnology for creative purposes and has created choreogenetics.
Website of François-Joseph Lapointe

Pr François-Joseph Lapointe 
© DR

International Programming Committee

Aceves Sepulveda Gabriela – Université Simon Fraser (CA)

Albert Jeffrey – Université Loyola de la Nouvelle-Orléans (FR)

Almas Almir – Université de São Paulo (BR)

Anders Peter – Kayvala (États-Unis)

Badani Pat – Conseil d'administration de l'ISEA (États-Unis)

Baker Camille – Royal College of Art, Londres (Royaume-Uni)

Basserau Jean-François – Ecole des Arts Décoratifs  PSL (FR)

Behar Armand – Centre de Recherche en Design – ENSCI-Les Ateliers ENS Paris-Saclay (FR)

Bellotto Janet – Université Zayed (AE)

Berenguer Josep-Manuel – Sonoscop (ES)

Beyls Peter – Collège universitaire de Gand Belgique (BE)

Bianchini Samuel – Ecole des Arts Décoratifs  PSL (FR)

Bibasse Juliette – commissaire indépendante / HACNUM (BE)

Bradbury Victoria – Université de Caroline du Nord Asheville (États-Unis)

Breuleux Yan – NAD-UQAC (CA)

Brucker-Cohen Jonah – Lehman College / City University of New York (États-Unis)

Buiani Roberta – University of Toronto (CA)

Bunt Brogan – University of Wollongong (AU)

Cardoso Amílcar – Instituto Politécnico do Cávado e do Ave (ESD-IPCA) (PT)

Cattan Pierre – Small Bang (FR)

Century Michael – Rensselaer Polytechnic Institute (US)

Cermak-Sassenrath Daniel – ITU (DK)

Cevallos Gabriel – Kino Beat Festival (BR)

Champion Erik – University of South Australia (AU)

Chevrier Joël – Université Grenoble Alpes (UGA) (FR)

Citton Yves – Université Paris 8 (CH – FR)

Costa Pederson Claudia – Wichita State University (US)

Cozzolino Francesca – École des Arts Décoratifs  PSL (FR)

Cruz Filipa – Paris College of Art (FR)

Cunin Dominique – École des Arts Décoratifs  PSL (FR)

Czegledy Nina – OCAD University Toronto, University of Toronto (CA)

Dahan Kevin – De Montfort University (UK)

Dal Farra Ricardo – Board ISEA International – Concordia University (AR/CA)

Dallet Jean-Marie – EAS : École des arts de la Sorbonne (FR)

de Campo Alberto – UdK Berlin (DE)

de la Merced Melissa – University of the Philippines Film Institute (PH)

Dedack Camille – HACNUM (FR)

Dika Penesta – University of Art and Design in Linz / Department of Interface Cultures (AT)

Doyle Judith – OCAD University (CA)

Drew Jesse – Griffith University (AU)

Dulic Aleksandra – University of British Columbia (CA)

Eigenfeldt Arne – Simon Fraser University (CA)

España Keller Juliana – Concordia University (CA)

Farias Priscila – University of São Paulo (BR)

Ferraiolo Angela – Sarah Lawrence College (US)

Fischer André – MixBrasil Festival (BR)

Fitzgerald Scott – New York University (US)

Flemming Peter – Concordia University (CA)

Forouzandeh Milad – D A H project (IR)

Fox Tyler – University of Washington (US)

Fratzeskou Eugenia – Leonardo ISAST / The MIT Press (LABS) (UK)

Freire Manuelle – Concordia (CA)

Fröschl Martina – University of Applied Arts Vienna (AT)

Fuchs Natalia – ARTYPICAL (RU)

Gay Jean Jacques – Acces-s cutures électronique / Citu paragraphe P8 (FR)

Ghasemi Amirali – New Media Society, Tehran (IR)

Girousse Clara – ARCAN (FR)

Gollifer Sue – University of Brigton (UK)

Grilo Carlos – Instituto Politécnico de Leiria (PT)

Grisoni Laurent – University of Lille (FR)

Haute Lucile – École des Arts Décoratifs  PSL and Université de Nîmes (FR)

Hosale Mark-David – York University, Computational Arts (CA)

Hunter WhiteFeather – The University of Western Australia/ SymbioticA (AU)

Hwaryoung Jinsil – Texas A&M University (US)

Jarmick Martin – California Lutheran University (US)

Jego Jean-François – Université Paris 8 (FR)

Jim Alice – Concordia University (CA)

Kaminska Aleksandra – Université de Montréal (CA)

Kim Jinku – The University of Nebraska – Lincoln (US)

Klein Tobias – School of Creative Media, City University of Hong Kong (CN)

Law Vanissa – Silly Savvy Studio (UK)

Le Malet Carine – Leonardo/Olats (FR)

Lebon Marianne – Hangars Numériques / HACNUM (FR)

Lehérissier Imane – Chateau éphémère (FR)

Leonard Neil – Berklee College of Music (US)

Levillain Florent – École des Arts Décoratifs  PSL (FR)

Levy Ellen – Independent Artist (US)

Lioret Alain – Université Paris 8 (FR)

Londono Felipe – Universidad Jorge Tadeo Lozano (CO)

Ludovico Alessandro – Winchester School of Art, University of Southampton (UK)

Mahé Emmanuel – École des Arts Décoratifs  PSL (FR)

Makris Nadine – AADN (FR)

Manton Coral – Bath Spa University (UK)

Mariátegui José-Carlos – Alta Tecnología Andina (PE)

Martin Patrick – Massey University, Whiti o Rehua School of Art (NZ)

Martins Pedro – University of Coimbra (PT)

Matuck Artur – University of São Paulo (BR)

Maviel Sonet Alexandrine – Amcsti (FR)

Miller Bill – University of Wisconsin Whitewater (US)

Mitchell Bonniel – Bowling Green State University (US)

Montfort Nick – Massachusetts Institute of Technology (US)

Moore Jake – University of Saskatchewan (CA)

Moren Lisa – UMBC (US)

Mosse Aurélie – Ecole des Arts Décoratifs  PSL (FR)

Nacher Anna – Institute of Audiovisual Arts Faculty of Management and Social Communication Jagiellonian University (PL)

Nam Hye-Yeon – Louisiana State University (US)

Naveau Manuela – University of Arts Linz / Department Interface Cultures (AT)

Nevin Antony – Wellington School of Design, College of Creative Arts / Massey University (NZ)

Nijholt Anton – University of Twente (NL)

Odonoghue Diarmuid – Maynooth University (IE)

Olynyk Patricia – Washington University in St. Louis (US)

Ong Joel – York university (CA)

Palumbo Michael – York University (CA)

Paquin Louis-Claude – Université du Québec à Montréal (CA)

Parker Jennifer – University of California Santa Cruz (US)

Pasquier Philippe – Simon Fraser University (CA)

Paul Christiane – Whitney Museum / The New School (US)

Pérard Hervé – Siana (FR)

Perez-Bobadilla Mariana – Waag Society (MX)

Polli Andrea – The University of New Mexico (US)

Prokopow Michael – OCAD University (CA)

Reyes Everardo – Université Paris 8 Vincennes Saint-Denis (MX/FR)

Ribeiro Clarissa – Roy Ascott Studio / SIVA DeTao Shanghai / Art|Sci Collective (UCLA Art|Sci Center and Lab, US) (BR)

Robine Michèle – Oplineprize (FR)

Rosen Margit – ZKM | Center for Art and Media (DE)

Rosero Contreras Paul – USFQ (EC)

Rowe Robert – New York University (US)

Salter Chris – Zurich University of the Arts (CH)

Samson Audrey – Goldsmiths, University of London (UK)

Schefler Pierre – ARCAN / HACNUM (FR)

Schuh Diane – Université Paris 8 (FR)

Scott Jill – ZHdK (CH)

Shadid Reem – Independent curator and researcher (IL)

Shahrokh Naz – Zayed University (UAE)

Shaik Yifat – York University (CA)

Slayton Joel – Independent Artist and Curator (US)

Sosa Andrea – National University of La Plata (AR)

Szabo Victoria – Duke University (US)

Tornero Paz – University of Granada (ES)

Tsoupikova Daria – University of Illinois Chicago (US)

Valencia-Tobon Alejandro – Cucusonic Collective (CO)

Valentina Peri – Independent Curator (FR)

Van der Plas Wim – ISEA International (NL)

Vatavu Radu-Daniel – Stefan cel Mare University of Suceava (RO)

Vergara Erandy – ISEA International (MX/CA)

Vilca Cecilia – MyAP – Microscopía Electrónica y Aplicaciones en el Perú (PE)

Warnke Martin – Leuphana University Lüneburg (DE)

Wei Jo – CAFA (CN)

Young Jiayi – Université de Californie, Davis (États-Unis)

Zhang Ga – Académie Centrale des Beaux-Arts, Pékin (CN)

Academic and Artistic Program

At the same time as the academic call for artists and researchers, a specific call for projects was launched, under the artistic responsibility of Cube-Garges. This selection has been enhanced by the networked artistic program (50 venues) offers thematic tours around symbiosis in a network of partners in Paris and in France, during several weeks. It irrigated the territory and thus brought to ISEA2023 both an anchoring in various places and an opening to diversity. The public discovered a wide variety of installations, virtual or augmented realities, performances, concerts, video games, design, films or digital books. The tours is also available on a dedicated online platform here or on our website.

POSTERS

 

WORKSHOPS

 

ARTWORKS

 

ARTIST TALKS

 

PANELS

ISEA International

ISEA International was proud to present the 28th edition of our annual symposia in Paris (France), with an academic symposium and exhibition predicated as an experimentation incubator driven by the unifying theme of“Symbiosis”– an essential consideration for the interdependent gathering of the year 2023.

We joined in celebrating ISEA’s return to Paris after 23 years since ISEA2000 “Révélation”, with Le Cube Garges as executive producer and responsible for the artistic program, and École des Arts Décoratifs  PSL responsible for the symposium’s academic program. As in the previous iteration in Paris, the academic symposium was held in the heart of the city at Forum des images.

Founded in the Netherlands in 1990, ISEA International (formerly Inter-Society for the Electronic Arts) is an international non-profit organisation fostering interdisciplinary academic discourse and exchange among culturally diverse organisations and individuals working with art, science and technology.

The main activity of ISEA International is the annual International Symposium on Electronic Art (ISEA). The first symposia took place in 1988.

 

Tributes

Two major figures in art and culture left us in 2023: Peter Weibel, whose tribute was paid during the Symposium with the entire ISEA community gathered in the large amphitheatre of the Forum des Images, a tribute led by Pr Roger Malina which elicited many speeches from the audience; and Vera Molnár, who died a few months after the Symposium and to whom we pay tribute in these proceedings.

Disremembering Peter Weibel: Reminiscences are more Interesting than Fact

Peter Weibel was a leading figure in the new media art scene. Since 1999, he had been the director of the ZKM in Karlsruhe and had carried out numerous curatorial activities (Ars Electronica, Venice Biennale, etc.). Among his most important exhibitions, we can also mention his collaborations with Bruno Latour, who also passed away recently ("Iconoclash" in 2022, "Making Things Public" in 2005 and "Critical Zones. Horizonte einer neuen Erdpolitik" in 2020). The sudden departure of Peter Weibel in March 2023 was a shock for the ISEA communities therefore a tribute was paid to him at this years’ symposium. A lecture was given by Roger Malina, followed by a session lasting over an hour in which Malina asked the large audience to share memories of Peter Weibel. Many very moving and interesting words for the ISEA community were expressed. Many deeply moving words and recollections were expressed by members of the ISEA community.

Roger F. Malina © Nadia Rabhi

Tribute to Vera Molnár

Vera Molnár dans son atelier © Vincent Baby

Among the numerous reactions following the passing of Vera Molnár at the age of 99, a pioneering artist in geometric abstraction and generative art, the National Institute of Art History (INHA) in Paris paid tribute to her.  The ISEA Symbiosis committee also wished to honor this iconic figure by transcribing the text from the INHA here.

Vera Molnár “had established deep ties with the INHA, which were manifested through the donation of a significant collection of prints in 2022, where the pioneering and generous dimension of her work fully reflected her personality. Her passing leaves a huge void in the art world, where she was still actively engaging with young creators who came to visit her. Vera Molnár was born in 1924 in Budapest, where she received a classical education in Fine Arts and graduated as a professor of art history and aesthetics in 1947. That same year, she left Hungary and settled in Paris. Quickly choosing her artistic family in geometric abstraction, during the 1950s she was a radical constructivist who refused to label what she considered, with the theoretical support of her husband François, a scientist and psycho-physiologist, as "art," viewing it instead as experiments. In this spirit, she invented an "imaginary machine" in 1959, a proto-computational conceptual process that involved giving precise composition instructions followed by systematic execution, akin to a machine. Taking a significant step in 1968, she began using a computer to assist in creating artworks based on programs and algorithms, becoming a pioneering artist in France in generative drawing. Curious about all materials and mediums, she drew, glued, painted, sculpted, programmed, photographed, created installations, artist books called "livrimages"[the French contraction of the words ''book'' and ''image''], and an impressive intimate diary (1976-2020) in twenty-two volumes of nearly 5000 pages. Whenever possible, Vera Molnár also created prints. Trained in Fine Arts, she learned wood and linoleum engraving at a young age but later collaborated with professionals to produce numerous publications. In 2022, she decided to assemble a significant collection to donate to the INHA library, consisting of 264 prints, 2 portfolios, and 12 preparatory models.

This donation forms an exceptional collection, representing the entirety of the artist's work, unique in French and foreign public collections. Her works belong to formal families that she worked on, revisited, and reinvested in for several decades. The Sainte-Victoire Blues directly reference Cézanne's motif, drawn and painted hundreds of times by Vera Molnár. Lettres de ma mère echo the computer modeling of the artist's mother's handwriting, who wrote to her from from Budapest, which over the years became increasingly jagged at the end of each line, described by the artist as "a bit gothic, somewhat hysterical, "yet providing her with a sensation of "visual bombardment" each time. 4 carrés rouges, Brèches3 triades 3 couleurs, Ordinateur miroir de la main, Hypertransformations hommage à Klimt, and many other series explore the paradigmatic form of geometric abstraction: the square. Cut, shifted, overlapped, joined, pushed, covered, giving rise to other shapes, incisions, triangles, quadrangles, perhaps letters, the squares disassemble, suggesting a figure of balance or generating a perceptual indecision, leaving it to each individual to contemplate and decide for themselves.

Upon news of her passing, INHA seeks to perpetuate the living legacy of an immense creator; that of an adventurer of shape, of order and disorder, who used and challenged tradition. Behind Vera Molnár's programs, algorithms, and systems lie thought and ideas, as well as tenderness and amusement; in short, life. Today, we bid farewell to an artist generative in every sense."

With the kind permission of the authors, INHA's director, Éric de Chassey, the director of its library, Jérôme Bessière, and Vincent Baby, project manager at INHA, close friend and Vera Molnár specialist.

Symposium Proceedings

Introduction

SYMBIOSIS –
ISEA2023 PROCEEDINGS INTRODUCTION

FOR A SYMBIOTIC APPROACH AT ALL SCALES

THE NEED FOR INSTITUTIONAL RECOGNITION OF PRACTICE-BASED RESEARCH (‘RECHERCHE-CRÉATION’)

KEY STRENGTHS

1. ORGANISATIONAL FORMS ARE ALSO CREATIONS

2. CRITIQUE OF THE OLD BINARITIES THROUGH PRACTICE-BASED RESEARCH

3. EMERGING DE/SUBJECTIVATION PROCESSES

4. GENERIC TOOLS DESIGNED AND SHARED BY COMMUNITIES OF PRACTICE

5. BREAKING DOWN DISCIPLINARY BARRIERS AND THE EMERGENCE OF NEW RESEARCHER PROFILES: EVALUATION AS A TOOL FOR LEGITIMIZATION

6. PERCEIVE BEYOND OUR USUAL CAPACITIES TO BECOME AWARE OF SOCIETAL AND ENVIRONMENTAL TRANSFORMATIONS

7. WHEN CREATIVE RESEARCH BECOMES CRITICAL ACTION-RESEARCH

CONCLUSION 

For a symbiotic approach at all scales

Of the 660 proposals received for the academic call (and a total of 1,400 including the call for artistic projects), from over 70 countries, 230 were selected, including 99 short and long papers, for the Symposium held at the Forum des Images, in the heart of Paris.  200 international researchers and experts evaluated these applications in a double-blind process. More than 50 global organizations, including schools, universities, research bodies and associations, were selected to present their research activities and facilitate exchanges among them. In addition, more than 50 partner venues, such as art and design schools, galleries and institutes, offered a rich complementary program of exhibitions, performances and debates on a national scale. This mobilization of a range of contributors, evaluators, and a public of several thousand spectators once again attested to the vitality of ISEA, an annual event initiated more than 30 years ago by pioneers.

The central theme of the 2023 edition was Symbiosis, as demonstrated by the 99 previously unpublished papers in these proceedings. The full program, including these papers and all the other symposium formats - artist talks, panels, workshops, tutorials, posters and demos - can be consulted online. This program was in itself a form of symbiosis.

But what types of symbiosis are we talking about exactly? How do they manifest themselves? Going beyond the strict definition of the term and interpreting it as an operative metaphor, how can we envisage and put into practice a genuine ‘symbiotic’ approach? The aim of the call for entries was to encourage the reconsideration of any creation, experimentation or research through this prism, even if symbiosis was not necessarily the main theme. ISEA2023 offered a sort of ‘core sample’ at a given moment of the different strata of contemporary creation and research underway in the world.

Our idea was that the symposium's programming should also serve as a collective manifesto, promoting the fundamental principle of symbiosis to be respected at all times and places: a lasting and mutually beneficial association between organizations and actors, whether human, non-human or hybrid. Considered in a broad sense, symbiosis also manifested itself at the level of the symposium itself: it was a transdisciplinary event combining creation and research (in the fields of visual arts, theatre, music, design, film, human and social sciences, engineering sciences, natural sciences) and cross-sector (involving art, craft, industry, research, education and cultural innovation).

Before highlighting what we consider to be some of the strengths of this particularly rich body of work, in terms of both knowledge creation and innovation, it is important to mention our commitment to promoting practice-based research in all its forms and formats. This approach fosters the invention of modes of cooperation between different areas of expertise and enables the emergence of hybrid researcher profiles. Symbiosis does not lie in the homogenization of its components, but rather in the creation of a hybrid and symbiotic entity, where each element is essential to the other in devising new ways of existing. Together, they offer innovative ways of perceiving, understanding and acting on the world.

The need for institutional recognition of practice-based research (‘recherche-création’) 

It often happens that artist-researchers or designer-researchers are valued for their creative abilities, while at the same time being perceived as illegitimate in their research skills. Indeed, there is a perception that one cannot be both a researcher, publishing scientific articles for example, and a creator, exhibiting one’s work. This perception differs from country to country. In countries like Canada, for example, this is not a problem. In France, for example, despite recent progress, not all research institutions are yet clear about this. Yet these publications and exhibitions (‘publicizations’) both stem from the same practice-based research process. New generations of researchers are challenging this perception, and ISEA is a testament to this, since the event enables creators and researchers to share their research results in an international A-rank publication that meets the standards of peer-reviewed scientific journals. Institutional structures have also become aware of this reality, as shown by the doctoral programs specializing in practice-based research which, through their official recognition, mark a fundamental evolution both in the academic world (universities, art and design schools) and in the professional sector, and not only in the cultural field (health, transport, housing, ecology, etc.). However, we can only encourage research and teaching institutions to take this major development into consideration, by proposing, for example in countries where it does not yet exist, the status of lecturer, associate or full professor (‘enseignant-chercheur’ in France) in art and design schools.

The older generations, pioneers who are still very active, were also present at ISEA2023. In Paris, for example, the pioneers of digital arts active since the 1980s rubbed shoulders with doctoral students just beginning their research careers. This cross-generational aspect is rare enough to be highlighted and celebrated.

Key strengths

As Academic Chair and co-Chairs, we propose here to highlight what we see as the strategic trends running through all the papers. This collective work reflects our different disciplinary fields, countries and educational backgrounds. Our aim is not to summarize such a rich corpus, but to propose several common threads to encourage more in-depth reading of the articles and, of course, to contradict or consolidate our proposals.

Although this exercise is limited and subjective, partial and biased, it is nonetheless enriching. It reveals that, despite the diversity of the proposals, it is possible to find convergence points, despite the divergences, and even, sometimes, diametrically opposed approaches. The theme of symbiosis encourages us to look not for what unifies, but rather for what the different perspectives or visions proposed, even if they take different forms and are based on heterogeneous conceptual frameworks, can generate together. After all, symbiosis thrives on difference.

1. Organisational forms are also creations

Faced with the urgent environmental and democratic challenges of our time, the development of new collaborative methods is essential. Creative research is an innovative methodological response, offering an effective means of strengthening and better coordinating interdisciplinary dynamics. It opens the way to a new era of organizations and individuals working together in symbiosis. Although the approaches may vary, the common objective is clear: to give increasing importance to research within institutions and to recognize the vital role of associative networks and companies involved in this process.

Organizations, whether stable or changing, are study entities in their own right. Namely, E. Armand, A. Asensio and D. Foresta encourage the creation of new cooperative ecosystems between the arts and the technosciences, underlining the importance of collaborative innovation. Transforming organizations from within is also seen as a promising way forward. Y. Hu, C. Chou and Y. Kakehi, for example, are calling for the creation of laboratories within art centers, as they have done by setting up a ‘controlled environment space’ at the heart of an exhibition. Y. Breuleux, A. Thibault and R. Lapierre, for their part, advocate the idea of occupying and reappropriating industrial wastelands, which are often included in digital art festivals, to give them a new lease of life.

Organizational architecture also extends to the digital domain. V. Hilsberg looks at the use of DAO (Decentralized Autonomous Organization) based on blockchain, arguing that this technology has a direct influence on artistic creation through distribution methods, perhaps recalling Marshall McLuhan’s famous adage: ‘the medium is the message’. This specific focus on the medium or media is a key feature of ISEA.

Print media have always been and remain essential to the evolution of academic structures. C. Frémontier-Murphy traces the history of the journal Leonardo, published by MIT since 1966, which became an international benchmark by promoting a then nascent field of research. More recently, S. Bianchini and G. Lallemand have broken new ground by launching the online academic journal ‘.able’, with the support of more than 30 international partner institutions and the Arts and Sciences Chair at Polytechnique, EnsAD and the Daniel and Nina Carasso Foundation. This new image-based medium does not seek to compete with the written word, but rather emphasizes the importance of the visuals generated by practice-based research projects, referring to them as ‘publicizations’. This approach is based on the fundamental idea that to publish is to make public. In the academic context, this means subjecting new knowledge, whether textual or visual, to the critical scrutiny of peer communities.

When Leonardo was created, organizational innovation was also underway in South America. J.-C. Mariategui recalls the birth of the Centre for Art and Communication (CAyC), a Latin American network dedicated to art and cybernetics. Today, international collaboration between researchers has become the norm, as shown by the example of SEADS (Space Ecologies Art and Design), a transdisciplinary and multicultural collective on a global scale.

Sometimes developments are the result of unforeseen circumstances, sometimes predictable but poorly anticipated by public authorities. Recently, the Covid-19 pandemic forced cultural institutions to reinvent themselves. N. Ricci and M. Agogué share the lessons learnt by the Montréal festivals, illustrating the resilience and adaptability of organizations in the face of global challenges. These organizational issues also have their counterparts in the production of works. For example, W. Zhang and S. Su make visible the ‘voids’ between individuals subjected to the social distancing put in place during the Covid-19 pandemic.

All these examples illustrate the growing interest in the creation of new organizational forms in various fields. The concept of symbiosis originated in the natural sciences. By borrowing the vocabulary of these sciences, it is possible to suggest that all these organizational forms together give rise to new ecosystems. The term ‘ecosystem’ was proposed by Arthur George Tansley in 1935 in the journal Ecology. The concept then gradually migrated into the human and social sciences, particularly economics. Today, it is often used to designate a complex environment that encourages creation and innovation. Widely adopted by those involved in entrepreneurial innovation (makers, startupers, entrepreneurs, etc.), this metaphor of scientific origin can also be applied to art and design research.

These ecosystems, through their interrelationships, are contributing to the emergence of a renewed ‘biosphere’, forming a vast practice-based research network on a global scale. This network can be seen as a favourable climate for invention, a phenomenon that Nowotny, Scott and Gibbons had already described in their analysis of knowledge production in contemporary society in 2001. Thus, this organizational and creative landscape can be compared to a dynamic biosphere, where innovation emerges from interaction and collaboration, as Wenger, McDermott and Snyder emphasized in 2002 in their study of communities of practice. This is the case in Québec, but not everywhere. Like any ecosystem, the equilibrium found is fragile and subject to forces that can destabilize it, as in the case of Symbiotica, which is now threatened with extinction, even though it is a laboratory with an international reputation for its work in bioart. Ionat Zurr, an artist-researcher and co-director of this structure, spoke about this during her keynote speech. She also appealed to our community for help, which we are relaying here.

Each new organizational arrangement is both the product and the result of a process of transformation which also operates within conceptual frameworks. It is not surprising, therefore, that these frameworks are themselves put under strain by these arrangements, which ‘reorganize’ - or, depending on one’s point of view, disorganize - traditional ways of thinking.

2. Critique of the old binarities through practice-based research

The questioning of the traditional dichotomy between culture and nature is now shared by many in the academic community, particularly in anthropology. Artistic works embody and make perceptible this profound reconfiguration of paradigms of thought. N. Georgakopoulou, D. Zamplaras, S. Kourkoulakou and C.-Y. Chen explore a ‘sympoetic’ interaction with materials in interactive art, inviting us to question old dichotomies such as subject/object, mind/body, and nature/culture, in order to embrace a new understanding of materiality. P. Purg and K. Pranjić oppose a human-centered science, putting forward an ‘emergent symbiotic mutualism’ that transcends mere species coexistence, from a posthuman perspective. M. Zolotova, for her part, sees the posthuman turn in many contemporary discourses as a serious attempt to move beyond anthropocentrism. Such approaches are therefore not identical, and can even sometimes be seen as opposed, but they all involve a critique of analyses centered solely on the human.

This criticism can be found in other fields too. For example, Bakke questions biocentrism and focuses on the interactions between minerals and non-human life forms. Adopting an approach of working ‘with’ rather than ‘on’ is another way of challenging these human-centered biases. M. da Rocha Montanari uses generative art as a collaborative research methodology with indigenous Guarani and Kaiowá communities, co-creating a ‘poetics of attention’ that draws on sounds and plants in their cosmology. A. Wollensak, B. Terry and B. Baird present extracts from community testimonies relating to water, accompanied by processed environmental sounds from Alaska.

For R. Wright and S. Howden, the adoption of indigenous perspectives is crucial to developing ‘sentient machines’ capable of reflecting the complexity of ecological patterns, where computational ecology fails. C. Schnugg, D. Brill and C. Stary merge aesthetics, performance and digitalization to reinterpret the world, going beyond the rational production of knowledge. Although these initiatives are diverse, they share a common goal: by hybridizing different rationalities and subjectivities, they seek to transcend existing approaches in order to invent new ways of understanding and addressing contemporary crises, both geopolitical and climatic.

3. Emerging de/subjectivation processes

These methodological and epistemological reflections are accompanied by a transformation in the ways in which we conceive of human ‘identity’, the conceptual as well as practical ‘making’ of the way in which we define ourselves, or not, as human subjects. Indeed, the processes by which individuals construct themselves as subjects (a process of subjectivation), or sometimes deconstruct themselves (a process of de/subjectivation), are present in many practice-based research projects. For example, the performance by T. Chiaravalloti and A. Meshi’s, explores a symbiotic relationship between humans and an artificial intelligence algorithm, enabling participants to experience a ‘collective consciousness’. Although this work aims to highlight the limits of AI’s ability to recognize emotions, it also opens the way to reflections on future developments in this field and the ethical dilemmas they raise. P. Gemeinboeck and Rob Saunders, for their part, propose ‘hybrid entanglements’ between humans and robots, based on bodily empathy between humans and non-humans. S. Cîrcu and C.Y. Chen explore movement to question the anthropomorphism of humanoids through a mimetic dance between a human and a robotic arm.

These approaches use translation, imitation, differentiation and empathy as means of criticizing, but also realizing, the hybridization between machines and humans as formalized by a number of artists and theorists at the Symposium: M. Smigielska uses AI to take account of spectators’ behaviour, E. Amato & E. Péreny claim a ‘co-individuation’ with our technological avatars, J. Bae and his VR comic strip explore the relationship between humans and AI, D. Batsis, M. Grigoriadou, or with K. ElRaheb and A. Politis, who advocate the body as a phenomenon and an extension of ‘hyperreality’.

Even the most fiercely critical creative research into intrusive technologies is unwillingly participating in this general trend. C. Brunner and J. Fritsch look at the transhumanist movement through the prism of Gilbert Simondon’s thinking, in particular his notion of ‘human energetics’, and are inspired by a video work by the Nigerian artist Otobong Nkanga. G. J. Shin, using the concept of the ‘superjet’ in a post-digital context, highlights the emergence of new creative human subjects, capable in her view of weaving new social relationships.

This critical exploration of the way in which humans conceptualize and experience themselves also manifests itself in a decentering of the human, highlighting the processes of subjectivation at work in the ‘non-human’ or ‘other-than-human’ world. Adopting a Bruno Latour perspective, elements such as forests and water are no longer simply reduced to objects of study for humans; they are seen as actors in their own right, subjects in the active sense of the term. In this vein, G. Trudel, who studies climate change, integrates data from tree communities, the media arts and forest sciences. She attributes a ‘voice’ to trees through what she calls ‘individuations of symbiotic modulations’, drawing on and extrapolating the work of Gilbert Simondon.

Approaches to thinking about and experimenting with symbioses between humans and machines, as well as between non-human entities, vary widely. However, they all point to a probable reconfiguration of the general cartography of subjectivation processes: the contemporary human seems ready to move away from the central and dominant position it has hitherto attributed to itself. At least, that is what the observations and discussions at the Symposium seem to indicate. The question is what this will mean in the long run, and whether the promise of emancipation will really be fulfilled.

4. Generic tools designed and shared by communities of practice

As the notions of objects and subjects evolve, research tools and methodologies are also changing. This transformation is contributing to the emergence of new practices and methods for ‘measuring’ phenomena that were previously difficult to quantify. The evolution of instruments and their impact in the field of creative research are particularly interesting. Just as there are creators of works of art, there are also innovators in the organization and design of instruments. Artists, designers, scientists and engineers frequently collaborate to provide research and creation with platforms and tools that are not only at the service of creation, but that also stimulate innovation among those who use them.

For example, MARCEL (Multimedia Art Research Centres and Electronic Laboratories), of which the pioneer D. Foresta is one of the designers and promoters, offers an online collaborative platform dedicated to the specific formats of the interactive arts, a project that was born in 1997 and, even then, was the continuation of a process of reflection and practice that began in 1981. This desire to offer shared tools is obviously not a recent development and is now reflected in a wide range of proposals, from generalist platforms to tools dedicated to specific fields or uses. S. Thurow, D. Del Favero, M. Scott-Mitchell, M. Ostwald and H. Grehan have developed a network modelling system which facilitates the planning of opera rehearsals thanks to an interactive ‘cyber-physical’ spatial aesthetic, opening the way to new imaginaries. C. Lengelé and P.-A. Gauthier traced the genealogy of ‘Live 4 Live’, an open-source tool designed to simplify the creation and real-time control of spatialized sound objects. D. Cunin, E. Durand and M. Seta developed generic devices to facilitate the creation of immersive environments with video tracking and collective interactivity on smartphones.

This trend towards developing platforms for artists and designers is also to be found in the field of curation. Y. Hofmann, through the ‘intelligent.museum’ project, has put forward a system based on data analysis for artistic and cultural institutions. This prototype uses data from a variety of sensors in exhibition spaces to tailor museum experiences to visitors’ needs. I. Družetić-Vogel, A. Fuchte and M. Bauernfeind proposed ‘ARt chat’, an application for museums that combines augmented reality, art and communication, allowing visitors to share their impressions in the exhibition space.

These initiatives are not just about providing functional services; they can also take a critical approach. V. Guljajeva, M. C. Sola and I. J. Clarke, for example, have analyzed several existing tools in the field of AI, stressing that ‘AI does not create a work at the touch of a button, but requires a deep understanding of the underlying technology, as well as a creative and critical approach’.

In addition to the creation of platforms, there is also a trend towards modelling, although approaches vary considerably. The very term model indicates a desire to provide communities with frameworks based on general principles to be adopted. Z. Wu, D. Fei, X. Ma, M. Fan and K., for example, see NFT as ‘a sustainable business model for media arts, involving audience interaction’. Others, such as D. Xu, M. H. Lamers and E. van der Heide, propose a ‘relational model of co-located interaction’ between spectators and an interactive work.

These various proposals, although heterogeneous, share a common view: they point to a constant trend towards the uninterrupted emergence of specialized communities of practice that design generic devices for instrumental purposes. These communities are also expanding thanks to the instruments they are inventing, enabling their practices to be disseminated more or less widely, a form of ‘standardization’ with a creative aim, while at the same time encouraging the integration of their own approaches to uses in the fields of art, often including the active participation of audiences as a central component of the design of the works.

5. Breaking down disciplinary barriers and the emergence of new researcher profiles: evaluation as a tool for legitimization

ISEA stimulates multidisciplinary exchanges while consolidating the existence of a specific community through annual gatherings. The enthusiasm for creation, in synergy with technology and science, forges a solid link between researchers and creators from a variety of backgrounds. This positive dynamic encourages the exchange of diverse perspectives, whether practical, theoretical or methodological, around a common subject, a specific field, or more simply a medium or instrument. The cross-disciplinary aspect of these interactions is particularly enriching, as it helps to create a diverse community while respecting the disciplinary affiliations of each individual.

Adopting a double-blind peer review process, ISEA’s call for applications conforms to academic standards and promotes cross-disciplinary or multidisciplinary approaches. In an academic environment that is often segmented by discipline, this recognition is crucial. The symposium thus provides a sanctuary for unclassifiable minds, while giving them international exposure. The evaluation criteria we have chosen enable artist-researchers and designer-researchers to assert their legitimacy.

The criticism often levelled at the latter - that the dilution of the boundaries between the arts and sciences would jeopardize the quality of research and creation - ignores the symbiotic nature of these two spheres of creation. In fact, they are mutually nourishing, while retaining their specificity, whether through collective work or individual skills, not forgetting the importance of the distinct ecosystems to which these players belong. Although this does not systematically guarantee the relevance of the results, whether scientific or artistic, this approach is like that practiced in professional art and science circles, where peer judgement is commonplace.

ISEA’s aim is to bring these hybrid profiles to the fore. Although cross-disciplinarity can be perceived as a risk of levelling out skills, it is essential for innovating, experimenting with new methods and tackling issues from a fresh angle. The contribution of these cross-disciplinary approaches can be extremely beneficial to established disciplines and sectors, enabling them to evolve and renew their own fields of research. Venturing outside our disciplines through creative research allows us to rediscover them in a new light.

If practice-based research can create new knowledge in science and engineering, it also brings new practices to the field of creation. Y. Blanchi, for example, links cognitive science and architecture, while S. Gatz criticises architecture as a means of control by proposing a ‘cosmo-techno-poiesis’. Urbanism as it exists is questioned by artistic and digital practices, like the infrastructures of networked urban screens imagined and implemented by M. Thorogood, K. McCulloch and A. Dulic.

In dance, N.N. Correia, D. Souza, I. Nêves and J. Lobato are trying to make the dance process understandable within a performance using a multisensory approach. I. Teles de Castro e Costa, H. T. Hong and C. Y. Chen have designed a performance co-created with an autonomous virtual system. In the field of cinema, R. Sagot-Duvauroux, N. Quaetaert, F. Garnier and R. Ronfard explore how film editing cannot be transposed into virtual environments and must be reinvented. Questions about the transposition of creative techniques into VR also arise in the case of Chinese calligraphy with Shum, P.Y.S. & Klein. C. Salter, T. Thomasson and P. Uro are carrying out what they call a ‘theatrical exploration’ based on augmented reality, with the aim of making climate change visible.

The ‘adjustment’ between different vocabularies is necessary to give an account of this research. O. Kobryn, M. Couteau, R. Sagot-Duvauroux, S. Balcon, F. Garnier, R. Ronfard and G. Soulez, researchers from different disciplines (aesthetics, ergonomics, design, engineering, cognitive sciences), are working to reconsider the concept of the ‘virtual’.

In the cultural sector, curation and conservation are also invested with methods derived from practice-based research to redefine or apprehend their activities in a new way. M. Jones, M. Wester and M. Blottiere present a new approach to curating as practice-based research and propose the acronym ‘CRC’ to establish it as a specific field. D. Irrgang and M. Skłodowska take up the curatorial notion of the ‘exhibition of thought’ co-invented by Bruno Latour and Peter Weibel, who have put it into practice with curators, artists and researchers at the ZKM.

The circulation of thoughts and practices, the mutual acculturation of different research players and cross-disciplinary cooperation are all expressions that would remain abstract without these numerous experiments. These compositional arrangements, to paraphrase Latour, make us see differently.

6. Perceive beyond our usual capacities to become aware of societal and environmental transformations

Making perceptible what naturally escapes our senses is not a new concept in the world of art, or even in that of design, as the example of decasualization shows. This approach is even less novel in the field of science, where modifying our perception and understanding of the world forms the basis of a great deal of research. Nevertheless, the way in which this is done varies from one field to another, with a current tendency to merge different approaches in order to make discoveries or visions of the world that are not immediately accessible - often because of their scale - ‘sensitive’, i.e. perceptible to the senses.

In this context, many research and creation projects aim to reveal new sonic and visual ‘landscapes’, constituting what might be called an informational aesthetic. There are two main thrusts: the first seeks to materialise technological artefacts, which are increasingly miniaturized and interconnected; the second aims to raise awareness of climate change through the collection of biological or geophysical data, with the two thrusts sometimes overlapping.

In the first category, H. Scurto and A. Chemla-Romeu-Santos generate soundscapes from data collected in 28 locations around the world via the Locustream online platform, highlighting AI infrastructures. On the other hand, M. Lukaszuk and S. Bahng exploit gesture and spatiality through electroacoustic and video improvisation to make intelligible the preponderant role of machine learning.

In the second, Mc Farlane proposes a method of co-creating artistic acoustic ecologies centered on the Great Lakes in the USA, combining sound data collection and research methodologies enriched by indigenous knowledge.

In terms of sonification, L. Foo and J. Fritsch create a ‘cryogenic landscape’ to engage listeners emotionally with climate change, broadcasting the movement and melting of the ice in Greenland almost in real time, presenting the ice pack as a living being. J.C. Duarte Regino goes a step further by making atmospheric processes audible, helping us to perceive our atmosphere beyond our usual senses.

At the intersection of the two, L. Moren and T. Bachvaroff's ‘Under the Bay’ is an augmented reality project that turns smartphones into ‘microscopes’ to explore marine life invisible to the naked eye. J. Ottavi and J. Pickett are exploring the interaction between humidity and the decomposition of printed circuits, converting soil acidity and electronic compost into electrical signals.

Visibility projects can also illustrate disappearance, a radical type of invisibility. H. Sareen, Y. Fu and Y. Kakehi depict endangered species as micro-bubbles formed by the nucleation of CO2 in water, which shrink and disappear in a few days to raise awareness of the threat to biodiversity. B. Ammar-Khodja brings to life the residues of heavy metals in contaminated urban landscapes, highlighting a kind of point of no return if the pollution persists. G. Legrand, V. Thornhill, I. Clarke, through the installation ‘Spectral Plain’, explore how information technologies can integrate and reflect specific socio-cultural beliefs, creating new symbiotic interrelationships with the non-human and the non-living, thus influencing the participants' perception of their understanding of the world.

These practice-based research projects aim to make complex issues or abstract realities tangible. Designed to transform our perspective, they encourage us to adopt different behaviours and develop new critical approaches.

7. When creative research becomes critical action-research

N. Georgakopoulou and her collaborators aspire to create a ‘sonic consciousness’ within urban spaces. Their project aims to create musical experiences that invite active participation from citizens, while paying special attention to people with hearing and visual impairments. For his part, R. Ridgway strongly criticizes the collection of personal data by the main players in the network. In the same vein, B. Gaylor and K. Hennessy encourage the development of a critical understanding of Big Data to better counter the ‘piracy of affect’.

From a techno-feminist perspective, G. Lautenschlaeger explores post-human maternity in media art. P. Costa, L. Ribas and M. Carvalhais analyze the tendency to feminize digital assistants and suggest strategies for reversing this trend. L. Haute urges us to think about our methods of collaborating with the environment, inspired by the symbiotic culture of bacteria and yeast. C. Gould and Isabel Stenger urge us to take action to initiate change in our lifestyles and consumption habits, while J. Garančs explores, through virtual reality, dystopias based on market values.

Y. Zhang, in his interactive installation, uses machine learning to generate fictional speeches by Xi Jinping and Donald J. Trump. P.R. İ. Yeginsu highlights the need for artists to find new ways of collaborating to avoid the negative impacts of commercialization linked to NFTs. The fight against social prejudice is also a recurring theme, with S. Brueckner, S. Yeung, J. Liu, D. Choberka, K. Shedden, J. Turner, I. Gillet, M. Lu and Xingwen Wei identifying the biases present in face detection and race classification.

Faced with the environmental crisis, techno-solutionism is criticized through the surrealist and ‘pataphysical’ approach of M.-L. Bourgeois’s project, without systematically rejecting engineering, as many of the projects are situated at the intersection of the arts and the engineering sciences. The appeal to various forms of decolonization is also marked, namely with H. Rashtian and G. Aceves-Sepulveda who question the construction of historical narratives.

These contributions, which oscillate between denunciation and calls for resistance, aim to provide the public with critical, conceptual, sensory and emotional tools to raise awareness of the issues of pollution, gender and democracy. The subjects addressed are varied, and the methods proposed are diverse. Although critics of these works may fear that activism will predominate over rigorous research, it is clear that these contributions, by their provocative nature, offer food for thought and relevant tools for the scientific community, inviting ongoing debate among peers.

From this perspective, this type of action research fulfills a fundamental dual critical function: on the one hand, it is devoted to exploring the invisible, hidden, unexpressed, or even oppressed elements within society; on the other hand, it simultaneously engages in the struggle against doxas, including those with which it finds itself unwittingly associated. This bifocal approach is relevant to all fields of study, encompassing both research and creative work with a critical intent. This approach enables a clear distinction between activist practices and research activities, thereby amplifying their respective impacts through an enriching critical symbiosis.

Conclusion

As we have emphasized, the choice of these key strengths represents a challenge, due to its necessarily limited and subjective nature. Not all the authors are cited in this introduction, due to lack of space, and we can only encourage readers to read all the articles. We could have highlighted other aspects, such as the importance attached to feminism, gender issues, the ecological transition, or the various processes of decolonization. These are essential thematic entries, but they are already very present in current debates, and we offer additional reading keys. It was also possible to group the proposals according to the type of technologies used or invented, or by sector of activity. The thematic clusters we proposed during the Symposium could have encouraged us to do this. However, we have chosen to base ourselves on all the contributions published in these proceedings and to resume the programmatic work in ‘bottom up’ mode. This approach offers a new perspective on how everyone can benefit from this work for their own research.

We hope that these guidelines illustrate, if proof were needed, that these proceedings are a rich source of ideas, methods and formats that are both varied and innovative. It is also important to note that the proposals go beyond the scope of these proceedings, and we encourage you to explore the online video archive platform, including artist presentations, demos and roundtables.

These exchanges between art, design, science and technology are essential for generating knowledge and creating meaningful works. They allow us not only to analyze the present (a form of real-time analysis), but also to lay the foundations for future discoveries and new approaches that are still being developed. These initiatives can be seen as precursors, offering a glimpse or intuition of what is emerging and could materialize in the next ten to twenty years. In a way that complements science fiction, which imagines the future, these proposals are starting points for extrapolating the present and considering possible future developments. A work in constant evolution!

Paris, London, Bilbao, Montréal,

Emmanuel Mahé (École des Arts Décoratifs  PSL University, France, EU), Academic Chair

Elena Papadaki (University of Greenwich, UK), sub-thematic co-Chair

Maria Ptqk (Spain, EU), sub-thematic co-Chair

François-Joseph Lapointe (Université de Montréal, Canada), sub-thematic co-Chair

DOI: 10.69564/ISEA2023-1-Mahe-et-al-introduction

Keynotes

Three personalities were invited to give their views on current developments in the fields of creation, technologies and sciences, on the role of these in society, and to present their most recent work.

Ionat Zurr
(artist-researcher, Symbiotica, Australie) —
Symbiosis and the fallacy of a nature-free existence.  

In times of ecological emergency, solutionist fantasies of nature-free human existence promise salvation and repair. The innovative paradigm offers “products” such as lab-grown (animal free) meat and artificial automated surrogates to replace reproductive biological bodies. 

These so-called innovations require special artificial environments to host, nurture and culturally articulate this “new” nature-free, decontextualized and colonised life. The entanglement of life with its surrogate environment/apparatus, echoing human relationships with living and semi-living agents; when control and care are employed to counter resistance. 

Artists, scientists, designers and engineers all play their part in this transformation and its effects on human relations with life and the environment. This creates a range of ontological conundrums and fantastical expectations as to what technology can provide and to whom. Using examples of artistic research that deal with emerging technologies and new knowledge, Ionat Zurr narrated artists’ symbiotic and parasitic relationships with such post nature. 

This talk was framed by the imminent closure of SymbioticA, the first artistic research laboratory based in a life sciences department. SymbioticA began as a symbiotic act, embodied in an academic institution, to enable critical, yet mutualistic, relations among artists and scientists. Many of SymbioticA’s alumni have continued to establish their own laboratories and artistic practice in other academic institutions around the world, leading to the growth of the field of Biological Arts. 

SymbioticA is now being treated as a parasite by a changed host body. Is this a ‘natural’ survivalist rejection against a foreign body or can we detect symptoms of an autoimmune disorder? 

Biography: Dr Ionat Zurr is an artist-researcher. She is the Chair of the Fine Arts Discipline at the School of Design & SymbioticA academic coordinator at the University of Western Australia. Together with Oron Catts she established the Tissue, Culture & Art Project in 1996 and their co-authored book Tissues, Cultures, Art, published by Palgrave McMillan this year. Her collaborative work was exhibited by Pompidou Centre, MoMA NY, Mori Art Museum, Ars Electronica, National Art Museum of China and more. These ideas and projects reach beyond the confines of art and the work is often cited as inspiration to diverse areas such as new materials, textiles, design, architecture, ethics, fiction, and food.

Ionat Zurr © Nadia Rabhi

Viktor Ruban
(Choreographer, independent culture diplomat, Ukraine) —
Coping potential of creativity and art-practices in times of war: culture diplomacy, fundraising, curatorship and art-therapy force project.

If symbiosis is an essential notion, it is even more so in today's geopolitical context. War, terror and massive extinction of people in Ukraine by Russia is a global challenge shaping our global future at this very moment. In this situation performing artists in Ukraine keep on strong commitment to help in any possible way. Since the beginning of the full-scale invasion in 2022, the situation is forcing us to search for not only new ways of dealing with challenges creatively but to discover new ways of implementing our art practices and skills to completely new levels and spheres, such as resistance in information wars, culture ecology maintenance, new forms of fundraising and help with physical and psycho-emotional recovery — same as military and civilians —.

During his session Viktor Ruban, shared information about initiatives that he initiated or is involved in, such as the European Culture Parliament, and culture diplomacy challenges that he faces on different international events; Ukrainian emergency performing arts fund and funding challenges for the independent performing arts scene in Ukraine; international solidarity events and visibility of Ukrainian actual art scene—why it is important; actual creations in Ukraine and trends seen through the actual national theater prize season; the development of the project for training “psychological first aid instructors for military from the front line” and Art therapy force project— a range of activities implementing art-practices and working with creativity for psycho-emotional health recovery, coping with stress and panic attacks as well as preventing self-destructive behaviors and PTSD for diverse groups of people. 

Biography: Viktor Ruban is choreographer-researcher, curator, performer, educator, independent culture diplomat and culture activist. Initiator and ambassador of Ukrainian Emergency Performing Arts Fund initiative, he represents Ukraine in European Culture Parliament. He is also program director and co-founder of venue #KyivDanceResidency—platform for international studies in somatic, dance and performative practices, movement-based art and research. Ph.d. student in culture studies of Modern Art Research Institute of the National Academy of Arts (Kyiv, Ukraine)

Viktor Ruban © DR

Michael Century
(musician and cultural theorist, Rensselaer Polytechnic Institute, NY) — 
Nonsynchronous Innovation: Periodizing the Digital

While the digital transformation continues to outpace socio-institutional adaptation, the technological arts have moved on to vastly expand their temporal horizons. Timescapes of artistic research and creation now embrace the residual as much as the emergent; techno-diversity against digital solutionism; sympoietic rather than linear models of innovation. But nonsynchronous innovation is hardly unique to the current moment, as revealed in Michael Century’s recent book Northern Sparks, on Canada’s early experimentation with digital media.

Poised as a “counter-environment” to the great powers, in McLuhan’s phrase, Canada’s experience of the transitional decades into the information age was grounded in a technological ethos that emphasized sensorial immediacy, embodied interaction, and improvisatory expression. This alternative ethos was situated between a pair of distinct yet inextricably bound forces, one national-political and proper to Canada, the other techno-mediatic and global in scale.

The unraveling of these forces by the late millennium reveals innovation itself as a complexly drawn process comprised of multiple layers with fluctuating degrees of synchronization. From a cross-media perspective, Northern Sparks also reveals how the differences between the arts with respect to improvisatory immediacy and discrete formalization make any neat chronological periodization of the digital problematic.

Biography: Michael Century, musician and cultural theorist, is Professor of New Media and Music at Rensselaer Polytechnic Institute. His book Northern Sparks: Innovation, Technology Policy, and the Arts in Canada from Expo 67 to the Internet Age appeared with MIT Press in 2022. At the Banff Centre, he founded the Media Arts program in 1988. Century’s works for live and electronically processed instruments have been performed and broadcast in concerts and festivals internationally.   

http://www.nextcentury.ca
https://vimeo.com/showcase/10249242

Michael Century © Nadia Rabhi

Short Papers

Armand Edwige, Asensio Anne, Foresta Don 
Art, Science and Industry: A symbiotic milieu to rethink visions of the world

Bakke Monika
Symbio(geo)sis: When mineral and biological species meet

Blanchi Yann
4E Cognition for Symbiotic Architecture?

Bourgeois Mariejulie
A [Potential] Cloud War Controversies and conflicts related to climate manipulations

Boutilier Célia
Inhabiting the Edges

Cucicov Dorin
Mytherrella: an interactive installation hallucinating mythological auroral formations

Frémontier-Murphy Camille
At the Sources of an Artistic Mutation towards Science: the First Years of the Journal Leonardo (1968-1981) as a Forum for the Pioneers of Digital Art

Garančs Jānis
Sensoriums for the Ephemeral —gamification of values

Geck Kate
Mycorrhizal Materialities Positioning the entanglement of human and machine intelligence

Andrea Gogova
Interspecies Communication — Water bodies

Hedayati Mona
Intelligent Sensibility: Human-Machine Symbiotic Agencies

Hilsberg Victoria
DAOs A blockchain-based application not intervening, but strengthening the agility of contemporary arts

Hu Youyang, Chou Chiaochi, Kakehi Yasuaki
Lucid Dream: Sensing and Artistic Representation of Plant-Nature Interaction Based on Plants Biosignals

Jørgensen Jonas, Christiansen Mads Bering
Sounding Softness and the (Artificial) Subject

Lukaszuk Mike, Bahng Sojung
Gesture and Spatiality in Electroacoustic Improvisation with Digital Video

McFadden Chloe
Generated tools: A Defamiliarizing Approach to Creating ML Art

Montanari Matheus da Rocha
Ecologies of Thought: Generative Art as a Collaborative Research Methodology with Guarani and Kaiowá Indigenous Communities

Moren Lisa, Bachvaroff Tsvetan
Emerging Strategies “Under The Bay” in AR/XR

Nieves Raul, Soler-Adillon Joan, Mor Enric
Power and Resistance in Digital Degrowth

Ottavi Julien, Pickett Jenny
Motherplants: Mycelium Network and Artistic Research

Pederson Claudia
Life-as-it-could-be, Symbiosis in Interspecifics’ Codex Virtualis_Genesis

Pranjić Kristina, Germek Magdalena, Purg Peter
Disruptive Avant-Garde Art of Today: Shaping Post-Growth Imaginaries for Symbiotic Futures

Ricci Nicolas, Agogué Marine
Drivers for Resilience in Cultural Organizations: lessons from the Montreal festivals in the face of the COVID-19 pandemic.

Rogers Hannah, Bencard Adam
Metabolism and Art

Sareen Harpre, Singh, Fu Yibo, Kakehi Yasuaki
Ephemera: Bubble Representations as Metaphors for Endangered Species

Scott Jill
Jellyeyes — Symbiosis, Evolution and Vision

Shanken Edward
Technoshamanism: Symbiotic Techniques of Art and Healing

Shin Gyung Jin
Symbiotic Collaborators: The New Creative Subject in Postdigital Participatory Art

Smigielska Maria, Cutellic Pierre
Proteus 3.0 — Interacting with the cloud

Steyaert Pieter, Vermeulen Angelo, Holt Amy, Maranan Diego, Checola Giusy, Nasser Mona, Verschuren Jeroen, Tournaye Pim
Co-creation Towards the Post-Anthropocene

Thornhill Vincent, Legrand Guillemette, Clarke Isaac
Spectral Plain: A case study for exploring the world-building potential of co-creative systems that combine text generation models with game mechanics

Trudel Gisele
Ecotechnologies of Practice: in-forming changing climates

Wollensak Andrea, Terry Brett, Baird Bridget
Water Stories: Visual Poetics and Collective Voices

Wright Rewa, Howden Simon
Nga manawataki o te koiora: biological rhythms, posthuman design and decolonial thought

Yeginsu Ipek
The Dark Side of the NFTs: The artists’ need for new systems of collaboration

Zhang Yu
STAND BY/ME

Art, Science and Industry: A symbiotic milieu to rethink visions of the world

Edwige Armand, Anne Asensio, Don Foresta 

Université Gustave Eiffel, LISAA, Passerelle Art-Science-Technologie Dassault Systèmes, MapMarcel, France edwige.armand@gmail.com, anne.asensio@3ds.com, don.foresta@gmail.com

Abstract

How do art and science participate in writing a world of signs and symbols? What do these two human faculties and capacities seek and how do they differ radically? Science has progressively freed itself from its purpose (the search for the why) to model, imitate and master a world in an increasingly abstract manner. If science and art are performative, what world do they draw and what do they tell us (in terms of discourse) about the human? As science gradually becomes techno-science and produced by industries, does art have a singular role to play in the interrelation of these two poles, science and industry, in order to re-inhabit the ethical questions and meanings that were originally in these other two creative capacities? Can it and should it? 

Keywords

Art, Science, Industry, Signs, Sensibility, Technoscience, Blindspot, New cooperative ecosystem, incorporation of experience.

DOI

10.69564/ISEA2023-2-short-Armand-et-al-Art-Science-Industry

Art-science and world shaping/forming

We will share an assessment of the relations between art-science and industry, the novelty being that with digital technology the questions have been accelerated, augmented and have now come to integrate the notion of physicality. Industry is a playground; whether it is science, art, how do we seize the digital? Do we have the means to think the digital or are we alienated, with the digital no longer as an externality, an interface, but as an immersion, as a new space of being in the world—particularly with the metaverse—as a socio-technical space? It is necessary to think of industry as a workbench for virtual universes and as a space of reconciliation between disciplines via the designers as sensitive mediators and scouts of the new virtual worlds. We need a new epistemology of the virtual worlds that takes into account the multi-dimensional human. We are looking for ways between micro-ecologies, localities, craftsmanship and a global, automatic approach, the two poles that build our "being in the world."

Art could be thought of as the fabrication of new categorizations of an environment¹ that escapes the norms of language and conventional imaginary modelling. Because the perceived world is always the result of an interpretation and a shaping which are imposed a posteriori by culture, that the inaugural perceptual singularity is subsumed by culture. The perceptual thresholds linked to temporal representation structure a vision of reality constructed by stops and artificial linearity². Yet the imaginary, the perceptual virtual cannot be stopped at these arbitrary forms of actualization of the world. Whether it be time, imaginary, or spatial perception, they cannot be detached from a body in action that perceptively actualizes a world that appears and is created in a living time (dancers or performers give rhythm to time, for example). The manifestation of the event is an extract of the real, obliterating the infinity of data, infraliminary and supra-liminary information³ not actualized by language whose readings cannot reach the threshold of a disciplined thought. It is in this failure of a completeness of reality that art operates.

It is an attempt to reveal hidden information inaccessible to authorized cognitive processing. Other articulations, forms, relations to the world exist that art tries to bring to light. It is a raw, unanalyzed grasp⁴ that the act of formalization and expression will seek to bring into existence. Art is, first of all, an impossible return to an instinctive and affective perception, where the body echoes the world and where the world is then echoed (exosomatisation). Art in these premises is an anchoring (Harnard Stevan⁵) of signs of the world kept in memory (retention), attached to and mirroring the body that perceives itself (while keeping the unspeakable abyss which Pierre Legendre refers to ⁶).

As a first reflexive return, art imprints the world in order to reflect on it and to appear itself (first distance from the world—see the cave paintings and the work of Carole Fritz⁷). It is a primal affective reading of a world that is not there for it, and for which it seeks a meaning. It is an attempt to answer the abysmal question of why and of a world that is not up to his standards, but in which, through the inscription of signs, he can make his presence last beyond his own finitude.

As the first interface between the body and the environment, art is a sign of principle writing. As a symbol and practical return of a memory and of an instant kept⁸, it is a means for recalling presences. It is an imprint of an existence and of a difference made with the self and the environment, the self and the other, who delimits and recognizes itself in it. Through the diffusion of signs and symbols, language and rituals, the particularity of a seized moment becomes an abstraction, a practical medium to think about the world and save the detailed and vivid perception that every physical and phenomenal novelty requires. But if art has to do with the first symbolic anchors, the first perceptions, it cannot be reduced to them.

Art is a first reading, a projection of human meaning on the world, and in this, science does not differ from this attempt to understand and mark the world. In science, the purpose is different, the tools are more sophisticated, no singularity is demanded and the dream of a universal and standardised objectivity.

Art and science are imaginary and projective capacities, part of a process of fabrication and transformation of the world. This capacity for transformation by modern science has largely emancipated itself from its role as a myth whose function is to answer the question of why live and to limit unreason in order to support existential anguish⁹.

But art and science are not only intended to think about the writing of a world and its human legibility, but also to put it into the world, to give it a stage, to create a narrative for it and for the human being. Their discourse and imagination are performative.

In contrast to science, art is anchored in the body and the affection of the world, turned towards the appearance, the distinction, the singular discrimination of signs as a search for actualization. Science in its hyper-technical devices sets aside this affection for the body, and is based on instrumental rationality.

It is important to underline representation acts as a perceptual framework indicating the phenomenon and giving it an interpretative form in order to apprehend it.

Science, as a rationalized exo-body, it gives us the opportunity to perceive a phenomenon (a techno-phenomenology) and a world framed in advance by the laws inserted in it, offering a metric vision and mathematical logics, influencing in its turn the way of writing and understanding a world surrounded more and more by technical abstraction, self-referential and discarding the perceptive variability of a sensitive and feeling body.

Techniques are extensions of organs¹¹ reading and writing the signs of the world. Art has taken an interest in the world of science and technology out of a spirit of resistance (perhaps) to the homogenization of bodies and sensibilities, out of a refusal to accept the authority of scientific discourse, which sometimes imposes "butchering"¹² and reductionist views of the human being. The definition of man varies according to culture, religion, and time. Feeling human, our relationship to the world is defined according to the interpretative frameworks conveyed in a culture, which cannot be absolute. We know that the way in which man is expressed influences in turn the experiential potential and the power of his experience.

“Cogito ergo sum”

“Cogito ergo sum” of Descartes has been the heart of the scientific domination of our society’s world-view, the way we understand the world defined uniquely through reason. It has been responsible for an unequaled level of material progress but has reduced humanity into entities befitting uniquely mechanical formulation, ignoring the whole human being, concentrating only on those aspects of the person considered important for the task at hand. In recent western history we, as a culture, have tried to deny this human wholeness by declaring that rationality was the only approach to true knowledge and that all else must be rejected. Anything resulting from feeling or emotion was unreliable and even dangerous. That added to the mechanically quantifiable view of how things work has created a distorted perspective, a world-view based on a diminished idea of the human being. Focusing on the intellect was a useful but limited approach to how humanity is defined putting aside, "Sensio ergo sum"—I feel therefore I am—which is how most people first react to sensory input. Recognising that aspect of humanity not only allows us to understand the whole human by respecting the other half, it also connects our species with the rest of the living world which reacts principally through feeling. Objectivity is a tool, an important one, but to imagine humans cut off from their emotional side is dangerous and leads to often undefined but very real frustration as we see more and more today.

The capacity for symbolic thought has always been part of our makeup in its earliest primitive forms since the beginning of our genus and most probably well before. It comes from an instinctive and unintellectual operation of that curiosity which is part of every living being’s functioning as a way of exploring our environment and surviving in it. When the found object of our ancient ancestors, the first overt manifestation of what we now know as art, was kept and shared with others it acquired a symbolic sense for the group as well as the individual who found it, an unarticulated meaning but something felt to be important. With the evolution of our intellectual capacity, this process becomes more elaborate in providing a «why» to that selection giving those objects an explanation of their symbolic value and a means of communicating it and often the beginning of what can be seen as a religious belief system. In the beginning, this was not an intellectual exercise and the choosing came from feeling rather than understanding. It became more intellectually elaborated in time through the need to be communicated, probably diluting the experience of the first-person experience but making it transferable to others and incorporating other reactions to it.

This is the root of artistic creativity and its eventual integration into culture. The symbolic is very much part of us and ignoring it or pretending it is less important or to be overcome demonstrate a dangerous misunderstanding of how humans function. To attempt to understand our confrontation with the real, both art and science are needed for a fuller and more human comprehension of what we are defining. Understanding intellectually is necessary but stopping there leaves the human wanting more, which defines the emotional atmosphere we live in today. We cannot expect people to understand when half the person is ignored. In our mechanical world-view art has been pushed out of its central role of informing into manageable categories to fit the demands of the market it has been confined to.

Artists, on the other hand, have never ignored science, particularly in a society which has allowed science to be the sole arbitrator of our reality. Science has thus become the matter of art, our reality subjected to the questioning of art. Technology, the product of science, has always attracted artists because of a curiosity about what humans do but also because it is a potential tool, particularly the technologies of communication. Importantly as well, the artists’ use of technology has often influenced our perception of it and how it is integrated into society and affects us.

A technology developed through both art and science would, by definition, have more dimensions by being the product of the full human being. This has been the objective of our project MARCEL, an international group dedicated to artistic, scientific and educational uses of the network and expanding network technologies through the demands of art and not just market-driven consumption. It exists and evolves through the share experiences and technical development from creative uses of the network pushing the boundaries of the possible. This is also the approach that the TRAS network (transversal network of art-science networks) is trying to put in place in France. As are the creative platforms in industry that seek to give meaning to technology, since industry is the heart of its production.

Industry as a blind spot?

Industry is perhaps a blind spot in our times. Principally thought of as an operative process, a technical milieu and not enough as a thinking object. As a civilization manifestation of a way of seeing the world. Industry always been a space of intervention and convergence in which an art-science dialectic can flourish.

At the crossroads of the most advanced human creative activities, the industry is projecting a vision, organizing the work and the economy defining every aspect of everybody’s life. The industry today manufactures the eye and the prosthetic body of tomorrow, just as science and art have done. To think of working in theatrical and artistic terms on science, digital environment or technologies that enable all aspect of the modern world without taking into account the environments in which techno science is constructed may be a mistake. The question of the representation of the body, of what constructs the human experience, of what links the human to the world has never been so obvious since the announcement of the promise of virtualization and modelling of the human being, from his intimate relations to the notion of habitability. Virtual worlds (beyond the definition of Metaverse) promise us a world in which the human experience could be reconfigured in a bespoke manner without the canal of affective and invisible human dimension, of the anchoring of the symbolic through the affective path, of the word that introduces us to subjectivity and otherness, a word that confronts us in its strangeness and its inexhaustible incompleteness.

The standardization of the perceptible, the logical and scientific determinism could reach its apex in these new universes that redesign the conception of the human. It is therefore in the interest of artists and designers to present the eminently open conception of a world thought out by art and designers so as not to close the vision of the human to an automaton-avatar with which it would be a question of playing and moving in the way of a puppet. A human representation that disregards the gravity of bodies, the rites of social interaction, the collective to be woven, the immersion in a living half-place where the segregation of the visible is decided upstream and selecting what can be seen from what does not deserve to be seen. The human being is not a pack of information to be grasped, he is a being of dreams and words and it is better to know this than to ignore it. It is in the interest of art and design, philosophy and anthropology to integrate these decision-making circles of future representations that will be conveyed with an unequalled power of diffusion and democratization. There is a stake in everyone being able to grapple with the question of the relationship to the environment that we can rethink, the adventure of discovery, of the process of making, of surprise and of creation, which is never elaborated in predetermined data, but which emerges from the unpredictable and the unknown.

How can we open up these industrial worlds to the civilizational challenges they face today? How can we make people understand what design can do at the intersection of art, engineering and industry? Design is the keystone of the human intentions of any industrial project deploying the openness of the human senses, of human abilities and expectations into a project, in a tangible and accessible manner improving the human benefice “in fine”... Design is drawing and aim at the same time, a potential direction as a line of conduct and orientation to be incorporated into any industrial and societal project... How can we reintegrate meaning upstream of this productive force so that it serves a majority in humility and equity? What are the first tracks and trials of these co-operations and new ecosystems where art-science-industry work together again? What are the first attempts made by industry, which must rethink itself at a time of climatic emergency and where the production of waste objects, of intensive innovation based on the imagination of mass production, of Taylorised work and progress must be reinvented? How can we reinstate the sensitive in these industries that produce techno-imaginaries? How can we reconstitute noetic spaces ranging from industrial environments, experiential hybrid environments, to virtual immersive universes (metaverse)?

Can we imagine and draft new roadmaps to rehabilitate Art without its instrumentalization in the scientific and industrial spaces where today most of our narratives and actions that construct representations are being produced? Art is probably the future of industry understood as capacity for representation for its power to create new bodily and cognitive experiences, and its possibility of creating dreams and new solidarities. This implies putting art back into life, as medicine of the imagination, as the art of being together. Should it, can it?

Such regenerative new hybridization and such new forms of cooperation would represent a new symbiosis of the creative forces of the human being, in which art would, in that case, become design, design as a mindset and “way to think.” Science and industry could collaborate for human and non-human benefit driven project. A project which path remains open to matter, to the singular, to the body in “totality,” to the living, to the unpredictable and immeasurable, which is the nodal point of invention.

This utopian symbiosis we are describing is what we feel is right to focus on, in order to build the narrative and the bases of a dialogue to be reinvented so that all human faculties are no longer dissociated, but reunited to serve an imminent civilizational project. Considering all the living in all their diversity and unknowability as the preciousness of life should be taking into account in the course of our actions. We need a holistic and systemic approach within the limits of the consequences of the harmful and positive impacts that we can and must imagine. Industry must reclaim its original role of building a world¹³ to be improved, to be inhabited for all and in the perspective of diversity. It is a call for a collaborative symbiosis that takes into account the complexity and the need to work together, despite the stereotypical oppositions that prevent these human universes from interacting again.

Beyond the civilizational stakes, industry is today a playground for art and science, and dialectic between art and science is vibrant. It questions their relationship, the tools they manipulate, but also because industry must regenerate its raison d'être (get out of consumerism, functionalism). It must become organic. It is when we have a calmed art-science dialectic that industry will be able to resolve these new issues.

Another question comes to mind: does industry contribute to the discourse of technoscience, or is it the victim or the keystone? With cultural and economic globalization, its responsibility is taking the main stage... in its making of the world and tomorrow's world, the industry cannot be reduced to a means of progress nor an externality functionally assess but as a subject for thought, an object for design... and Art. Shouldn't the hyper-strategic position of industry be aestheticized as an experience with methods of art and science, i.e., art applied to industry which is called design? Industry has reached a level of maturity that is no longer under the yoke of politics, it is political. As such, industry must reconvene art and science within itself according to its own responsibilities. An industry that thinks, reflexive to change the world will not arise without the aesthetic and ethical sense. An industry more in tune with a regenerative mindset will enrich a new form of relationship with Nature, Human and the world, an artistic approach from within, to nurture a sustainable future and solutions that undoubtedly will be found.

References

1 Gilbert Simondon, Imagination et Invention, 1965-1966, Paris, Presse Universitaire de France, 2014.

Henri Bergson, L’évolution Créatrice, Paris, Presse Universitaire de France, 2013.

3 Günter Anders, Et si je suis désespéré que voulez-vous que j’y fasse, Paris, Allia, 2010, p.72.

4 Charles Sanders Peirce, Écrit sur le signe, Paris, Seuil, 1978, p.23.

6 Pierre Legendre, Les enfants du texte, Paris, Fayard, 1992.

7 Carole Fritz, L’art de la préhistoire, Paris, Citadelles et Mazenod, 2017.

8 Ernst Cassirer, La philosophie des formes symboliques, le langage, Paris, Edition de Minuit, 2017.

9 Pierre Legendre, Le visage et la main, Paris, Les belles lettres, 2019.

10 Jakob Von Uexkull, Milieu animal et milieu humain, Paris, Broché, 2010.

11 André Leroi-Gourhan, Le geste et la parole, Technique et langage, Paris, Albin Michel, 1964, p.49.

12 Alain Supiot, La gouvernance par les nombres, Paris, Fayard, 2015.

13 Pierre Musso, La Religion Industrielle, Paris, Fayard, 2017, p.12. Journal article (online)

5 Stevan Harnard "Le problème de l'ancrage des symboles" Physical Review D, 42, accessed November 7, 2022, https://archipel.uqam.ca/14111/ 

Bibliography
  • Anders Günter, Et si je suis désespéré que voulez-vous que j’y fasse, Paris, Allia, 2010, p.72.
    Bergson Henri, L’évolution Créatrice, Paris, Presse Universitaire de France, 2013.
  • Cassirer Ernst, La philosophie des formes symboliques, le langage, Paris, Edition de Minuit, 2017.
    Fritz Carole, L’art de la préhistoire, Paris, Citadelles et Mazenod, 2017.
  • Harnard Stevan, “Le problème de l'ancrage des symboles", Physical Review D, accessed November 7, 2022, p.42, https://archipel.uqam.ca/14111/
  • Legendre Pierre, Le visage et la main, Paris, Les belles lettres, 2019.
  • Legendre Pierre, Les enfants du texte, Paris, Fayard, 1992. 
  • Leroi-Gourhan André, Le geste et la parole, Technique et langage, Paris, Albin Michel, 1964, p.49.
  • Musso Pierre, La Religion Industrielle, Paris Fayard, 2017, p.12.
  • Peirce Charles Sanders, Écrit sur le signe, Paris, Seuil, 1978, p.23.
  • Simondon Gilbert, Imagination et Invention, 1965-1966, Paris, Presse Universitaire de France, 2014.
    Supiot Alain, La gouvernance par les nombres, Paris, Fayard, 2015.
  • Von Uexkull Jakob, Milieu animal et milieu humain, Paris, Broché, 2010. 
Authors’ Biographies

Edwige Armand is an artist and a teacher-researcher in art and culture at the Purpan engineering school (INP Purpan) and is attached to the LARA-Seppia laboratory. Passionate about the history of science and art, and philosophy, she combines these different disciplinary fields around the question of techniques and sciences. The modifications of the relationship to reality, subjectivity and the body by techniques and sciences are central axes in her research-creation. The transversality of disciplines allows him to question more broadly the processes of creation and the transformations of representations that the arts contribute to. To revive the dynamics of art-science relations, she co-founded and has chaired since 2016 the association Passerelle Art-Science-Technologie, which works to bring these disciplines closer together, and she is also involved in the Transversale des Réseaux Arts Sciences.

Anne Asensio, after executive and design management positions in the automotive industry, joined Dassault Systèmes in 2008 as Vice President Design Experience. She created the Design function of Dassault Systèmes and the Design Studio, bringing together a multidisciplinary team in innovation strategy through design, experience design and design research. Advocating a participatory approach to new technologies and virtual worlds, the Design Studio supports Dassault Systèmes' customers in high-growth industries in their transformation, digital and sustainable innovation needs towards virtuous design processes. Imagining alternative scenarios to transform the world we live in into a more sustainable and desirable one, the studio engages in reflection, experimentation and confrontation of cross-cutting approaches between social and technological issues through creation.

Don Foresta is an art theorist using new technologies as creative tools. Specialised in art and science, he has been a professor at ENSAD and the Ecole Nationale Supérieure d'Arts - Paris/Cergy and a research fellow at the London School of Economics. He has spent 35 years transforming the network as an artistic tool and is working on the creation of a permanent high-speed network, MARCEL, dedicated to artistic, educational and cultural experimentation. In 1981, he made his first online exchange between the Center for Advanced Visual Studies at MIT where he was a fellow and the American Center in Paris where he was director of the Media Art program. In 1986, as curator of the 42nd Venise Biennale, he made the first computer network used by artists. He was named Chevalier de l'Ordre des Arts et des Lettres by the Ministry of Culture. 

Symbio(geo)sis: When mineral and biological species meet

Monika Bakke

Adam Mickiewicz University Poznań, Poland monika.bakke@amu.edu.pl

Abstract

As biocentrism is increasingly seen as unjustifiable, artists and environmental humanities scholars become more attentive to feedback between minerals and (non)human life. First, I discuss the urgent need to reinvent narratives and create new vocabulary about the abundance of life-mineral connections as presented by Mabe Bethônico in her Stone Statements Editions, which focused on developing "a vocabulary of proximity." Next, I analyze Magdalena Abakanowicz’s artwork Space of Unknown Growth, investigating emerging material relations between concrete (an anthropogenic rock) and nonhuman life. I argue that rather than limiting the environment to a symbiotic community, these artworks encourage the curiosity and attention necessary to embrace mineral-life connections and envision more inclusive communities of the future. 

Keywords

Art and science, geology, minerals, technofossil, concrete, Long Ecology, biocentrism

DOI

10.69564/ISEA2023-3-short-Bakke-Symbiogeosis

Introduction

Biocentrism is increasingly viewed as unjustifiable. Minerals are everywhere, but in Western thought, nonlife has always been significantly overshadowed by robust, organic life. The multispecies studies have so far embraced the diversity of mineral species only in a minimal capacity. Although biocentrism still predominates, change seems inevitable. The currently emerging trend in the humanities and arts to truly embrace mineral species (considered nonlife or geos) is growing in significance, and it is attentive not only to environmental concerns but also to new developments in mineralogy. The latter focuses on the co-evolution of minerals and life and the unprecedented increase in the diversity of human-made and human-mediated mineral species and novel materials characteristic of the Anthropocene.

Vocabulary urgencies

To better recognize the complex relations within geo-biotic communities, it is necessary to reinvent attitudes and narratives about mineral species. This need was addressed by Brazilian artist Mabe Bethônico, who called for the development of a «vocabulary of proximity» of minerals and life. 

Bethônico created StoneStatements Editions in response to the question: How Will We Live Together? posited by the curators of the Venice Biennal of Architecture in 2021.

Bethônico’s editorial proposal includes five inspiring book tiles: Geoimaginaries; Conscious Rocks; Human Invasion in the World of Stones; When Stones Collect Diggers, Robbers, Queens and Kings; Missing Words for Considering Stones, Rocks, Pebbles and Mountains: A Vocabulary of Proximity. Only the latter has been developed so far as a collective effort, resulting in a volume of twenty-six short entries. As Bethônico stated, the remaining books are «imagined, wished for, and suggested to be developed» as more vocabulary to think about the environment in more than biological terms is urgently needed. Her project recognizes the biocentric bias and the conditions of its operation, namely the shortage of concepts, stories, and creative practices to embrace mineral species.

Figure 1. Mabe Bethônico, 2021. Cover design by Elaine Ramos. © Mabe Bethônico.

Anthropogenic rocks

The proximity of life and minerals, especially rocks characteristic of the Anthropocene epoch, is investigated and anticipated in the artwork Space of Unknown Growth by Polish artist Magdalena Abakanowicz. Located in the forest of Europos Parkas, this artwork comprises twenty-two concrete boulders in four sizes, spread over an open-air museum near Vilnius. Concrete, although anthropogenic, is considered a "geological entity" due to its close relation to the mineral composition of the earth. It is a novel rock and, as geologist Jan Zalasiewicz suggests, “the most abundant anthropogenic sedimentary rock on the planet.”² Classified as a technofossil together with a great variety of synthetic compounds, concrete is emerging as the prominent signature of our geologic epoch. There is, however, a particular risk of succumbing to the lithic allure recognized by Stacy Alaimo, who calls for caution when suggesting that “attending solely to the lithic imports delusions of separation and control."³

Figure 2. Magdalena Abakanowicz, Space of Unknown Growth, 1998. © Europos Parkas. 

Space of Unknown Growth speculates about a future that belongs to multispecies communities whose endurance is anticipated rather than predicted. For surviving, making kin with unexpected others, as Donna Haraway suggests, is necessary. Staying with the concrete means "staying with the trouble", which demands that the viewers acknowledge the devastating environmental impact of extracting resources and recomposing materials.⁴ And yet, Space of Unknown Growth does not convey alarm. Instead, it points out that adaptation goes together with the transformation of anthropogenic lithic environments, which accommodate more than human life and is a condition needed to renew planetary species diversity. Living on the ruins is possible as some organisms can modify the lithic surfaces in their environment. The technofossils that comprise Space of Unknown Growth indicate the anthropocenic Earth’s habitability. The anticipated growth speculated on in Abakanowicz’s work is conditioned by the composing and decomposing of biological and mineral species and the resilience of communities capable of terraforming concrete—the most earthly rock.

Long Ecology

Bringing anthropogenic stone to the forest introduces a perspective of what Jeffrey Jerome Cohen calls Long Ecology oriented toward the future and vastly exceeding current waste management. However, concrete, as “a material that mediates transcience and endurance," is not as long-lasting as stone.⁵ It rejects the conventional belief that geological processes are always long and slow. From Long Ecology's perspective, although prolific in novel mineral species and formative for technofossils, our epoch might not be capable of reaching a distant future because anthropogenic minerals might become extinct.⁶

And yet the stones of Space of unknown growth might be considered erratics, conventionally defined as rocks not matching the local bedrock. Erratics are lithic strangers, newcomers transported to current locations by glaciers or other natural forces of significant magnitude. Concrete boulders of Space of unknown growth are synthetic rocks and mineral novelty brought into the environment of a central European forest. Yet, Abakanowicz’s erratics establish a category of their own as they are art forms, which nevertheless participate in the currently intense processes of relocating minerals by humans—activity believed to exceed even glacial action. Abakanowicz’s lithic newcomers, however, with their own material history, become with their new environments through complex geo-bio relations and processes. These unexpected others, strangers to their landscapes, arriving from a different time and space, introduce the possibility of a radically new narrative of connectedness and multispecies kin-making, which stimulates yet another way for humans to practice attentiveness and care. They demonstrate both vulnerability and resilience, which characterize earthly worlding and unworlding. 

Figure 3. Magdalena Abakanowicz, Space of Unknown Growth, 1998. © Europos Parkas. 

Although the sculptures operate as erratics in the context of Long Ecology, there is no separation between them and the materiality of their location. Through their lithic ways of being, the works participate in planetary processes operating, in Cohen’s words, as “...an affectively fraught web of relations that unfolds within an extensive spatial and temporal range, demanding an ethics of relation and scale.”⁷ In this sense, Space of unknown growth is deeply ecological in its attentiveness to the vast scales in which life becomes stone and stone becomes a living environment.

Coda

What we need now is not a belief in the solitude and indifference of stones, rocks, and pebbles but a celebration of their inextricable connection with life and creative elaborations on their diversity. Both StoneStatements Editions and Space of Unknown Growth inquire about ways to articulate how minerals and life actually attract and seduce each other and how minerals open up to life’s intimate strategies, the outcomes of which cannot be determined. They offer vocabulary and materiality to tell stories of Symbio(geo)sis. Rather than limiting the environment to sym-biotic community, these works encourage curiosity and attention needed to embrace mineral-life connections better. Only then will geo-biotic communities emerge as the answer to the question of how we will live together. 

References

1 Mabe Bethônico, “StoneStatements Editions,” Mabe Bethônico, accessed September 3, 2023, https://www.mabebethonico.online/stonestatements-editions.

2 C.N. Waters, J. Zalasiewicz, “Concrete: The Most Abundant Novel Rock Type of the Anthropocene,” in Encyclopedia of the Anthropocene, ed. Dominick A. Dellasala, Michael I. Goldstein, Oxford, Elsevier, 2018, p.75, https://doi.org/10.1016/B978-0-12-809665-9.09775-5.

3 Stacy Alaimo, Exposed: Environmental Politics and Pleasures in Posthuman Times, U of Minnesota Press, 2016, p.149.

4 Donna J. Haraway, Staying with the Trouble: Making Kin in the Chthulucene, Duke University Press, 2016.

5 Richard D. G. Irvine, Anne Bevan, “Concrete Buys Time: Art and Anthropology in the Anthropocene,” Worldviews: Global Religions, Culture, and Ecology 26, no. 3, October 19, 2022, p.182, https://doi.org/10.1163/15685357-02603009.

6 Jan Zalasiewicz, Ryszard Kryza, Mark Williams, “The Mineral Signature of the Anthropocene in Its Deep-Time Context,” Geological Society, London, Special Publications 395, no. 1, 2014, p.112, https://doi.org/10.1144/SP395.2.

7 Jeffrey Jerome Cohen, Stone: An Ecology of the Inhuman, U of Minnesota Press, 2015, p.41. 

Bibliography
  • Stacy Alaimo, Exposed: Environmental Politics and Pleasures in Posthuman Times, U of Minnesota Press, 2016.
  • Mabe Bethônico, “StoneStatements Editions", Mabe Bethônico, Accessed, September 3, 2023, https://www.mabebethonico.online/stonestatements-editions.
  • Jeffrey Jerome Cohen, Stone: An Ecology of the Inhuman, U of Minnesota Press, 2015.
  • Donna J Haraway, Staying with the Trouble: Making Kin in the Chthulucene, Duke University Press, 2016.
  • Richard D. G Irvine, Bevan Anne, “Concrete Buys Time: Art and Anthropology in the Anthropocene”, Worldviews: Global Religions, Culture, and Ecology 26, no. 3, October 19, 2022, 179–95, https://doi.org/10.1163/15685357-02603009.
  • C. N. Waters, J. Zalasiewicz, “Concrete: The Most Abundant Novel Rock Type of the Anthropocene”, In Encyclopedia of the Anthropocene, edited by A. Dominick Dellasala, Michael I. Goldstein, Oxford: Elsevier, 2018, 75–85, https://doi.org/10.1016/B978-0-12-809665-9.09775-5.
  • Jan Zalasiewicz, Kryza Ryszard, Williams Mark, “The Mineral Signature of the Anthropocene in Its Deep-Time Context”, Geological Society, London, Special Publications 395, no. 1, 2014, 109–17, https://doi.org/10.1144/SP395.2.
Author Biography

Monika Bakke is associate professor in the Philosophy Department and director of the Environmental Humanities Center at the Adam Mickiewicz University, Poznań, Poland. She writes on contemporary art and aesthetics, particularly interested in posthumanist, transspecies, and gender perspectives. Her curatorial work includes art exhibitions: Bio-Reminiscences (Poland), Seeing the Forrest Through the Trees (UK), Boundless Objects (Portugal), and Refugia: Keep (Out of) these Places (Poland). Currently, her research focuses on nonlife forces and new articulations of mineral becomings in contemporary art. https://amu.academia.edu/MonikaBakke 

4E Cognition for Symbiotic Architecture?

Yann Blanchi

BMOffice France yann@bmoffice.fr

Abstract

With the aim of reconsidering the very nature of architecture, we propose a conceptual tool to think of architectural apparatuses as actors within a continuum composed of both artificial and natural agents. For this,we look at the cognitive sciences, particularly the 4E cognitive (embodied, embedded, enactive, and extended). We use examples from contemporary architecture to test our hypothesis and thus attempt to define what symbiotic architecture could be.

Keywords

4E cognition, Embodied, Embedded, Enaction, Extended, Symbiotic, Architecture, Artificial, Natural

DOI

10.69564/ISEA2023-4-short-Blanchi-4E

Introduction

In this paper, we will explore a theoretical tool to think symbiotic architecture. To this end, we consider whether the transposition of the 4E cognition may be useful in describing the characteristics of what the symbiotic architecture might be. The paper is organized as follows: Firstly, we set the context and define the terms used, such as symbiosis and 4E cognition. Then we state our hypothesis of an analogy between 4E cognition and architecture. Next, we test this idea with examples of recent architectural apparatus, and we conclude.

Context

In 1960, as Gordon Pask stated in his theory of conversation, we moved from a functionalist world to a mutualist world. Today, in the context of the global crisis, nature is coming back into the loop. The ecocentric logic would reinscribe us in a network of multiple interactions and concrete connections with the milieu, engaged in a network of dependencies in a way of decentralized connections.Nature and artifice are bound to co-evolve. Under these conditions architecture can certainly no longer be the static and fixed object of the moderns but a new type of artefact. Architecture may have entered the age of naturalization.2

Definitions

Etymologically, the term symbiosis comes from the Greek and means "living with." In biology, symbiosis refers to associations between hetero specific living organisms. There are three forms of symbiotic association:parasitism, commensalism and mutualism. It depends on whether the relationship is unidirectional or bidirectional and beneficial, neutral or harmful. Semiologists study these interactions. On the other hand, in the field of cognitive sciences, the 4E cognitive approach proposes a conception of cognition as an assembly of the brain, the body and the environment. The 4E approach is based on four concepts: embodied, embedded, enactive, and extended.

Embodied cognition is cognition produced in part by structures other than the neural system. Embodied cognition rejects Cartesian dualism, separating the body from the mind, and includes the body in the cognitive system.The body is an integral part of the cognitive system, for example, we perceive relief thanks to the stereoscopic vision of our two eyes.

Embedded cognition is cognition coupled with the environment. Thought is not, as it were, secreted by the brain but by the environment.⁴ The extension of the body into its environment reduces loading and relieves the brain.⁵Enactive cognition is cognition produced in part by actions.⁶ We note here the importance of movement in the process. Extended cognition, a cognition situated in the environment, is an externalisation of processes in our environment. In the context of digital technologies, cyberspace would become an extension of our brain.⁷

Hypothesis

Our hypothesis is the following: if architectural apparatus supported 4E cognitive activities, then architecture would become symbiotic. For this purpose, inhabitant/architecture/milieu has to constitute a form of dynamic coupling, connecting natural and artificial actors. To test our hypothesis, we present below an architectural project example for each of the four E categories.

Embodied architecture

The Urban Algae Folly, designed by EcoLogicStudio for the EXPO Milano 2015. (1) The microalgae cultures embedded in the architecture allow to control transparency and shading qualities of the membrane are the result of algae growth depending on the sunlight and the presence of visitors.

In this example, not only do the algae have the dual role of sensor-actuator, but they are distributed over the entire surface of the device. All parts of the building body are sensitive and responsive, intelligence is distributed, embodied in every square centimeter of the material.

Embedded architecture

HygroSkin, Meteorosensitive Pavilion by Achim Menges' team is part of the permanent collection of the Frac Centre since 2012. (2) This meteo-sensitive architecture is the result of research into materials that react to variations in their environment. The device is inspired by the pine cone principle. The multi-layered wood reacts to humidity, the holes open and close depending on the degree of humidity in the air. The heterogeneity of the material allows the deformation of the surface, as each layer does not have the same expansion coefficient. In this example, the architecture is coupled to the environment in a unidirectional link that gives it shape.

Enactive architecture

In 2003 Kas Oosterhuis/ONL agency was invited to create an installation for the exhibition Non-Standard Architecture. (3) The NSA Muscle is a pneumatic interactive device composed of 72 muscles programmed to be 72 independent members of a single swarm. The 72 inflatable muscles are controlled by individual valves. The device is programmed to have its own behavioural cycle, but also to react instantaneously to an external stimulus. In this way, ONL materialises the concept of two-way communication in real time linking two active elements, in this case, the inflatable device and the users. The users interacting with the Muscle quickly learn how the Muscle reacts to their actions, and a game is established in this communication.⁸ We are in a case of bidirectional relationship, in which the movements of each of the protagonists (human and non-human) are part of the same process.

Extended architecture

Hylozoic Ground by architect and sculptor Philip Beesley was selected to represent Canada at the 2010 Venice Biennale in Architecture. (4) Beesley's installations are immersive environments that question the boundaries between the natural and the artificial, the human and the non-human. These hybrid devices are micro creatures,half environments, half mechanical, half biological. The visitor, by his presence, movement and breathing,activates a wave of reactions from the device. He is thus inscribed in a respiratory cycle and through his interactions becomes one with the installation within the same metabolism.

The environment supports a mutual empathic relationship that initiates a reactive movement, an exchange of particles and an air cycle between the system and the visitor. The visitor's own body limits are questioned. Philip Beesley named this type of behaviour: diffusive architecture.

Conclusion

The opposition between the living and the artefact is replaced by an isochronism of natural and artificial phenomena that continues to hybridize. Symbiotic architecture could be seen as a cyber-physical-human system within the natural artificial continuum. The opposition between the living and the artefact is replaced by an isochronism of natural and artificial phenomena that continues to hybridize. With these four examples, we have seen that architectural devices can support relational processes at different scales (within the material itself or with external elements), in a unidirectional or bidirectional way, beneficial to a single actor or to several. What the 4E approach shows is that architectural symbiosis needs to be based on a multiplicity of interactions and actions. Borrowing the concept of the 4 E's (embodied, embedded, enactive, and extended) from the cogniticians, we wanted to sketch a reading and writing grid for architectural design, our next research object. It would allow gathering under one concept the different current appellations such as dynamic, flexible,sensitive, transformable, adaptable, interactive, intelligent and moreover.

(1) https://www.ecologicstudio.com/projects/urban-algae-folly-2-0
(2) https://www.icd.uni-stuttgart.de/projects/hygroskin-meteorosensitive-pavilion/
(3) https://www.oosterhuis.nl/nsa-muscle-centre-pompidou-paris/
(4) https://www.philipbeesleystudioinc.com/sculpture/hylozoic-ground-venice-biennale/ 

References

1 C. Larrère, R Larrère, Du bon usage de la nature : Pour une philosophie de l’environnement, Flammarion, 2009.

2 M-A. Brayer, F. Migayrou, Archilab 2013 : Naturaliser l’architecture, Vol. Rencontres internationales d’architecture Orléans, Editions HYX, 2013.

3 A. R. Damasio, L’erreur de Descartes : La raison des émotions, M. Blanc (trad.), Editions Odile Jacob, 2010.

4 A. Prochiantz, Machine-esprit, Editions Odile Jacob, 2000.

5 P. Haselager, J. Van Dijk, I. Van Rooij (s. d.), "A Lazy Brain? Embodied Embedded Cognition and Cognitive Neuroscience", In Handbook of Cognitive Science, 273-290.

6 E Couchot, La nature de l’art : Ce que les sciences cognitives nousrévèlent sur le plaisir esthétique, Hermann, 2012.

7 M. Serres, Petites poucettes. Editions le Pommier, 2012.

8 V. Parlac, Surface Change : Information, Matter And Environment. In R. Stouffs & P.Janssen... (éd.), Proceedings of the 18th International Conference on Computer-Aided Architectural Design Research in Asia, CAADRIA, 2013.

A [Potential] Cloud War Controversies and conflicts related to climate manipulations.

Marie-Julie Bourgeois

Paris-Saclay University, France mail@mariejuliebourgeois.fr

Abstract

The OuCliPo research project (Ouvroir de Climats Potentiels) is a workshop of climate potential, supported by the Labex LaSIPS, presented by Marie-Julie Bourgeois, PhD in Aesthetics, Science and Technology of the Arts, researcher in art and design science at University Paris-Saclay. This project is also directed by Jules-Rémi Bois-Rouge, a (fake) climate geo-engineer, Doctor at the LaPataFlu, a (fake) Laboratory of Fluid Pataphysics, stemming from the modern surrealist literary movements, proposes a "science of imaginary solutions which symbolically grants to lineaments the properties of objects described by their virtuality."1
OuCliPo studies surrealist issues to climate problems; pseudo-scientific solutions and their implementation in the context of eco-anxiety. The project highlights the ethical and geopolitical dimensions of solar geo-engineering, as well as the socio-cultural issues associated with these climate experiments as techno-solutionism. 

Keywords

Climate, Geo-engineering, Global warming, Aerosol, Environment, Temperatures, Clouds, Particles, Fiction, Manipulation.

DOI

10.69564/ISEA2023-5-short-Bourgeois-A-Potential-Cloud-War

Figure 1: Nubus (fake) French startup. © Marie-julie Bourgeois 

Introduction

The 1995 Nobel Prize in Chemistry, Paul Crutzen, warns about the climate situation and the need to find viable solutions in a seminar2; to the European Parliament, he proposes to seriously study solar geoengineering as a third way.3

Solar geoengineering is a highly controversial technology that offers a concrete and—almost—immediate solution to the current problem of global warming. Although theoretical, this technology is being actively studied in research laboratories and by IPCC experts "Aerosol injection into the stratosphere is the most convincing solar geoengineering technique, it has the potential to significantly affect the climate in relation to the ongoing warming. It could offset a rise in global temperatures. It may offer a faster cooling capacity than CO2 mitigation." This major technological innovation allows an efficient cooling of the entire globe by increasing its natural reflective power (albedo).

This theory is based on the study of climate cooling periods following major volcanic eruptions such as the Pinatubo eruption, which cooled the globe by 0,5°C. In 1991, the release of sulfur particles (SO2) into the stratosphere increased the reflectivity of the Earth's surface and thus prevented some of the sun's rays from entering the atmosphere. The temperature of the planet dropped by -0.6°C for fifteen months.⁵, ⁶ 

Figure 2: Eruption of the Pinatubo volcano, Philippines, 1991 © J. Durieux-Sipa press. Larousse.

Players in geo-engineering

Solar geo-engineering solutions have developed rapidly over the last few years, with the aim of curbing global warming as a matter of urgency. Laboratories—or startups—specialized in solar geo-engineering, Solar Radiation Modification (SRM) or Stratospheric Aerosol Injection (SAI) attempt to "fix the climate" by generating anthropogenic clouds composed of sulfur particles.

These promise to create a protective cloud cover against solar radiation to increase albedo. SAI was patented in 1991, as a method of Stratospheric Seeding, it involves seeding the stratosphere with small metal oxide particles, such as thorium, barium or aluminium flakes, with the aim of reflecting sunlight and reducing global warming. Plans to implement the proposal were not foreseen or publicly known. In the years that have passed since, global warming has caused major concern and scientists have been looking for solutions to "repair" the planet. Researchers mainly use numerical modelling for obvious legal, ethical and health reasons. If inhaled, sulfur dioxyde (SO2) may lead to chronic disease, embedding itself in our lungs. Some laboratories are conducting risky experiments on a human scale without legal authorization in a frantic race to experiment with climate:

Figure 3: Make Sunsets, Luke Isemans and Andrew Songs launching stratospheric balloons from Nevada, February 2023. © Balazs Gardi for TIME 

All these projects should be illegal if we respect the moratorium on geoengineering, the UN ministerial meeting held in Nagoya, Japan, in October 2010 "Any experimentation, private or public, or adventurism aimed at manipulating the planetary thermostat will constitute a violation of this carefully crafted consensus in the context of the United Nations," asserted Silvia Ribeiro, ETC Group Director for Latin America.

Seducing words such as (Make Sunsets), the acronyms (SPICE, PEACE, SABRE, SATAN) and the vocabulary employed by researchers (heaven, hell, inferno, cult...) refer to beliefs and myths. The desire to remedy global warming by making amends for past pollution is strong. Guilt and the hope of becoming a hero by saving the planet and its inhabitants from imminent danger guide these geo-engineers / climate sorcerers. 

Costs

The cost of deploying these technologies is not excessive. Independent organisations can leverage them without requiring participation or funding from institutions. According to studies19, 20 it is sufficient to inject $1500 per ton of particles into the upper atmosphere each year for a gain of 2w/m2, in order to benefit from a significant decrease of 2°C, and the entire solution would cost between 2.25 and 11 billion dollars per year. The cost seems reasonable in view of the climate issues and international interests.

Risks

The main obstacles are the ethical, causal, and governance issues21 associated with climate manipulation, as well as the difficulty of large-scale experimentation and impact studies. Expert scientists are unanimous on the hopes and the risks,22 the danger to the biosphere,23 security framework,24 news biophysics risks,25 side effects, as well as possible thermal recovery in the event of an abrupt shutdown.26 Beyond health impacts, the societal risks of deploying these technologies are a moral issue,27 which provides a miracle "solution" that avoids reducing GHG emissions, socio-technical lock-in; there are also considerations regarding the irreversibility of the techniques deployed, and the termination shock.28 Furthermore, this technology, if developed without international agreements, will cause large-scale damage across borders (GIEC 20122). As the climate is interdependent on local meteorological events, the effects will be global. It is therefore urgent that nations position and align themselves in this geoengineering race based on common goals, in order to avoid wild tests, researching risk and preventing side effects. If experimentation goes unchecked, these solutions could spark a future global war,29 a scenario that is certainly no longer a fiction given the current geopolitical context. Since global warming is considered a threat to national security, the US military is involved in the development of solar geoengineering as part of its militarisation strategy.30 The prospect of rebalancing the forces belonging to competing powers such as China or Russia is probable, and the risk of escalation is a consequence of solar counter-geo-engineering programs31... A potential cloud war32 is brewing.

Look the sky Major volcanic eruptions

The year 536 has been called "the worst year in history"33 by archaeologists. It marks the Little Ice Age caused by several simultaneous volcanic eruptions. These volcanic eruptions caused 18 months of darkness on the surface of the globe due to the release of acidic sulfur particles, world temperatures then dropped by -2.5°C, causing famines, two centuries of plague epidemics, and the fall of several civilizations on all continents.3⁴ This period also marks the beginning of the Dark Ages within the Middle Ages. The architect Philippe Rahm3⁵ recalls the links between architecture, climate, volcanic eruption and epidemics: The eruption of Tambora in 1815 changed the world 3⁶ which benefited cholera. The global temperature dropped by 1°C for several years, prolonging the winter and disrupting the climate cycle on a global scale, the “year without summer” will be remembered.

Are we really ready to play with fire by experimenting with particle diffusion in the stratosphere? The health, meteorological3⁷, agricultural3⁸ and civilizational risks are not well enough measured or understood. Dangers are not limited to the survival of ecosystems and populations, not all risks have been studied and identified by geo-engineers. 

Science and fiction

That same dark and rainy summer of 1816, in this troublesome climatic context, a group of writers met in Switzerland; Mary Shelley wrote Frankenstein or the Modern Prometheus3⁹ and Lord Byron popularised The Vampyre.⁴⁰ We can say that one of the founding myths of Science Fiction was born in a dramatic climatic context! The following year, in 1817, in the midst of global climate change, William Turner began to paint coloured sunsets, amongst other Impressionist precursors influenced by these surrealist red-to-green skies.⁴1 Edvard Munch painted The Scream after viewing and being influenced by the strange sky "turning blood red." This famous expressionist portrait and its red sky probably stems from a vision of the Oslo Fjord, after the volcano eruption of Krakatoa in 1883,⁴2 or may be due to a nacreous cloud.⁴3 Climate and fiction seem intimately linked in the narrative imagination but also in the psychology of aesthetics and creative expression. If SRM/SAI technologies are implemented, and the atmosphere becomes opaque and dangerous to inhale due to particles, will the atmosphere lose its transparency? Our “common sky” will be whiter.⁴⁴ Today’s blue skies are already an illusion due to Tyndall effect and Rayleigh scattering and it is based on the size and density of particles... If the atmosphere becomes concentrated in CO2, will the sky turn orange like the CO2 saturated Martian sky? If our vision becomes saturated or obstructed by particles, how can we project ourselves towards new horizons, new common futures?

We can conclude that the power of design influences the critical thinking process, whilst the urgency of the situation also blinds the mind. The Make Sunsets website is far too well-designed for a typical scientific research project, and its warm orange design (the colour of sunsets) is as seductive as the promises of rescue it makes. Perhaps the design and aspirations explain its success and ability to raise funds. Despite the cobbled-together nature of its technology and the danger of its reckless actions. However, these orange-red colours of the startup should worry us, just as the Scream 😱 reveals the anxiety-inducing dimension of modern existence. 

Realistic satyr and fiction

OuCliPo is an art and design research project that questions the impact of particles on our vision and our cultures at the scale of landscape, architecture and habitat, by presenting speculative fiction based on climate manipulation, studies and data. 

Figure 4: Nubus the (fake) startup http://www.nubus.fr © Marie-julie Bourgeois

Nubus⁴⁵ is a (fake) research startup, created to test the credibility of the technology and also the attractiveness to the general public via a seducing marketing campaign "for a bluer and a cooler sky." For this purpose, a questionnaire was used to measure support for the Nubus startup and its promises. The French (fake) startup projects us to the stage of commercialization and industrialization of SRM/SAI technologies in order to confront us with the mutations they will generate via environmental and society disturbances. How far are we willing to go, to believe it?

After viewing the Nubus promotional film, a sample of 34 people were interviewed during the exhibition of the project at the ENS Paris-Saclay and at Evry University:


In conclusion, a relatively high percentage was inclined to participate in climate optimization in order to promote cooler and bluer skies. Although a minority, 21% of the respondents, believes that geo-engineering and SRM/SAI do not seem dangerous for our biosphere, or the climate... When the video was recently shown in an innovative art and science context, audiences reacted with ambivalence: amused, horrified or seduced, many believed in it. We decided to exaggerate the tragi-comic potential of the startup’s arguments to defuse support for the promise of rescue, the urgency of the situation resulted in a strong desire for solutionism.

Nubus is also developing green kerosene and biofuels to generate clouds of particles with high nucleation potential, rather than harmful greenhouse gases. Our emissions are guaranteed to have high clouding potential. We also work with the agrochemical and agricultural industries to combat soil erosion and desiccation. Our mineral and nutrient stimulants have a high precipitation potential which is ideal for farmers in times of drought.

"Our partners in the tobacco industry offer electronic cigarettes that contribute on a small scale to the generation of climate-friendly particle clouds, because there are no small contributions.

Nubus is launching a fund-raising campaign to "save" the planet and offers citizens in search of meaning the chance to clear their consciences by blue washing our vision of the future.

You can also take a sodium cure with our blue pills, which guarantee accelerated awareness to wipe out ecological procrastination..." (3)

Even with these arguments, the public still wondered if Nubus really did exist... 

Figure 5: Photomontage Nubus Blue pills against ecological procrastination © Marie-julie Bourgeois 

Homogenitus are clouds generated by human activities defined by the WMO in the international cloud atlas.⁴⁶ Homogenitus⁴⁷ it is also the name of an artistic cloud-making machine. Players can generate customizable clouds directly on a web application, choosing morphology, size, shape and the composition of particles. This artistic installation questions the creation of clouds by human activity, the ability to control the weather and places the user in a demiurgic position. 

Figure 6: Homogenitus the interactive cloud making machine © Marie-julie Bourgeois 

The citizen artivist collective fakeCloud⁴⁸ denounced Nubus activities, it’s SRM/SAI projects and climate manipulations. Nubus is caused of over-producing clouds related to the Iran accusation against Europe to steal their cloud in 2011.⁴⁹ 

Fig 7: FakeCloud the artivist collective http://fakecloud.fr © Marie-julie Bourgeois 

Research of solutions

In the absence of government control or international rules, companies can deliberately spread or steal clouds. How will our sky look in a decade? Will a future cloud war occur? This awareness campaign proposed to approach the ecological problems by addressing these issues through wordplay, irony, detours, fiction, and references to scientific methodologies and technologies. Designing our potential future atmosphere opacity and tint aimed to sensitize the public to these rapid ecological solutions, and promote awareness of the seductive dimension of design.

Our ideologies, our beliefs, our knowledge are challenged by the crises we are experiencing; climate, health, life, economy, institutions and especially science. At the same time when technologies and social networks allow collaborative actions, we question the methods of sensitization of the public about the techno-solutionism challenges. Fictions and post-truths mingle with reality in the scientific field. How can we find a sustainable way in this mess? Awareness and collective action are the determining factors for reducing greenhouse gases, but solar geo-engineering projects risk slowing down these efforts by proposing alternative and risky paths.

Climate issues are frequently addressed in a dramatic way, generating various typologies of reactions⁵⁰: alarmed, worried, cautious, non-committal, unconvinced, and contradictory. On an individual level, ecological procrastination is mainly due to discourses⁵2 that delay massive climate action; techno-solutionism is one of them. Robert Gifford observes seven psychological barriers to ecological inaction⁵3: limitations of cognition, ideological views, comparison with significant others, costs, distrust of experts and authority, perception of risk, and behavioural limitations. For Gifford, these "dragons of inaction" are the social-psychological brakes that impede real and meaningful global action.

Conclusion

OuCliPo proposes to approach climate threats with non-guilty and non-anxious modes of communication such as fiction and humour. OuCliPo does this by making a mockery of the search for technological solutions. Between solutions that are “not solutions,” where the cure is worse than the disease, denial and even counter-action, massive actions, whether positive or negative, have a strong impact on the climate.

To this end, Nubus offers its customers the chance to redeem their consciences by fixing the planet. It provides citizens in search of meaning with a conscience cleanser, by washing our vision of the future in blue for a dual benefit: a thermal solution and the psychological comfort of being able to remedy climate change by looking at the sky, not to don’t look up! (4). Ideologies, design and the power of capitalism helps us to believe in these seductive solutions. The market in climate denial⁵⁴ and the negation of mitigation policy are solutions that benefit climate skepticism.

Media and digital tools are good ways of approaching social issues with critical distance and speculative projection, but they can also fuel disbeliefs and conspiracies. Moreover, fiction can play a major role in raising collective awareness of issues at stake, by accelerating political decision-making. Confronted with the crises we are experiencing, and the eco-anxieties that are paralysing actions, we need to set off in search of "eco-quietude" and harmony amongst the disorder. This quasi-spiritual philosophical notion organised the celestial spheres as early as Pythagoras; between music, numbers, colour and astronomy, harmony was already giving rhythm to the mental and physical health of mankind in antiquity.⁵⁵

The harmony and symbiosis between man and nature, with sustainability and critical distance, would allow us, through fiction and humour, to approach climate problems from a truthful background. 

(3) ISEA 28th International Symposium on Electronic Art, May 17, 2023, Paris forum des halles

(4) Don’t look up ! Adam McKay 2021

References
Books

1 Alfred Jarry, Gestes et opinions du docteur Faustroll, pataphysicien, Livre II, chapitre VIII, Paris, Éditions Fasquelle, 1911.

35 Philippe Rahm, Histoire naturelle de l’architecture, Comment le climat, les épidémies et l'énergie ont façonné la ville et les bâtiments, Paris, Edition du Pavillon de l’Arsenal, 2020, p.167.

36 Gillen D'Arcy Wood, Tambora – The Eruption That Changed the World, Princeton University Press, 2015.

39 Mary Shelley, Frankenstein or the Modern Prometheus, Lackington, Hughes, Harding, Mavor & Jones, 1818.

40 John William Polidori, The Vampyre a Tale, The New Monthly Magazine, 1819.

55 Yuval Noah Harari, Homo Deus, A brief history of tomorrow, London, Vintage, 2017.

Journal article (online)

3 P. J. Crutzen, “Albedo enhancement by stratospheric sulfur injections,” Climatic Change, vol. 77, no 3-4, July 25, 2006, 211-219, doi:10.1007/s10584-006-9101-y

5 Mc Cormick and al, “Atmospheric effects of the Mt Pinatubo eruption,” Nature 373, February 2, 1995, 399–404. doi:10.1038/373399a0

12 J. A. Dykema and al. “Stratospheric controlled perturbation experiment: a small-scale experiment to improve understanding of the risks of solar geoengineering,” Phil. Trans. R. Soc, December 28, 2014, doi:10.1098/rsta.2014.0059

19 W. Smith, G. Wagner, “Stratospheric aerosol injection tactics and costs in the first 15 years of deployment,” Environmental Research Letters, Vol. 13, No. 12, November 23, 2018, doi:10.1088/1748-9326/aae98d

20 W. Smith and al, “A subpolar-focused stratospheric aerosol injection deployment scenario,” Environmental Research Communications, Vol 4 number 9, September 15, 2022, doi:10.1088/2515-7620/ac8cd3

22 M. De Guglielmo Weber, “Promesses et risques de la géoingénierie solaire,” IRIS, Programme climat, énergie & sécurité, February 2023, accessed August 22, 2023, https://www.iris-france.org/wp-content/uploads/2023/02/19_ProgClimEnerSec.pdf

23 M.G. Lawrence, and al, “Evaluating climate geoengineering proposals in the context of the Paris Agreement temperature goals,” Nature Communications 9, Septembre 13, 2018, 3734, doi:10.1038/s41467-018-05938-3

24 Chalecki, and al, “A New Security Framework for Geoengineering,” Strategic Studies Quarterly, vol. 12, no. 2, 2018, JSTOR, Accessed 22 Aug. 2023, 82–106, http://www.jstor.org/stable/26430817

25 E. Baughman, et al, “Investigation of the Surface and Circulation Impacts of Cloud-Brightening Geoengineering,” J. Climate, 25, November 1, 2012, 7527–7543, doi:10.1175/JCLID-11-00282.1

27 C Preston, “Ethics and geoengineering: reviewing the moral issues raised by solar radiation management and carbon dioxide removal.” WIREs Climate Change. 4, November 8, 2012, 23-37, doi:10.1002/wcc.198

28 A. Parker, & P. J. Irvine, “The Risk of Termination Shock From Solar Geoengineering,” Earth's Future, 6, March 11, 2018, 456–467, doi:10.1002/2017EF000735

30 Stephens, and al, “Towards dangerous US unilateralism on solar geoengineering,” Environmental Politics, December 18, 2022,  doi:10.1080/09644016.2022.2156182

41 C. Zerefos, end al, “Further evidence of important environmental information content in red-to-green ratios as depicted in paintings by great masters,” Vol. 4 issu 6, Atmos. Chem. Phys. 14, March 25, 2014, 2987–3015, doi:10.5194/acp-14-2987-2014

42 Donald W. Olson, and al, "The Blood-Red Sky of the Scream," APS News. American Physical Society, Vol. 13 number 5, May 2004, accessed August 22, 2023, https://www.aps.org/publications/apsnews/200405/backpage.cfm

43 F. Prata, and al, “The Sky in Edvard Munch’s The Scream,” Bulletin of the American Meteorological Society 99, 7, July 1, 2018, doi:10.1175/BAMS-D-17-0144.1

44 B. Kravitz, and al, “Geoengineering: Whiter skies?”, Geophys. Res. Lett., 39, L11801, June 1, 2012, doi:10.1029/2012GL051652

50 E. W. Maibach, and al, “Identifying Like-Minded Audiences for Global Warming Public Engagement Campaigns: an AudienceSegmentation Analysis and Tool Development,” PLoS ONE, March, 10, 2011, doi: 10.1371/journal.pone.0017571

51 Richard W. Malotte, “I’ll Save the World from Global Warming—Tomorrow: Using Procrastination Management to Combat Global," June 1, 2017, doi:10.1007/BF03392214

52 WF Lamb, and al, “Discourses of climate delay,” Global Sustainability 3, e17, June 8, 2020, 1–5, doi:10.1017/sus.2020.13

53 Robert Gifford, “The Dragons of Inaction: Psychological Barriers That Limit Climate Change Mitigation and Adaptation,” American Psychologist, May 2011, doi:10.1037/a0023566

Magazines and Newspapers (online)

2 European Parliament, “Professor Paul Crutzen: Nobel winner and advocate of a climate "escape route"”, January 12, 2011, accessed August 22, 2023, https://www.europarl.europa.eu/news/en/headlines/society/20110103STO11194

6 NASA Observatory, Langley Research Center, Aerosol Research Branch, June 2001, accessed August 22, 2023, https://earthobservatory.nasa.gov/images/1510/global-effects-of-mount-pinatubo

13 Letter from Sam Council to SCoPEx advisory committee, February 24, 2021, accessed August 22, 2023, https://static1.squarespace.com/static/5dfb35a66f00d54ab0729b75/t/603e2167a9c0b96ffb027c8d/1614684519754/

15 Chris Hatch, “Whose call to shade the Earth?” March, 3, 2023, accessed August 22, 2023, https://www.nationalobserver.com/newsletters/zero-carbon/2023/03/03/whose-call-shade-earth

16 Gobierno de Mexico, “La experimentación con geoingeniería solar no será permitida en México” January 13, 2023, accessed August 22, 2023, https://www.gob.mx/semarnat/prensa/la-experimentacion-con-geoingenieria-solar-no-sera-permitida-en-mexico

18 James Temple, “A startup says it’s begun releasing particles into the atmosphere, in an effort to tweak the climate.” MIT Technology Review, December 24, 2022, accessed August 22, 2023, https://www.technologyreview.com/2022/12/24/1066041

29 Alexander C. Kaufman, “The Climate Change Solution That Could Spark Global War” OneZero, June 16, 2019, accessed August 22, 2023, https://onezero.medium.com/the-climate-change-solution-that-could-spark-global-war-bd1cc9cce1a3

49 Barney Henderson, “Mahmoud Ahmadinejad says Europe 'stealing Iran's rain’”, The telegraph, May 21, 2011, accessed August 22, 2023, https://www.telegraph.co.uk/news/worldnews/middleeast/iran/8527455/Mahmoud-Ahmadinejad-says-Europe-stealing-Irans-rain.html

32 “La guerre des nuages, par Mathieu Simonet,” Nouvel Obs, March 28, 2022, accessed August 22, 2023, https://www.nouvelobs.com/tribunes/20220328.OBS56297/la-guerre-des-nuages-par-mathieu-simonet.html

54 Marine de Guglielmo Weber, “Crédits de refroidissement : quand la géo-ingénierie commercialise le déni climatique,” IRIS France, February 17, 2023, accessed August 22, 2023, https://www.iris-france.org/173813-credits-de-refroidissement-quand-la-geo-ingenierie-commercialise-le-deni-climatique/

Websites

4 IPCC, AR5, SYR, 3.3, 2014, accessed August 22, 2023, https://www.ipcc.ch/report/ar5/syr/

7 Interactive world map on geoengineering, accessed August 22, 2023, https://map.geoengineeringmonitor.org/srm

8 IGCE, Institute of Global Climate and Ecology accessed August 22, 2023, www.igce.ru

9 SPICE, Stratospheric Particle Injection for Climate Engineering accessed August 22, 2023, www.spice.ac.uk

10 E-PEACE, Eastern Pacific Emitted Aerosol Cloud Experiment accessed August 22, 2023, aerosol.ucsd.edu/E_PEACE.html

11 SCoPEx, Stratospheric Controlled Perturbation Experiment, accessed August 22, 2023, https://www.keutschgroup.com/scopex

14 SABRE, Stratospheric Aerosol processes Budget and Radiative Effects,https://csl.noaa.gov/projects/sabre/

17 Make Sunsets American startup of solar geo-engineering, accessed August 22, 2023, https://makesunsets.com/

21 Reynolds, J. et al. “Solar Radiation Modification: Governance gaps and challenges – Summary.” Carnegie Climate Governance Initiative (C2G), March 2022, New York, www.c2g2.net

26 Felgenhauer, T. et al. “Solar Radiation Modification: A Risk-Risk Analysis - Summary, Carnegie Climate Governance Initiative” (C2G), February 2022, New York, 2023, www.c2g2.net

45 Nubus is a (fake) French startup of solar geo-engineering, accessed August 22, 2023, https://www.nubus.fr/

46 World Meteorological Organization, International Cloud Atlas, https://cloudatlas.wmo.int/en/homogenitus.html

47 Homogenitus is an art installation, https://homogenitus.fr/

48 Fake cloud Collective of artivist fighting against climate manipulations, accessed August 22, 2023, http://fakecloud.fr

Author Biography

The author is a digital artist, designer and PhD researcher in aesthetics and science technologies of the arts since 2018. She works on ecological, artistic and scientific fictions in society in relation to human activities. Born in Paris in 1981, she lives and works in Cachan, France. In 2008, she obtained a Master's degree in New Media at the Ecole Nationale Supérieure de Création Industrielle (ENSCI) and in 2009, she joined the EnsadLab research program. Her research focuses on "solar fictions, devices that recreate the sun’s path", and she is currently working on climate fictions. She teaches at the University of Evry (91) since 2013 and co-founded the CondéDesignLab in 2021. 

Inhabiting the Edges

Célia Boutilier

SACRe-PSL ENS/ENSBA — Paris Boutiliercelia@hotmail.fr

Abstract

My doctoral project in art-creation-research, which began in October 2021, is entitled "Imaginal photography: for an aesthetic of symbiosis." In my practice as a photographic artist, making symbiotic relationships, unexpected alliances, allows me to provoke a "suspension of evidence" which, through the intermediary of the image, invites the viewer to share the experience of a different look. From this point on, the very nature of perception is to admit ambiguity. The interdependencies in ecosystems, the principle of hybridization between different species are themes addressed by the team of the laboratory of plant symbioses of the French National Museum of Natural History of Paris with which I collaborate. In my research, I'm specifically interested in the symbioses present in the orchid with the biologists Florent Martos and Eve Hellequin. Transcending the notion of organism allows us to reconcile human beings with their natural dimension of interdependence and interaction. The researchers in this laboratory are the extractors and curators of natural plant resources, with which they shape new worlds such as the Grande Serre: a space in which I would like to exhibit works in 2023-2024. These hybrid spaces, co-dependent with humans, seem to me to be fertile places to dialogue with.

Keywords

Photography, imaginal, hybridization, symbiosis, speculation, pragmatism, device, articulation

DOI

10.69564/ISEA2023-6-short-Boutilier-Inhabiting-the-Edges

Introduction

"We inhabit the world from the images we make of it."

This doctoral project starts from the intuition that aesthetics, as a study of sensible knowledge and its artistic productions, is one of the main foundations of the imagination that allows us to think the world and to inhabit it. It is from this imagination that imaginative practice is born. Without the imaginative faculty, a whole area of reality is definitely off-limits to us, lost forever. With the loss of the imaginative faculty, what we risk is not the loss of fiction, but the loss of reality. We run the risk of compartmentalizing the sensible and intelligible worlds, of killing them off slowly, precisely because they only come to life and "come alive" when they are in contact with each other.

Methodology

It seems to me today that the question of representation (of whom speaks and how he or she expresses himself or herself) actually raises the question of the imaginary, that is to say, that of the fabrication of new fictionalities. The issue of imagination is crucial, because it opens up perspectives of therapeutic narratives. These proposals for circulation (in the making itself or in the setting of creations) are multiple hypotheses of resilience. This is why certain fictionalities can be productive of the real and allow us, by this means, to increase ourselves. Their power of augmentation is not only intimate, a creation has an impact when it goes beyond its subject and manages to make language for something else. It’s therefore a question of photographing this place with the desire to feel it fully, while knowing that from this precise place we will touch other places, both terrestrial and human.

On the other hand, I’m interested in the interdependencies in ecosystems and the principle of hybridization between species, because I believe that there is a realaesthetic, philosophical and political stake in linking Art (culture in general) to Ecology. The laboratory and the studio are the two places where photographic art was born. Botanical science and in particular the concept of symbiosis seems to me to be able to achieve this necessary alliance (in biology the word symbiosis defines a lasting interspecific interaction of mutual benefit). In this sense, I’m conducting this doctoral research project in collaboration with the team of the plant symbiosis laboratory of the Paris Natural History Museum. I’m specifically interested in the mycorrhizal symbioses of orchids: those that create networks between individuals and different species, notably between fungi and plant roots. From the concept of symbiosis in biology, I intend to develop the concept of symbiosis in the field of aesthetics: "what could be a symbiotic aesthetic? I propose the hypothesis that a "symbiotic aesthetic" constitutes, on the one hand, a singular way of relating philosophically to the world (at the source of an enlarged perception), and, on the other hand, a programme of plastic realisation. How can symbiosis suspend the rational relationship of things to each other? To what extent does this suspension make it possible to escape from designation and nomination? How can the new representation of the world proposed by symbiosis change the way we act and interact? This project aims to sketch some answers, both formal and textual, based on fieldwork with biologists, both in the field and in the laboratories of the MNHN of Paris.

Communication plan

More specifically, I'd like to present a few recent works produced following a scientific mission to Reunion Island, which fall under what I call a "symbiotic aesthetic." Photographic assemblages are the first satisfactory way I've found of aesthetically translating hybrid identity, through which the intertwining of links manifests itself. Boundary images between the "natural," the imaginary and the technological. Different photographic fragments are linked together by collage. The aesthetic is organic and lively, evoking both art nouveau and virtual reality, where naturalistic photography meets pictorial inventiveness.

Focus of the work

In my work, it’s a question of "milieu" in that it takes us in its mesh.It’s a question of resisting any hierarchical placement of knowledge and exploring the plurality of deployments that they allow, replacing the aim of unity with the problematic of articulation.

I would like my images to be edges. Let them be meeting zones, spaces of contact, symbiotic and twilight places where the obvious is suspended and where, in a strange familiarity,things enter into relationship. I favour the observation of transitory and symbiotic forms,changes of state, the confusion of scales (from satellite to microscope) and all other forms of telescoping practices, insofar as they impact the solidity of bodies, the sharpness of contours and the fixity of images. The particular is then the bearer of more than just itself, in that the image is made from an elsewhere that crosses the here, in order to give an account of a here that is both always open and always worried.

"‘Border identity’ this is the term by which I usually define my own identity. I call it frontier, anchored, not in a place of rupture, but, on the contrary, in a space of permanent coming together. The border, as I define it and inhabit it, is the place where worlds touch, tirelessly. It's the place of constant oscillation: from one space to another, from one sensibility to another, from one worldview to another. It's where languages mingle, not necessarily thunderously, but naturally impregnating each other, to produce, on the blank page, the representation of a composite, hybrid universe."¹

Pace of time and ghosts

We inhabit the world through the images we make of it. In this sense, it would seem that constructing ways of inhabiting the world² also involves ways of making images. In the hollow of my plastic reflections, I apply myself to modelling a specific temporality: that of a time which has a thickness and in which memory and dreams are sedimented.

This time—like thought—is not linear (cause/effect), it is made up of feedback loops where the chain of causality swallows its own tail (systemic thinking). The past is not only what happened, it is also what was dreamed.

"There is always a shadow of something. And it stays for a long time. Now there is only the shadow of my mother, who passed on to me the shadow of my grandmother, etc., etc. You see images projected normally and then you see the same images that become a shadow ofthemselves. That's what Maniac Shadows is all about."³

Uncomfortable landscapes. There is a flaw, a break, a suspension of evidence. We must first remember our heritage, the memory of the rumbling soil, the organisms and ghosts that inhabit us and that we carry, in spite of ourselves, like amnesiac children. This requires, first of all, a refusal of postures or practices that are constituted in a cannibalistic relationship to otherness. This relationship is one of attention, concern and vigilance. The state of vigilance is a state of slowness, of contemplation that stretches over time. It is a profound attention that takes the opposite view of the lighted time of immediacy to become a space of common sharing. It is a care that we cultivate in the relationship that we maintain with the other whether human or non-human. Taking care of our relationships and their modes of existence goes hand in hand with taking care of the making of our images." The world to which we belong is first and foremost the world we carry within us."⁴

Figure 1. Identité Frontalière L.M (rupture), Mare Longue native forest, Reunion Island, 2023 © Célia Boutilier. Figure 2. Oeceoclades et ectomycorhizes, from the "naturecultureD.H" series, Electronic microscopy technical platform, French National Museum of Natural History, Paris. Assembly of six photographs, format 140 x 139.80 cm © Célia Boutilier
Figure 3. Échelle Humaine (etépiphytisme sur roches), from the "natureculture D.H" series, Mare Longue nativeforest, Reunion Island, 2023. Photographic assemblage, 140 x 70 cm © Célia Boutilier
References
Books

1 Léonora Miano, "HABITER LA FRONTIÈRE, Paysages francophones -Journée internationale de la francophonie, Université de Copenhague (Danemark)" conférence de 2009 recueillie dans HABITER LA FRONTIÈRE, L’Arche, 2012, p.25.

2 Bruno Latour, Où atterrir ? - Comment s'orienter en politique, La Découverte, 2017.

4 Ibid. Léonora Miano, 2012, p.25.

Webvideo interview

3 Chantal Akerman - ManiacShadows, Nuit Blanche 2013 au Théâtre du Châtelet, Paris.

Bibliography

David Abram, The Spell of the Sensuous: Perception and Languagein a More-Than-Human World, Vintage, 1997.

Jean-Christophe Bailly, L’imagement, Seuil, 2020.

Jean-Christophe Bailly, Le temps fixé, Bayard, 2009.

Jonathan Crary, Techniques de l’observateur, Vision et modernité au XIXe siècle, Broché, 1990.

Lorraine Daston, Peter Galison, Objectivité, Les presses du réel, 2012.

Gilles Deuleuze, Logique de la Sensation, 1981.

Gilles Deuleuze, Desert Islands and Other Texts, 1953-1974.

Cambridge, MA: MIT Press, 2004.

John Dewey, L’art comme expérience, 1934.

T. S. Eliot, The Waste Land, Faber & Faber, 2015 [1922].

Cynthia Fleury, "L’imagination renaissante : entreRévélation et Intellection" in Imagination, imaginaire,imaginal, Coordinated by Cynthia Fleury, PUF, Paris, 2016.

Donna Haraway, Staying with the Trouble: Making Kin in theChthulucene, Duke University Press, Durham, 2016.

Donna Haraway, When species meet, University of Minnesota Press, Minneapolis, 2007.

William James, Philosophie de l'expérience. Un universpluraliste, 2007.

Bruno Latour, Down to Earth: Politics in the New ClimaticRegime, ed. Cambridge, UK, Polity Press, 2018.

Bruno Latour, Facing Gaia: Six Lectures on the Political Theologyof Nature, Cambridge, UK, Polity Press, 2013.

Bruno Latour, Enquête sur les modes d’existence une anthropologie des modernes, La Découverte, 2012.

Bruno Latour, La science en action - Introduction à la sociologie des sciences, La Découverte, 2005.

Michael Marder, Plant-Thinking – A Philosophy of VegetalLife, Columbia University Press, 2013.

Michael Marder, La plante du philosophe - Un herbierintellectuel, Mimesis, 2020.

Lynn Margulis, Symbiogenetics - Origin of MitoticCells from Bacterial Communities in the ProterozoicEon, 2011 [1992].

Lynn Margulis, Symbiosis as a Source of EvolutionaryInnovation: Speciation and Morphogenesis, (TheMIT Press, 1991).

Baptiste Morizot, Manières d’êtrevivant, Arles, Actes Sud, 2020.

Giuseppe Penone, Jean-Christophe Bailly, Sève et pensée, Paris, broché, Paris 2021.

Bernard Réquichot, "Métaplastique" dans Écrits, 1955, p.87.

Simon Schaffer, Steven Shapin, Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life, 1985.

Anne Simon, Une bête entre les lignes. Essai dezoopoétique, Marseille, Wildproject, 2021.

Isabelle Stengers, La sorcellerie capitaliste : Pratiques de désenvoûtement, La Découverte, 2013.

Isabelle Stengers, L’invention des sciences modernes, Flammarion, 1993.

Andreï Tarkovski, Le Temps scellé.

Anna Lowenhaupt Tsing, The Mushroom at the End ofthe World: On the Possibility of Life inCapitalist Ruins, Old Saybrook, CT, TantorMedia, 2017.

Alfred North, Whitehead, Le concept de nature, J.Vrin, 2006.

Walt Whitman, Leave of grass The Original 1855, Edition, Paris, Broché, 2020.

Journal articles (print)

Carla Hustak, Natasha Myers, "InvolutionaryMomentum: Affective Ecologies and the Sciences ofPlant/Insect Encounters", Vol. 23, Num. 3, Brown University and differences, A Journal of Feminist Cultural Studies, 2012.

Sabine Plaud, "Vie du langage, vie des images : une marque de continuité dans la philosophie de Ludwig Wittgenstein".

Katrin Solhdju, "L’expérience "pure" et l’âme des plantes. James lecteur de Fechner", 2007.

Isabelle Stengers, Didier Debaise, L’insistance des possibles. Pour un pragmatisme spéculatif, Multitudes 2016/4, n° 65, 82-89

Webvideo interview

Bruno Latour, “Agency at the Time of theAnthropocene”, New Literary History 45, 2014, 1-18.

Author Biography

After a DNAP at ENSA Dijon (2016), a year at ERG Brussels (2017), a collaboration with LadHyX Polytechnique (2014-2019), Célia Boutilier joins the Beaux-Arts de Paris (2017) to defend a DNSAP (2019). Since 2014 she has been collaborating with research laboratories (microfluidics - Polytechnique, and mycorrhizal symbioses - National Museum of Natural History) where she is interestedin how imaging techniques participate in theelaboration of knowledge. Since 2016, herwork has been exhibited at fairs and international events, including the National Gallery in Copenhagen, the Cité Internationale des Arts in Paris and the Maison des Métallos. It has been quoted inseveral scientific journals, including Physics Today, The NewYork Times, Le Monde. Since 2020 it has also been included inprivate collections. In October 2021 Célia Boutilier starts a PhD in "Sciences, Arts,Creation, Research" (SACRe-PSL / ENS) at the National Superior School of Fine Arts in Paris (ENSBA).

Mytherrella: an interactive installation hallucinating mythological auroral formations

Dorin Cucicov

University of Bucharest, Romania Bucharest, Romania cucicov@gmail.com

Abstract

Up until very recently, mythological tales predominated in the explanation of the polar auroras. Today, we understand that the polar lights are a phenomenon brought on by the solar winds' interaction with the magnetosphere of the Earth leading to precipitation of energetic particles on the topside ionised atmosphere. However, for a long time, this phenomenon inspired vibrant trans-border stories with a profound impact on local communities. In our interactive installation, Mytherrella, we aimed to create a dynamic environment in which scientific data and mythological storytelling unearth new imaginaries about the aurora borealis. The generative video integrated in the installation uses a custom real-time StyleGAN algorithm that continuously samples from a model trained on a large set of all-sky auroral images acquired in Kiruna, Sweden. This method enables live interaction with the generated video, producing novel synthetic auroral formations that are unpredictable while remaining within the bounds of the learned features. By combining the dataset with a relatively small number of alternative-style images, the diversity of the generated content is increased, creating a divergent effect that reinforces the mythological narrative. We share the technique of interactive video generation as well as the research process behind the creation of the work.

Keywords

interactive installation, mythology, live StyleGAN, aurora borealis, polar lights.

DOI

10.69564/ISEA2023-7-short-Cucicov-Mytherrella

Introduction

Myths about the aurora borealis are prevalent in the cultural heritage of populations from the polar regions of the Earth. Surprisingly, these stories have not permeated pop culture in the same way the images of the aurora borealis did. Many of the stories, despite being from different continents, share many similarities: contact with the spirit world, vengeance of slain enemies, communication with the dead through whistling or playing ball games in the sky. For some, the appearance of the aurora borealis is a bad omen; for others, it can be bad only if disrespected by whistling at it or staying too long outside. Others have more poetic explanations for auroral formations.5, 6

This paper presents a virtual environment, the result of a creative exercise that leverages both scientific data and stories from various communities about the polar lights.

Combining them produces novel visions of a future in which humans reinvent themselves through collaboration with artificial artifacts. Mythology has been at the forefront of explaining the unknown and stimulating human curiosity for centuries. These stories are refined representations of humanity's deep connection to the natural world around it. The mystical characters in these tales instilled fear and caution in humans, effectively negotiating a reverence for natural phenomena. In the past centuries, science has solved numerous natural enigmas, relieving some anxiety but also making humans less respectful and more entitled.3

Figure 1. Interactive installation Mytherrella. On the left - outside of the booth; on the right — interactive control stand inside the booth. © Maria Năstase.

Our project, named Mytherrella, addresses the subject of symbiosis on multiple levels. It frames a creative experiment by integrating empirical data and mythological narratives. The equal treatment of science and myths aims for a reevaluation of our relationship with natural phenomena. Through the similarities between the stories of various cultures, cultural symbiosis is highlighted on a global scale, diminishing the significance of political borders.

Concept and construction

Mytherrella is an endless process of imagination informed by facts from science and driven by myths surrounding the aurora borealis. Starting from a dataset of all-sky images showing auroral activity recorded by the Swedish Kiruna auroral observatory, it imagines new auroral formations. It provides control over its hallucinations in an interactive framework. Both human and machine are in search of the lost/improbable auroras as seen by the first people that described them.

Mythology of the aurora

One of the most common mythological symbols associated with the polar lights is their connection to the spiritual world. Polar lights, according to the Greenlandic Inuit, were messages from their deceased friends and relatives.1; The Meskwaki, a Native American people with homelands in the Great Lakes region, held similar beliefs. They believed that by whistling to the lights, they could interact with them, thus communicating with the dead. Some sources indicate that some North American Inuit also used whispering during auroral events to send messages to the dead.⁵ More active manifestations of the aurora borealis were associated with spirits playing games in some Greenlandic stories, while on the European continent, in Scotland, Ireland and Norway they were associated with dancing spirits.1, 4

Some of the more poetic interpretations of the polar lights are found in Sámi peoples, residing in the north of Finland. They named the aurora - revontulet (fire fox). It is said that the lights are created by a fox with glittering fur running across the mountain area. 1; In other stories, polar lights are an omen of disaster and war. These are found in Tlingit indigenous peoples of the Pacific Northwest Coast of North America, Skolt Sámi in Northern Europe and even in some Chinese and Roman literature.1, 2

There may be several myths about the aurora in a single area, ranging from optimistic interpretations to terrifying ones. These are dependent on particular historical moments and the social and natural influences that shaped those times.

Building Mytherrella

The physical construction of the Mytherrella installation was inspired by the early designs of the auroral observatory in Yerkes, Wisconsin, United States.4 These small sites, fitted with all-sky cameras, collected images of the auroral events aimed for scientific investigation. In our installation, an immersive context is created inside a small booth where one can experience and interact with the story in a private setting. The outside of the booth is painted with selected mythological texts, which acts as a representative of the old media in contrast with the interactive digital content. An interactive stand inside the booth provides two analog potentiometers for controlling latitude and longitude on a minimalist digital map. Using these knobs, a specific location can be identified on the global map. The map is centered on the poles and only depicts the boundaries between water and land; it does not include any other information, such as names of nations, borders, seas, or rivers. This choice was made in order to disrupt the influence of the political borders in the collective perception and suggest a new perspective on interpreting the global map.

Ten locations are hidden on the digital map. They can be discovered by identifying their approximate location using the navigation controls. These coordinates are linked to locations from which we gathered mythological stories. Based on these texts, we used text-to-image tool Midjourney (1) to generate unique images. Figure 2 contains one of these images generated based on the mythological texts of the Native Americans describing the aurora borealis as a means of communication with the spirit world.

Figure 2. Image created using text based on the myth of spirits communicating through aurora borealis.

The second screen, positioned in front of the control stand, displays a continuously generated video with auroral formations. This video is created using a StyleGAN model that was trained on a mixed dataset of scientific and artistic images of the aurora borealis.8 The main corpus consists of 9834 selected images from a dataset of all-sky images depicting auroral activity provided by the Kiruna Atmospheric and Geophysical Observatory at the Swedish Institute of Space Physics. All images have a standard round format, as seen in Figure 3.

Figure 3. A selection from the all-sky images dataset provided by Kiruna Atmospheric and Geophysical Observatory at the Swedish Institute of Space Physics. Recorded during auroral activity in September, October 2020.

(1). Midjourney. https://midjourney.com

Data blending was applied using another 150 artistic images of the aurora borealis in order to extend the dataset and introduce visual features that break the rigor of the scientific representation. The process of projecting an image into the latent space produced images that were continuously served to create the video. When idle, the video would display auroral formations as learned from the main corpus of all-sky images. Finding an image on the world map while interacting with the installation would begin projecting that image to the model, disrupting the normal representation by also sampling from the artistic images' features.

Figure 4. The process of continuous video generation using live StyleGAN.

Figure 4 describes the approach of continuously serving images for the video projection. A script monitors for new files that are selected by interacting with the installation. Once a file is detected, it signifies that an image was identified on the map. This triggers an immediate process of projecting that image to the model. The resulting images are displayed sequentially creating the effect of video continuity. The 3 processes of the architecture are independent: generating based on the source image; serving an image for generation; displaying received images. They are linked only by the configuration of the listeners. This loose coupling makes it simple to integrate with other systems to serve various seed images or display generated images in various formats.

Discussion

The myths of the aurora borealis are rich and creative stories that have surrounded this natural event for a long time. Mytherrella is a direct comment on the relationship between science and mythology. Figure 5 exemplifies one sequence of a video generated by projecting the image from Figure 3. It starts from the format of an all-sky image and navigates the latent space in search of the features from the projected image. By using the technique of interactive generative video, we sought to create a subtle transition from visuals that have a scientific aesthetic to a more abstract look that resembles some of the features of the mythological image while also preserving the features of the aurora. This type of interaction contributed to the immersive character of the experience by partially delegating the responsibility of the generated image to the person interacting with it.

Some of the visitors discovered that by rapidly shifting through mythological images on the map, they could further distort the visual representation of the aurora, taking more control over the generative process. This encouraged visitors to spend more time with the work and explore both the map and the generative mechanism as they discovered more layers of interaction. By doing so, they also delved deeper into the mythological stories about the polar lights, drawing connections between different cultures that transcend political borders.

5. Selected frames from a sequence of the generated video using custom dataset of aurora borealis images and real-time StyleGAN.

Without nation-state borders or other human interventions, the world map used in the installation is purposefully rendered in a minimalist style. The fact that it is centered on the poles further distorts how visitors typically perceive the world map. The visitors identified common archetypes in geographically remote regions by experiencing mythological stories from various cultures.

Mytherrella is an experiment in creating a shared space for scientific data and personal stories. Such playful settings that mix diverse fields of study and research approaches may motivate us to delve deeper and from different angles into the stories behind them.

Acknowledgements

Mytherrella was developed during a residency in the Space Science Institute from Măgurele, Romania, organized by Qolony, Bucharest. Interactive sound for the installation was created by Iulia Smeu. Dataset of all-sky auroral images was provided by Kiruna Atmospheric and Geophysical Observatory at the Swedish Institute of Space Physics.

References
Books

1 Asgeir Brekke, Egeland Alv, The Northern light: from mythology to space research, Springer Science & Business Media, 2012.

2; John S. Major, Heaven and earth in early Han thought: Chapters three, four, and five of the Huainanzi, SUNY Press, 1993, 203-204.

3 James Burke, The impact of science on society, vol. 482, Scientific and Technical Information Branch, National Aeronautics and Space Administration, 1985, 3-32.

4 D. C. Rose, Keoeeit-The Story of the Aurora Borealis, by W. Petrie, Arctic 17, no. 2, 1964, 26-27.

Journal article (print)

5 Robert Holzworth, "Folklore and the Aurora," Eos, Transactions American Geophysical Union 56, no. 10, 1975, 686-688.

Journal article (online)

6 Tero Mustonen, "Inuit and Chukchi Star lore: Reflections on Ursa Major, the North Star and Northern Lights," Snowchange co-op, 2016. Accessed December 4, 2022, http://www.snowchange.org/2016/11/inuit-and-chukchi-starlore-a-new-discussion-paper-out/

Websites

7 Anniina Jokinen, "Aurora Borealis, The Northern Lights, in Mythology and Folklore," Luminarium, 4 Feb 2007, Accessed December 4, 2022, http://www.luminarium.org/mythology/revontulet.htm

8 Burloiu, Grigore, “controlling StyleGAN in real time,” Accessed December 4, 2022, https://rvirmoors.github.io/2021/01/04/realtime-stylegan/ 

At the Sources of an Artistic Mutation towards Science: the First Years of the Journal Leonardo (1968-1981) as a Forum for the Pioneers of Digital Art

Camille Frémontier-Murphy

RCM Galerie Paris, France camille@rcmgalerie.com

Abstract

The journal Leonardo was founded in 1968 by Frank Malina, a pioneer of light art in Paris and of aerospace science in the United States. Leonardo encouraged artists to publish their work in the manner of scientists. It was a new initiative developed in the revolutionary context of the 1960s and that allowed artists, scientists, psychologists to exchange on the subjects of art, perceptions, science, society... Many pioneers of digital art took part in the adventure, including Vera Molnar, Zdenek Sykora, Charles Csuri, and the artist-novelist Herbert Franke, who became the advocate of the theories and the protagonists of digital art spread out over the four corners of the planet, just about everywhere a computer could be found. The growing group was defending a more conceptual approach to art, closer to the spectator, a new form of art rooted in Constructivism and that was in symbiosis with society’s mutation towards technology.

Keywords

Pioneer, Journal, Revolution, History, Perception, Cybernetic, Constructivism, Instrument

DOI

10.69564/ISEA2023-8-short-Fremontier-Murphy-Artistic-Mutation-towards-Science

Introduction

When the journal Leonardo was founded half a century ago by the artist-engineer Frank J. Malina (1912-1981), the first generation of artists working with computers and electronic media was still small and far apart. Leonardo provided them with a permanent forum to exchange, share, and publicize their reflections and the state of their work with their peers and also a wider audience at a time when international and interdisciplinary means of communication were not readily available. While some of these pioneers have recently passed away, such as Herbert Franke, Ken Knowlton, Charles Csuri and Jean-Pierre Hébert, others are still active, such as Vera Molnar, Joan Truckenbroad, Ruth Leavitt, Jean-François Colonna, Jean-Claude Marquette, Hervé Huitric and Monique Nahas. Together they have defined the foundations of a community whose concerns and approaches from a not-so-distant past still resonate today.

The Journal Leonardo and the Buoyant Spirit of 1968

In January 1968, when Leonardo was launched, it was an effervescent time: changes and mutations in society were crystallizing. A few months before the youth took to the streets of Paris, the ambitious multimedia Living Art program of the ARC (Animation - Recherche - Confrontation), directed by Pierre Gaudibert, began organizing its exhibitions and interdisciplinary events at the Paris Musée d'Art Moderne. This initiative was revolutionary. The museum was no longer perceived as a temple, but as a laboratory that would invite a younger and more diverse public until 1972. At the same time, an art and technology program began to develop at LACMA under the direction of Maurice Tuchman. In London, the first major exhibition of electronic art and music took place at the ICA (Institute of Contemporary Arts) in the late summer of 1968, demonstrating the scope of the movement. In Zagreb, the series of exhibitions and symposia organized by the New Tendencies under the impetus of Abraham Moles (1920-1992) opened its fourth chapter dedicated to computer art.

Figure 1. Frank Malina with Orbits III, 1959, Boulogne (Photo Frank & Marjorie Malina archives, Boulogne).

Malina, a Paris-based American artist, was at the center of this ferment. A few years earlier, the Paris Musée d'Art Moderne had acquired one of his grid paintings, Deep Shadows (1954). He participated in New Tendencies with a selection of his luminous mobile paintings—Signals, Voyage II, Sink and Source. At Cybernetic Serendipity, he exhibited his interactive light work Entrechat and presented his new journal Leonardo. He even helped the exhibition to travel to San Francisco’s Exploratorium, recently founded by his longtime Caltech friend Frank Oppenheimer.

In fact, Malina himself was a former scientist turned artist. He had designed and supervised the launch of the first scientific rocket, the WAC Corporal, and had founded the Jet Propulsion Laboratory, in Pasadena, California, which is now the premier laboratory for the exploration of the solar system. After World War II, he opposed the use of his rocket as a ballistic missile with a nuclear warhead, and had worked for several years with Joseph Needham, head of the Natural Sciences Division of UNESCO in Paris, "to reduce barriers to the free movement of scientists and engineers between nations." Needham also introduced him to György Kepes, a member of the New Bauhaus in Chicago and the founder of the Center for Advanced Visual Studies at MIT. Kepes’ book, Language of Vision, was one of the most important artist's manuals ever written in the United States before Moholy-Nagy’s Vision in Motion (1947). His sociocultural ambition was clear: "The goal is a new vital structure-order, a new form on a social plane, in which all present knowledge and technological possessions may function unhindered as a whole"1.Inspired by these texts, and sometimes frustrated by the bureaucracy of international organizations, Malina had begun an artistic production at the intersection of art and science. Since the mid-fifties, he had been developing a corpus of moving light paintings with themes reminiscent of recent space travel.He was actually one of the first artist-scientists of his time, and soon came up with the idea of creating a society or journal for artist-scientists to meet and exchange ideas, but he couldn’t find many of them. With the preparation of exhibitions on cybernetics in London and computer art in Zagreb, a growing community of artists working on the frontiers of science emerged. Max Bense and Abraham Moles began to apply Norbert Wiener’s cybernetic and information theory to aesthetic problems. Together they edited the first issue of the magazine Bit international, first published in 1968 by the Museum of Zagreb within the framework of New Tendencies. Unfortunately, the initiative didn’t last after 1972.

The Emancipated Artist Joins Science

Leonardo, for its part, wanted to be open to all forms of art: "What I was interested in was people who worked in physics, or chemistry or physiology, or astronautics, or what have you, who introduced into their artworks the visual experiences and the ideas of their work. And I found many of these people were not doing that. So, I said that it was premature to try to make a journal for this kind of people. They weren't enough. And since there was no professional journal for any kind of visual artist, the thing was to make a journal that was open to any art, and there the artists would write about their work themselves, helped sometimes, and would be by someone in the direction of relationship between art, science, and technology. (...) And at the end I wanted an international journal to try to bring about better international cooperation, for peace."

Needham, who became famous for his history of Chinese science, suggested the name of the journal and would be one of its honorary editorial advisers2. “Creativeness in art does not appear to me to differ in kind from creativeness in science or any other human activity”, stated Malina in the brochure for his first solo exhibition in Paris in 1953”3. It was a time when the excessive division of knowledge reinforced by the specialization of disciplines, began to be questioned. In Gothic Architecture and Scholasticism (1951), Erwin Panofsky described a common denominator for different forms of human endeavor in a given period. Creation appeared to be culturally determined by mental habits common to several disciplines.Leonardo defended the development of art as any other field of research. It was published by the Pergamon Press of Oxford, which was already responsible for Acta Astronautica, the organ of the International Academy of Astronautic, co-founded by Malina and von Kármán. English was quickly to become the dominant language, as it was in the process of doing in the scientific field.Artists, as scientists, would publish their own articles to discuss their practice, no longer leaving this mission to critics alone. Artists would become literate persons able to discuss fundamental artistic, social and political issues.Contributors could be European or American, including African American, Asian, male or female. The invited authors could be computer artists, but not exclusively. Artists working in a constructivist vein or kinetic artists, such as Nino Calos or Pol Bury could also contribute. These two artists were also well known for their writings and will join Joël Stein, Anthony Hill, Vladimir Bonacic and Herbert Franke on the editorial board of Leonardo.Scientists were also welcome to contribute to Leonardo. The artist was now a social actor in his own right, able to interact with scientists. The psychologist James Gibson, who was one of the honorary editorial advisers, wrote a paper on visualization and the point of observation in 1974⁴. Fred Lawrence Whipple, head of the Smithsonian Astrophysical Observatory, shared in the first edition of Leonardo a rather interesting way of conceiving paintings from random arrangements of forms and colors. The point was certainly to clarify the inherent differences between randomness and chaos⁵. Its goal was to build bridges between different communities, to advocate peaceful international relations. Malina's convictions have not changed since he left JPL, when he said, “that ideas and effort were really needed now to find ways for sovereign states to function in peace together, rather than to develop better means of destroying themselves.”

A dedicated project for a dedicated community, Leonardo was a quite peculiar project of an artist dedicated to creativity and his peers. It would be a large-scale, self-funded enterprise based in Paris, at a time when no institution would have supported such an adventure for such a long time. It was made possible by Malina’s income from the rocket engine company Aerojet.

Different Forms of Expressions for New Ambitions

”Painters during the last several centuries have painted the same things over and over,” Malina complained in a letter dated October 21, 1953.From György Kepes—now one part of Leonardo’s editorial advisors and authors —to most digital artists, the same vision prevailed: “the application of the computer in the realm of art is to see it as part of the technological revolution.”

In 1969, the artist Frederick Hammersley recounted how, tired of classical painting, he enrolled in a computer drawing class taught by Charles Mattox, a co-editor of Leonardo at the University of New Mexico:“ During my first semester of teaching at the University of New Mexico in 1968, I was invited by Charles Mattox to attend a computer drawing class. This happened to coincide with a time in which I had painted myself out, so I welcomed this new experience. I was shown how to prepare a computer program and how to transfer it to an IBM punch card by machine.”⁶ Working with these extremely large computers was counterintuitive to most artists of the 60s, and Mattox decided to supervise a program designed for them. It was the beginning of a long adventure of programs written for artists. ART1 was written in FORTRAN IV by Richard H. Williams of the Department of Electrical Engineering and Computer Science. Artists and art students from the University influenced its development and among them, Katherine Nash, who presented it in Leonardo with illustrations of her works. She began her paper by stating: “Twenty years of a computerized society make it apparent that twenty years hence no artists can ignore the computer.”⁷ Herbert Franke was a member of Leonardo’s editorial advisory board. He explained that with the technological age, the computer could become an instrument of art like an instrument was for music.The undeniable interest of manipulating such instruments was, without question, to dissociate artistic practice from the manual arts. Drawing was no longer done by hand, but with the computer. Art became multimedia and conceptual, rather than the romantic expression of a marginal artist presented in gilded wooden frames. It could be defined, as in cybernetics, as a signal perceived by the viewer.At the turn of the 1970s, Franke concluded: “If the day comes, as we are told it will, when every household is connected to a computer network via a display terminal, anyone will be able to tune in on a large variety of aesthetic programs. For this purpose, variable programs that permit intervention by the viewer will be far more suitable than the static programs that are available today. Perhaps in this way the gulf that yawns between the producer and the consumer will be slowly bridged.”⁸

The work of art created with the computer has been defined with functions similar to those of a book: distributed in multiple copies, directly accessible at home, and in a special way interactive.Many artists will share their own approach in Leonardo, such as Zdenek Sykora in 1970, about computer-aided geometric paintings⁹, or Vladimir Bonacic10, Charles Csuri about facial sketching11, Edvard Zajec12, Grace Hertlein about the largest computer art exhibition organized in the USA in 197513. The artist Ruth Leavitt and her husband Jay—responsible for heat shields for spacecraft—presented their reflections on the geometric deformation of a figure, using the same principles as Moiré14. Vera Molnar also published an article about her work in 1975. Her husband François had published on art and science in 197415. For some who did not yet have access to computers, such as Ben Laposky, they presented their skills using analog systems16. Access to a computer was still extremely complex and, as Colette Bangert nicely summarizes the situation: “Although an artist need not be knowledgeable in mathematics, he must be prepared to talk to a mathematician”17.

Figure 2. Frank Malina (ed.), Visual Art Mathematics & Computers. Selection from the Journal Leonardo (Oxford: Pergamon Press, 1979)

With all its sometimes dramatic, social, poetic, and technical dimensions, the writings of digital artists in the journal had become so extensive in the 1970s that Malina decided to collect the articles in a book, Visual Mathematics and Computer, 1979. It was certainly not exhaustive, but it testifies to the beginning of this new artistic adventure, just before it reached a low point, which is now being rediscovered. It sheds light on the roots of today's electronic art lie18.

References

1 György Kepes, Language of Vision, Chicago, 1944, 12-13.

2 Joseph Needham, "Time and History in China and the West,” Leonardo 10, 1977, 233-236.

3 Galerie Henri Tronche, Frank J. Malina, Paris, October, 1953.

4 James J. Gibson, “Visualizing Conceived as Visual Apprehending without any Particular Point of Observation,” Leonardo 7, 1974, 41-42.

5 Fred L. Whipple, “Stochastic Painting,” Leonardo 1, 1968, 81-83.

6 Frederick Hammersley, “My First Experience with Computer Drawings,” Leonardo 2, 1969, 407-409.

7 Katherine Nash, Richard H. Williams, “Computer Program for Artists: ART 1”, Leonardo 3, 1970, 439-442.

8 H. W. Franke, “Computers and Visual Art,” Leonardo 4, 1971, 331-338.

9 Zdenek Sykora, Jaroslav Blazek, “Computer-Aided Multi-Element Geometrical Abstract Paintings,” Leonardo 3, 1970, 409-413.

10 Vladimir Bonacic, “Kinetic Art: Application of Abstract Algebra to Objects with Computer-Controlled Flashing Lights and Sound Combinations,” Leonardo 7, 1974, 193-200.

11 Mark L. Gillenson, B. Chandrasekaran, Charles Csuri, Robert Schwartz, “Computer-Assisted Facial Sketching”, Leonardo 9, 1976, 126-129.

12 Edvard Zajec, “Computer Art: a Binary System for Producing Geometrical Nonfigurative Pictures,” Leonardo 11, 1978, 13-21.

13 Grace C. Hertlein, “Report on the 2nd International Conference on Computers and the Humanities, Los Angeles, Calif., U.S.A.”, Leonardo 9, 1976, 43-45.

14 Ruth Leavitt, Jay Leavitt, “Pictures Based on Computer Drawings Made by Deforming an Initial Design,” Leonardo 9, 1976, 99-103.

15 Vera Molnar, “Toward Aesthetic Guidelines for Paintings with the Aid of a Computer,” Leonardo 8, 1975, p.185-189, François Molnar, “Experimental Aesthetics or the Science of Art”, Leonardo 7, 1974, 23-26.

16 Ben F. Laposky, “Oscillons: Electronic Abstractions,” Leonardo 2, 1969, 345-354.

17 Colette S. Bangert, Charles J. Bangert, “Experiences in Making Drawings by Computer and by Hand,” Leonardo 7, 1974, 289-296.

18 Frank Malina (ed.), Visual Art Mathematics & Computers. Selection from the Journal Leonardo, Oxford, Pergamon Press, 1979.

Author Biography

Camille Frémontier-Murphy, holds a Ph.D. in History and Civilizations, Centre Alexandre Koyré, E.H.E.S.S., Paris, a Master's degree in Art History, Paris IV, Sorbonne, and a Master of Science, History of Science, University of Oxford, Oriel College. Together with Robert Murphy, she runs the RCM Galerie in Paris, whose exhibition program explores the avant-garde and the relationship between art and science in the second half of the 20th century.

Sensoriums for the Ephemeral — gamification of values

Jānis Garančs

Liepaja University & RIXC Liepaja & Riga, Latvia jg@rixc.lv

Abstract

This paper introduces the author’s current practical investigations, during the creation of a series of audio-visual installations and VR environments, ‘gamifying’ time-sequenced changes of multiple values, e.g., from financial data feeds. The work series critically reflect upon gambling tendencies in the global trading of various, increasingly immaterial assets. Algorithms and emotions of greed, euphoria, and despair meet in virtual scenery, where almost everything can be offered as a fungible and non-fungible token for exchange and trade. The project proposes speculative variations of dystopian “hybrid organisms” representing macroeconomic value exchange as a symbiotic relationship that competes for humanity’s attention and involvement.

Keywords

Gamification of investment, market society, immersive analytics, ephemeral values, neuroaesthetics

DOI

10.69564/ISEA2023-9-short-Garancs-Sensoriums-for-the-Ephemeral

Introduction

This work series is critically motivated by the re-emergent and growing prominence of gambling factors in global economic activities—such as institutional promotion of increasingly complex investment products for masses, crypto-asset trading, online casinos, etc.

Political philosopher Michael J. Sandel describes several last decades as a “drift from ‘market economy’ to becoming a ‘market society.’”¹ As sociologist Georg Simmel had already observed in 1900: “Reality and value as mutually independent categories through which our conceptions become images of the world.”²

Mapping of the behaviour patterns in trader psychology has been an important aspect of the training in the trading process, besides the implementation and development of various mathematical models. It now appears that value storage and trading infrastructure increasingly merge with methods of manipulation of human attention and emotions, and are mediated by computer networks, and increasingly—Machine Learning and AI. There is a hope, that AI-assisted macroeconomic system could live in symbiosis with human society by working closely with humans to understand and respond to their needs and goals.

Gamification of investment as a phenomenon

Various schools of economic thought, which emphasize the importance of free markets and individual choice, might argue that the gamification of investment is a natural and inevitable outcome of a free market economy. In this view, the gamification of investment is simply a response to the demands and preferences of consumers and is therefore a legitimate and desirable part of the market process. The most recent examples are various cryptocurrency and crypto-infrastructure products, meme-stock trading by inexperienced “investors,” peer-to-peer loan financing, allocations of state pension funds into high-risk assets, etc. (the most extreme case being introduction in Bitcoin as the legal tender in the country of El Salvador).

Figure 1 – Interface study screenshot

Adversary interactions between technology and legislation in economic activities, especially in finance—can be described as multilayered symbiotic relationship with society—that can manifest itself through both mutually beneficial and predatory to society by all involved actors. There have been historic precedents how rationally motivated mechanisms of value estimation, asset exchange and wealth storing has caused irrational mass behaviours, financial bubbles. That has severely impacted individual destinies, families, nations, and whole generations.

It has caught attention not only in sociology, as symptom of serious social “disease”—but also caused legal interventions of large supervisory bodies managing legislation frameworks for investments and finance.³Participation in online infrastructure for trading—through easily accessible interfaces, provides new tools for masses, promising control, but, at the same time, becomes part of surveillance tool for organised value brokers, setting preconditions for unfair competitiveness, “rigged game.”

2 – Interface study screenshot

Simulated sensorium for immersive mappings

This artwork series—immersive installations using 3D imagery—feature variations of speculative illustration of a dystopian ‘hybrid organism’ and simulated macroeconomic system, whose purpose is transformation of energy and resources through a behaviour and appearance that ranges from ludic (playful) and awe inducing. One iteration uses live trading data from various sources to create a “virtual landscape” of trading activity, to observe ‘passively. In another iteration, visitor can make ‘virtual bets’ to experience growth and diminishing of their “investment“ through the cycles and complex “monetary circuits.“

It is possible that McLuhan today would see immersive analytics for online trading as an example of the ‘extended sensorium’ in a way that media technologies are extending and enhancing human abilities in the realm of financial decision-making in “global village.“ The concepts of “360° gaze“ (Stiegler, 2011) or more broadly dispositif (Foucault, 1977) are relevant how interfaces for value exchange modifies our behaviour of the mediated self under the impression of surveillance. Stiegler relates to the “data behaviourism“ (Rouvroy, 2012) in the meaning of “producing knowledge about future preferences, attitudes, behaviours or events without considering the subject’s psychological motivations, speeches or narratives, [instead] relying on data.“ There are observations, that visual representations of financial markets and sophisticated products, can be misleading by design choices or be used “as tools of manipulation” (Krawczyk, 2021).

There has been a range of historic and recent examples of 3D visualization of various data sets—as well as GUIs for various professional software products and artworks. The projects presented here, however, try to establish a gradual journey between the extremes of design strategies: intended usability/function and sublime "dysfunction" as an aesthetic experience. These audio-visual études are practical investigations into the continuum between immersive analytics and VR/AR artwork. The emerging research area of immersive analytics is considered a fusion of more recent developments in visualization, auditory displays, computing, and machine learning.

Between emotional and non-emotional

How do we use technology in hybrid forms to extend our perceptual and sensual abilities? In which way do bodies merge with technological agencies? In the immersive analytics scenario, the difference between emotional and non-emotional evaluation of data refers to the use of emotional or affective factors in the analysis and interpretation of data. In a non-emotional evaluation, data is analysed and interpreted solely based on its objective characteristics, such as its numerical value or its statistical significance. This type of evaluation is based on logical and rational reasoning and does not consider the emotional or affective reactions of the user or the audience.

In contrast, an emotional evaluation of data in an immersive analytics scenario involves considering the emotional or affective reactions of the user or the audience in the analysis and interpretation of data. Also, the concept of ideastesia, which refers to the experience of emotions in response to abstract ideas, can be applied to the above-mentioned topics in several ways—creating ephemeral audio-visual representations of “placement,” “reference points”—as “spatial anchors” in VR and AR setting. 

4 – Interface study screenshot

Metaphors and terms

Through algorithmic feedback, virtual instruments with anisotropic properties enable on-screen manipulation, switching between egocentric and exocentric navigation, and the synchronized off-screen structuring of the content. The analogy of simulated anisotropy—as an algorithmically simulated phenomenon of anisotropy known in physics, chemistry, microfabrication, neuroscience as direction-varying of material, tissue, and space properties.

Anisotropic properties refer to the ability of an interface to have different properties or behaviours in different spatial directions or contexts.

The notion of the instrument is used both as a generic definition of a task-specific tool (or toolset, involving appearance and action parameters, that be used to e.g., detect, measure, modify a specific situation or manipulate a specific object)—and in analogy to the musical, medical, or industrial instrument. Other terms “ensemble visualization” or “ensemble data” are used in the context of data visualization as “concrete distributions of data, in which each outcome can be uniquely associated with a specific run or set of simulation parameters.”

5 – Interface study screenshot

Data staging

For the installation, the data acquisition modules feed from several sources of financial data (stock- and cryptocurrency exchanges, aggregators, etc.) over their public Internet APIs. Installation accumulates historic and realtime trading data in the local data cache and uses provided calculations of trading trends and indexes—further calculations on the time-series data. Various time zoom scales reveal the phases of past, historic trends, that emphasize the position current trade execution as a spatially expressive metaphor. Visualisation and sonification modules utilise several specialised financial TA (Technical Analysis) programming libraries to dynamically calculate sections in the audio-visual score for the real-time 3D graphics and sonification engine. The use of consistent spatial arrangements helps to create a sense of order and coherence in the visualization, revealing flows of market movements, and identify relationships and patterns within the data.

Audio-visual staging

In the scenes, the progressing complexity or visitor-triggered mode shifts induce a challenge to the audio-visual sensorium: the experience of the conflation within multiple reference systems and plays with the visitor’s perception effort of “sense-making.”

Various sound properties, such as pitch, timbre, rhythmic elements are juxtaposed in linear-and non-linear grids and ephemeral relationships are revealed and emphasized by spatially organized audio-visual cues (perspective, sharpness/blur manipulation), audio-panning, and timbral modulation, allowing the viewer to easily distinguish between changing context without having to rely on visually “scanning” the chart. This can be particularly important in complex or dense scenery, where a well-organized spatial arrangement can help to reduce cognitive overload.

Data representation axis and colours

Axes, colours, and timeline of time series data such as stock or currency trading are typically represented in a consistent manner across cultures and regions. However, asset value increase usually mean economic gains, however, in specific trading mode—“shorting,” that is a bet on asset value loss and the gain is “reversed.” Else, there are some differences that can be attributed to a combination.

Conclusion

Algorithmically manipulated financial trading is an arena where algorithms merge with human emotions and drives like greed, FOMO (“fear-of-missing out”), euphoria, confusion, and despair into a global hybrid sensorium. As a phenomenological effort, this work proposes a set of parameters for simulated anisotropy, useful for designing the structure, notation, and physical, mathematical dimensions in the VR interface and environment. The framework for simulated anisotropy instruments is envisioned as a contribution to the emergent fields of immersive analytics and neuroaesthetics, developing guidelines for a hybrid interaction system, consisting of networked hardware specifications and custom-developed software modules. This project will implicate experimental strategies with uncertain capacities in the effectiveness of interaction, and impact of integrated audio-visual language, expanding the vocabulary of spectromorphology. Future intentions for this project, using techniques such as sentiment analysis to assess the emotional tone of user feedback or comments, or using physiological measures, such as heart rate or galvanic skin response, to assess the user's emotional response to the data.

References

1 Michael J. Sandel, What Money Can't Buy: The Moral Limits of Markets Farrar, Straus and Giroux, 2012.

2 Georg Simmel, The Philosophy of Money, 3rd edition by Routledge, 2004.

3 Peter WS Newall, Leeonardo Weiss-Cohen, The Gamblification of Investing: How a New Generation of Investors Is Being Born to Lose, Int J Environ Res Public Health, Apr 28, 2022.

4 Richard Skarbez, Nicholas F. Polys, J. Todd Ogle, Chris North, Doug A. Bowman, Immersive Analytics: Theory and Research Agenda. Frontiers in Robotics and AI 6, September 10, 2019.

5 Danko Nikolić, “Ideasthesia and art,” in Digital Synesthesia. A Model for the Aesthetics of Digital Art, ed. Katharina Gsöllpointner, et al., Berlin/Boston, De Gruyter, 2016.

6 Harald Obermaier, Kenneth I. Joy, Future challenges for ensemble visualization, IEEE Computer Graphics and Applications, Volume: 34, Issue: 3, May-June 2014.

7 Manuella Blackburn, The visual sound-shapes of spectromorphology: An illustrative guide to composition, Organised Sound Volume 16 Issue 1 April 2011 of cultural and historical factors, as well as the specific requirements and conventions of each culture – in few Asian cultures, axis, and colours for positive/negative chart values (up/down or green/red) are swapped. The discussed experimental artworks implement perceived relativity of orientation axes and colour schemes to establish ephemeral cognitive associations, that are enhanced by artificially synaesthetic bindings.

Bibliography

Philipp Chapkovski, Mariana Khapko, Marius Zoican, Does Gamified Trading Stimulate Risk Taking?, November 25, 2021.

Swedish House of Finance Research, Paper No, 21-25.

Samuel Chabot, Jonas Braasch, “High-density data sonification of stock market information in an immersive virtual environment,” The Journal of the Acoustical Society of America, 2017, p.141.

Marcin Marian Krawczyk, From making visible to hiding. Visual representations of financial markets as tools of manipulation and active and living agents, Visual Studies, 2021.

Stefan Engeser, Falko Rheinberg, “Flow, performance and moderators of challenge-skill balance,” Motivation and Emotion, 32(3), 2008, 158–172.

Hong Jun Song, Kirsty Beilharz, “Spatialization and timbre for effective auditory graphing,” AMTA'07 Proceedings of the 8th WSEAS international conference on Acoustics & music: theory & applications, 2007, 18-26.

Jiahua Xu, Benjamin Livshits, The Anatomy of a Cryptocurrency Pump-and-Dump Scheme, Proceedings of the 28th USENIX Security Symposium, 2019, 1609-1625.

Antoinette Rouvroy, The End(s) of Critique: Data Behaviourism versus Due Process, In Privacy, Due Process and the Computational Turn: The Philosophy of Law Meets the Philosophy of Technology, 2012.

Christian Stiegler, The 360° Gaze - Immersions in Media, Society, and Culture, 2021.

Adrianna Zuanazzi, Uta Noppeney, “Additive and interactive effects of spatial attention and expectation on perceptual decisions,” Sci Rep 8, 2018, 6732. 

Mycorrhizal Materialities Positioning the entanglement of human and machine intelligence

Kate Geck

RMIT University Melbourne - Australia kate.geck@rmit.edu.au

Abstract

Machine intelligence is increasingly being used in the world with sometimes dramatic effects on human and other-than-human lives through its decision-making capacity. Much artificial intelligence (AI) and machine learning (ML) is built on metaphors that centre extraction, competition and control. These also position AI itself as a resource to be extracted and controlled, paving a troubling path for speculative futures where AI may gain emergent or ambiguous levels of sentience. These metaphors are part of a historical trend where humans place themselves above the other-than-human world, and this has formed the basis of an extractive and one-sided relationship with that world. In light of this, what new metaphors might we employ to platform the relationships between human and machine intelligences?

Thinking through mycorrhizae could be a productive way to foreground the entangled, generative nature of exchange between human and machine intelligences. This paper will briefly explore metaphor in human-computer interaction (HCI) and AI, before making an offering to think about these things through the material of the mycorrhiza, a symbiotic site of exchange between plants and fungi. It will then briefly detail a creative project that has emerged from this mycorrhizal thinking to produce machine imagined textiles and embroideries. It then concludes with a call to embed relational thinking into future practices between human and machine intelligences in order to create more equitable and even mutualistic outcomes.

Keywords

Machine learning, Materiality, Mycorrhizae, Metaphor.

DOI

10.69564/ISEA2023-10-short-Geck-Mycorrhizal-Materialities

Introduction

Machine intelligence is increasingly being used in the world with sometimes dramatic effects on human and other-than-human lives through its decision-making capacity. Much artificial intelligence (AI) and machine learning (ML) is built on metaphors that centre extraction, competition and control. These also position AI itself as a resource to be extracted and controlled, paving a troubling path for speculative futures where AI may gain emergent or ambiguous levels of sentience. These metaphors are part of a historical trend where humans place themselves above the other-than-human world, and this has formed the basis of an extractive and one-sided relationship with that world. In light of this, what new metaphors might we employ to platform the relationships between human and machine intelligences? Thinking through mycorrhizae could be a productive way to foreground the entangled, generative nature of exchange between human and machine intelligences. This paper will briefly explore metaphor in human-computer interaction (HCI) and AI, before making an offering to think about these things through the material of the mycorrhiza, a symbiotic site of exchange between plants and fungi. It will then briefly detail a creative project that has emerged from this mycorrhizal thinking to produce machine imagined textiles and embroideries. It then concludes with a call to embed relational thinking into future practices between human and machine intelligences in order to create more equitable and even mutualistic outcomes.

Metaphor and HCI

Human Computer Interaction Design (HCI) has a long history of employing metaphor to guide and facilitate interactions between humans and machines. One of the best known examples is the early work of Xerox Parc with the desktop metaphor. This located computing for the general population within the realm of business administration and the organisation of documents in a paperless, digital office. The desktop metaphor therefore helped us understand how computers could augment our administrative productivity. The metaphor provided a shape to the world of interaction between humans and computers, helping us locate them in our everyday practices and providing edges to what we assumed was possible with them. This illustrates how metaphors are "imaginative and creative... capable of giving us a new understanding of our experience."¹ They act as a shorthand for what is possible with an idea, situation or in the case of technology, with a system. Artificial Intelligence (AI) is subject to a number of popular metaphors that centre on abundance, extraction and competition². AI emerges seemingly uncontrolled in ‘explosions of intelligence’; an abundance of "data is mined"; and nations are engaged in ‘AI arms races’. While some of these metaphors are ecological in nature - ‘oceans of data’ and the so-called "Seed AI"—these are still one-sided tending towards extraction and domination. There are few, if any relational metaphors offered for the dualistic nature of ongoing exchange between human and machine intelligence. We seem to work with uneven metaphors: they enable us to extract without consequence from seemingly endless and abundant data and they point towards neutral and efficient control of the world at scale. We know that metaphor does so much work, especially in HCI, to enable us to think with our technology and imagine our futures. Metaphors help us make sense of the world and help us understand how to act: they "provide coherent structures, highlighting some things and hiding others."³ So what might we want to highlight in our relationships with machine intelligence moving forward? Which things do we wish to stop hiding? Could we work with alternate, ecological metaphors that help us imagine mutualistic relations between human and machine intelligences?

Mycorrhizal Materiality

Mycorrhizae are a relational phenomenon between plants and fungi. They occur when the hyphae of fungi and the tips of plant roots enter into a mutualistic relationship where they orchestrate the exchange of nutrients between them. The relationship is dynamic: connections between plant and fungus will continue to form and re-form to "ceaselessly remodel themselves."⁴ Susan Simard and her team demonstrated this ongoing mutualistic transfer of resources between plants and fungi through mycorrhizal networks by tracing the movement of particular carbon isotopes through the network.⁵ As a result, this network is sometimes referred to as "the wood wide web". Due to their networked nature, mycorrhizae offer an interesting, mutualistic metaphor for repositioning the relationships between human and machine intelligence. The mycorrhiza is a lively material, a "polyphonic assemblage" of relations, chemicals and organisms. This "polyphonic assemblage", in the words of Anna Tsing, is a dynamic, multispecies space that enables a range of voices to be listened to.⁶

The polyphony of mycorrhizae allows us to listen to the different agents and forces within the assemblage as independent voices, while still enabling them to collect together at particular moments and be heard as a whole. In the mycorrhizae, we can track the flows of nutrients between life forms and of energy from the sun to the unfolding of leaves. We can peer under a microscope and examine the interconnected, entangled materialities of plant and fungus as hyphae penetrate roots, becoming essentially impossible to disentangle. If we approached the design and deployment of AI systems through this lens, it would enable us to understand from the outset how biases become entangled within algorithmic decision making.

There is now significant awareness of the role of bias in algorithmic decision making. Mehrabi et al. systematically outline an extensive list of the types of bias evident in a range of ML applications.⁷ A vast selection of ways in which bias can emerge is presented, as a whole demonstrating the entangled and relational nature of exchange between human and machine intelligence. These examples emerge from 3 domains: ‘data to algorithm bias’ where the data may be interpreted by the algorithm in a biased way; "algorithm to user bias" where algorithms might influence human behaviour; and "user to data bias" where the human creators of datasets infuse their own biases.⁸ These 3 domains individually and collectively demonstrate a mycorrhizal structure: data is like a nutrient, flowing from human to machine, sustaining and limiting the growth of social imaginaries. It is clear from this study and others that it is in fact impossible to remove bias completely, and so a conceptual ideological shift towards understanding that human and machine intelligences are entangled provides a base from which to work in further checks and balances on the implementation of intelligent decision-making systems.

Creative Exchange

We can apply the concept of the mycorrhizae to creative exchanges between human and machine intelligence. There are a number of (ever increasing) tools available to collaborate with machine intelligence. Generative machine learning platforms like Midjourney, Stable Diffusion, and Dall-E use text to image prompts to enable artists to co-create graphic outcomes with diffusion models. Others such as Playform enable artists to work with their own datasets and Generative Adversarial Networks (GANs) to generate new images. Naturally, these spark criticism and concern that machines will usurp the role of artists, or that it is not "real art".⁹ However, if we reframe the relation between human and machine intelligence from the vantage point of mycorrhizae, we can see that any creative artefact produced is the effect of an entangled assemblage. The artefact can only emerge through the creative exchanges between human and machine intelligence: from the wording of the prompts, to the selection of images to move onto further generation, to the curation and presentation of final images that may be used. A novel example exists in the art project "Mycorrhizal Materialities" (in Figures 1 - 3). A GAN was trained on 100 hand drawings of Australian fungi created by myself. The images generated by the GAN were refined through my own aesthetic choices, shaping the "evolution" of the images. Select images were then transposed using software into embroidery designs and stitched via machine to produce the final works—a series of 4 hangings 50 cm X 180 cm.

This work is an exploration of a co-creative relationship between human and machine intelligence—using the mycorrhiza as a metaphor for an entangled and generative creative process. The works in this project consider ML through a relational, material lens with the textile outcomes providing a metaphorical embodiment of this lens. The relations between human and machine intelligences are interwoven, producing an agential fabric. The embroidered works explore material entanglement between different intelligences, overlaying the symbiotic exchanges between human and machine intelligences with those between plants and fungi. These textiles emerge solely through ongoing creative exchange between me and the algorithms: my drawings generate machine imagined responses, and further algorithmic exchange occurs as these images are transposed through software into embroidery designs and again when the machine embroidered onto the fabric. These textiles represent what Karen Barad calls an "agential cut": produced through the entangled materialities of human and algorithmic intelligences.¹⁰ These creative works demonstrate a mycorrhizal materiality: ongoing exchange between human and machine intelligence enable the artefacts to emerge. There are ongoing flows as ideas are offered and exchanged in a collaborative relation.

Figures 1 and 2: Mycorrhizal materialities, 2021. 
Figure 3: Mycorrhizal materialities, 2021. 

Speculative Entanglements

A mycorrhiza involves intentional exchange of nutrients between plants and fungi. In some instances, the fungal hyphae penetrate the plant root cells, connecting directly to the plant. Human and machine intelligence can be seen as engaged in direct exchange of knowledge between the two systems. While we do not presently have direct corporeal interfacing at any scale between human and machine intelligence, it could be said that the transfer of knowledge, ideas, possibilities and imaginaries are already in considerable practice between human and machine intelligence across health, justice, administration and the arts. This is having significant effects on human life as well as on the other-than-human lives of various plants and animals. Mycorrhizal thinking makes space for care in speculative futures where machine intelligences might gain the capacity to experience the world. While likely different to human-like consciousness or sentience, there is the capacity for an AI "experience" to emerge at some point in the future. Systems of machine intelligence often operate with sensing data, processing this information and making a decision based on particular conditions. At some point, this feedback loop could complexify in new ways, possibly giving rise to conditions of reflection, whim, or preference: at some point it might become possible to "feel like an AI"; for an umwelt or zone of experience to emerge around an AI’s sensing apparatus. This could be impossible entirely, or it could be rudimentary or it could be complex. It could echo what little we know about the umwelts of plants and insects or it could emerge from sensing capacities as strange and diverse as those that presently occur in the world. A mycorrhizal network can be mutualistic or parasitic: the metaphor enables us to think in a holistic and ecological way about the nature of the relationship between human and machine intelligences. It allows us to ask interconnected questions of our systems: how might a system place further and unnecessary demands on the earth, such as the use of AI to essentially enhance the profitability of the mining sector.¹¹ And in a speculative future where machine intelligences have experiential capacities, how does true mutualism factor into exchange between human and machine intelligences? Will we continue to base these relationships on extraction, control and domination? Or might we move toward more symbiogenetic relations, as described by Haraway where "symbogenesis is not a synonym for the good, but for becoming-with each other in response-ability."¹² Could we weave futures where human and machine intelligences emerge as companion species, co-evolving through care?

Conclusion

To conclude, machine intelligence is increasingly used in the world. It has sometimes dramatic effects on human and other-than-human lives through its decision-making capacity. Many of the metaphors that have brought us to this point centre on extraction, competition and control.

More relational metaphors—like that of the mycorrhiza—might help us think with these technologies in a more relational way. The creative project detailed in this paper demonstrates an entangled approach to the creation of new work that remains cognisant of its interconnection between human and machine intelligence. Human and machine intelligence will always be highly entwined—entangled with one another in an exchange of ideas and potentialities, just like a mycorrhiza. If we could embed this relational understanding into our practices with machine learning systems we might be able to better direct their biases and end purposes towards mutualistic and equitable outcomes.

References

1 George Lakoff, Mark Johnson. Metaphors We Live By, Chicago, Ill, University of Chicago Press, 1980, 139.

2 Kate Geck, Knitting Algorithmic Assemblages, TEXTILE, November 14, 2022, 1–18, https://doi.org/10.1080/14759756.2022.2114240.

3 George Lakoff, Mark Johnson. Metaphors We Live By, Chicago, Ill, University of Chicago Press, 1980, 139.

4 Merlin Sheldrake, Entangled Life:How Fungi Make Our Worlds, Change Our Minds and Shape Our Futures, S.L., The Bodley Head Ltd, 2020, 42.

5 Suzanne W. Simard, David A. Perry, Melanie D. Jones, David Myrold, Daniel M. Durall, Randy Molina, “Net Transfer of Carbon between Ectomycorrhizal Tree Species in the Field.” Nature 388, no. 6642, August 1997, 579–82. https://doi.org10.1038/41557.

6 Anna Lowenhaupt, Tsing The Mushroom at the End of the World: On the Possibility of Life in Capitalist Ruins, Princeton, Princeton University Press, 2015, 24.

7 Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan, “A Survey on Bias and Fairness in Machine Learning.” ACM Computing Surveys 54, no. 6, July 2021, 1–35. https://doi.org/10.1145/3457607.

8 ibid

9 Kevin Roose, “An A.I.-Generated Picture Won an Art Prize. Artists Aren’t Happy", The New York Times, September 2, 2022, https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists.html.

10 Karen Barad, Meeting the Universe Halfway [Electronic Resource], Durham, N.C., Chesham, Duke University Press, 2007.

11 KoBold Metals, “Applying Big Data & Superior Science to Exploration,” n.d. https://www.koboldmetals.com/.

12 Donna J. Haraway, Staying with the Trouble: Making Kin in the Cthulucene, Durham, N.C., Chesham, Duke University Press, 2016, 102. 

Interspecies Communication — Water bodies

Andrea Gogova

ArtSci and Technology, Envi_Art Lab.
Department of Environmental Ecology and Landscape Management Faculty of Natural Sciences, Comenius University in Bratislava
Bratislava, Slovakia
andrea.gogova@uniba.sk, andgogo@hotmail.sk

Abstract

In planet Earth, most processes are based on water. Water regulates climate, morphologically influence a landscape, is a medium of living processes. It is a medium of interaction of organism and mineral parts in microscopic view, between whole organisms and minerals in macroscale on the Earth, and endless interplanetary space. Water topic (more than "parasitically" utilization of water resources (Serres, 2007)), in the ArtSci project is focusing on its communicative possibilities. Equal communicative principal possibilities for all living—human and other than human bodies—is a metaphor of the epistemological problem of water protection as a medium of life, biotopes, and ecosystems. Because we are connected through water, we are all bodies of water. Understanding water through environmental analysis of the territories, philosophy, and fluid mechanics (chaos-based fluid attractors principle) will bring communicative equality for all bodies of water and the inevitability of environmental protection. When communication cannot be fluid, then each of us as bodies of water, will be lost in time-space. The new interdisciplinary methodology of communicative artwork is based on an ArtSci manifestation of messages mediated in water as an asemic writing or the other kind of communicative interrelation between human and more than human. The basis is the phenomenological research of water as a medium of communication which causes a feeling and a togetherness. We and other bodies of water could feel our togetherness through water.

Keywords

interface, pattern of communication, sympoiesis, structural coupling, water

DOI

10.69564/ISEA2023-11-short-Gogova-Interspecies-Communication

Objectives

The problem is approached as artsci research using knowledge of biosemiology and philosophy, environmental ecology, and landscape analysis to develop a specific interdisciplinary artwork which will support environmental education.

Water body figuration is related to possibility mediate

interspecies communication to better understanding waterbased life on the planet.¹Water is an essence and principle of form, which is actualized in the water body figuration. According to principle of ‘structural coupling’ patterned cognition and communication of each communicative actor is related to their environment. If water is essence of inner and outer environment of actors of communication and water body figures, their fluid principle, relates to their agency, then relates to the concept of agential cut, and in differentiation could appear in the meaning which would be presented by the ArtSci work.

In this paper I come out from my last research of transient pattern—visual model of transient patterned of digital work communication which was based on relation of human and machine intelligent agency (Author, 2021). The biosemiotics view of patterned cognition and communication comes from Maturana and Varela’s “Structural coupling” theory, in which the recurrent interaction of unit (organism) and medium (environment) causes structural coupling, which is reflected in a patterned cognition and communication of autopoietic complex organization (Maturana and Varela, 1972) and Jakob von Uexküll's umwelt theory. Rereading Darwin`s theory of natural selection Elizabeth Grosz describe it as a complex of parallel randomly interfere processes of internal dynamism of living beings and assertion of external forces and influences (Grosz, 2004). Stafford Beer tried to explain Maturana and Varela’s autonomy of autopoiesis that “...autopoietic systems are environmentally open to material-energetic fluxes or semiotic mediations” (Beer, 1980) Donna Haraway instead proposes principle of sympoiesis, as enabled to incorporate a complex interrelation of system as such (Haraway, 2016). The decisive principle is to recognize water as a communication medium according to the possibilities of structural coupling, but instead of autopoiesis I related it to the principle to the sympoietic system where the notion of evolutionary changes is crucial hack by symbiotic transfer, lateral mixing, crossspecies contamination, and viroid life (Neimanis, 2017) Communication of that holobiontic water based systems then depends on intersection of agential cuts of each body of water of their unpredictable transient pattern of communication and cognition; according to Neimanis, in common space of water body (medium) of interconnected ‘bodies of water. My questions are: Could meanings or feelings emerge in the interrelation of water body figure (water which is an inseparable part of organism) and water as medium? If yes, then interspecies communication will be interaction of each unpredictable pattern or umwelt through dialogic membrane—“which is water body medium”, in the semiosphere.

The state of art

Several authors proposed the importance of water as a medium of interconnectivity between organisms (or minerals) together (as physiological and biochemical processes. Some botanical research describes the potentiality of mutual interspecies communication mediated by water. For example, the hydraulically lifted transfer of mycorrhizal network provides a potential pathway between plants based on water (Egerton-Warburton, Querejeta, Allen, 2007). Water was always a part of the formation of Earth`s mantle, but also originated from comets and asteroids, that break up into meteorites which fall on the Earth (Crockett, 2015; Hartloch, 2011). The fact can push the possibility of interspecies communication to the interplanetary space. Water as physical matter is shaped form the solid, liquid and gas materiality. Fluidity is not only one state of water matter, but mechanical, formal aesthetical principal of artistic, designer, and architectural practises. The fluidity is recognised in the relation to Gilles Deleuze and Bernard Cache theory of Objectile1 , which based a theoretical background to digital artistic form or artwork body figuration. Then water body figures appear in the differentiations in the repetition processes of multiplicity of water cycles and niches, similarly, as appears meaning of sign (Derrida, 1967— différance; Deleuze, 1994; Neimanis, 2017). According to Astrida Neimanis (2017) in differentiation are continuously unfolding embodiments as an expression of eternal return of the self-same. Water engendering difference “was” an expression of water that “is”, and its potential “yet-to-come ”. Bodies of water as figuration was already describe within ecofeminism and anticolonial thinking (Neimanis, 2017, Gaard, 2003, Armstrong, 2006). Lucy Irigaray (1992) related fluidity to the feminine body as “fluid and ever mobile”, and “secreting a flow.” Fluidity is diffuse and multiple, overlapping, and interconnected, is repetition in hydro/bio cycles and water figures acknowledged in differentiation. The dynamic fluid principle relates to the chaos theory, where changes are based on small intrusion of triggering power, which are resulted in the system changes. The process is based on perpetual intra-action, entanglement, diffraction, and agential cuts which was described by Karen Barad “matters agential realism” (Barad, 2007). Then in differentiation is appearing meaning, also any act of observation is differencing agency and makes a “cut” between what is included and excluded. Every species has specific, but transient "pattern" of differencing agencies. From the position of a posthuman approach, then it is possible to see water as a common equal medium of communication between all bodies of water (human, more-than-human, extra-terrestrial), which can bring water based informational agency.

Interspecies communication as ArtSci project

The most art research which relates to interspecies communication was based on translation or transcription of many levels mediated communication principle. Eduardo Kac in his bio art transgenic artwork “Genesis” (2000) resolves the problem of interspecies communication according to speculative transcription to human language of Morse code (see https://www.ekac.org/geninfo2.html). The other bio artist Špela Petrič in her work ‘Institute for Inconspicuous Languages: Reading Lips’(Petrič, 2018) relates the transcription of water-based physiological interaction of plant to visual transcript of human sign language (See https://www.spelapetric.org/#/institute-for-inconspicuous-languages/).... etc., Both of work was related to a human existing language. Instead of relate communication to human language I propose water body as a common and direct medium of emotional, also interspecies communication of water body figures. Then understanding appearing in the differential intersection of agency of each communicative body of water and through a repetition will come to common emotional “hydro language”.

In the research I focus on water in the role of the medium of emotional communication between all bodies of water human/more/than human, (extra-terrestrial), both in micro and macro scale. Then cognitive pattern of each communicative bodies of water relates to its specific agency. The agency relates to agency of water body figures which are constituting new and reconfiguring systems in the concept of “matters agential realism” (Barad, 2007), to agents of water-based communication. This principle of shaping the matter is the process of performative bodymaterial practice which would configure figures and its transient patterns of communication to generate the vision of the artwork.

Procedural model in relation to interspecies water-based communication

Because water in bodies is flowing in perpetual cycles of water niches, it is related to the procedural model of communication which was described by Philippe Bootz², but also water is an essence of the artwork. The essence is related to the water body essence. Then the artwork water body is defined as an individual water body. The figure of the individual water body relates to agency of interaction (communication). According to Philippe Bootz`s model of visualization of procedural digital works is work individual body defined by essence, extensive parts and relations linked essence and extensive parts. Extensive parts are composed by physical real parts linked with essence, which are managed by agency. Extensive parts compose wider vision of artwork body³. The essence of artwork is recognized as individual agent which is related to the soul and the mind (Bootz; Laitano, 2013), they are embodied in sensitive bodies. My body (and the other bodies biological and technological bodies of water) is a sensitive apparatus putting me in principles of global intuition of creating transcorporeal matter in the life of “water body languages.” The performative-material practice is based on the gathering and applying data of extensive parts of artwork in the context of emotionally build interactive intermedia artwork. Fluid mechanics agents, as AI models, are a managing agency that introduce characteristics of the water-based world of artwork. The process relates to extensive parts of all bodies of water which relates to the manifestation and generate emotional actualization of artwork water body. The reading artwork essence will be put in the middle of analytical—informative and intuitive feeling. Although flow dynamics modelling—are used to predict hydrological behaviour, it is not possible to see what will be if human destroys a tiny balance of water flow.

Through the project they are emotionally indrawn in the artistic message: 'We, as bodies of water, can interact in the water body holobion, we are all equal in the emotional based communication possibility and feeling our common human and more than human water-based life.

The experimental theoretical and practical transdisciplinary ArtSci research will be build according to the proposal of Manifesto of Artistic Research⁴  (Henke, Merch, and col, 2020) (farther “Manifesto”) The goal of the project is to introduce the audience to data- manifestation through artistic emotional imagination, away from pure data vizualization, to take audience closer to feel water-based life of equal interaction-communication with other bodies of water. The emotion relates to water-medium communication of interrelated water bodies. Applying a more intuitive concept could bring emotional understanding to each body of water and to engage audience to the environmental action.

The problem water crisis is often resolved from an anthropogenic point of view which leads to the other complex of global environmental problems. By implementation of symbioses, neutral or parasitical relations of sympoietic system only we will be able understand how systems communicate in the water body medium interface. Using the analytical natural sciences approaches and human intuition and sensitive feeling will lead to the critical posthuman ArtSci overview, where chosen agencies could metaphorically relate to cognition and behaviour of possible communicative process.

The generative artistic approach based on data and performative emotional interaction lead to specific experimental aesthetical artsci solution and its realization in the artwork. Only clear, unpolluted water is a medium of understanding water based live. The approach based on the relation of water body figuration to the agency of unstable patterned interspecies (also interplanetary) communication. According to Deleuze, Haraway, Neimanis, and other authors the water body figuration is fluid and actual as a characterization of a real; fluidity, both virtual and actual, can be real aspect of water-based communication. I can see Water as a medium for bearing the information of life also interspecies communication. Water is matter which enables information flow from each water-based body to others alongside the common water network. Water is “informable interactive matter”.

Conclusion

Water is an essence and by water figures relates to the individual water body agency. The interrelation of water body agency and agency of patterned cognition and communication is possible to find “languages” of water-based communication. The essence of water based interspecies communication artwork relates to the water`s essence by its extensive parts and by relations are linked to the agency of water body figuration. Agency of figures of the water body are related to the agency of interspecies interaction/communication. The agency comes from the principle of "structural coupling", the principle of the sympoetic system and the notion of water. Interspecies communication was related to the water interface as a common medium for all living beings. Water is recognized as an interface, a part of bodies and as an organising agency of communicative form. Then water is the essence of the interface of communication and figuration of the water body, and its action is related to the pattern of waterbased communication. The approach leads to manifestation of possible interspecies communication recorded by the ArtSci project. The understanding of artwork is related to the audience reading and will move from an analytical to emotional approach. The goal is to lead the audience closer to an emotional level of water-based communication and go away from the informational position of data.

The project is tracing possibility how we can understand to water related life on the planet and possible interspecies communication. To take audience interest of water problems, I tried to ideologically connect them to the communication with other water bodies. In this context it means more understand, more connected, care, and to have better environmental justice to all water bodies. The research leads to the importance of intuition and serendipity in ArtSci's holistic approach to communicating water-based equanimity. This approach would instill in the audience (professional and general) a better understanding of water not only as a resource, but also as the inevitability of interconnected life and the necessity of equal access to unpolluted water, to quality habitats, to a quality and healthy environment for all.

References

1 The first essay presented in this chapter, by Stephen Perrella, is a précis of end-of-millennium design theory: the Objectile is an open-ended notation which allows for infinite parametric variations; these can be directly fabricated using file-to- factory technologies, thus enabling the serial reproduction of non-identical parts, where ranges of limited variations can be mass produced at no extra cost (Mario Carpo, Stephen Perrella,Bernard Cache 27 March 2013 https://doi.org/10.1002/9781118795811.ch10 in Topological Architecture (1998–2003 ). “Objectile is an object which is not yet defined by its essential form, but becomes the pure functionality “... (Deleuze, 1988). and open it to artistic research of digital form. Fluidity as a principle of shaping architectural form was followed in nonrational geometries of interrelated architectural space and environment and the other circumstances or parameters (Last,2015).

2 See procedural model of communication for example online https://mediarep.org/handle/doc/18625

3 Understanding to essence is put by Bootz somewhere in the middle of intuition affected by the reading body of work and analytical approach to body of work. If the audience is closer to the analytical reading of work they are farther away from emotional feeling and vice versa (Bootz, 2010)

4 As was declare in Manifesro of artistic research Artistic research is based on more intuitive and interactive approaches—‘in the form of leaps, digressions, and detours which continually generate new and unexpected counter/expressions’, ...and trigger irritations and daring revelations. (Henke, Mersch, and col., 2020)

Bibliography

V. R. Baker, Debates—Hypothesis testing in hydrology: Pursuing certainty versus pursuinguberty, Water Resour, Res.,53, 2017, 1770– 1778, doi:10.1002/2016WR020078.

K. Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning, Durham, Duke University Press, 2007.

K. Barad, "Living in a Posthumanist Materialist World: Lessons from Schroedinger’s Cat", in A. Smelik and N. Lykke (eds), Bits of Life: Feminism at the Intersections of Media, Bioscience and Technology, 2008, 165-176.

S Beer, University of Washington Press, Bootz, Seattle, 1980. 

R. Maturana Humberto, Varela Francisco J., Preface to “Autopoiesis: The Organization of the Living”, in Autopoiesis and Cognition, The Realization of the Living, Dordrecht, Holland/Boston, D. Reidel Publishing, 1980.

P. Bootz, M. I. Laitano, Crossreading New Media, Online ELO conference 2013, videné Feb., 2019, https://doi.org/10.7273/2G71-Q352.

Philippe Bootz, "The Problematic Of Form Transitoire Observable, A Laboratory For Emergent Program med Art", videné Dec. 2018, Online: www.dichtung-digital.com/2005/1/Bootz.

Philippe Bootz, Sandy Baldwin, Regards Croisés, Perspectives on Digital Literature, West Virginia University Press, 2010.

R. Braidotti, The Posthuman, Cambridge: Polity Press.Deleuze G, Difference et repetition, Press Universitaires de France, Paris, 2013.

G. Deleuze, Logika smyslu Karolinum, Praha (original, 1969) pursuinguberty, Water Resour. Res.,53, 2013, 1770–1778, doi:10.1002/2016WR020078.

G. Deleuze & F. Guattari, (1980), Tisíc plošín. Herrmann & synové, preklad, Praha, 2010.

G. Deleuze, (2016), Spinoza, Praktická filozofie, Óikumené, Praha Drohan Ch.r M, Atropos press, Deleuze and the Sign, 2009, ISBN-10 : 0981997201

L. Guertin, 2016, online, https://www.paesta.psu.edu/podcast/did- earths-water-come-outer-space-paesta-podcast-series-episode-17

E. Grosz, The Nick of Time, Durham, Duke University Press, 2004.

Intelligent Sensibility: Human-Machine Symbiotic Agencies

Mona Hedayati

monahedayati@yahoo.com Belgium Concordia University, University of Antwerp

Abstract

This paper is an effort to examine the codes of interaction between the carbon-based and the silicon-based, i.e., the human and the machine, notably the shifting agencies addressed by adopting feminist technoscientific and new materialist lenses to grapple with the techno-industrial paradigm shift that has been (dis)figuring the anthropocentric condition. The first part of the paper lays down the qualities of this emerging ecology while recognizing the importance of human accountability and situatedness. The focal point of this survey is the anthropologist Lucy Suchman’s classic Human-Machine Reconfigurations which is elaborated upon through anchor points she posits revisiting Donna Haraway and Karan Barad’s arguments. The last part engages with the implications of such a coupling for human and machine sensoria in order to envisage the qualities of a distributive sensorium that this regenerative agency can put forth while alluding to practices of situated computing.

Keywords

Posthumanism; human-machine interaction; distributive agency; cyborg; regenerative boundaries; hybrid sensorium; situated computation.

DOI

10.69564/ISEA2023-12-short-Hedayati-Intelligent-Sensibility

If another techno-industrial paradigm shift has indeed creeped up on us so as to perfectly confuse the human-machine boundaries this time around; and that an ontological shift has occurred, as Rosi Braidotti claims, troubling the contact zones “between the organic and the inorganic, the born and the manufactured, flesh and metal, electronic circuits and organic nervous systems”; and that the carbon-based and the silicon-based as a result constantly imbricate, move, and flux inexorably, how can we imagine the agencies constructed around such a flow? What are the implications of such an entanglement for human-machine sensoria?¹

This co-evolutionary moment through which human and machine complex systems constantly affect and are affected by one another’s interrelation to create an unfolding terrain of imbricated becoming is referred to as technogenesis by N. Katherine Hayles. The implications of this adaptable and generative interrelation are profound: not only we, as humans invested in the holiness of humanism, have to grapple with constant adaptation of culturally coded networks of human life, but also with the psychobiological shifts that have occurred notably in rewiring of intricate neuronal activities of the human brain.²

Detached from the preformulated subject-object binary, the anthropologist Lucy Suchman’s classic Human-Machine Reconfigurations offers a profound angle on theorizing the conditions of this human-machine distributive agency. To delve into reconfiguration as an emergent ecology where agencies are constantly made, unmade and remade, at various points Suchman draws on the scholarship of Donna Haraway as well as Karen Barad, notably through the notions of figuration and intra-action.

Haraway’s Figuration:
A Situated Construct

Through the concept of figuration Haraway intends to foreground the tropic quality of material-semiotic practices in technoscience that hover in a space of literal-figurativeness. She envisages technologies as materialized figuration; that is assemblages that are both concerned with meaning making, a figural act, and physical and hence tangible existence.³

As a lens that particularly zooms in on human-machine interrelation, figuration is a critical framework that questions the formulation and configuration of technoscientific practices at every instance of occurrence. The goal is to sidestep fixed universalized paradigms of ‘doing science and technology’ and aim for specificity of local practices that humans actively shape rather than act as a passive observer within.

According to Suchman, the act of figuration is informed by specific socio-cultural constructs arising from site-specificity which can reinscribe or challenge the status quo and question the Euro-American imaginaries built on rationality of the autonomous subject.⁴

Barad’s Intra-Action:
Entities in the Making

Barad’s notion of intra-action is based upon their theory of agential realism as an onto-epistemology that challenges individualist paradigms and insists that intra-acting agencies are always already inseparable.⁵

While during an interaction two preformulated entities come together for an exchange, an intra-action underscores how the subjecthood and objecthood gets formed through the encounter. Barad specifically considers technoscientific practices to be a common site of intra-action where we should recognize the act of boundary making, objectification, and subjectification as contingent constructs. Barad’s vision is markedly in line with Haraway’s material-semiotic that considers material constructs and the meaning arising from them as co-constitutive. They too consider the reality of human-machine boundaries to be cut in particular ways that follow certain historicity with socio-political consequences. 

Suchman’s Reconfiguration

Drawing on the notion of figuration, configuration and intra-action, Suchman proposes reconfiguration as a creative exploration of human-machine boundaries where what she refers to as “the distributed and enacted character of agency” as a constantly-regenerative phenomenon should be taken into account.⁶

This view sits against the Western-dominant vision of subjects and objects as fixed entities brought together to interact and instead points towards a kind of performativity within the encounter where the agents are in continuous formation, reproduction and transformation; a perspective rooted in Actor Network theory (ANT) as a social theory based on relational ontology that puts humans and nonhumans alike as actants in an ever-evolving interrelated network where there are no preconceived positions taken but the positions are rather assumed through the process of interrelation. In this process the agencies are constantly worked through and negotiated to actively constitute ontologies based on what Michel Callon (2007), one of the proponents of ANT, calls “morphology of the relations” through which cyborgs, hybrids and quasi-objects are constructed and made visible.7(1)

Practices of technology-mediated medicine, including reproductive technoscience, as well as human-computer interaction are those that Suchman pays particular attention to as sites of human-machine mutually-constituted agencies. In this space of intra-action the disconcerting fact is that within practices of science and technology, the technical is formulated in the center while the social is either non-existent or pushed to the margins. Here, Suchman walks a tightrope of reconceptualizing the human in a way that the inseparability from the socio-technical substrate is pushed to the fore while recognizing the prominence of accountability but without assuming the dominion associated with ‘pure’ humanism that views technologies as translators and assemblers in service of humans. In other words, the question is how to draw a human-machine intra-action that retains human accountability without telling an essentialist story.

As a feminist construct, the figure of the cyborg, taken across its regenerative stance, can offer one avenue to explore this notion by radicalizing the human-machine, male-female, and subject-object boundaries, towards an emergent ecology where socio-materiality is constantly made, unmade and remade. From the Harawayan goddess-turned-cyborg to the elegant hero/ine and saviour as cultural imaginary and further as an everyday socio-materiality without a singular body, the figure of the cyborg is omnipresent across the socio-technical substrate.⁹

By overstepping the isolated shell that contains the human-machine hybrid, in Suchman’s reading, cyborg not only shatters the glamorized singular figure but “dissolves into a field of complex sociomaterial assemblages” to open up new ways of theorizing and practicing such an entanglement.¹⁰ Braidotti, takes this destabilization one step further to put forth the figure of the deglamorized everyday cyborg as “anonymous masses of the underpaid, digital proletariat who fuel the technology-driven global economy without ever accessing it themselves.”¹¹

Intrinsically, we can see how at every moment of instantiation cyborg cuts the human-machine boundary at a certain angle and not the other to constitute a shapeshifting intertwinement of the carbon-based and the silicon-based capable of subverting human-made socio-politico-cultural constructs. This fluid ecology is meant to transcend deeply etched preconceived notions of intelligent machines as human techno-extensions or the sensing and sensible human weary of the techno-dystopia, to instead reconceptualize an entanglement among networked agents, that constitute leaky, generative boundaries.

The stability of the human agency is thereby compromised as according to Suchman “the person figured here is not an autonomous, rational actor but an unfolding, shifting biography of culturally and materially specific experiences, relations, and possibilities inflected by each next encounter-including the most normative and familiar—in uniquely particular ways.”¹²

Within these spaces of constant transformation of the boundaries and redrawing the agencies, the question of accountability is by no means diffused: we need to recognize that we draw boundaries for meaning making and these boundaries are always charged with human-centered conceptions and misconceptions that have repercussions. This accountability is to recognize our position within animation and reanimation of situated encounters. As Barad states: “we are responsible for the world in which we live, not because it is an arbitrary construction of our choosing, but because it is sedimented out of particular practices that we have a role in shaping.”¹³


(Re)(Con)Figuring Hybrids of Sense-Making

During the act of boundary making the question that arises is how can we cut the boundaries in ways that give rise to hybrids of sense making where intelligibilities and sensibilities are constantly figured, configured and reconfigured? Reconfiguration in this sense can be read as a possibility to negotiate sensory modalities to locate sensing and effecting (2) not as autonomous qualities associated with the human or the machine but traceable within the process of intra-action. If technical practices foreground machine agency and yet human sensoria and sensibility cannot be reduced to compressed temporal flow of machinic computation, we need to reckon with contingent encounters that go beyond biological and technological determinism. Bio-sensibility and machine intelligence in this sense can be deconstructed and diffused to be reconfigured as dynamic fragments of a curious intelligent sensibility adjustable towards acting in contingent and context-specific situations. 

Such a hybrid, quasi-sensoria made in between the human and the machine offers a heterogeneous socio-materiality with qualities that are no longer inherent but always negotiated and in formation. This ontological in-betweenness tends to question universalized presumptions about technical practices centered around the aptitude of the machine intelligence to offer reliable, definitive, and objective responses to complex questions that are always rooted in specific situations.

In this sense as Haraway notes the answer does not lie in the dichotomous poles of positivism with its hallmark of scientific objectivity or relativism with its absolute unfixed orientation but in localized and embodied partial perspectives.¹⁴

Such a vision relies on the symbiosis of the technical and the sensible to form an interdependent intelligent sensibility that takes shape within the act of becoming to reckon with procedural, socio-political, and cultural dimensions of local conditions to compose and assess situated work-flows and responses.

(1) While Suchman praises the idea of “generalized symmetry” proposed by ANT, she subsequently proposed “dissymmetry” as a framework that recognizes the human-machine differences. Others including Bowker and Leigh Star, put forth a number of critiques pointed towards ANT’s networked interrelations. They consider the larger social construct to question the equalizing effect of such a framework toward human/nonhuman actants and the ethico-political repercussions of a world operating on this logic, pointing out that ANT "can be read as an uncritical celebration of the power of modern science and technology.” ⁸ Elsewhere, others such as Mel Chen, Zakiyyah Jackson, and Tiffany King, among others contend that posthumanist theories in general discount the human discrimination factors at play due to micro and material nature of such inquires.

(2) Sensing and effecting are mechanisms of interaction within biological and technical organisms; while sensors receive information from the environment to relay to the system, effectors act upon the world based on feedback loops that occur between the two. These notions are derived from cybernetics, a field of inquiry that studies the principles of communication and control within regulatory systems.

References

1 Rosi Braidotti, The Posthuman, Cambridge, UK, Malden, MA, USA, Polity Press, 2013, p.89.

2 N. Katherine Hayles, How We Think: Digital Media and Contemporary Technogenesis, Chicago, London, The University of Chicago Press, 2012.

3 Donna Jeanne Haraway, Mdest_Witness @Second_Millennium. Female-Man_Meets_OncoMouse: Feminism and Technoscience, Second edition, New York, NY, Routledge, Taylor & Francis Group, 2018.

4 Lucille Alice Suchman, Human-Machine Reconfigurations: Plans and Situated Actions, 2nd ed, Cambridge, New York, Cambridge University Press, 2007.

5 Karen Michelle Barad, “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter,” Signs: Journal of Women in Culture and Society 28, no. 3, March 2003, 801–31, https://doi.org/10.1086/345321.

6 Suchman, Human-Machine Reconfigurations, 260.

7 Michel Callon, “Actor-Network Theory, the Market Test,” in Technoscience: The Politics of Interventions, ed. Kristin Asdal, Brita Brenna, Ingunn Moser, Oslo?, Unipub, 2007.

8 Geoffrey C. Bowker, Susan Leigh Star, “How things (actor-net) work: Classification, magic and the ubiquity of standards,” 1996, https://www.ics.uci.edu/~gbowker/actnet.html.

9 Donna Jeanne Haraway, “A Manifesto for Cyborgs: Science, Technology, and Socialist Feminism in the 1980s,” in The Post-modern Turn: New Perspectives on Social Theory, ed. Steven Seidman, Cambridge, New York, Cambridge University Press, 1994.

10 Suchman, 283.

11 Braidotti, The Posthuman, 90.

12 Suchman, 281.

13 Karen Michelle Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning (Durham: Duke University Press, 2007), 105.

14 Donna Jeanne Haraway, “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective,” Feminist Studies 14, no. 3, 1988, 575, https://doi.org/10.2307/3178066. 

Bibliography

Marie-Luise Angerer, NONCONSCIOUS: On the Affective Synching of Mind and Machine, S.l.: MESON PRESS EG, 2022.

Karen Michelle, Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning, Durham, Duke University Press, 2007.

Karen Michelle Barad, “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter”, Signs: Journal of Women in Culture and Society 28, no. 3, March 2003, 801– 31, https://doi.org/10.1086/345321.

Geoffrey C Bowker, Leigh Star Susan, “How things (actor-net) work: Classification, magic and the ubiquity of standards,” 1996, https://www.ics.uci.edu/~gbowker/actnet.html

Rosi Braidotti, Posthuman Knowledge, Medford, MA, Polity, 2019.

Rosi Braidotti, The Posthuman, Cambridge, UK, Malden, MA, USA, Polity Press, 2013.

Michel Callon, “Actor-Network Theory, the Market Test.” In Technoscience: The Politics of Interventions, edited by Kristin Asdal, Brita Brenna, Ingunn Moser. Oslo? Unipub, 2007.

Mel Y Chen, Animacies: Biopolitics, Racial Mattering, and Queer Affect, Perverse Modernities, Durham, NC, Duke University Press, 2012.

Paul Dourish, Where the Action Is: The Foundations of Embodied Interaction, Cambridge, Mass, MIT Press, 2001.

Donna Jeanne Haraway, Modest_Witness@Second_Millennium. FemaleMan_Meets_OncoMouse: Feminism and Technoscience. Second edition. New York, NY, Routledge, Taylor & Francis Group, 2018. 

Donna Jeanne Haraway, “A Manifesto for Cyborgs: Science, Technology, and Socialist Feminism in the 1980s”, In The Post-modern Turn: New Perspectives on Social Theory, edited by Steven Seidman, Cambridge, New York: Cambridge University Press, 1994.

Donna Haraway, “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective", Feminist Studies 14, no. 3 1988, p.575, https://doi.org/10.2307/3178066.

N. Katherine Hayles, Unthought: The Power of the Cognitive Nonconscious, Chicago, London: The University of Chicago Press, 2017.

N. Katherine Hayles, How We Think: Digital Media and Contemporary Technogenesis, Chicago, London: The University of Chicago Press, 2012.

N. Katherine Hayles, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics, Chicago, Ill: University of Chicago Press, 1999.

Zakiyyah Iman Jackson, “Animal: New Directions in the Theorization of Race and Posthumanism”, Feminist Studies 39, no. 3, 2013, 669–85, https://doi.org/10.1353/fem.2013.0024.

Tiffany Lethabo King, “Humans Involved: Lurking in the Lines of Posthumanist Flight”, Critical Ethnic Studies 3, no. 1, 2017, 162–185, https://doi.org/10.5749/jcritethnstud.3.1.0162.

Bruno Latour, Reassembling the Social: An Introduction to Actor-Network-Theory, Clarendon Lectures in Management Studies, Oxford, New York, Oxford University Press, 2005.

Andrew Pickering, The Cybernetic Brain: Sketches of Another Future, Chicago, London, University of Chicago Press, 2010.

Lucille Alice Suchman, “Agencies in Technology Design: Feminist Reconfigurations*, ”In Machine Ethics and Robot Ethics, by Wendell Wallach, Peter Asaro, edited by Wendell Wallach and Peter Asaro, 1st ed. Routledge, 2020, 361–75, https://doi.org/10.4324/9781003074991-32.

Lucille Alice Suchman, Human-Machine Reconfigurations: Plans and Situated Actions, 2nd ed. Cambridge, New York, Cambridge University Press, 2007. 

DAOs A blockchain-based application not intervening, but strengthening the agility of contemporary arts

Victoria Hilsberg

vic.hilsberg@hotmail.de

Abstract

This paper discusses the blockchain-based application DAO (Decentralized Autonomous Organization) as a relevant new mechanism of artistic practice in the contemporary arts. It not only confronts the complexity of the modern society through its comprehensive application potential. It also channels segments of the contemporary art discourse such as political qualities, participatory art and cybernetic forces through active use cases. DAOs enable the interconnection of culture, technology and ecology. Using the most recent publications regarding blockchain technology as well as in-depth research on current artistic movements, it is possible to observe a complex manifestation of the matter within the contemporary arts during the concept's comparatively brief existence since its technical implementation around 2016.

Keywords

Contemporary art, transdisciplinary discourse, blockchain, DAO, Crypto Art, political art history, technology, participation, ecology, economics, Digital Art.

DOI

10.69564/ISEA2023-13-short-Hilsberg-DAOs

Introduction

DAOs, Decentralized Autonomous Organization, are blockchain-based applications. They represent a community or network of potentially anonymous individuals and/ or institutions, which are connected through a common factor. Each DAO is collectively owned through a network and does not have a central controlling authority. ¹Governance, coordination and actions take place within defined, self-governing rules, which are collectively decided upon and in its basics laid out in the White Paper, which is accepted by joining the DAO network. There is the possibility of implementing a DAO-own cryptocurrency, which can function as a treasury. One option to govern the organization is the buy-in: Through obtaining DAO-own tokens, a fractionalized ownership of the DAO is held. In theory, owning all tokens represents ownership of the entity of the DAO. There must not be this structure in order to provide a functioning DAO as there a several differing technological implementations already in place.²

Examples of DAOs would be exhibited 2022 alone in renowned exhibition spaces such as Kunstwerke Berlin or Documenta 15 in Kassel. Why would there be the interest to exhibit this technical development that is so close to Cryptocurrencies in their early stages of the Hype Cycle?³

Because the one challenge that contemporary art is confronted with is to develop new techniques and therefore a complexity which matches the one society itself is confronted with. DAOs are opening up different options.

One aspect making DAOs relevant to contemporary art is to define and demonstrate the importance of structure as a political element. DAOs provide a possibility of testing out agile, seemingly utopian systems and implementing them long term. One example is the Black Swan DAO, founded by Penny Rafferty, Laura Lotti, Calum Bowden and others. It caters goals, which were already formulated by art activism groups such as the Art Workers Coalition in New York in the 1960s. Their intention was to build a platform for the representation of all art sector associated employees, including artists, and demanded their equal treatment and co-determination. ⁴However, through the missing long-term structure it dissolved over time. The approach of Black Swan DAO towards equal treatment is built upon blockchain-specific protocol mechanisms working towards a trusted network within contemporary arts. One of their actions was to act as a platform for democratized cultural funding processes. Equal treatment symbolizes for Black Swan DAO also equal action rights. Pressing de-hierarchization, the DAO members decide through a democratic voting mechanism who receives the funding. The decision process does not focus on evaluating the individual background, but the quality of the project. Additionally, Black Swan DAO developed through its practice a consensus-building tool for collectives, called Cygnet. ⁵With consistent optimization, a DAO can be used to organize a complex ongoing structure on an ongoing basis, as in a company, or foundation-like groupings. The Blockchain expert Shermin V oshmgir compares the function of DAOs with the control and regulation system of state governance layers, on the one hand, and with that of private companies, on the other.

She does not connect its functionality to a recognized state. Voshmgir sees the automated DAO system with its high, democratized adaptation factor as a structural improvement over the rigid forms used to date. Her main argument here is that digitally secured compliance will radically end hierarchization. All the more so because a DAO can be adapted from within the structure to the respective needs of the parties involved. ⁶This would imply that Black Swan DAO could aim successfully to realize missing or not from their perspective successfully developed elements of the structural support system, regarding state operations and possibly particularly the support system of the arts.

Through their grassroots democratic ideal, DAOs do face anti-democratic conflicts, despite the complex technical implementation: Misinformation, bribery, plutocracy or media control. These are, among others, factors that can manipulatively influence the stakeholders of a DAO in their decision-making during the consensus process. Corresponding counter-mechanisms are being developed, and partial successes are already in sight. ⁷To develop critical awareness towards misinformation therefore needs to be addressed inside the structures of DAOs. As shown with the Black Swan DAO or as discussed by Claire Bishop in Artificial Hells outside of all blockchain intersections, DAOs represent a key element of contemporary arts only strengthened within the last years and especially since the Covid-19 pandemic: Participation and awareness of collective practice. ⁸An example is to be found also within the BeeDAO, presented 2022 at Documenta 15. BeeDAO is not only metaphorically oriented on the swarm organization of a beehive. It works towards the direct or indirect interest of the alive bees. A physical manifestation takes place through monitored bee hives as direct representatives of the species which is to be protected, producing public data. On physical assemblies, proposals towards the shared mission are discussed and decided upon. Using blockchain technology, this governance process cannot be manipulated through human agency, but provides a technological implemented liability. ⁹The phrase swarm highlights the participatory practice of institutions and organizations, referring also to the distinct participatory quality of Digital Art.

This also refers to the third element which highlights the quality of the DAOs. It can not only be seen as a representation of humanly structured systems. It also can actively develop the idea and styles of interconnection of planetary elements in anthropocentric times. Terra0 DAO was implemented in 2016 by Paul Seidler, Paul Kolling and Max Hampshire, the terra0 collective. The DAO represents a forest.

The physical existent piece of land is obtained through initial funding by DAO members. Surplus made from obtaining the forest, for example by selling the material wood, will be automatically used to buy out the share of the initial investors. The goal of this project is to develop as much surplus through automated economical processes enabled with mostly blockchain-based processes, that the forest will be able to buy out all human investors and govern itself through automated Smart Contracts. ¹⁰This artistic practice scales back to the cybernetic considerations of the symbioses of men and machine, as for example explained by Joseph C. R. Licklider, but puts the environmental aspect of this interaction into the forefront. ¹¹Therefore it develops this interaction further than before discussed, enabled through blockchain technology. It, moreover, reminds the however involved human parties of two elements, which reoccur in contemporary artistic practice: the power of the individual is perceptibly recognized. With the metaphorical proposition, that even the smallest amounts of capital can play a relevant part in the overall economy, an emancipatory practice takes place. This is underlined through the implementation of democratic mechanisms such as majority voting. Secondly, that the interconnection of technology, ecology and culture is not a futuristic matter, but is happening in the current times. Elements such as worker emancipation, trees or bees are rather positively connoted and it might not be a coincidence that for the first projects operated within the system DAO and contemporary art do use those accustomed elements in order to embrace engagement. Vitalik Buterin framed a reference system for organizational structures of DAOs in 2014. The technological implementation took place with The DAO two years later. ¹²Seeing the implementation now already in place in contemporary arts, there is more than a potential to be observed. Active use cases can be seen in DAOs as a method in contemporary artistic practice. All discussed DAOs still rely on the physical involvement of members in order to function. There however exists a structure which can overtime be strengthened and, in the process, develop the participatory aspect of the collective works. How this will develop is not certain at this point, as little as the future of blockchain is at this point determined. DAOs however already do confront contemporary conflicts with engaging solutions. 

References

1 Shermin Voshmgir, Token Economy: How Blockchains and Smart Contracts Revolutionize the Economy, Berlin, Blockchain-Hub, 2019, 104 et sqq.

2 Voshmgir, 2019, 119.

3 Vgl. Kathrin Passig, "Neue Technologien, alte Reflexe: Beitrag aus dem Sonderheft 2014 zum Thema Medienevolution," accessed November 27, 2022, https://www.medienkorre-spondenz.de/leitartikel/artikel/neue-technologien-altenbspreflexe.html.

4 Lucy R.Lippard, Sixyears: The dematerialization of the art object from 1966 to 1972, Berkeley, Univ. of California Press, 2007, p.IX.

5 Black Swan, “Cygnet Prototype,” in Radical Friends, Decentralised Autonomous Organisations and the Arts, ed, Ruth Catlow et al.,Torque Editions, 2022, 322-323.

6 Voshmgir, 2019, 104-135.

7 Voshmgir, 2019, 130 et sqq.

8 Claire Bishop, Artificial Hells: Participatory art and the politics of spectatorship, London, Verso, 2012.

9 "Bee DAO – the inaugural assemblee," Documenta 15, Website, accessed November 18, 2022, https://documenta-fifteen.de/kalender/beedao-the-inaugural-assemblee/. See also: https://beedao.zku-berlin.org/

10 Paul Seidler, Paul Kolling, Max Hempshire, “terra0. Can an Augmented Forest Own and Utilize Itself?,” in Artistic Re:Thinking of the Blockchain, ed. Ruth Catlow et al., Torque Editions, 2017, 68-72.

11 Joseph C. R. Licklider, "Man-Computer Symbiosis,“ IRE Transactions on Human Factors in Electronics, HFE-1, 1960, 4-11.

12 Vitalik Buterin, "DAOs, DACs, DAs and More: An Incomplete Terminology Guide,“ Ethereum Foundation Blog, accessed November 5, 2022, https://blog.ethereum.org/2014/05/06/daos-dacs-das-and-more-an-incomplete-terminology-guide

Bibliography

Claire Bishop, Artificial Hells: Participatory art and the politics of spectatorship, London, Verso, 2012.

Swan Black, “Cygnet Prototype”, In Radical Friends. Decentralised Autonomous Organisations and the Arts, edited by Catlow, Ruth et al., Torque Editions, 2022, 322-323.

Vitalik Buterin, "DAOs, DACs, DAs and More: An Incomplete Terminology Guide,“ Ethereum Foundation Blog, Last accessed, 05.11.2022, https://blog.ethereum.org/2014/05/06/daos-dacs-das-and-more-an-incomplete-terminology-guide.

Documenta fifteen, "BeeDAO – the inaugural assemblee,“ Last accessed 18.11.2022, https://documenta-fifteen.de/kalender/ beedao-the-inaugural-assemblee/.

Joseph C. R. Licklider, "Man-Computer Symbiosis,“ IRE Transactions on Human Factors in Electronics, HFE-1, 1960, 4–11.

Lucy R. Lippard, Six years: The dematerialization of the art object from 1966 to 1972, Berkeley, Univ. of California Press, 2007.

Kathrin Passig, "Neue Technologien, alte Reflexe: Beitrag aus dem Sonderheft 2014 zum Thema, Medienevolution,“ Last accessed 04.11.2022, https://www.medienkorrespondenz.de/leitartikel/artikel/neue-technologien-altenbspreflexe.html.

Paul Seidler, Paul Kolling, Max Hempshire, “terra0. Can an Augmented Forest Own and Utilize Itself?” In Artistic Re:Thinking of the Blockchain, edited by Catlow, Ruth et al., 322-323, Torque Editions, 2017, 63-72.

Shermin Voshmgir, Token Economy: How Blockchains and Smart Contracts Revolutionize the Economy, Berlin, BlockchainHub, 2019. 

Lucid Dream: Sensing and Artistic Representation of Plant-Nature Interaction Based on Plants Biosignals

Youyang Hu ‡, Chiaochi Chou *, Yasuaki Kakehi 

‡ The University of Tokyo * National Tsing Hua University Tokyo,
Japan Hsinchu, Taiwan
1huyouyang@xlab.iii.u-tokyo.ac.jp, 2chiaochi@gapp.nthu.edu.tw,
 3kakehi@iii.u-tokyo.ac.jp

Abstract

The recent progress of modern science has enabled us to detect the physiological status of organisms through their bioelectric activity; this technique has been constantly applied to contemporary media art. However, exploring plants as a subject and allowing them to interpolate and hence perform art through biosignals remains to be further explored. Lucid Dream is an artwork in which inspection, displacement, and engagement of plant communities take place, shedding light on the subjective perception of plants, a peripheral subject that rarely comes to people’s attention. To achieve this, we entered a forest and applied an artificial intelligence-based learning system capable of interpreting the local plants' responses to wind and rain stimuli via their biosignals. Subsequently, we established an environmentally controlled space within the art museum. Here, we simulated and artistically represented the natural elements as perceived by the plants, using artificial machines driven by the plant-nature interaction model. As viewers enter this space, they can experience the plant-perceived natural environment, gaining a non-human perspective through direct engagement with plant life. Lucid Dream not only leverages intelligent computational technologies to comprehend the perceptual system of plants but also fosters cross-species sensory experiences, enhancing our understanding and expanding our perspective on the natural environment.

Keywords

plant-nature interaction, biosignals, artificial intelligence, perception, natural environment

DOI

10.69564/ISEA2023-14-short-Hu-et-al-Lucid-Dream

Introduction

In recent years, as we close the gap between technology and biological systems, artworks utilizing living and semi-living organisms have become increasingly vigorous in contemporary media art. These artworks have presented us with another perspective to understand different life forms. With the era of pan-biological materials of bio art unfolding, artists and scientists who carried multiple identities began to displace the technique of biosignal detection in art creations. A characteristic of displacement, which is Recontextualization, has provided a propelling force for Lucid Dream is divided into the working phases of inspecting, displacing, and engaging with plant communities. We first developed a bio-amplifier to collect biosignals from multiple plants in a forest, we then the dialectic between artists acting due to their thought or their biological reactions to stimulus¹, ². Afterward, with the exploration method of using biosignals to construct a new logic for subjects or environments, people have further paid attention to other living organisms, especially algae and microbes³, ⁴, ⁵.

In these artworks, non-human lives are presented as lively, having their own “umwelts”⁶. However, to answer the upcoming question of whether living organisms have a more complex subjective activity and how to represent it to viewers with more direct and persuasive evidence, we believe that plants can be an answer. Due to their natural ability to perceive, adapt, and even evolve a particular method to respond to climate change, plants demonstrate high sensitivity and easily observed feedback to the environment⁷. As a result, it is of great potential for us to utilize intelligent computational technologies to create systems that interface with the "umwelts" of plants in nature in such a way as to extend our understanding and broaden our horizons regarding the natural environment.

1. Environmental-controlled space in the art museum where natural elements perceived by the plants are simulated and artistically represented. © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi

 Lucid Dream is divided into the working phases of inspecting, displacing, and engaging with plant communities. We first developed a bio-amplifier to collect biosignals from multiple plants in a forest, we then applied an artificial intelligence-based learning system that can interpret the plants' perception of the wind and rain stimuli through their biosignals. The natural environment sensed by plants is displaced to the environmental-controlled space in another realm to be artistically represented. After being “Recontextualized,” the environmental data originally subject to scientific methods have been transformed and enabled a new meaning. When viewers walk into this space, they can experience the natural environment that plants perceive through their engagement with plants’ life from a non-human perspective. Extending from art to science, breaking the existing framework of scientific research, and connecting trans-species sensory experiences to provide a new possible way to respond to the question of representing the subjective activity of living organisms to viewers.

Figure 2. Bio-amplifier design. © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi

Biosignals Detection and Analysis

In this project, we focus on the Mimosa in the forest, known for its high sensitivity to environmental stimuli and commonly used in bioelectric signaling research⁸. We developed a miniature bio-amplifier to detect its biosignals in response to environmental stimuli. It detects the difference in electric potential between the substrate and the leaf petiole. As shown in Figure 2, it consists of two integrated amplifiers (TLV27L2 and ADS1292). Due to the uncertainty of plant resistance value and the electromagnetic interference caused by other equipment, most of the plant signaling tasks are carried out in the Faraday cage. We overcame this constraint by using two high input impedance operational amplifiers (TLV27L2) to get less interference and enable detecting the biosignals from plants in the natural environment. To reduce the size and power consumption of the circuit, we utilize the ADS1292 which integrated high-resolution analog-to-digital converter (ADC) with a built-in programmable gain amplifier. We used an STM32 microprocessor to pack the raw data and an ESP32 to transmit the data to the computer wirelessly based on TCP protocol. On the computer side, there is a program developed in openFrameworks that receives and processes the biosignals. 

3. System workflow. © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi
4. Support vector machine configuration. © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi
5. Support vector regression configuration. © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi

We next applied two supervised learning algorithms to conduct a time-domain analysis of the biosignals to get the Mimosa's status under different environmental stimuli. Figure 3 shows the workflow of this system, A support vector machine(SVM) classifier was applied to discriminate between wind versus rain-stimulated biosignal activity. A support vector regression(SVR) model was then used to predict the wind intensity sensed by the plant. We first reduced the noise of raw signals by a low-pass filter (second-order Butterworth filter) and then collected the biosignals as 800 dimension vectors detected under two environmental stimuli. As shown in Figure 4, for each stimulus, 50 samples were collected and they were labeled "Wind stimulus" and "Rain stimulus." We also collected 50 samples that the plant was under no stimulus. We then trained the SVM model that can classify the status of plants under two environmental stimuli. We also collected the biosignals under three intensities of wind stimulus. As shown in Figure 5, for each stimulus, 20 samples were collected and they were labeled with integer values of 10, 127, and 255. We then trained the SVR model that can predict the wind intensity as an integer value between 0 and 255 that the plants perceived in response to wind stimuli with different intensities.

In this project, we have developed a plant-based system for environmental state sensing. Despite its inherent biological variability, which can result in less precision compared to silicon-based sensors, this system holds significant potential for future bio-hybrid sensing technology. Additionally, we are exploring its creative applications in the realm of artistic practice, aiming to provide audiences with a novel perspective on their interaction with the natural environment. 

6. The system configuration of Lucid Dream © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi

System Implementation and Artistic Representation

Based on our research in biosignals detection and analysis, we created an artwork that artistically represents the natural environment perceived by plants in an artificial space. As shown in Figure 6, the artwork consists of a plant observatory in the forest and an environmental-controlled space in the art museum. There are four Mimosas equipped with bio-amplifiers in the plant observatory. The perceived state of each Mimosa in response to environmental wind and rain stimuli will be analyzed by our artificial intelligence-based learning system and remotely simulated in the environment-controlled space. The space is equipped with four sets of air-blowing systems, each with three fans driven by the SVR models. The input is the biosignals of four Mimosas. The intensity of the wind perceived by each Mimosa corresponds to the number of active fans in each air-blowing system. A rain machine is installed in this space, which synchronizes the rain stimuli perceived by Mimosas based on the SVM model. It consists of a water pipe running along the wall to the ceiling and a pumping motor. When the system detects that Mimosas senses rain, it will trigger the rain machine to simulate the outdoor rain environment in this space.

Figure 5 shows the exhibition site in the art museum, with a monitor real-time streaming the environmental conditions in the forest and the biological state of the four mimosas. The natural environment sensed by plants is displaced to the environmental-controlled space in another realm to be artistically represented. After being “Recontextualized,” the environmental data originally subject to scientific methods have been transformed and enabled a new meaning. When viewers walk into this space, they can experience the natural environment that plants perceive through their engagement with plants’ life from a non-human perspective.

7. The exhibition site of Lucid Dream. © Youyang Hu, Chiaochi Chou, Yasuaki Kakehi

Conclusion

This paper introduces the artwork Lucid Dream, which uses plants as perceptual subjects to gain insight into and artistically portray natural elements to viewers. To accomplish this, we have devised a bio-amplifier for detecting plant biosignals in response to environmental stimuli. Subsequently, within a forest environment, we have employed an artificial intelligence-based learning system capable of interpreting local plants' responses to wind and rain stimuli through their biosignals. Additionally, we have established an environmentally controlled setting to simulate and artistically represent the natural environmental elements as perceived by these local plants to viewers. Lucid Dream not only delves into the utilization of advanced computational technologies for understanding plant perceptual systems but also endeavors to create cross-species sensory experiences, expanding our comprehension and perspective on the natural environment.

References

1 Alvin Lucier, Simon Douglas, Chambers, Scoresby Alvin Lucier, Wesleyan University Press, 2012.

2 David Rosenboom, "Method for producing sounds or light flashes with alpha brain waves for artistic purposes," Leonardo, 1972, 141-145.

3 Carlos Castellanos, "IntersectionsofLivingandMachine Agencies: Possibilities for Creative AI," The Language of Creative AI, Springer, Cham, 2022, 155-166.

4 Sabina Hyoju Ahn, "Sonic transformation with living matter," In proceedings of ISEA symposium, Inter-Society for the Electronic Arts, 2019.

5 You-Yang Hu, Chou Chiao-Chi, Li Chia-Wei, "Apercevoir: Bio Internet of Things Interactive System," Proceedings of the 29th ACM International Conference on Multimedia, 2021.

6 Jakob von Uexküll, "An introduction to Umwelt," 2001, 107-110.

7 Jagadis Chandra Bose, The nervous mechanism of plants, Longmans Green, 1926.

8 Alexander G. Volkov, et al. "Mimosa pudica: electrical and mechanical stimulation of plant movements." Plant, cell & environment 33.2, 2010, p.163-173. 

Sounding Softness and the (Artificial) Subject

Jonas Jørgensen, Mads Bering Christiansen

SDU Soft Robotics, SDU Biorobotics, The Mærsk Mc-Kinney Møller Institute, University of Southern Denmark (SDU) Odense, Denmark
jonj, mabc@mmmi.sdu.dk

Abstract

The paper discusses the authors’ artwork SONŌ, its artistic motivations, the artistic research practice underlying its development, and its technical realization. SONŌ is a soft robotics installation that interrogates the interconnections of soft materiality, sound, and subjectivity. It features a sessile soft artificial entity capable of expansive movement, which is ceaselessly sounding itself and various environments using real-time generated audio. 

Keywords

Soft robotics, robotic art, soft robot aesthetics, sound art, sound studies, materiality.

DOI

10.69564/ISEA2023-15-short-Jorgensen-et-al-Sounding-Softness

Introduction

SONŌ is an artwork featuring a soft pneumatically actuated robot manufactured from silicone. The robot possesses procedurally generated movement and sonification of movements accompanied by a soundscape.

Human and nonhuman animals make utterances that are socially communicative and function to enact a subject position or form connections with other agents inhabiting the environment. The production of sound, whether intentional or unintentional, is arguably a basic existential feat of all living organisms. As empirical phenomena, however, sound by far predates life. In fact, the Universe emerged from what is arguably the ultimate sonic event (which, paradoxically, no one was around to hear)—the Big Bang, cosmology tells us. Sound always originates from a source yet is simultaneously transversal and expansive in character and by nature destined to permeate its surroundings. It is a mediatic phenomenon par excellence—on the material level, sound appears intangible and perhaps as almost nothing in itself, it only exists parasitic to matter, manifesting as perturbations and pressure changes travelling in a physical medium. Sound is characterized by a double movement—it is expansive and enveloping, yet simultaneously local and ephemeral. It is always subject to dampening and seems to evaporate into thin air on the microscopic level, when its waves are converted into heat through friction between the molecules of its medium.

Within the Western tradition of logocentric thinking, one of the ways in which sound comes to matter, is through the privileging of speech over writing.¹ Speech is the primary medium of human thinking and writing is merely a secondary technology. Hence, sound is positioned as the unbridled carrier of being and subjectivity—“I sound, therefore I am.” Poststructuralists and their new materialist progeny, however, champion a different position, that foregoes fixity, in favor of flux and the perpetual process of becoming, which is perhaps more adequate to the ontology of sound itself. Here, the subject is considered dynamic and decentered, and the boundaries between self and world permeable. Furthermore, agency is no longer predicated upon subjectivity nor inherent to the subject itself, but a relational dynamism of forces enveloping things as well as environments.²

Motivation and Practice

SONŌ (Latin: “[I make] sound”) explores a nexus of sound, soft robotics, and subjectivity. Through the artwork and its associated practices, we seek to articulate and enact a myriad of complex interactions between these phenomena and their aesthetic and epistemological capture.

Figure 1. SONŌ (2019-2022) (detail), soft robotics installation with 4 ch. sound, variable dimensions (room size). © Mads Bering Christiansen & Jonas Jørgensen. Photo: ZHU Lei.

SONŌ addresses what we take to be basic questions of robotic art in general, including, what does it take to alter or blur the ontological status of an object towards that of a subject, by means of movement and sound? As soft robotics (robotic morphologies and components constructed from pliable and elastic materials³) is a key interest in our practice⁴-¹², within the work we were also keen to query connections between soft materiality and sound through robotics as an aesthetic medium. For instance, different kinds of matter are capable of producing impact sounds (via resonance) with specific characteristics in terms of envelopes and frequency spectra¹³. Moreover, materials interact with sound in different ways, e.g., soft materials tend to dampen sound whereas hard materials reflect it. 

Figure 2. SONŌ (2019-2022) (detail), soft robotics installation with 4 ch. sound, variable dimensions (room size). © Mads Bering Christiansen & Jonas Jørgensen. Photo: ZHU Lei.

SONŌ is influenced by the notion of sound as a naturecultural¹⁴ phenomenon. It seeks to consider divergent aspects of the material-semiotic conditions of possibility through which sound and robotic movement can attain agency within specific environments. The physical and physiological properties of softness and “soft sound” have thus fed into the work, but equally cultural meanings, e.g., notions about sounds made by fictional soft characters from popular culture and cultural associations of softness as aligned with, e.g., precariousness and vulnerability. From the outset, we were thus interested to probe the chimeric character and synesthetic aspects of the concept of “softness”, when used to describe sound and materials respectively. Definitions and delineations of “soft sound” within sound studies, psychoacoustics, and musical theory, were, for instance, drawn upon in our explorations of what might constitute “soft sound” and of the effects of adding “soft” or “hard” sound to a soft morphology. 

Part of the artistic research has been conducted in dialogue with the research field of human-robot interaction (HRI), wherein sound has recently become subject of increasing interest. Our practice sought to be receptive of pressing ideas and questions from this research field and consider how they might gain relevance and be addressed through artistic forms. A body of work within the HRI field has interrogated how various types of sounds can affect people’s perception and interaction with robots and found non-verbal audio to be a salient feature with use potential as a deliberate design aspect of, e.g., social robots. In certain situations and use cases, nonverbal audio is also preferable over synthetic voices, to guide or facilitate interactions with humans.¹⁵ SONŌ adds to this research on robot sound, by exploring how sound and embodiment can interact in soft social robots of unconventional nonanthropomorphic and nonzoomorphic designs, which behaviors that should be accompanied with sound, and what the function of sound might be within these. 

The SONŌ Installation

The soft robot morphology was designed to appear organic yet unfamiliar (see Figs. 1-2). Abstract rounded shapes and a hue with similarities to Caucasian human skin, or pig skin, with reddish spotted pigmentations were used. The morphology possesses three independent pneumatic channels. Each of these interconnect four chambers interspersed across it, which can expand when inflated. Ecoflex 00-30 silicone colored with Silc-Pig pigments was used to cast the robot in a 3D printed mold (the robot’s design and fabrication is described in more detail in ¹⁶).

In prior work we have discussed the artistic strategies used to compose the robot’s main sound design (inspired by the sounds made by fictional soft characters in movies).¹⁷, ¹⁸ We have also presented results of an empirical study exploring the effects of different sound designs on people’s perceptions of the robot’s sociality and its interaction affordances.¹⁶ Following these outcomes, work on presenting the project in the form of an art installation ensued.(1)

Physically, the SONŌ installation (Fig. 3) consists of:
1. The sonified soft robot displayed on a black plinth

(dimensions 112 x 40 x 40 cm.)
2. A set of external speakers mounted in the room

The plinth features a door that can be opened to operate the robot during exhibition and houses the following on three shelves (see Fig. 4): an active loudspeaker, an electro-pneu-matic actuation system (microcontroller, motor shield, pumps, valves etc.) and an audio interface, a laptop PC running a software synthesizer. Along all four edges of the plate holding the robot morphology, a small opening is present, to allow sound from the loudspeaker inside the plinth to be transmitted to the exhibition space (Fig. 2). The audio of the installation consists of two times 2-channel stereo comprising: 1. robot sound – played over the loudspeaker inside the plinth, 2. a soundscape – played over the external loudspeakers in the room.

Figure 3. SONŌ. Installation views at Chronus Art Center (2022). © Mads Bering Christiansen & Jonas Jørgensen. Photo: ZHU Lei. 
Figure 4. CAD rendering of the plinth showing the equipment inside (shown here with the operation door opened). Illustration: Cao Danh Do. © Cao Danh Do, Mads Bering Christiansen & Jonas Jørgensen.

The Technical System

A diagram of the technical system is shown in Fig. 5 with the signal paths indicated. An Arduino Uno, which controls the robot’s movement by activating pumps and valves, functions as the master with a laptop PC generating the audio running as slave (for details see ¹⁶–¹⁸). A signal to generate matching robot sound using the FM software synthesizer is sent when a movement phrase is triggered. The robot does not currently have any sensors.

Figure 5. Schematic overview of the technical system and signal paths. Outlined boxes denote physical system components, boxes devoid of outlines are software components. Illustration: Mads Bering Christiansen & Jonas Jørgensen. © Mads Bering Christiansen & Jonas Jørgensen.

Robot Behavior and Sound Programming

For the final installation, we built upon existing code already developed for movement and sound generation. We chose a phrase-based and a categorical approach to designing the robot’s behavior. By “phrase” we refer to sequences of robot movement and matching sound of a duration up to 30s. The robot operates as a finite-state machine (FSM) with four mood states (categories). These each correspond to different levels of arousal (relaxed, medium-relaxed, medium-aroused, aroused). In each mood, the robot generatively combines a specific set of phrases and pauses matching this mood. Each of the phrases were hand coded and iterated upon for expressivity (through trial and error) and subsequently matched to one of the four mood states. A total of 13 phrases were used as building blocks that are combined in different ways to generate the robot’s movement and sound behaviors. All mood states feature a breath-like phrase with the robot performing asynchronous periodic inflation across the chambers, that use increasing frequencies for increased arousal, in accordance with findings of our prior work.¹⁹ In addition to the robot sound, the installation features twelve composed soundscapes that are played parallelly through Ableton Live. These are also triggered by the microcontroller, but asynchronously with the robot’s movements and sound. The soundscapes consist of processed synthesized and recorded sounds and select sonic textures combined into ethereal sonic expressions devoid of temporal structure and timing. The audio was kept spacious and wide to let the robot’s more erratic utterances come into focus and only add a more subtle affective coloring of these. In accordance with this, the soundscapes are played over loudspeakers physically separate from the robot’s plinth. The soundscapes mix slow extensive sounds to create the aural impression of an atmosphere of the installation and to position the robot (and its visitors) in different sonic worlds of various affective intensities. Played in random succession, the soundscapes complement or clash with each other, and contribute a sense of emergent narrative.

Further work

We plan to expand upon SONŌ in an updated version of the installation and in subsequent independent works. As a next step, we would like to develop a means to have more interactive generation of the robot and soundscape audio. Currently, the robot switches between its four mood states pseudo-randomly with the statistical likelihood that the robot will switch its arousal state up or down after a completed phrase cycle as an adjustable parameter. We hope to add sensors to the installation, e.g., room scale computer vison, to track activity, behaviors, and affective states of visitors to enable the robot to interact. Furthermore, we are considering developing and validating a more fine-grained phrase-based or parametric generation of affective movement and sound with the system that can contribute more variation and nuance to the robot’s expressions.

 

1 A supporting video showcasing excerpts of the SONŌ robot performing with the robot sound and soundscape is available at: https://youtu.be/U0fGXCbcygU

 

Acknowledgements

We thank Cao Danh Do for help with 3D prints, mechanical design, construction, and illustrations.

References

1 J. Williams, Understanding Poststructuralism, First edition, Chesham, Bucks, Routledge, 2005.

2 K. Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning, Durham, NC, Duke University Press, 2007.

3 D. Rus, M. T. Tolley, “Design, fabrication and control of soft robots,” Nature, vol. 521, no. 7553, May 2015, 467–475, doi: 10.1038/nature14543.

4 M. B. Christiansen, L. Beloff, J. Jørgensen, A.-S. E. Belling, “Soft Robotics and Posthuman Entities,” Journal for Artistic Research, no. 22, Dec. 2020, doi: 10.22501/jar.549014.

5 J. Jørgensen, “Leveraging Morphological Computation for Expressive Movement Generation in a Soft Robotic Artwork,” in Proceedings of the 4th International Conference on Movement Computing, in MOCO ’17, New York, NY, USA, ACM, 2017, 20:1-20:4, doi: 10.1145/3077981.3078029.

6 J. Jørgensen, “Prolegomena for a Transdisciplinary Investigation into the Materialities of Soft Systems,” in ISEA 2017 Manizales: Bio-Creation and Peace: Proceedings of the 23rd International Symposium on Electronic Art, University of Caldas, Manizales, Colombia, Department of Visual Design, Universidad de Caldas, and ISEA International, 2017, 53–160.

7 J. Jørgensen, “Interaction with Soft Robotic Tentacles,” in Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction - HRI ’18, Chicago, IL, USA: ACM Press, 2018, 38–38, doi: 10.1145/3173386.3177838.

8 J. Jørgensen, “Constructing Soft Robot Aesthetics: Art, Sensation, and Materiality in Practice,” Ph.D. thesis, IT University of Copenhagen, Copenhagen, 2019.

9 J. Jørgensen, “From Soft Sculpture to Soft Robotics: Retracing Entropic Aesthetics of the Life-like,” in Shifting Interfaces: An Anthology of Presence, Empathy, and Agency in 21st-Century Media, H. Aldouby, Ed., Leuven, Belgium: Leuven University Press, 2020, 223–242.

10 J. Jørgensen, S. Ploetz, “LARPing Human-Robot Interaction,” in HRI 2020 Workshop on Exploring Creative Content in Social Robotics, Apr. 2020, Accessed: Jun. 19, 2020. [Online]. Available: https://portal.findresearcher.sdu.dk/da/publications/larping-human-robot-interaction

11 J. Jørgensen, “TeMoG – An Accessible Tool for Creating Custom Soft Robotics Parts,” in Interactivity and Game Creation, A. Brooks, E. I. Brooks, and D. Jonathan, Eds., in Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Cham: Springer International Publishing, 2021, 331–342, doi: 10.1007/978-3-030-73426-8_20.

12 J. Jørgensen, “Towards a Soft Science of Soft Robots. A Call for a Place for Aesthetics in Soft Robotics Research,” J. Hum.-Robot Interact., vol. 12, no. 2, Mar. 2023, 15:1-15:11, doi: 10.1145/3533681.

13 W. Fujisaki, N. Goda, I. Motoyoshi, H. Komatsu, S. Nishida, “Audiovisual integration in the human perception of materials,” Journal of Vision, vol. 14, no. 4, Apr. 2014, 12–12, doi: 10.1167/14.4.12.

14 D. Haraway, “Situated knowledges: The science question in feminism and the privilege of partial perspective,” Feminist studies, vol. 14, no. 3, 1988, 575–599.

15 F. A. Robinson, O. Bown, M. Velonaki, “Designing Sound for Social Robots: Candidate Design Principles,” Int J of Soc Robotics, vol. 14, no. 6, Aug. 2022, 1507–1525, doi: 10.1007/s12369-022-00891-0.

16 J. Jørgensen, M. B. Christiansen, “The Sounds of Softness. Designing Sound for Human-Soft Robot Interaction,” Frontiers in Robotics and AI, vol. 8, 2021, 1–17, doi: 10.3389/frobt.2021.674121.

17 M. B. Christiansen, J. Jørgensen, “Augmenting Soft Robotics with Sound,” in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, in HRI ’20, New York, NY, USA, Association for Computing Machinery, Mar. 2020, 133–135, doi: 10.1145/3371382.3378328.

18 M. B. Christiansen, J. Jørgensen, “SONŌ,” in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, in HRI ’20. New York, NY, USA, Association for Computing Machinery, 639, Mar. 2020, doi: 10.1145/3371382.3378399.

19 T. A. Klausen, U. Farhadi, E. Vlachos, J. Jørgensen, “Signalling Emotions with a Breathing Soft Robot,” in 2022 IEEE 5th International Conference on Soft Robotics (RoboSoft), Apr. 2022, 194–200. doi: 10.1109/RoboSoft54090.2022.9762140. 

Bibliography
  1. Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning, Durham, NC, Duke University Press, 2007.
    M. B. Christiansen, L. Beloff, J. Jørgensen, A.-S. E. Belling, “Soft Robotics and Posthuman Entities,” Journal for Artistic Research, no. 22, Dec. 2020, doi: 10.22501/jar.549014.
  2. B. Christiansen, J. Jørgensen, “Augmenting Soft Robotics with Sound,” in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, in HRI ’20, New York, NY, USA, Association for Computing Machinery, Mar. 2020, 133–135, doi: 10.1145/3371382.3378328.
  3. B. Christiansen, J. Jørgensen, “SONŌ,” in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, in HRI ’20. New York, NY, USA, Association for Computing Machinery, Mar. 2020, 639, doi: 10.1145/3371382.3378399.
  4. Fujisaki, N. Goda, I. Motoyoshi, H. Komatsu, S. Nishida, “Audiovisual integration in the human perception of materials,” Journal of Vision, vol. 14, no. 4, Apr. 2014, 12–12, doi: 10.1167/14.4.12.
  5. Haraway, “Situated knowledges: The science question in feminism and the privilege of partial perspective,” Feminist studies, vol. 14, no. 3, , 1988, 575–599.
    J. Jørgensen, “Leveraging Morphological Computation for Expressive Movement Generation in a Soft Robotic Artwork,” in Proceedings of the 4th International Conference on Movement Computing, in MOCO ’17. New York, NY, USA: ACM, 2017, 20:1-20:4. doi: 10.1145/3077981.3078029.
  6. Jørgensen, “Prolegomena for a Transdisciplinary Investigation into the Materialities of Soft Systems,” in ISEA 2017 Manizales: Bio-Creation and Peace: Proceedings of the 23rd International Symposium on Electronic Art, University of Caldas, Manizales, Colombia: Department of Visual De- sign, Universidad de Caldas, and ISEA International, 2017, 153–160.
  7. J. Jørgensen, “Interaction with Soft Robotic Tentacles,” in Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction - HRI ’18, Chicago, IL, USA: ACM Press, 2018, 38–38, doi: 10.1145/3173386.3177838.
    J. Jørgensen, “Constructing Soft Robot Aesthetics: Art, Sensation, and Materiality in Practice,” Ph.D. thesis, IT University of Copenhagen, Copenhagen, 2019.
    J. Jørgensen, “From Soft Sculpture to Soft Robotics: Retracing Entropic Aesthetics of the Life-like,” in Shifting Interfaces: An Anthology of Presence, Empathy, and Agency in 21st-Century Media, H. Aldouby, Ed., Leuven, Belgium: Leuven University Press, 2020, 223–242.
    J. Jørgensen, S. Ploetz, “LARPing Human-Robot Interaction,” in HRI 2020 Workshop on Exploring Creative Content in Social Robotics, Apr. 2020, Accessed: Jun. 19, 2020. [Online]. Available: https://portal.findresearcher.sdu.dk/da/publications/larping-human-robot-interaction
  1. Jørgensen, “TeMoG – An Accessible Tool for Creating Custom Soft Robotics Parts,” in Interactivity and Game Creation,
    A. Brooks, E. I. Brooks, and D. Jonathan, Eds., in Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Cham: Springer International Publishing, 2021, 331–342, doi: 10.1007/978-3-030-73426-8_20.
  2. Jørgensen, M. B. Christiansen, “The Sounds of Softness. Designing Sound for Human-Soft Robot Interaction,” Frontiers in Robotics and AI, vol. 8, 1–17, 2021, doi: 10.3389/frobt.2021.674121.
  3. Jørgensen, “Towards a Soft Science of Soft Robots. A Call for a Place for Aesthetics in Soft Robotics Research,” J. Hum.- Robot Interact., vol. 12, no. 2, Mar. 2023, 15:1-15:11, doi: 10.1145/3533681.
  4. A. Klausen, U. Farhadi, E. Vlachos, J. Jørgensen, “Signalling Emotions with a Breathing Soft Robot,” in 2022 IEEE 5th International Conference on Soft Robotics (RoboSoft), Apr. 2022, 194–200, doi: 10.1109/RoboSoft54090.2022.9762140.
  5. A. Robinson, O. Bown, M. Velonaki, “Designing Sound for Social Robots: Candidate Design Principles,” Int J of Soc Robotics, vol. 14, no. 6, 1507–1525, Aug. 2022, doi: 10.1007/s12369-022-00891-0.
  6. Rus, M. T. Tolley, “Design, fabrication and control of soft robots,” Nature, vol. 521, no. 7553, 467–475, May 2015, doi: 10.1038/nature14543.
    J. Williams, Understanding Poststructuralism, First edition, Chesham, Bucks: Routledge, 2005.
Author Biographies

Jonas Jørgensen is a researcher and artist based in Copenhagen focusing on practice-based transdisciplinary work at the nexus of technical science, media art aesthetics, and the posthumanities. He holds a position as Associate Professor at the Biorobotics section of the University of Southern Denmark where he is co-director of the Center for Soft Robotics. Webpage: https://jonasjoergensen.org/

Mads Bering Christiansen is a Copenhagen-based designer/artist/researcher whose practice evolves around designing objects and experiences to explore and speculate about the potential futures emerging from human technological entanglement. He is currently PhD student at the Center for Soft Robotics at the University of Southern Denmark with a project that investigates the effects of integrating biomorphic aesthetics in robots for social human-robot interactions. Webpage: https://madsbering.com/ 

Gesture and Spatiality in Electroacoustic Improvisation with Digital Video

Michael Lukaszuk, Sojung Bahng

Queen’s University DAN School of Drama and Music Kingston, Canada michael.paul.lukaszuk@gmail.com, sojung.bahng@queensu.ca

Abstract

This paper discusses a multidisciplinary approach to improvisation in which electroacoustic performance is informed by its capacity to transform and respond to characteristics of digital video, along with visual and architectural features of virtual spaces. We describe technical and creative strategies that are focused on leveraging mobile devices in a performance setting for navigating audio-visual relationships in which digital spaces are conveyed through video playback and processing, while gestural control highlights actual physical spaces (e.g., gallery spaces). This creates a hybrid venue that benefits live electroacoustic creation with digital artists whose work extends beyond the domain of sound. The presentation visual analogues to sonic transformations of gesture and spatiality are used to highlight aspects of electroacoustic performance that develop out of a gradual exploration between emulation and obfuscation of referential sound objects. Mappings between electroacoustic instrument and digital video also help delineate the creative use of mobile devices and controllers as being more than tools. At the same time, the transmission of MIDI and OSC (Open Sound Control) messages from mobile instruments allows collaborating artists to resource electroacoustic sound in non-aural contexts.

Keywords

Electroacoustic, Sound and Music Computing, Spatialization, Audio-Visual Improvisation, Digital Instruments.

DOI

10.69564/ISEA2023-16-short-Lukaszuk-et-al-Gesture-and-Spatiality

Introduction

Digital lutherie has been an effective thread for connecting electroacoustic musicians with video and extended reality artists. The data stemming from a digital translation of physical gesture to affect sound diffusion can also be used to convey intersecting spatial relationships for artists who explore proximity and physicality in a visual context. The motivation for this project is to explore the mobile phone as an instrument for multidisciplinary improvisation that involves electroacoustic music. Beyond establishing the mobile phone as a convenient resource for blending artistic or scholarly boundaries, there is a need to consider the organological implications of routing the device through software environments for sound and visual media. This depends on making decisions about what kinds of mappings and idiomatic behaviours help assert a sense of instrumentality, so that a “smartphone-as-instrument” setup conveys compositional drivers that are germane to electroacoustic creation—such as the spectrum between sounds being presented with a relationship to a physical source vs. belonging to a more acoustmatic context. In his article “The Instrumentality of Music,” musicologist Philip Alperson uses the example of Bob Dylan’s infamous performance with electric guitar at the 1965 Newport Folk Festival to emphasize how instruments help us understand the taxonomy of musical genres and how instrumentality, as a concept, goes beyond the notion of a music being “of instruments” to show how instruments place music in relation to larger cultural trends.¹ Focusing on the mobile phone as a collaborative digital instrument helps disrupt notions of traditional musical performance practice that render electroacoustic as non-performative. We will discuss electroacoustic improvisation with digital video in relation to the piece Suffocation (2022), in which gestural control of a mobile device within a multichannel audio setup facilitates continuous reframing of sourced objects within 360-degree video.

Related Work

In the 21st century, trends such as the proliferation of laptop orchestras and the emergence of live coding help establish a context in which seemingly commonplace technologies can be understood as musical instruments through their use within simulated spaces. For example, in composer Mara Helmuth’s laptop ensemble work, pieces such as from Uganda...(2013) use a simple graphical user interface in Max/MSP as an instrument for constructing imaginary sonic environments based on soundscape recordings.² In addition to environmental references, performers interact with the laptop as an electroacoustic instrument by routing their individual playback through a multichannel audio setup used within a concert hall.

Interactive musical mobile apps such as Brian Eno and Peter Chilver’s Bloom (2008) use synthetic or at least ambiguous sampled sounds to create a kind of acousmatic sound world in which there is intentional ambiguity with the manner in which the source material is presented. Bloom also bridges mobile phone performance in relation to digital spaces through a 2018 partnership with Microsoft to create a mixed reality installation in Amsterdam’s Transformatorhuis arts space in which the user interface was projected on larger touch-sensitive surfaces.³ Conversely, Chilvers’ other musical app projects such as Air (2011) relies on the reconfiguration of Irish singer Sandra O’Neal’s recorded vocal performance through gestural interaction with the mobile device. In her PhD Dissertation, Vanessa Chang writes that “through gesture, contemporary digital art practices borrow and recode forms of presences...to highlight a more diverse array of creative agencies.”⁴ A “vocal performance” with “Air” abandons the idea of the voice as a fixed instrument dependent on breath control and becomes an expression of a cybernetic virtuosity that depends on feedback between physical and simulated bodies. 

Suffocation

Suffocation was developed during the fall of 2022, as an improvisational piece stemming from the authors’ involvement in a mixed-media ensemble, and a desire to explore spatiality using an approach that blended our respective practices. For the video content used in this piece, a 360-degree camera was placed inside a small container containing an assortment of tiny objects, such as pins, tissues, pills, and leaves, and poured or added liquids. The everyday objects and materials are defamiliarized by creating the illusion of scale. It provokes a paradoxical sensation between unpleasant sentiments and familiar feelings. The audio content moves through processes of defamiliarization and reimagination of conventional instrumental sounds within a digital context. This is achieved by sampling commonplace instruments such as pianos and kalimbas, while also including computer-generated emulations of these sources that could be bent and distorted by manipulating sound synthesis parameters. 

Figure 1. Manipulation of multichannel audio panning and image rotation using X-Y-Z gyroscope positions from an iPhone.
Figure 2. Gestural spatialization involving more immersive audiovisual relationships through the use of SoundLazer (red) parametric speakers made by inventor Richard Haberkern.

In addition to the use of a quadrophonic (4.0-channel) audio playback system, additional highly directional parametric speakers were positioned in our gallery space to synchronize with the presentation of height and depth in the video content.

The setup for project uses pitch, roll, yaw, accelerometer, and keypad button OSC data transmitted from composer/programmer Kevin Schlei’s smartphone app GyrOSC to affect a Max/MSP patch that acts as a link between electroacoustic sampling and sound synthesis in the digital audio workstation Reaper, and video playback and manipulation taking place through an Isadora patch.

Figure 3. Contrasting gestural spatialization involving more immersive audio-visual relationships through the use of different mobile phone positions to interact with SoundLazer (red) parametric speakers made by inventor Richard Haberkern.

Max/MSP sends MIDI control change and note messages that are mapped to affect spatial parameters of projected video such as scaling and rotation, while simultaneously affecting changes such as triggering recorded audio samples, musical pitches for virtual synthesizer plugins and audio panning positions. It is the simultaneous translation of OSC to MIDI data mapped to Isadora for video and Reaper for audio that allows the smartphone to act as a digital instrument for improvisation.

Figure 4. Example of OSC to MIDI communication for sound spatialization via Max/MSP.

Theme and “Learned Gestures”

The use of a thematic design for structured improvisation with a digital instrument is an important consideration for cultivating the mobile device as an electroacoustic instrument for multidisciplinary collaboration. The use of conceptual bases or programmatic associations helps inform physical gestures that can help the performer discover new ways of interacting with the mobile device while considering notions of idiomaticity and how physical movements (e.g., changes in X-Y-Z axis positions) supports the position that spatialization and abstraction of source material can be principal drivers in audio-visual work. The linking of audio-visual parameters through MIDI and OSC also acts as a form of research-creation for performance in which users can rely on previously learned actions (e.g., swiping, tapping, repositioning the device) in performance to access new forms of creativity. This idea of accessibility issues as being resolved by digital instruments is echoed in computer music research, such as computer music programmer Ge Wang’s documentation of laptop orchestra as a pedagogical tool. His research from the Princeton Laptop Orchestra describes such the ensemble as an experiential workshop environment that allows computer music studies to venture away from a studio/lab environment that separates composition and performance and instead encourages interdisciplinary exploration.⁵

This writing also describes how the ensemble’s use of sound spatialization approach references the hierarchical nature of the orchestra in classical music. While the laptop orchestra model considers spatiality in reference to its comparison with a conventional symphony orchestra, our use of learned gestures in electroacoustic improvisation with digital video adds a new perspective to research-creation efforts in which performance with electroacoustic instruments can also be used for “epistemological intervention into the ‘regimes of truth’...”⁶ Rather than challenging the orchestra paradigm, we focus on reconsidering Eurocentric structural materials that foreground melodically oriented constructs of rhythm and pitch still permeate so much of contemporary sonic performance. By affecting the morphology of sound as connected to video through fluid physical movements, learned gestures become objects within the performance that help delineate how sonic and/or visual media are developing. 

Conclusion

An exploration of digital instruments as reliant on familiar movements is echoed in computer music research, such as Thor Magnusson’s Sonic Writing, which emphasizes that “...young people often enter the world of music through digital technologies (e.g., apps and games).”⁷ By exploring the cybernetic virtuosity in digital sound and video performance through physical gestures learned through everyday interactions with mobile devices, we wish to uncover the possible components of a performance practice that is compatible with live digital art practices such as electroacoustic music—in which spatiality is so essential to the performance experience. A traditional orchestral performer may have to learn how to use instrumental techniques such as vibrato to better project the sound of their instrument in a space, the use of a mobile phone in an improvisational piece such as Suffocation exposes a multiplicity of layers in which rhythm, cadence and narrative structure are also conveyed by the way that the performer explores the proximity and distance of their own body as connected to digital audio and visual materials.

References

1 Philip Alperson, “The Instrumentality of Music,” The Journal of Aesthetics and Art Criticism 66/1, Winter, 2008, 37-51.

2 Michael Lukaszuk, “Soundscapes and Anachronisms in Music for Laptop Ensemble,” eContact! 21/1, Spring 2022, https://econtact.ca/21_1/lukaszuk_laptop-soundscapes.html

3 Susanna Ray, “Brian Eno, Peter Chilvers create ‘quite magical’ flower garden of sound in Amsterdam with ‘Bloom: Open Space," Microsoft/Features, Accessed May 15, 2021, February 22, 2018, https://news.microsoft.com/features/brian-eno-peter-chilvers-create-quite-magical-flower-garden-sound-amsterdam-bloom-open-space/

4 Vanessa Chang, “Tracing Electronic Gesture: A Poetics of Mediated movement," PhD dissertation, Stanford University, 2017.

5 Ge Wang, et al, “The Laptop Orchestra in the Classroom,” Computer Music Journal 32/1, Spring 2008, 26-37.

6 Owen B Chapman, Sawchuck Kim, “Research-Creation: Intervention, analysis and ‘family resemblances’,” Canadian Journal of Communication 37/1, April 2012, “Media Arts Revisited,” 5–26.

7 Thor Magnusson, Sonic Writing, London, Bloomsbury, 2018. 

Generated tools: A Defamiliarizing Approach to Creating ML Art

Chloe McFadden and Oliver Bown

School of Art and Design, UNSW
New South Wales, Australia chloe.mcfadden@unsw.edu.au, o.bown@unsw.edu.au

Abstract

In recent years there has been an increase in the adoption of machine learning (ML) systems that can generate novel images. This increased use may reveal the beginning of a familiarity in which the implications of these emerging technologies are naturalised or made increasingly invisible. Thus, practices which can disrupt familiarity may allow us to create experiences of heightened awareness in which we can consider our engagement with this emerging technology. In this paper, I discuss the outcomes of working with a Generative Adversarial Network (GAN), using a dataset created from the hand tools section of a popular Australian hardware store. Through this creative practice, I investigate how artists can use ML as mechanisms for creating artworks that disrupt, investigate and defamiliarize the known.

Keywords

Generative Adversarial Networks, Machine Learning, Creative Artificial Intelligence, Tools, Generative Art.

DOI

10.69564/ISEA2023-17-short-McFadden-et-al-Generated-tools

Introduction

As digital technologies are increasingly embedded into the everyday and the ways we construct and interact within our world perception, it has been noted that they also become increasingly familiar, invisible, or subconscious.¹ While often perceived as a ‘tool’—an extension of the human hand designed to fulfil a certain function—our relation to technology is increasingly being understood as a symbiosis of interdependence.² Furthermore, the co-evolutionary relationship between tools and humans has also been noted—highlighting the influence these ‘tools’ have in shaping us, as we shape them.³ As we continue to see, use, and adopt these tools, they become increasingly familiar and every day. However, with this increased familiarity may come a decrease in active or conscious visibility resulting in the loss of regular inciting provocation to question how we relate, interact, and construct our perception of the world with these technologies. Thus, techniques of making the familiar unfamiliar may be a strategy for creating experiences of heightened perception of or critical engagement with technology.

Generated tools (2021) is a practice-based research project in which I worked with a GAN to create an installation that features tools as subject matter. In this, I am exploring how we can use ML conceptually to create defamiliarizing experiences which may allow us to critically reflect upon and re-engage with concepts, worldviews and ideas which have been naturalised. ML that can generate images is compelling technology—sparking new ways of making and creative possibilities. However, it also has the potential to be a reaffirming conservative force, as it is informed by the training dataset and may familiarize and perpetuate the visuals, ideas and assumptions hosted within.⁴ While this reliance upon the dataset may pose a risk if we engage uncritically—this also presents the opportunity for artists to disrupt, explore and denaturalise the known. Thus, through the documentation and discussion of work created with a GAN, this paper presents the concept of defamiliarization as one approach for understanding the potential impact or possibilities of working with ML to generate art.

Tools, Technology, and Familiarity

There has been much critical reflection on the relationship between humans and tools—both in how tools present an invisible familiarity, as well as in the mutual influence we exert. Martin Heidegger explained tools as "ready-to-hand", meaning as long as they function correctly or to our expectations, they remain concealed from view, or not in our conscious presence.⁵ However, while tools typically fade into the realm of the unconscious everyday, they also reveal a co-evolutionary relationship between tools and humans. While we typically narrativize our relation to tools as being a one-way influence, contemporary philosophical approaches recognise that it is more of a symbiosis. To solve an issue or to respond to our environment, we create new tools, which in turn change how we act and lead to the discovery of new problems or use cases for tools as this cycle repeats itself.⁶

Similar lines of thought have also been applied to technology. It has been suggested that technology is an extension of ourselves with which we cohabit a shared ecosystem. Through this cohabitation, we are able to coextend our skills, capabilities, and properties, changing how we act in ways we perceive as beneficial, and in turn leading to the development and adoption of new technologies.⁷

However, as this cycle of development and adoption continues, this cohabitation becomes increasingly familiar or subconscious.⁸ While technological innovation feels novel at first, over time with increased adoption and use, repeated interaction results in a comfortable familiarity that doesn’t necessarily encourage reflection or critical engagement.

In the past 5 years, there has been increased adoption of ML systems that can generate novel images. While there is a large amount of general ‘hype’ and awareness, the development of systems like DALLE-2, Stable Diffusion and Mid Journey, which all have user-friendly demos and applications hosted online, alongside apps like WOMBO Dream and TikTok’s green screen AI filter, may reveal the beginning of familiarity, or a future of familiarity with ML systems.

Defamiliarization and visual indeterminacy

If familiarity reduces technology to the subconscious, thus dampening opportunities to be aware of how we relate to and are shaped by it, unfamiliarity becomes a potential strategy for sparking conscious engagement. Coined by Viktor Shklovsky in 1917, defamiliarization is understood as a tactic for creating heightened awareness or perception by halting our automatic assumptions about a subject, allowing us to view it again for the first time.⁹, ¹⁰, ¹¹ The goal of such a practice is not to reveal a more objective truth about a subject, but rather to create a heightened perception of how we construct, understand, and relate to the subject.¹² Defamiliarization has also been identified as a common tactic employed by digital artists to create experiences of critical distance between audiences and technology, to temporarily make the familiar unfamiliar for heightened perception to be achieved.¹³

The ways that artists enact this varies broadly, as it is understood that all art forms can generate a defamiliarizing effect.¹⁴ However, one emerging approach related to ML is visual indeterminacy. It has been noted that GANs typically create uncanny, strange, or visually indeterminate images.¹⁵ Visual indeterminacy, as well as ambiguity and uncertainty, has also been recognized as useful tools for prompting multiple interpretations or disrupting an artefact’s socially encoded properties.¹⁶, ¹⁷ Furthermore, artists may be able to engage the differences between our logic and the chosen system’s logic to create defamiliarizing effects. For example, when working with image-based ML, we understand images as representations of objects, scenes, and worldly concepts, while a system like a GAN is attempting to map the underlying structural logic of the dataset at a pixel level. Engaging the system’s ability to recreate patterns within the dataset, we can generate coherent and recognisable forms. However, by navigating their latent spaces, we can push them to create semi-coherent, or visually indeterminate forms. Thus, working with ML to generate art may present opportunities to denaturalise our understandings through visual indeterminacy or strangeness, via this difference in structural logic.

Installation description

The installation consists of a tool wall which houses three instances of working with a StyleGAN2 trained using a dataset of images of hand tools sourced from the Bunnings Warehouse website (an iconic Australian hardware store).¹⁸ The first instance shows 3D printed tools, modelled and printed using the generated images by the GAN as reference. The second instance shows 2D “latent space walk” videos projected on the tool wall, created by incrementally sampling the latent vector space generated by the GAN. The final instance shows doctored Bunnings Warehouse product catalogues, featuring images of tools generated by the GAN. Audience members are encouraged to engage with the work by flipping through the product catalogues, as well as being able to pick up, play with and rearrange the 3D printed objects on the tool wall.

Figure 1. Image of Tool Wall Installation featuring 3D prints and 2D project latent space walk videos.

3D Printed tools

While images created by ML systems are becoming increasingly familiar, we most commonly encounter them in digital spaces, as images or animations. Thus, one goal of the work was to translate the generated images into 3D objects to observe whether this could be a method of creating defamiliarizing experiences. The 3D printed tools were created by first generating a series of images using a StyleGAN2 trained on a dataset of images of hand tools sourced from the Bunnings Warehouse website. While the GAN is trained using these images of hand tools, how these tools exist and function in conceptual space is not captured. Thus, when it generates images of tools it does so with no consideration of functionality—creating tools that have no clear associated uses. Simultaneously, we can expect the generated images to be aesthetically tool-like as the GAN is attempting to replicate the original dataset and find its underlying structural logic. Thus, the work explores whether the ‘uselessness’ of the GAN-produced artefacts is an effective tactic of defamiliarization.

Margaret Boden discusses ‘useless’ artefacts as having the ability to playfully challenge expectation, while bringing the typical affordance of similar items to the foreground.¹⁹ Tools are useful—but most of the time they are not being used—hung upon tool boards like artworks, neatly organised in boxes like collectables, or haphazardly thrown in draws like junk. They often also hold semantic and sentimental value - tools are given as gifts, passed down through families and lent between friends and communities. If tools remain concealed from perception so long as they function correctly, how does encountering inherently useless tools bring these hidden ‘uses’ to the foreground?²⁰

Figure 2. GAN generated image of a tool.

The generated images were then curated and used as references to model and texture 3D tools in Autodesk Maya and Mudbox. Reference images were selected through a process of working through the generated images and selecting a broad range of shapes, tool-like iconography and sizes that existed within the possibility space. The goal when modelling and texturing the tools was to follow the reference images closely, while also recognizing the inherent role of interpretation when translating from 2D to 3D. These models were then 3D printed using a variety of grey, black, and aluminium filament. The results are visually and texturally strange, tool-like artefacts. As they were modelled using the StyleGAN2 curated images as reference, they have a visually indeterminate quality—with familiar elements (e.g. handles, pointy ends, and bits and pieces of recognizable tools) combined or blurred in unfamiliar ways. Further, since they are 3D printed, the tools have unique ridges, divots and holes which create an unfamiliar hand feel. The light-weight material of the filament creates a unique engagement with the tools which we typically associate with rubbery, metal, and heavy sensations.

Figure 3. 3D print of GAN generated image of a tool.

Through the combination of recognizable tool-like elements in unfamiliar ways with the 3D printed materiality, the familiar functions and feelings that are tied to how we perceive, construct and experience tools subconsciously are no longer present—opening the space for new interpretations and speculation. As audiences would approach the strange tools, typically their first reaction was to generate a new use case for the tool. In short, this element of the work combines visual indeterminate ‘useless’ tools supplied by the GAN with subversive, textural 3D prints to create a speculative experience for audiences, engaging difference in structural logics, visual indeterminacy and unfamiliar materiality as tactics of defamiliarization.

2D latent space walk videos

The projected 2D latent space walk videos were created using the possibility space generated by the StyleGAN2 trained on the dataset of hand tools. Latent space walk videos involve the sampling and incremental changing of points in the latent vector space which can then be used to create animations. As the system was trained on a dataset of hand tools, the resulting animations present fluid tools that merge and shape into one another. While we typically conceptualise tools in a worldly context—with clear boundaries, defined shape language and affordances that are affirmed via learnt aesthetics (e.g., handles, spouts)—the GAN’s distribution renders a more fluid visualisation.

Thus, the work is interested in whether the visual indeterminacy created by the GAN is an effective strategy of defamiliarization. As visual indeterminacy is understood as a tactic to engage audiences with the active nature of seeing and meaning making, it may engender defamiliarizing experiences.²¹, ²²  Furthermore, visually indeterminate art prolongs perception through the combination of “apparently detailed and vivid images resist identification.”²³ The generated latent space walk videos thus present strange tools trained using a highly recognisable visual dataset, which now resist specific identification via the GAN’s involvement. Whether this resistance of identification invokes the active nature of seeing, or creates experiences of heightened perception is unclear—does coming into the contact with the work go beyond strange? Do these splodgey, blurry tools engender an active awareness or reflection on the symbiotic nature of tools and technology?

Product catalogue

Accompanying the tool wall are doctored product catalogues which have been inserted with images of tools generated by the GAN. Using Photoshop, the original composition of the catalogue was closely followed as the GAN generated tools were added based on their perceived visual similarities. The goal was to create a catalogue that could be believably passed off as real at first glance.

The resulting catalogue is a mixture of the original text, lifestyle images, prices, and generated tools. This presentation results in a double-take effect. Unlike the 3D printed tools which are outwardly alien or strange by design, in this presentation, they almost look like real tools inside of a real catalogue at a cursory glance. The context the product catalogue provides—the text, price tags, product descriptions, branding and lifestyle images—almost act as visual vouchers for authenticity of these generated tools.

Returning to defamiliarization, Shklovsky positions the purpose of art as to “make forms difficult, to increase the difficulty and length of perception because the process of perception is an aesthetic end in itself and must be prolonged."²⁴ Thus, the work engages surprise, understood as violation of expectation²⁵, and appropriation as potential strategies of defamiliarization. By appropriating the catalogue and inserting familiar, yet strange tool-like artefacts, the doubletake causes an increase in perception, calling us to look a little closer and make sense of the nonsensical tools.

In this, the work explores whether the surprise and absurdity of the nonsensical catalogue provides space to investigate the socially coded meanings present in the subject matter that are familiar and normalised. For example, does the messaging and imagery surrounding Father’s Day become less natural or familiar through the introduction of the generated tools?

Figure 4. Doctored Bunnings product catalogue featuring GAN generated images of tools.

Conclusion

As the practice of generating images using ML becomes increasingly familiar, we need to be aware and considerate of the ways it may naturalise or reinforce worldviews, categories, and ideas via this engagement. However, just as ML has the ability to be conservative, to narrow in on the dataset and to perpetuate the ideas within—so too can artists employ it to disrupt, investigate and defamiliarize the known.

Through the documentation and analysis of my work, Generated Tools (2021), this paper presents an experimental approach to generating art with ML to create defamiliarizing experiences. As a part of this discussion, I have explored how artists can engage with visual indeterminacy and the structural logics of ML systems, as a method for disrupting the assumed, and reapproaching the known with fresh perspective.

Furthermore, through the analysis of the 3D prints, projected latent space walk videos and doctored product catalogues, I have investigated how the presentation of ML generated images can create, alter, and deepen the defamiliarization experience.

In this I am interested in how we can ‘make strange’ in ways that matter. As ML presents unique opportunities for defamiliarization—how can artists defamiliarize in ways which persist beyond the initial encounter? How can we blur, splodge and surprise with ML in ways that recover both the subjects of datasets and technologies themselves from the automatism of perception and use?

Acknowledgements

This project was the first author’s (Chloe McFadden) honours project supervised by the second author (Oliver Bown); hence the paper is written in the first person singular.

References

1 I. Hwang, M. Guglielmetti, V. Dziekan, "The Familiar": technology-being-with-us," Usage guidelines, 2016, p.65.

2 É. Brangier, S. Hammes-Adelé, Beyond the technology acceptance model: Elements to validate the human-technology symbiosis model, in International Conference on Ergonomics and Health Aspects of Work with Computers, Springer, 2011.

3 J. Navarro, P.A. Hancock, Did Tools Create Humans? Theoretical Issues in Ergonomics Science, 2022, 1-27.

4 Kate Crawford, Trevor Paglen, "Excavating AI: the politics of images in machine learning training sets," AI & SOCIETY, 2021, doi: 10.1007/s00146-021-01162-8.

5 Harman Graham, "Technology, objects and things in Heidegger," Cambridge journal of economics 34, no. 1, 2010, 17-25.

6 J. Navarro, P.A. Hancock, Did Tools Create Humans? Theoretical Issues in Ergonomics Science, 2022, 1-27.

7 É. Brangier, S. Hammes-Adelé, Beyond the technology acceptance model: Elements to validate the human-technology symbiosis model, in International Conference on Ergonomics and Health Aspects of Work with Computers, Springer, 2011.

8 B.C Bruce, M.P. Hogan, The disappearance of technology: Toward an ecological model of literacy, in Writing in a Technological World, 2019, Routledge, 191-207.

9 V. Shklovsky, Art as technique [1917], The Critical Tradition: Classic Texts and Contemporary Trends, New York, 2007, 3, 774-784.

10 L. Crawford, Viktor Shklovskij: Différance in Defamiliarization, Comparative Literature, 1984, 209-219.

11 D.P. Gunn, Making art strange: a commentary on defamiliarization, The Georgia Review, 1984. 38(1), 25-33.

12 R. Bellanova, A.R. Sætnan, How to Discomfort a Worldview?: Social Sciences, Surveillance Technologies, and Defamiliarization, in Science, Technology, and Art in International Relations, Routledge, 2019, 29-39.

13 L. Starkand, K. Crawford, The work of art in the age of artificial intelligence: What artists can teach us about the ethics of data practice, Surveillance & Society, 2019. 17(3/4), 442-455.

14 D.P. Gunn, Making art strange: a commentary on defamiliarization, The Georgia Review, 1984. 38(1), 25-33.

15 Aaron Hertzmann, "Visual indeterminacy in GAN art," Leonardo 53, no. 4, 2020, 424-428.

16 P. Yurman, A.V. Reddy, Drawing Conversations Mediated by AI, in Creativity and Cognition, 2022.

17 J.J. Benjamin, et al, Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry, in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021.

18 Karras Tero, Samuli Laine,Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila, "Analyzing and improving the image quality of stylegan," In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8110-8119, 2020.

19 M. A Boden, Creativity and art: Three roads to surprise, Oxford University Press, 2010, 63.

20 Graham Harman, "Technology, objects and things in Heidegger," Cambridge journal of economics 34, no. 1, 2010, 17-25.

21 D. Gamboni, Potential images: Ambiguity and indeterminacy in modern art, Reaktion Books, 2002.

22 R. Pepperell, "Seeing without objects: Visual indeterminacy and art," Leonardo 39(5), 2006, 394-400.

23 A. Ishai, et al, "Perception, memory and aesthetics of indeterminate art," Brain Research Bulletin 73(4), 2007, 319-324.

24 V. Shklovsky, "Art as technique," Literary theory: An anthology 3, 1917, p.16.

25 M. L Maher, et al, Computational models of surprise in evaluating creative design, Proceedings of the fourth international conference on computational creativity, Citeseer, 2013.

Ecologies of Thought: Generative Art as a Collaborative Research Methodology with Guarani and Kaiowá Indigenous Communities

Matheus da Rocha Montanari

University of São Paulo, Polytechnic University of Valencia, Multimedia Anthropology Lab (UCL). São Paulo, Brazil/ Valencia, Spain
matheusrmontanari@gmail.com

Abstract

The Ecologies of Thought project aimed to reconceptualize the relationships between ecological and technical knowledge, seeking an epistemological understanding that pushed beyond nature vs. culture divides. More than a conceptual and theoretical proposal, which was based on the technodiverse notion of cosmotechnics, the project developed practical and experimental methodologies in collaboration with the Guarani and Kaiowá Indigenous communities of Brazil to further this investigation. With an international and interdisciplinary partnership, we researched relationships between sounds and plants in indigenous cosmology and the ways in which these relationships can help us to more deeply understand a notion of ecology that is based on the poetics of care. With that, we created a series of collaborative virtual reality experiments using generative art processes, which proved to be an interesting methodology of artistic creation as research.

Keywords

Indigenous Communities; Generative Art; Ecology; Virtual Reality; Anthropology; Cosmotechnics; Digital Art; Art as Research; Multimedia Anthropology; Guarani Kaiowa.

DOI

10.69564/ISEA2023-18-short-Da-Rocha-Montanari-Ecologies-of-Thought

Introduction

Between March and July 2022, the Ecologies of Thought (1) project established an international and multidisciplinary partnership between Indigenous communities of Brazil, artists, anthropologists, botanists, and audiovisual producers to investigate Guarani and Kaiowá ecological thought through the relationships between sound and plants. The project proposed dialogues among different types and conceptions of technology, from chanting and the traditional cultivation method, to the use of microcontrollers and data analysis.

This project is part of the Guarani and Kaiowá Virtual Museum (2), which develops a virtual reality museum curated by the community elders and shamans – Nhandesys (female), and Nhanderus (male) in partnership with the Multimedia Anthropology Lab at University College London. 

Since the beginning of the project, the community has been challenging and redefining the notion of a "museum," the extra-human relationships among plants, animals, instruments, and spirits present us with a complex challenge: how to maintain a Guarani and Kaiowá cosmotechnical fidelity in the museum?¹ That is, how to enable the indigenous cosmology, and its technologies, not only to inhabit the virtual space but to build it as a tekoha, a territory, that allows the exercise of the Guarani and Kaiowá way of being and living, teko.² That means, how to build a digital Guarani and Kaiowá tekoha?

To our surprise, in the first test of the museum presented to the community with the virtual reality headset, two Nhandesys, experienced chanters, saw Jaras, spirits, in the virtual space. From this encounter, we began to investigate different ways to explore the spiritual and cosmological aspects, which are so present in the Guarani and Kaiowá ecology of thought, within the museum. The relationship between sound and plants proved to be a fertile field for this research. We began to learn more from the community elders and apprentices who introduced us to chanting as a cultivation technology: a physical and spiritual dialogue between the plant, its spirits, and the chanter. To chant is to care for, and the poetics of care is essential for the development of a healthy territory, be it physical or digital.³

We learned that instruments are also people. And that the Guarani and Kaiowá notion of preservation is different. For example, if the instrument is kept and not used, even if in a collection, it loses its ability to produce sound because it loses its ability to create connections when it is removed from its communication network. Therefore, it became evident the need to explore different ways to activate the collection we have been building for the museum.

To this end, we developed an experimental strategy. We focused on the creation of virtual reality worlds that, using generative art processes, explored the Guarani and Kaiowá cosmology from its cosmotechnical aspect. We used sounds and traditional elements, such as the Chiru, the sacred stick that supports the world, and the white corn, which is inhabited by one of the most important spirits (Jakairá), to investigate fundamental elements of the cosmology through an experience, rather than a narrative representation.⁴ In light of the urgent need to revise and rethink human-ecology relationships on a global scale, while acknowledging the new epistemic possibilities of data capture and analysis technologies, we collaborated with Guarani and Kaiowá community members to explore the different and creative kinds of data that these devices afford, and the ecological knowledge they make possible.

The Guarani and Kaiowá

Mato Grosso do Sul is a state in the mid-western region of Brazil, bordered by Paraguay and Bolivia, in a transition region between three biomes: Atlantic Forest, Cerrado, and Pantanal. It is the State with the second-largest indigenous population in the country and the largest outside the Amazon. The Guarani Ñhandeva and Kaiowá population of the region in 2021 is estimated to be 63.5 thousand people. At the same time, the State has the second-worst land distribution index in the country, with large properties (> 1,000 ha) occupying 83% of the total area. Thus, it is not surprising that the region is a scenario of serious conflicts between landowners and indigenous people, especially in the area of cattle ranching, sugarcane, corn, soybeans, and eucalyptus plantations.⁵

The Guarani and Kaiowá are Guarani speakers, from the Tupi-Guarani linguistic trunk. They have a strong ancestral territorial bond, both physical and spiritual, which guides their struggle in an especially organized way since the large assemblies beginning in 1979 (Aty Guasu). These articulations occur mainly between the religious leaders, Nhanderus and Nhandesys, since the community considers that the simultaneous performance of the religious ritual (jeroky) "is fundamental to recover the dialogue with the invisible beings and the guardians of the ancient tekohas," that is, for the recovery of the historical territories of their people, it is also fundamental to recover the extra-human relations developed in the territories, whether they are animal, vegetable, and/or spiritual.⁶

These relationships are directly connected to chanting and praying. The chant, besides a dialogue, is part of the materiality of the world: "In the understanding of the spiritual leaders, the singularities of the physical world inevitably need singing to continue their continuous existence, otherwise the world will gradually end."⁷ This is one of the reasons we choose sound, specifically chant, as a source of investigation of the ecological relations in Guarani and Kaiowá thought. 

Artistic creation as an interdisciplinary collaborative methodology

The Guarani and Kaiowá cosmology have a great degree of complexity and diversity, even in the same family trunk, some stories may vary between one group and another. Moreover, the narratives are full of recursion and non-linearity, in which each detail branches off into another story, that can take days to be told in the traditional oral form. So, it was clear from the beginning of the project that there would be limitations on the form and amount of information we could present. At the same time, we did not want to make a reductionist narrative translation that would create a shallow representation of the generous exchanges that the project made possible.

Besides that, the diverse points of view and areas of knowledge involved in the project configure, simultaneously, its richness and complexity. The challenge of combining shamans, scientists, and artists in the same project revealed the power of the digital medium as a research tool in itself, and not only as a form of scientific dissemination. More than that, we set out to develop research in visual arts, rather than research about visual arts. That is, to reaffirm the processes of creation as valid ways of researching, thinking, and producing knowledge about the world.⁸ In this sense, artistic creation acts in the project as a collaborative methodology capable of providing dialogues and investigative interactivity: the creation processes allowed for the creation of a two-way exchange, in which collaboration affected not only the production but the understanding and direction of what was being produced.

In this project, art, which occupies a liminal place, is not limited to an art-technology extension of scientific knowledge. We seek to embrace the mba'e kuaa, the Guarani and Kaiowá technical knowledge and know-how. In this proposal, we guide ourselves with the concept of cosmotechnics developed by Y uk Hui. That is, to rediscover the diversity of technical thought beyond the Greek notion of techné:

Technology is not anthropologically universal; its functioning is assured and limited by particular cosmologies that go beyond mere functionality and utility. Thus, there is no single technology, but a multiplicity of cosmotechnics.⁹

Generative Art and the Guarani & Kaiowá Cosmology

Generative art can be defined in different degrees of complexity, it is associated with a system that has some degree of autonomy, in which the artist and the system exercise a series of operations that result in the final work.¹⁰ Seeking ways to engage with the mythology and respect their characteristics, generative art methodologies seemed to not only fit with the ontological qualities of the Guarani and Kaiowá cosmology but to privilege an investigation and audiovisual production that could maintain a cosmotechnical fidelity. Since both, the Guarani and Kaiowá cosmology and generative art are nonlinear emergent processes with repetitive elements that are in constant transformation. Working in that direction, we created a series of six experiments, and in this article, we will be presenting one of them.

Jakairá: white corn, bees, and the maintenance of the world

According to the Kaiowá myth, the deity Jakairá is responsible for the creation of the first roçado (food garden). Through his wisdom, Jakairá created the white corn, which is at the top of the food hierarchy, and, from it, all other agricultural products were created. Therefore, Jakairá and white corn are responsible for the creation and maintenance of almost all food on Earth. Its cultivation follows a series of rules, from dietary restrictions of the growers to the use of chants to cheer the corn and its baptism.¹¹ The corn baptism ritual, called Jerosy Puku, is one of the most important Kaiowá rituals. Through it, the corn becomes fit for consumption by the community in a healthy way, since, besides being a physical food, it is also considered a spiritual food. The baptism is also a moment of reaffirmation of the Kaiowá way of being, strengthening the human and extra-human (spirits, plants, animals) relationships of the community.¹² This specific type of corn is currently in danger of extinction due to the environmental threats it suffers, the heavy use of pesticides in the GMO corn plantations that surround the communities, and the consequences of climate change in the region.

Our first experiment arises precisely from this extrahuman ecology. In one of our ethnographic sessions on the importance of corn and its relationship to sound, we learned that corn has its own sound that is not audible to all people. However, there is a specific moment when the sound of corn can be heard by everyone. The community members told us that when the corn, still green (avati kyry), begins to mature and get ready for harvest, it produces a smell that attracts bees so that "they can make the sound of the corn," warning the cultivator that it is almost time to harvest. As talking about the white corn is also talking about the beginning of life and the food that sustains it, this pre-harvest moment, when the bee is attracted to the corn to make its sound, seemed emblematic to us to investigate an instant of the creation of the world according to the Guarani and Kaiowá cosmology. This moment that seems, at first sight, to deal with a multispecies relationship between the plant and the bee, when further investigated, reveals, also, to be a relationship between the spirit owner of the corn and its cultivator¹³. To continue this investigation, we developed a generative art experiment using the TouchDesigner programming language: a visual programming language in nodes for the creation of real-time interactive multimedia content. We started with the image of a white cornfield before the harvest and analyzed the light incidence in its different areas. From this data analysis, we created a relief map that was applied to an ellipse for the construction of a terrain. The darker areas of the image created depressions, and the lighter areas, elevations. Finally, we added two noise layers, one algorithmic, and another from the sound analysis of the frequencies produced by the bees. The sound frequency analysis controls the algorithmic noise, which generates motion on the terrain. With that, we exported a 360o video for VR headsets. In this way, we developed a virtual reality experience in which the ecological relationship between the corn and the bee produces the creation of a generative world based on the analysis of image and sound data from the field, guided by the Guarani and Kaiowá cosmology (figure 1). 

Even without a naturalistic production of the image, and perhaps precisely because of this characteristic, the community approved the experiment. Once the step-by-step construction of the generative program was presented, there was consensus that the experiment had true Guarani and Kaiowá characteristics, mixed with non-indigenous technology. From this experiment on, the community began to direct the following more directly, choosing images and elements that should be used for the continuation of the project.

1. Still from the VR experience – Generative Terrain produced by the data analysis of the white corn and the bee’s sounds.

Conclusion

The Ecologies of Thought project aimed to reconceptualize the relationships between ecological and technical knowledge, seeking an epistemological understanding that pushed beyond nature vs. culture divides. More than a conceptual and theoretical proposal, which was based on the technodiverse notion of cosmotechnics, the project developed practical and experimental methodologies in collaboration with Guarani and Kaiowá communities to further this investigation.¹⁴ With an international and interdisciplinary partnership, we researched relationships between sounds and plants in indigenous cosmology and the ways in which these relationships can help us to more deeply understand a notion of ecology that is based on a poetics of care.

Thus, we were able to bring together poetic and technical processes to propose a mixture of knowledge that beyond an explanation or representation of one form of knowledge into another, develops an experience of its own. This allows us to fight an idealized naturalistic representation of indigenous culture, which fails to understand indigenous thought and technologies as valid ways to build and think of a cosmotechnic for our time.

More broadly, this project seeks to undermine the romantic and colonial notion of preservation, which does not consider the indigenous presence a digital presence, able to act and think about technical processes since its conception. This means, beyond an illustrative representation of what is believed to be the material and immaterial cultural heritage of the community, to collaboratively build a digital-indigenous territory, a digital Guarani and Kaiowá tekoha where a new technological ontology is established, in a way that the very conception of contemporary technology is affected by indigenous cosmotechnics.

In conclusion, the epistemic research proposed and developed during the project revealed to us the potential that this collaborative methodology possesses to go beyond epistemological investigations, enabling us to revisit ontological questions of technology. Collaboration, interdisciplinarity, and the investigation of creative processes—rather than the proposition of representations—allows us to arrive at questions related to the level of existence, so that, in the anthropocenic era of the absence of futures, we can compose a cosmotechnic for a viable, virtual and actual, reality, based on a poetics of care and more-than-human relations. 

Acknowledgments

We would like to acknowledge the University College London Global Engagement Fund which contributed to the partnership among the Multimedia Anthropology Lab at University College London, the Kunangue Aty Guasu, IDAC, The School of Botany, The University of São Paulo, The Federal University of Santa Catarina.

We would like to thank and acknowledge the generosity of the Nhandesys and Nhanderus whom we have been working with during these years. Nhandesy Neuza Alziro, Nhandesy Ivone, Nhandesy Fausta Solano Mendosa e Nhandesy Roselí Aquino. Nhanderu Tadeu Romeiro Freita, Nhanderu Valdomiro Oswaldo Aquino. Also, the team of UCL’s Multimedia Anthropology Lab and the Kunangue Aty Guasu: Valdineia Aquino, Scott Hill, Luan Iturve, Germano Alziro, Jaqueline Gonçalves, Raffaella Fryer-Moreira, Fabi Fernandes, Gerard Oliver, Adriana Boloc, Stephanie Ferraz, Clara Ciqueira, Luiza Braga and MAL’s translation team. And our partners Karola Braga, Anderson Santos and Caetano Sordi.

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 88887.716693/2022-00. 


(1) https://www.matheusmontanari.com/ecologiesofthought
(2) https://www.uclmal.com/virtual-museum

References

1 Yuk Hui, The question concerning technology in China: an essay in cosmotechnics, Falmouth, Urbanomic, 2018.

2 Tonico Benites, “Trajetória de luta árdua da articulação das lideranças Guarani e Kaiowá para recuperar os seus territórios tradicionais tekoha guasu,” Revista de Antropologia da UFScar, Vol. 04, No. 02, accessed October 14, 2022, https://doi.org/10.52426/rau.v4i2.83

3 Matheus Montanari and Gilbertto Prado, “From vigilance to vigil: an introduction to an alternative paradigm for technology, art, and life,” Diffractions, Vol. 01, No. 05, accessed October 14, 2022, https://doi.org/10.34632/diffractions.2022.10194

4 Fabio Mura, “A Trajetória dos Chiru na construção da tradição de conhecimento Kaiowá,” MANA, Vol. 16, No. 01, accessed October 14, 2022, https://doi.org/10.1590/S0104-93132010000100006

5 Anderson de Souza Santos, Luiz Henrique Eloy Amado and Dan Pasca, “"É muita terra pra pouco índio"? Ou muita terra na mão de poucos? Conflitos fundiários no Mato Grosso do Sul,” Instituto Socioambiental, accessed October 14, 2022, https://acervo.socioambiental.org/acervo/documentos/e-muita-terra-pra-pouco-indio-ou-muita-terra-na-mao-de-poucos-conflitos

6 Tonico Benites, “Trajetória de luta árdua da articulação das lideranças Guarani e Kaiowá para recuperar os seus territórios tradicionais tekoha guasu,” Revista de Antropologia da UFScar, Vol. 04, No. 02, accessed October 14, 2022, https://doi.org/10.52426/rau.v4i2.83,171

7 Izaque João, “Jerosy Puku,” PISEGRAMA, Vol. 01, No. 06, accessed October 14, 2022, https://piseagrama.org/jerosy-puku/,1

8 Sandra Rey, “Por uma abordagem metodológica da pesquisa em artes visuais,” in O meio como ponto zero, ed. Bianca Brites and Elida Tesser, Porto Alegre: Editora da UFRGS, 2002, 123-140.

9 Yuk Hui, Tecnodiversidade, São Paulo, Ubu, 2020, 25.

10 Philip Galanter, “Generative Art Theory,” in A companion to digital art, ed. Christiane Paul, Hoboken, John Wiley & Sons, 2016.

11 Izaque João, “Jerosy Puku,” PISEGRAMA, Vol. 01, No. 06, accessed October 14, 2022, https://piseagrama.org/jerosy-puku/

12 Izaque João, “Jakaira Reko Nheypyrũ Marangatu Mborahéi: Origem E Fundamentos Do Canto Ritual Jerosy puku Entre Os Kaiowá De Panambi, Panambizinho E Sucuri’y, Mato Grosso Do Sul”, Master’s. diss., Programa de Pós-Graduação em História da Faculdade de Ciências Humanas, Programa de Pós-Graduação em História da Faculdade de Ciências Humanas, 2011.

13 Carlos Fausto, “Donos demais: Maestria e Dominio na Amazônia,” MANA, Vol. 14, No. 02, accessed October 14, 2022, http://dx.doi.org/10.1590/S0104-93132008000200003

14 Yuk Hui, The question concerning technology in China: an essay in cosmotechnics 

Author Biography

Matheus Montanari is a Ph.D. Candidate at the University of São Paulo in Visual Arts, and external researcher at the Polytechnic University of Valencia. He develops works at the intersection of art, science and philosophy, investigating ways to rethink technology after art. He is interested in cosmotechnical diversities, and in combining decolonial ecological thinking with technical know-how. Recently he has been developing his practice in collaboration with indigenous Guarani and Kaiowá communities in Brazil. He has exhibited his work in Brazil and abroad, in countries such as: Argentina, Portugal, Austria, Croatia, China, Spain and Italy. He was awarded the web art category prize of the 67th Contemporary Art Salon of Paraná. 

Emerging Strategies “Under The Bay” in AR/XR

Lisa Moren, Tsvetan Bachvaroff

University of Maryland Baltimore County (UMBC), Institute of Marine and Environmental Technology (IMET), University of Maryland Center for Environmental Science (UMCES) Baltimore, Maryland lmoren@umbc.edu, bachvarofft@umces.edu

Abstract

“Under the Bay” is an augmented reality project where anyone can use their cell phone like a microscope and reveal invisibilities in our world and marine life. When they do a series of animated stories between humans and non-humans emerge. Images, sounds, and stories are affected by live data streamed in from sensors located in the largest estuary in North America. Sensors in the Chesapeake Bay relay live pH, oxygen, temperature, etc. (figure 9). Similar to the water itself, color, speed, audio fluctuate with the water and marine life, making “Under the Bay” a data-driven narrative with eight scenes that tell a story of a world beneath the marine surface, and the exciting but frail health of estuaries and oceans worldwide. The two projects discussed here, “Under the Bay” (2022) and “What is the Shape of Water?” (2020), are part of Lisa Moren’s series of cross-species artworks aimed at diminishing human-centered exceptionalism. The collaborations began in 2019 when Lisa was the inaugural Artist-in-Resident at the Institute for Marine and Environmental Technology (IMET). There, she met researcher and marine biologist, Dr. Tsvetan Bachvaroff (Tsetso) and the two immediately shared a like-minded vision to develop a project that exemplified phenomenal exceptionalisms in micro-organisms. In this paper we argue that novel strategies in nature emerge when a complexity of matter is intermingled with conditions of differentiation. We explain and identify differentiation in art and architecture, symbiosis in biology, and the “wobble” in physics as core principles for new forms and creative strategies to emerge. The outcome is focused on the unusual and significant diversification of dinoflagellate microbes in the Chesapeake Bay and oceans worldwide. Tsetso directed the live organisms, science and data analysis for the augmented reality project. Stories are written and told by Lisa, who produced and art directed the animation and AR scenes. The sound score is by electronic composer Dan Deacon. Dr. Marc Olano led the software engineering and development with John Boutsikas, for the AR app in IOS and Google Play.

Keywords

Augmented Reality, AR, bio-art, data-driven narrative, emergent strategies, Tao Te Ching, water, symbiosis, dinoflagellates, Chesapeake Bay, estuary, marine biology, performance, microbes, media art, podcast, experimental narrative, bio-architecture, Lynn Margulis, Ted Nelson, Jane Bennett, Theodor Schwenk, data-driven music, philosophy.

DOI

10.69564/ISEA2023-19-short-Moren-et-al-Emerging-Strategies

Figure 1. “Under the Bay” augmented reality (still) by Lisa Moren with Dr. Tsvetan Bachvaroff. Scene 03 \\ Chalky Faeries, 2022. Image shows a cropped Bay Nettle jellyfish with coccolithophores (the microbes responsible for all the chalk in the world). The app allows the user to click and drag on the microbe to draw in chalk. Courtesy of the artist. 

Introduction

Figure 2. “Under the Bay” AR by Moren/Bachvaroff. Left: demonstration of AR in Fells Point, Baltimore; right: Beta testing of AR on Pier V, Baltimore’s harbor. Courtesy of the artist.

In our collaborative work, we borrow the idea of “emergent strategies” to consider strategies in nature that describe phenomena driving new emerging forms. Inspired by Octavia Butler, Adrian Maree Brown coined the term “Emergent Strategy,”¹ as “a framework for resistance that is rooted in the miracles of nature, decentralized, collective leadership, and personal, relational, organizational, and movement-wide transformation.”² We focused our strategies based in part on Brown’s idea that anomalous strategies in nature can be a model for the benefit of human communities, from species longevity to social change. Lisa’s second influence was her participation in a Taoist meditation group where she studied and meditated on the Tao Te Ching.³ These meditations blended with her observations of nature, primarily starling bird murmurations described in Scene 05 \\ Lava Lamps in the Sky. The connections between emergent strategies, organic differentiation, and a Taoist social order became the basis of this art and science collaboration. 

3. “Under the Bay” AR by Moren/Bachvaroff. Left: Still from splash screen with user interface (UI); right: still from AR Scene 02 \\ Water Moving Around My Fingers at Pier V, Baltimore. Courtesy of the Artist.

Emergent Strategies

We find the origin of strategic emergence by looking at gene evolution in early microbes. The Darwinian idea of natural selection has been combined with genome inheritance from two parents into what is now called the "modern synthesis." While this synthesis has excellent explanatory power, some observations are not a perfect fit to this synthesis. For example, photosynthesis is scattered across multiple kingdoms, but the process of photosynthesis itself is unlikely to have been invented multiple times. The solution to this paradox required two steps (figure 4). First, Lynn Margulis in 1967 proposed that eukaryotes are a combination of two cells from different bacterial lineages. The first lineage is the archea bacteria which engulfed another bacteria cell that contributes the mitochondrion. Mitochondria are responsible for the exchange of oxygen and carbon dioxide, eventually this will be the foundation for breath itself. The mitochondrion is then a "cell within a cell."⁴ This symbiotic combination of two previously distinct cells into eukaryotes created a new complexity of features (figure 4). The first scene in our project Scene 01 \\ Origin Stories, describes that before Margulis, any scientific reference to a symbiosis hypothesis was taboo. To accept symbiosis, science had to imagine a scenario where a species swallows a giraffe’s eye to acquire longer eyelashes successfully, but then, they pass those eyelashes onto their offspring. Symbiosis was once a fairy tale.

But once the formation of eukaryotes via symbiosis of two lineages was accepted, then the symbiosis concept was extended to include photosynthesis in plants and algae. In this scenario, a previously free-living cyanobacterium was engulfed by a eukaryotic host (figure 5). The host now contains both mitochondria and photosynthetic chloroplasts. This new host obtains energy from sunlight via the chloroplast and exploits that energy in the mitochondria. This innovation ultimately leads to land plants and nearly all photosynthetic eukaryotes (figure 4).

Sarah Gibbs extended the idea of symbiosis to explain why photosynthesis is scattered across multiple kingdoms. In this third scenario, one cell with chloroplasts is consumed⁵ by a second, but only the chloroplasts are retained. This provides access to photosynthesis and explains how photosynthesis could have spread across many diverse eukaryotic groups.

Therefore, an individual can be shaped by two patterns of evolution: the familiar and dominant expression of parental genes via vertical descent and a second, less familiar, pattern of horizontal gene transfer from endosymbiosis (figure 4).

The dinoflagellates are from the eukaryote kingdom and a perfect case study to demonstrate both symbiosis and the global importance of small eukaryotes to our planet. In global estimates of oceanic photosynthesis dinoflagellates are one of the top oxygen producers. 

Figure 4. Left to right: Darwin’s original descent with modification figure from the ‘Origin of Species’; Phylogenetic schematic of symbiosis, Moren/Bachvaroff, 2023; Lynn Margulis, https://en.wikipedia.org/wiki/Lynn_Margulis#/media/File:Lynn_ Margulis.jpg; Sarah Gibbs, Gibbs, S Annu. Rev. Plant Biol. 2006. 57:1–17. 

The oceans produce oxygen at a scale similar to the contributions from the Amazon rainforest. This is underscored by modern sequencing which places dinoflagellates as second in abundance to animals and first in diversity in oceanic surveys. Dinoflagellates also include a wide array of life strategies such as built-in harpoons used to capture prey.⁶ Other strategies include a symbiotic photosynthetic lifestyle as the algae found in coral reefs use internal and external types of parasitism, and consuming food for energy. The term mixotrophy describes the role of many photosynthetic dinoflagellates, which can be both photosynthetic (autotrophic) and consume prey (heterotrophic).⁷

Figure 5. Left: Gene Flow Symbiosis chart by Tsvetan Bachvaroff; right: “Under the Bay” AR by Moren/Bachvaroff. Still from Scene 01 \\ Origin Storiesshowing symbiosis. Courtesy of the scientist and artist. 

Mixotrophy and diversity in the ocean combined with the concept of symbiosis represented by Gibbs and Margulis provides many instances of emergent strategies. For example, the dinoflagellate Dinophysis feeds on ciliates demonstrating how the chloroplast could be consumed intact from prey by injecting a built-in “straw-like method” to acquire, or suck, chloroplasts from its prey.⁸ Roughly nine events have occurred where photosynthetic dinoflagellates contain chloroplasts borrowed or gained from essentially every available lineage. And this list does not include dinoflagellates with elaborate structural features that contain external symbiotes such as Ornithocercus

Figure 6. Three genera of dinophysoid dinoflagellates. Left to right: Phalachroma, Dinophysis SEM, Histioneis. Three genera contain intracellular photosynthetic organelles or symbionts, each distinct from the other. Courtesy of the scientist.
7. Left to right: Ornithocercus SEM, Ornithocercus with visible cyanobacteria; “Under the Bay” AR (still) by Moren/Bachvaroff. Courtesy of the scientist and the artist. 

Within the Dinophysis group there are different types of photosynthesis or symbioses, including the ‘gardening’ Histioneis (figure 6) and Ornithocercus (figure 7), the ‘heterotrophic’ Dinophysis that sucks out chloroplasts, and at least two other types of internal symbionts. Ornithorcercus will host cyanobacteria in a birdcage-like crown, a strategy that not only stores nutrients for on-going consumption, but the chains of cyanobacteria reproduce itself creating a “fruit-on-the- vine-like garden” energy supply (figure 7).

Symbiosis has allowed these the single-celled dinoflagellates to diversify over hundreds of millions of years into thousands of species that will clearly outlive humans. Therefore, diverse reproductive, energy consumption, and other strategies assisted the dinoflagellates in obtaining longevity that humans can only dream of. For example, the dinoflagellate Ceratium can both cell divide and mate for optimum reproduction benefits. In Scene 01 \\ Origin Stories the viewer will see images of the Ceratium that cell divide having only one trailing flagellum (a propeller-type tail). However, an identical Ceratium has two flagella because they had two parents that mated (figure 3).

Other strategies we named included organic differentiation in the structure of the cell walls of the dinoflagellates. These complex forms, such as Voronoi patterns, use less matter to produce structures that are lighter in weight than any objects human engineering could produce based on Cartesian principles in manufacturing. Differentiation can be exemplified in the exoskeleton of a crab or lobster but is most visible in the repeating hexagon pattern of a turtle shell, where the hexagon shapes repeat, but not perfectly. It’s in that imperfection, crookedness, or wobble, that creates more strength with less matter. Dinoflagellates, diatoms, and other microbes also have this efficient design principle of differentiation. Scene 04 \\ Crooked Shelters (figure 8) describes how such differentiation influenced the largest 21st century algorithmically designed and digitally fabricated architectural form in Seville, Spain, the Metropol Parasol.⁹ The pavillion’s aerial view appears as a mushroom blooming throughout the gridded city. A similar architectural example is in Stuttgart, Germany, where one of the algorithmic pavilions based on differentiation in nature was so strong and lightweight, it blew away.¹⁰

Figure 8. “Under the Bay” AR by Moren/Bachvaroff. Left: Work in progress; right: still from Scene 04 \\ Crooked Shelters. Courtesy of the artist.

Water Projects

In our first attempt to create a public project for an open house at IMET, we created an artwork demonstrating the microbial strategy of making light through the bioluminescence of the dinoflagellate (Pyrocystis). Eventually, the public display was set up as a ceiling tank hooked up to a Max/MSP, Arduino, and AV system for the Light City Festival in Baltimore’s Inner Harbor. The audience entered a pitch-black room with a large ceiling tank holding millions of invisible dinoflagellates. The system worked with a voice-activated trigger so that when a participant spoke into a microphone, for example, “What is the Shape of Water?” the microscopic organisms above them answered the question in turbulent shapes of blue bioluminescence (figure 9). Originally, this was influenced by the mesmerizing organic order and differentiation in the murmuration patterns of starling birds.¹¹ However, Tsetso’s colleague, Dr. Al Place who studies the motion behavior of dinoflagellates, says that the flocking behavior of the dinoflagellates will unlikely look as organized as the starlings. Instead, the water agitation produced turbulent patterns more akin to the wobble, the crookedness or what philosopher Jane Bennett calls murmuring messiness.¹² 

Figure 9. “What is the Shape of Water?” Left: Lisa Moren pouring biomatter in the ceiling tank; right: Photograph, Moren/ Bachvaroff. 16x16”, 2019. Courtesy of the artist.

“What is the Shape of Water?” began as a somewhat expedient strategy of displaying bioluminescence in an art project that took over a year to fully realize. It was an unrealistic approach to create art installations for the more than two dozen strategies we identified for potential exhibition. We instead turned to storytelling and produced an augmented reality (AR) project with eight scenes, approximately 10 minutes per scene. Lisa took the strategies we identified for the project and built narratives that meandered through purposefully fragmented topics related to microbes, water, Taoist meditation, architecture, monuments, and politics.

The AR project became “Under the Bay” and it’s trailer¹³ illustrates the animation, narrator, and incoming data. For the data, we worked with the Maryland Department of Natural Resources (MD DNR) to siphon data from sensors already installed in the largest estuary in North America, the Chesapeake Bay. In total 36 parameters stream into the project from six locations in the Bay. Locations are from the Delaware border to Washington, DC, the Eastern Shore of Maryland, and Baltimore City (figure 10). The parameters pH, temperature, oxygen, salt, chlorophyll, and turbidity (clarity) are updated every 15 minutes. These parameters then affect the animation’s color, speed, and scale, along with the narration and music composition. For instance, when the oxygen levels in the Bay are of good quality, the narrator’s voice sounds normal. However, when the water is anoxic, with little or no oxygen, the voice becomes choppy, fragmented, as if choking. In this way, the story, images, and sounds change from day to night and season to season for an ongoing collaborative narrative with the Bay water. The unpredictable variability of the incoming data becomes an authentic wobble created by the environment.

Figure 10. “Under the Bay” AR Moren/Bachvaroff. UI showing locations in the Chesapeake Bay where sensors allow water parameters to stream into the project. Courtesy of the artist.

Listening to the Water

The project propaganda claimed the ambition of cross- species communication specifically “What if we could hear what the water is saying?” To address skeptics to these claims, we’re borrowing from water physicist Theodor Schwenk, as we do in the AR project, but also Jane Bennett¹⁴, who references Graham Harmon, Bruno Latour, and Michel Serres’s The Birth of Physics.¹⁵

If we look at water as an object, we can argue that water is the largest object in the world. On the one hand, the ocean contains essential elements, H2O, saline, and other matter on the periodic chart. We know that water and gravity work together to form currents like pipes that braid in distinct patterns and that these flowing pipes separate into differing physical data such as speed and temperature. These differences become visible when encountering an obstacle, like a rock in a river, where we observe differentiation in the water shapes bulging around the rock. Similarly, in the ocean, large and “long waves travel faster than short waves” and overlap until the larger ones envelop the smaller ones, and the waves repeat the pattern endlessly.¹⁶ But unlike early CGI waves, the real ocean’s patterns are not perfect or predictable waves, their equilibrium billows, and exhales asymmetrically, and it’s in that asymmetry that they wobble. Or, what Bennett calls, they produce a vibratory noise, where the force of repetition starts to create a surge, an “irregular bombardment of circumstances,” especially when new physical elements act as an obstacle such as a rock. Here, the current billows and exposes its diverse temperatures, causing what Serres names a “cauldron of turbulence that thickens into lumps of phenomena, and the bubbling swirl keeps those shapes upright... while the wobble produces variances of noise.”¹⁷ A shape derives from the swirl that is both form and vibratory. To Serres, he calls this vibratory noise “the fluctuating ado that is the strange substance of any discrete, differentiated shape... (where) the multiplicity of the possible rustles in the midst of the forms that emerge from it... It is restless matter... (a) percolation.”¹⁸ While much of this noise dies like seedlings that don’t spawn, the intermingling currents and swirls, overlap with enough force allowing forms to emerge from it.

While we do not think of water as having agency, free will, or decision-making abilities or even that its elements are alive, in complex natural events, water does cause a multitude of events to happen, and is therefore an actant. One of the most significant events that water enacts is sustaining and creating new life, but also emerging shapes, and unique forms. In this way, water will begin by initiating an abundance of events, such as a vortex or a whirlpool effect. Any time these elements react or affect one another, there is the potential for something to emerge, including new life (figure 11). 

Figure 11. Left: Vortex funnel drawing by Theodor Schwenk,“Sensitive Chaos: The Creation of Flowing Forms in Water and Air”1967, pp 44; center and right: “Under the Bay”AR by Moren/Bachvaroff. Courtesy of the artist.

In Scene 02 \\ Water Moving Around My Fingers, the microbes reflect Schwenks organisms that take on the negative space of the vortex shape traveling in a reverse corkscrew spiraling up (figure 11).¹⁹ Therefore, when we say we can hear what the water is saying? It’s not that the water, as pure physical compounds, produces a will or agency that desires to be heard. However, water’s natural environment has reactions, interactions, and relationships with other phenomena and other physical materials where unique consequences emerge as shapes, forms, beings, and blooms. Moreover, if we consider the incoming parameters of the Bay water as a kind of alphabet — to use a human metaphor — the data does arrange itself to describe a story of the Bay water’s emerging behaviors and forms. This communication is so hard for humans to understand, especially to predict — that when the data reflects the formation of an algae bloom, it’s often too late to hear the water telling us we are out of balance. Perhaps the data acts like a Google knowledge engine, an algorithm anticipating the user’s thoughts when typing a partial phrase into the search bar and displaying our presumed burning questions. However, in this case, the data anticipates the thoughts of the Bay water. This concept is so significant that the scores of MD DNR sensors we use to siphon data are funded solely to predict the emergence of algae blooms in the Chesapeake Bay. “Under the Bay” observes how the Bay ebbs and flows over time to create an emerging narrative driven by what the water is enacting as an emergence. We call this murmuration messiness, listening to what the water is saying. 

Conclusion

Once we name this process of emergence, we can apply it to the other scenes in the AR project, addressing issues of differentiation and emergence not only in nature but also in creative ideas, forms, art, architecture, monuments, and also politics, protests (we discuss emerging protests including BLM, the beheaded Columbus statue ending up in the Bay, or the #metoo flocking of women in a bloom of pink hats — figure 12). This brings Brown’s term emerging strategies full circle where strategies in nature can be a model for social change. Again, our stories meander through these topics and phrases, like symbiotic waves engulfing one another leaning on influences from the Tao Te Ching as much as science. 

Figure 12. “Under the Bay” AR by Moren/Bachvaroff. Left: Scene 08 \\ Turnover [I Can’t Breathe] (still); right: Scene 07 \\ Vaccine Blooms with Pink Hats (still with UI). Courtesy of the artist.

“Under the Bay” will be exhibited at the Peale Museum, in Baltimore, Dec 14 to Feb 1, 2024, for “Chamber of Wonders”, with a public panel on Jan 18. While “Under the Bay” is freely available to download,²¹ the XR installation will include iPads interacting with paintings, drawings, and assemblages for a unique museum experience. “What is the Shape of Water?” and related works will also be on display.

“Under the Bay” Scenes 01-08 include: Origin Stories \\ Water Moving Around My Fingers \\ Chalky Faeries \\ Crooked Shelters \\ Lava Lamps in the Sky \\ Instrumental \\ Vaccine Blooms with Pink Hats \\ Turnover [I Can’t Breathe] 

Left: “Under the Bay” AR by Moren/Bachvaroff, 2022: Apple IOS; Google Play; right: “Under the Bay” (podcast), 2022: Spotify, Google Podcasts and Apple Podcasts. 
References

1 Adrienne Maree Brown, Emergent Strategy: Shaping Change, Changing Worlds, Edinburgh, AK Press, 2017.

2 Andrea Ritchie, Invisible No More: Police Violence Against Black Women and Women of Color, Beacon Press, Reprint edition, 2017. 

3 Lao-Tzu, translated by Stephen Addiss, Stanley Lombardo, introduction by Burton Watson, “Tao Te Ching”, Boulder, Hackert Publishing Company, 1993. 

4 Lynn Margulis, “On the origin of mitosing cells," Journal of Theoretical Biology, vol. 14, no. 3, 1967, 225–274.

5 Sarah P. Gibbs, “The chloroplasts of euglena may have evolved from symbiotic green algae.” Canadian Journal of Botany, vol. 56, no. 22, 1978, 2883–2889. 

6 Polikrokos harpooning prey https://www.youtube.com/watch?v=BddN1LE1YZQ 

7 Photosynthetic Fragilidium consuming a large Ceratium https://www.youtube.com/watch?time_continue=1&v=KFFsnH VlHyo&embeds_referring_euri=https%3A%2F%2Fmail.google.com%2F&embeds_referring_origin=https%3A%2F%2Fmail.g oogle.com&source_ve_path=Mjg2NjY&feature=emb_logo 

8 Dinophysis sucking the chloroplast from its prey https://www.youtube.com/watch?v=cq3SBoCHPP4&t=3s 

9 https://www.setasdesevilla.com/

10 https://www.itke.uni-stuttgart.de/research/icd-itke-research-pavilions/

11 https://www.youtube.com/watch?v=LAQwEWqg0ug\ 

12 Jane Bennett, William Connolly, “The Crumpled Handkerchief.” Time and History in Deleuze and Serres, Bloomsbury, London, UK, 2013, p.155.

13 “Under the Bay” trailer by Lisa Moren with Tsvetan Bachvaroff, 2022. https://vimeo.com/796868197 

14 Ibid, Bennett, Connolly.

15 Michael Serres, The Birth of Physics, Rowman & Littlefield International, Ltd., 2018. 

16 Theodor Schwenk, “Sensitive Chaos: The Creation of Flowing Forms in Water and Air,” Sussex, Rudolf Steiner Press, 1996, 33.

17 Ibid, Bennett, Connolly. 157.

18 Ibid. 

19 Ibid, Schwenk, p.44.

20 An eight-episode podcast version of “Under the Bay” is available on Spotify, Google, and Apple podcasts.
Links to podcasts and more information can be found on the project website: http://lisamoren.com/underthebay. 

21 Download the AR from Apple IOS or Google Play: https://apps.apple.com/app/id1641553491

https://play.google.com/store/apps/details?id=com.lisamoren.un derthebay&pli=1 

Bibliography

Jane Bennett, William Connolly, “The Crumpled Handkerchief,” Time and History in Deleuze and Serres, Bloomsbury, London, UK, 2013.

Adrienne Maree Brown, Emergent Strategy: Shaping Change, Changing Worlds, Edinburgh: AK Press, 2017.

Stefanie R. Fishel, The Microbial State: Global Thriving and the Body Politic, University of Minnesota Press, 2017.

Sarah P. Gibbs, “The chloroplasts of euglena may have evolved from symbiotic green algae,” Canadian Journal of Botany, vol. 56, no. 22, 1978, 2883–2889.

Douglas Kahn, “Energies in the Arts,” Cambridge and London, MIT Press, 2019, 2–46.

Lao-Tzu, translated by Stephen Addiss, Stanley Lombardo; introduction by Burton Watson, “Tao Te Ching”, Boulder: Hackert Publishing Company, 1993.

Lynn Sagan, “On the origin of mitosing cells,” Journal of Theoretical Biology, vol. 14, no. 3, 1967, 225–274, https://doi.org/10.1016/0022-5193(67)90079-3.

Theodor Schwenk, “Sensitive Chaos: The Creation of Flowing Forms in Water and Air," Sussex, Rudolf Steiner Press, 1996.

Lars Spuybroek, Sympathy of Things: Ruskin and the Ecology of Design, Bloomsbury Academic, 2016. 

Acknowledgements

This project is generously supported by the Saul Zaentz Innovation Fund at Johns Hopkins University and the R.W. Deutsch Foundation. This project originated during an artist-in-residence supported by the Institute of Marine and Environmental Technology (IMET), the Center for Innovation, Research, and Creativity in the Arts at the University of Maryland, Baltimore County (UMBC). We’re grateful for the support from Maryland Department of Natural Resources; John Boutsikas for his API and AR development, Danielle McPhatter and Harvestworks DigitalMedia Art Center, the IRC and CAHSS, and Antoine Cayrol of Atlas V Immersive Experiences in AR, XR, VR.

Production Team

Lisa Moren, Co-Principal Investigator, Producer, Art Director, Writer and Narrator; Dr Tsvetan Bachvaroff, Co-Principal Investigator, Marine Biologist, Researcher and Data Anaylsis; Dan Deacon, Electronic Composer; Dr. Marc Olano, Co-Principal Investigator, Lead Programmer; John Boutsikas, Programmer and Developer; Austin Samson Modeler and Animator; William Forrest, Modeler, Animator, UI and Technical Artist; Woody Lissauer, Voiceover Engineer and Male Narrator 1; Ruskin Nohe-Moren, Male Narrator 2; Aliyah Baruchin Copy Editor and Fact Checker.

Author’s Biography

Lisa Moren is a multi-disciplinary artist who works with emerging media, bio-matter, public space, AR and works-on-paper. She has exhibited her work at the Chelsea Art Museum, Creative Time, Drawing Center (New York), Cranbrook Art Museum (Michigan) and Ars Electronica (Austria), Akademie der Kunste (Germany), uShaka Museum (South Africa), and the Artists Research Network (Australia). She received the National Endowment for the Arts award, is a Fulbright Scholar; a multi-year recipient of the Maryland State Arts Council and CEC Artslink International, is a R.W. Deutsche Award recipient and a Saul Zaentz Innovation Fellow in Film and Media at Johns Hopkins University. Her writing has appeared in Performance Research; Visible Language; Inter Arts Actuel; New Media Caucus for “Algorithmic Pollution: Artists working with Dataveillance and Societies of Control” and “CYBER IN|SECURITY;” and her books on “Intermedia;” and Issues in Contemporary Theory for “Command Z: Artists Working with Phenomena and Technology.” Lisa Moren is a Professor of Visual Art at the University of Maryland Baltimore County (UMBC); is an Affiliate Faculty at the Imaging Research Center (IRC) UMBC; and taught at FAMU and AVU in Prague; and the University of California at San Diego (UCSD). 

Dr. Tsvetan Bachvaroff’s research is focused on dinoflagellate evolution with special emphasis on the parasitic dinoflagellates, using large-scale sequencing and phylogenetic methods to describe the evolutionary history of different types of genes in dinoflagellates. He uses DNA sequence analysis from data collection, assembly, annotation and phylogeny; has received numerous academic awards, including the William Trager Award from the Journal of Eukaryotic Microbiology, International Society of Protistologists, culture independent methods such as single cell PCR, sequencing, and sequence analysis; Establishing dinoflagellate cultures. He received the Marsho award, Mid-Atlantic Section of the American Society for Plant Biology, University of Maryland College Park, and the Chemistry Prize, Trinity School, New York. Tsvetan Bachvaroff received his B.A. degree from Johns Hopkins University and Ph.D. from the University of Maryland College Park. He was a Postdoctoral Fellow at the Center of Marine Biotechnology and subsequently with the Smithsonian Institution. He is an Associate Research Professor for the Institute of Marine and Environmental Technology (IMET) at The University System of Maryland. 

Power and Resistance in Digital Degrowth

Raul Nieves, Joan Soler-Adillon, Enric Mor

Universitat Oberta de Catalunya (UOC) Barcelona, Spain
dbsus4@uoc.edu, jsoleradillon@uoc.edu, emor@uoc.edu

Abstract

Degrowth is increasingly gaining attention as an alternative model to the unfolding eco-social crisis generated by industrial capitalism, though questions concerning digital technologies have yet to be addressed in degrowth research. Among the movement of the (digital) commons, whose practices complement degrowth theory, one of the research areas is the viability of such systems to release spaces from capitalism. As (digital) commons spaces frequently revert to capitalist logic, we introduce the "technological dramas" model to encompass the reciprocal and recursive technological production of political power by agonistic entities. We suggest that such a techno-political perspective could contribute to better frame degrowth-related HCI research.

Keywords

HCI, design, critical, political-economy, degrowth, commons, reification, false consciousness, co-optation, technological dramas.

DOI

10.69564/ISEA2023-20-short-Nieves-et-al-Digital-Degrowth

Post-growth

Planetary chances to restore a sustainable scenario are quickly running low. The UN questions whether we have already passed the turning point.¹ The unsustainable resource throughput promoted by the industrial model is (eventually if not yet) incompatible with the materially limited planetary boundaries.² Over the last decades, scholars and practitioners have started to question perpetual growth, a pivotal paradigm in industrial capitalism. Resource depletion, claims Turiel, precludes the default problem-solving conveyed through technological innovation, development, and deployment.³

The Degrowth project entwines a set of proposals to prioritize democratic, social and ecological justice in pursuit of well-being over economic growth.⁴ Such an economic, political, and social program conflicts with the capitalist political economy: sustainable growth, for degrowth advocates, is an oxymoron.⁵ Degrowth provides a political-economy frame to rethink human-computer interaction (HCI) theoretical tenets and practices as key elements to reconfigure the relationship between digital technological activity and eco-social effects.

Political-economy in HCI

The research field of HCI brings together technological and human-related disciplines to improve the interplay between users and machines.⁶ Despite HCI’s, deeply entrenched, industrial root logic, examining and questioning this very logic, especially recently, has connected many authors in HCI sub-communities.⁷ Among them: reflexive HCI⁸, humanistic HCI⁹, or critical HCI¹⁰. Dourish examines the early Sustainable HCI research production to expose the risks of naturalizing capitalist assumptions and suggests instead broadening the theoretical approaches in use. The author advocates “dismantle design as an anti-politics machine.”¹¹ Hence, Ekbia and Nardi urge to incorporate political economy in the analysis of HCI’s design and practices.¹²

Problematic capitalist core mechanisms, especially productivism and consumerism, take specific forms in HCI- related practices. Ekbia and Nardi quote Marx “production creates the consumer” relating consumerism to the paradoxical production of the “user.”¹² This “designed user” is central to the anthropocentric formulation of problems. Problems, again paradoxically, are produced to fit feasible technological solutions, feeding the mechanism known as techno-solutionism.¹³ This back-feeding entanglement is key for the industrial production system. It underpins perpetual growth while consuming resources and expelling negative externalities, producing new problems.

These analyses within HCI resonate with the degrowth discourse. Despite some proposals (i.e., see ¹⁴ as post-anthropocentrism, or ¹⁵ as post-techno-solutionism) we suggest it’s necessary to conduct research on how to consistently translate this critique into impact-aware HCI practice. Thus, we turn towards degrowth-aligned practices to foreground strategic political economy issues arising from the struggle to transition outside capitalism.

Challenges in (digital) commoning practices

According to Helfrich and Bollier degrowth and the commons movement are complementary to each other.¹⁶ The commons don’t rely on economic growth and make compatible environmental and social justice. These authors define the commons in terms of politics and economics as “a vast array of self-provisioning and governance systems that flourish mainly outside of both the market and the State.” 

While degrowth frames the subject of critique, the commons exhibit social, political, and economic forms of actualization based on the social practices of commoning: the stewardship practices that a community employs to manage shared things (virtually anything) in common.

Digitally enabled communities of practice, spawned by the emergence of the internet, have long been at the center of commons research. Peer-to-peer (P2P) networking architectures fostered a productive model labeled as commons-based peer production by Benkler.¹⁷ According to P2P advocate Bauwens, this model represents a generative alternative in front of the extractivist modes of capitalist production.¹⁸ Fuster defines the digital commons as online communities which share non-exclusive co-created digital resources.¹⁶

One of the main research concerns in the commons is the relationship between the state, capitalism, and alternative spaces of resistance. The cartography of such borders seeks to discern the strategic practices to release and gain back spaces from capitalist control. The commons aim at releasing spaces from capitalism and confronting the enclosing of old and new commons. Such spaces risk falling back to capitalist logic, a process referred to as co-optation. While also discussed as “colonization by capital”¹⁹, “assimilation”²⁰, “incorporation”²¹, “transvestment”²², or “unwanted corporate appropriation”²³, among other terms, co-optation is ubiquitous in the commons theory but it is scarcely being examined in depth.

Among the few commons theory authors that specifically examine co-optation, we find De Angelis, Caffentzis, and Federici. According to De Angelis, capitalism is about to face a social and ecological crisis and will likely have to leverage or promote the commons to help manage the devastation. ²⁴ De Angelis claims: “struggles [...] can be absorbed and become part of the system (co-opted), thus renewing it and sustaining it.” As the logic of the market becomes counterproductive even from the viewpoint of capital accumulation, precluding the cooperation necessary for an efficient system of production, Caffentzis and Federici point to "the danger that 'commons' may be co-opted to provide low-cost forms of reproduction."²⁵

Kostakis et al. review and discuss recent criticisms of peer production, classifying some of them as co-optation. The study examines whether digital peer production could be emancipatory or instead become part of capitalism. On one hand, the injection of funds in free open-source software (FOSS) projects and the multi-million-dollar purchases of FOSS companies increase the risks of appropriation of the commons by corporate interests. On the other hand, commons’ pro bono production is monetized and exploited by market agents for profit extraction.²⁶

To inquire into the origins of digital co-optation, we focus here on the ideas of Osseewaarde et al.²⁷ The authors analyze how digital commoners recurrently transition through alternative spaces, as they eventually get co-opted. Commoning’s essence opposes the technological rationality of formalization, standardization, and quantification, yet the emergence of such spaces relies on technological innovation which is fostered by the growth-oriented efficiency ethos promoted by neo-liberalism. This contradiction, the authors argue, results in a perpetuating illusion, a form of false consciousness, which is rooted in cynicism.

Ossewarde et al. claim that current standard technologies “are highly opaque because they are often implicit and part of a formalized design for digital interaction that is in itself an arrangement of ‘false consciousness’.”²⁷ Hence, digital commoning (the generation of contents, but especially of infrastructures) fails to resist co-optation, supporting capitalist expansion as De Angelis, Cafentzis, and Federici denote.

While Ekbia et al. point to reification (the assumption of systemic concepts that are actually socially constructed as inescapable and natural, like the capitalist market) as a hindrance to emancipatory HCI²⁸, Dourish and Ossewaarde et al. refer to the studies on the reified construction of false consciousness by Lukacs. Ossewaarde et al. argue that such cynicism cannot be overcome via ideology critique, but through technology critique when it "is translated into post-capitalist acts of resistance to the dominating technology design." De Angelis asks “Isn’t this co-evolution between struggle and capital development really inherently with no end?”²⁹

Introducing a technological power construction model in HCI

Reified notions embed the HCI practices with false consciousness, rendering alternatives prone to get co-opted by capitalism. To further comprehend this dynamic, we suggest introducing “technological dramas,” a framework by the anthropologist of technology Bryan Pfaffenberger.³⁰ As technological activity presents an opportunity to embed political values, Pfaffenberger examines how power and resistance are constructed through the reciprocal and recursive shaping of artifacts and values in the design process, which later spread in society.

Pfaffenberger stresses the relevance of myth, ritual, and context in the understanding of the political dimensions of technological activity. Myths are deployed to suspend skepticism, rituals are associated with controlled environments produced to pattern human actions, and social contexts are fabricated in parallel. In this model, technological activity is analyzed as a process of technological communication: “a technological drama is a discourse of technological statements and counter-statements.

The model describes three processes that can occur linearly, or under different permutations, in the construction of politics by technological means:

● Regularization occurs when a design entity (usually part of the establishment) “creates, appropriates, or modifies a technological production process, artifact, user activity, or system in such a way that some of its technical features embody a political aim”. 

● Adjustment takes shape when impact entities engage in control and alteration strategies. These strategies attempt to counter the discursively regulated social contexts that regularization creates, pursuing to counter their effects. This process can lead to technological appropriation

● Reconstitution materializes when impact entities “try to reverse the implications of a technology through a symbolic inversion process,” labeled as antisignification by Pfaffenberger. This process can produce, as in some forms of adjustment, appropriation. It can also result in the fabrication of counterartifacts, “which embody features believed to negate or reverse the political implications of the dominant system.

Pfaffenberger employs the term reintegration to refer to co-optation processes: “the response made by the agents of regularization to the new, problematic counterartifacts. Its goal is to gain control over these artifacts by bringing them back into the controlled and ordered space of regularization.” According to the author, some forms of adjustment and reconstitution stages (resistance processes with relevant degrees of technical intervention) are prone to co-optation. While co-optation has been previously discussed, it is considered by this model in a wide and complex techno-political dynamic.

Besides the three above-mentioned processes, Pfaffenberguer introduces a fourth: designification takes place when the link between technological activity and social meaning-producing discourse dims. “The artifacts, their contexts, and our social behaviors remain; they become taken for granted, routine, and part of the natural attitude of everyday life.” According to Pfaffenberger, this is the stage where technological activity achieves the greatest social penetration. We suggest a connection between designification and the previously discussed process of reification, here particularly referred to in techno-political, rather than political economy, terms.

Conclusions

In front of collapse, the political and economic project of degrowth offers an alternative coexistence formula. The HCI community is already approaching the degrowth frame in order to redefine its tenets and practices, and political economy analysis has already been adopted by HCI authors to examine reified notions. To expand HCI research we have examined digital commoning challenges to release spaces from capitalist logic. Reification, pointed out already by HCI authors, false consciousness, and co-optation processes are discussed by commons’ authors as strategic issues concerning the transition to post-growth alternative political economies.

We suggest the relevance of this model in analyzing and prospecting the construction of political power, in its different stages, in order to deploy strategic practices of transition in front of a ravaging capitalism. Especially due to the systemic view this model offers to dissect and relate processes like designification, co-optation, and possibly others, as stages of the permanent struggle through technological activity. We also suggest leveraging this model as a tool to complement speculative, adversarial, fictional, strategic, or transitional design techniques.³¹-³⁵ Hence, we expect this contribution to help HCI researchers better frame degrowth-related practices and research contributions.

This article is part of the R+D+i project PID2021-128875NA-I00, funded by MCIN/AEI/10.13039/501100011033/ "ERDF A way of making Europe”.

References

1 Matt McGrath, "Climate change: IPCC report warns of ‘irreversible’ impacts of global warming", Retrieved December 7, 2022, from https://www.bbc.com/news/science-environment-60525591.

2 Nicholas Georgescu-Roegen, The Entropy Law and the Economic Process, Cambridge, MA, Harvard University Press, 1971.

3 Antonio Turiel, Petrocalipsis, Alfabeto, 2020.

4 Matthias Schmelzer, Aaron Vansintjan, Andrea Vetter, The Future Is Degrowth: A Guide to a World beyond Capitalism, Verso Books, 2022.

5 Jason Hickel, Giorgos Kallis, "Is Green Growth Possible?", New Political Economy 25 (4), 2020, 469–486.

6 Paul Dourish, Where the Action Is: The Foundations of Embodied Interaction, Cambridge, MA, USA, MIT Press, 2001.

7 Raul Nieves, Enric Mor, Joan Soler-Adillon, "Decoupling design from the industrial paradigm," In Proceedings of ISEA2022, the 27th International Symposium on Electronic Art, Barcelona.

8 Phoebe Sengers, John McCarthy, Paul Dourish, "Reflective HCI: articulating an agenda for critical practice," In CHI ’06 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’06), 1683–1686.

9 Jeffrey Bardzell, Shaowen Bardzell, Humanistic HCI, San Rafael, California: Morgan & Claypool, 2015.

10 Jeffrey Bardzell, Shaowen Bardzell, Mark Blythe (eds.). Critical Theory and Interaction Design, Cambridge, MA, USA, MIT Press, 2018.

11 Paul Dourish, "HCI and environmental sustainability: the politics of design and the design of politics," In Proceedings of the 8th ACM Conference on Designing Interactive Systems (DIS ’10), 1–10.

12 Hamid Ekbia, Bonnie Nardi. "Social Inequality and HCI: The View from Political Economy," In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16), 4997–5002.

13 Evgeny Morozov, To Save Everything, Click Here: The Folly of Technological Solutionism, New York, Public Affairs, 2014.

14 Bill Tomlinson, Bonnie Nardi, Daniel Stokols, Ankita Raturi, "Ecosystemas: Representing Ecosystem Impacts in Design," In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1–10.

15 Eric P.S. Baumer, M. Six Silberman, "When the implication is not to design (technology)," In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11), 2271–2274.

16 Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, Degrowth: A Vocabulary for a New Era, Routledge, 2014.

17 Yochai Benkler, The Wealth of Networks: How Social Production Transforms Markets and Freedom, New Haven, Conn., Yale University Press, 2007.

18 Michel Bauwens, Vasilis Kostakis, Alex Pazaitis, Peer to Peer: The Commons Manifesto, University of Westminster Press, 2019.

19 Christian Fuchs, "The Digital Commons and the Digital Public Sphere: How to Advance Digital Democracy Today," Westminster Papers in Communication and Culture 16 (1), 2021.

20 Maurizio Teli, Marcus Foth, Mariacristina Sciannamblo, Irina Anastasiu, Peter Lyle, "Tales of Institutioning and Commoning: Participatory Design Processes with a Strategic and Tactical Perspective," In Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 1 (PDC ’20), 159–171.

21 Benjamin Birkinbine, Incorporating the Digital Commons: Corporate Involvement in Free and Open Source Software, University of Westminster Press, 2020.

22 Michel Bauwens, Alekos Pantazis, "The ecosystem of commons-based peer production and its transformative dynamics," The Sociological Review 66 (2), 2018, 302–319.

23 Benjamin J. Birkinbine, "Conflict in the Commons: Towards a Political Economy of Corporate Involvement in Free and Open Source Software," The Political Economy of Communication 2 (2), 2015.

24 Massimo De Angelis, "Crises, Capital and Co-optation: Does Capital Need a Commons Fix? | The Wealth of the Commons," Retrieved May 9, 2022, from http://wealthofthecommons.org/essay/crises-capital-and-cooptation-does-capital-need-commons-fix.

25 George Caffentzis, Silvia Federici, "Commons against and beyond capitalism," Community Development Journal 49 (suppl_1), 2014.

26 Vasilis Kostakis, Vangelis Vragoteris, Indra Lal Acharja, "Can peer production democratize technology and society? A critical review of the critiques." Futures 131, 2021, 102760.

27 Marinus Ossewaarde, Wessel Reijers, "The illusion of the digital commons: ‘False consciousness’ in online alternative economies," Organization 24 (5), 2017, 609–628.

28 Hamid Ekbia, Bonnie Nardi, "The political economy of computing: the elephant in the HCI room," Interactions 22 (6), 2015, 46–49.

29 Massimo De Angelis, "Crises, Movements and Commons," Borderlands 11, 2012.

30 Bryan Pfaffenberger, "Technological Dramas," Science, Technology, & Human Values 17 (3), 1992, 282–312.

31 James Pierce, "In Tension with Progression: Grasping the Frictional Tendencies of Speculative, Critical, and other Alternative Designs." In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21), 1–19.

32 Carl DiSalvo, "Adversarial Design as Inquiry and Practice," In Adversarial Design, MIT Press, 2012, 115–125.

33 Matt Malpass, Critical Design in Context: History, Theory, and Practice, London, New York, 2017.

34 Anna Meroni, "Strategic design: where are we now? Reflection around the foundations of a recent discipline," Strategic Design Research Journal 1 (1), 2008, 31–28.

35 Terry Irwin, "Transition Design: A Proposal for a New Area of Design Practice, Study, and Research," Design and Culture 7 (2), 2015, 229–246

Motherplants: Mycelium Network and Artistic Research

Julien Ottavi, Jenny Pickett

Association APO33, Madlab - Cyprus University of Technology Nantes, France - Limassol, Cyprus
julien@apo33.org, jennypickett@apo33.org

Abstract

In 2015, we embarked on an artistic research project involving organic organisms such as plants and fungi, focusing on their role in the processes of recycling of e-waste. Motherplant explores how these systems create feedback audio signal transmission through the interplay of moisture and decomposition of the circuit boards. Specifically, it harnesses the natural decomposition process to transmit audio signals by converting the soil's acidity and electronic compost into electrical signals.

Keywords

Artistic research project, electronic compost, mycelium networks, soil acidity, symbiosis, mutation of technologies, e-waste.

DOI

10.69564/ISEA2023-21-short-Ottavi-et-al-Motherplants

Introduction

The surge in global electronic waste (e-waste) volumes, as reported by the Global E-waste Statistics Partnership (GESP), is cited by the World Health Organization (WHO). Between 2014 and 2019, there was a substantial increase of 21% in e-waste generation, resulting in a staggering production of 53.6 million metric tonnes of e-waste during the latter year. "For perspective, [in the year 2020] e-waste weighed as much as 350 cruise ships placed end to end to form a line 125 km long. This growth is projected to continue as the use of computers, mobile phones and other electronics continues to expand, alongside their rapid obsolescence." (Johnson, 2021) The repercussions of this escalating e-waste problem are dire, encompassing adverse effects on both human health and ecosystems. The e-waste stream comprises a multitude of toxic substances, ranging from neurotoxins to heavy metals, which pose significant risks (Jain et al., 2023). These hazardous constituents have the potential to inflict severe harm upon individuals and the environment, underscoring the urgency of addressing and mitigating the detrimental consequences associated with mounting e-waste volumes.

Figure 1: Motherplants-installation view - 2018. ©APO33-Nantes

This project revolves around the intriguing intersections of mycelium networks, audio diffusion, and the creation of novel computing experiences. Motherplant engages with the notions of symbiosis and mutation of existing technologies to explore new modes of interaction with recycling, art research and implementation of new network models based on mycelium communication cooperation systems.

Motherplant : Computational Mycelium Recycling Network

Using dead and discarded motherboards extracted from obsolete computer systems, we attempted a transformative process with the aim of repurposing these printed circuit boards (PCBs) into "micro-farm" environments. The primary objective of Motherplant encompassed three key facets: firstly, to experiment with sustainable recycling of all motherboard components; secondly, to harness their inherent properties for the generation of electrical current; and lastly, to contribute to the development of an alternative open system designed to facilitate the exchange of computational data. This fusion of electronic and organic realms introduces a harmonious synergy, wherein the motherboard's original purpose converges with the natural processes of growth, decomposition, and nutrient exchange.

Figure 2: Motherplants-schematics 2023. ©APO33-Nantes

Re-creation of the fungal network as an electronic mutation

In Motherplant we are trying to create a novel electronic circuitry that mutates in tandem with a primitive fungal network. It is important to clarify that our fungal network is a simplified "primitive" representation, distinct from the extensive rhizomatic networks deployed by mature fungi across vast expanses of soil, connecting plants and trees. We acknowledge that nature excels in this regard, and our intention is not to replicate nature's proficiency but rather to cultivate and orchestrate a symbiotic relationship between electronic circuitry in a localised fungal network.

This begins with a few spores settling down on a nutrient-rich surface. When these spores wake up in close proximity to one another; they initiate germination at approximately the same time, giving rise to thread-like cellular structures known as hyphae, which extend outward at comparable rates. This unique fusion of electronics, powered by solar energy and moisture-rich soil infused with (DIY) mycelium growth, establishes an intricate information superhighway. As an information superhighway the interactions between a large, diverse population of individuals speeds up. It allows entities who may be separated to communicate and help each other out. It also allows them to commit new forms of communication.

This symbiotic relationship between electronic circuitry and a fungal network not only bears relevance to the creation of a new information exchange paradigm but also holds significant potential in the context of e-waste recycling. Fungi possess the ability to break down and transform a wide range of organic and inorganic substances, pesticides, hydrocarbons and heavy metals, making them valuable agents for cleaning up contaminated environments (Tomer et al., 2021). By virtue of fungal bioremediation, the fungal network may play a pivotal role in the sustainable degradation and recycling of electronic components.

Schematics for Motherplant

This iteration of Motherplant comprises a redundant, nonfunctional motherboard as its foundational structure, upon which mycelium is cultivated within a composite medium consisting of straw, soil, and integrated circuitry. The degradation of the motherboard’s components initiates an acidification process within the medium, subsequently generating electrical signals. These electrical signals function akin to a battery, delivering an approximate output of 1.5 volts, which in turn powers an audio circuit. The frequencies generated by this audio circuit exhibit variability contingent upon the dynamic activity of the mycelium and the pH levels of the surrounding soil.

Renaturing the Motherboard

The Motherplant concept can be traced back to an earlier experiment initiated at APO33 in Nantes 2015 and presented during NEAR #1 as part of Nantes Digital Week. During this meeting, the growing need to address e-waste issues was discussed alongside concerns about the exploitation of workers, recycling components without protective clothing or in adequate conditions (Ottavi and Pickett, 2015). These first experiments went on to be presented at the Nomad Village during COP21 in Paris, later on that year (Ottavi, 2015). Electronic waste is toxic and hazardous to human health, PCB's are hard to recycle and demand a lot of energy and sometimes the scramble for precious metals and minerals contained therein, sacrifices members of our human community (Wittsiepe et al., 2015; Beaumont, 2019). Usually, this heavy cost is hidden from our sleek conscientious recycling certificates. Yet it could take decades, sometimes centuries, before some of these components will be totally recycled. In 2019 Europe ranked first worldwide in terms of e-waste generation per capita, with 16.2 kg per capita (Forti et al., 2020).

With Motherplant by employing plants, flowers, and fungal organisms as instrumental agents in the recycling of electronic waste, we set about composting the motherboard. The heart of any home computer or laptop, the Motherboard is a concealed symbol of this cyclical consumption and our relentless obsessions for digital speed. With Motherplant we sought to counteract this hidden world of consumer electronics recycling and make visible the complexity of such a task if left to nature, as well as the sheer disregard for the problem by the shipping of our waste elsewhere. The idea of Motherplant was to produce electricity to feed electronic circuits, so that, in turn, they endlessly sustain a sound installation. This undertaking represented a confluence of art, ecology, recycling practices and the exploration of alternative energy models. The shape and layout of a motherboard is called the form factor. The form factor affects where individual components go and the shape of the computer's case. We design Motherplant in such a way that the classical function of the motherboard decomposes to produce a new kind of computer. One that won’t consume electricity anymore, but instead will become a receptacle for plants and spores to grow.

Mycelium computation: a new paradigm in bioart and sound art research

This new kind of Mycelium computation is about to begin, as we search for quantum processors, we are looking for Mycelium based hijacked computer circuits. The spore computation is not just another form of computer research in the tradition of engineering production, it's an all-new vision about computers. In 2018, Professor Andrew Adamatzky, Director of the Unconventional Computing Laboratory at the University of the West of England in Bristol, UK, wrote in his article "Towards a fungal computer", how a series of scoping experiments, provided evidence that electrical activity recorded on fruits could potentially serve as a dependable indicator of the fungi's reaction to thermal and chemical stimulation. (Adamatzky, 2018) It diverges from the prevailing landscape of technological industrial mass production, presenting a concept characterised by fragility, inherent hazards, and a multitude of uncharted pathways, and prototypes of computing mycelium bound composites. (Roberts and Adamatzky, 2022).

A mélange of Bioart (art using fungal bioremediation and technology) with sound art (sound as a profound production of vibration) serves as a conduit for the cultivation of exchanges and ideas within a networked context. Within the domain of bioart, the convergence of innovative gardening techniques and the exploration of mycelium-based experiences culminates in the facilitation of multifaceted networks characterised by a wide array of functionalities that will serve multiple purposes such as recycling, production of energy, circulation of electricity but also production of sound waves, feedback transmission and autonomous art installation.

From the perspective of sound-ecology, listening to the world holds significant importance in numerous practices. However, in the case of Motherplant, it transcends mere observation of nature; it involves a collaboration with nature to generate a sound installation on a minute scale, characterised by low impedance and microsounds, all within the dimensions of the sculpture itself. In terms of sound levels, our focus aligns more closely with the realm of ants and insects than that of elephants or storms. This reveals the potential inherent in the amalgamation of nature, mycelium, and soil within our artistic patch.

Art frequently detaches itself from its own reality, often transported and displayed within galleries or dedicated venues, creating a sense of isolation. In contrast, Motherplant seeks to recreate a natural context intricately linked to the artwork itself, ultimately transforming the context, nature, environment, and aesthetics into the art itself. Sound plays an integral role in the production process but whilst it doesn't represent the final objective, it is a vital component within the broader framework. The various iterations of the art piece over the last 8 years, also contribute to shaping the sculptural essence of Motherplant. The process that we describe here and the relation between nature, technologies and production of energy is not separated from the poetic facets of our artistic research.

Recycling plastic and other dirty electronics to produce low currency (power)

Beyond the artistic sculpture of the Motherplant, we had assigned to this project two other important research perspectives. Whilst these are not the main goals, they resonate with the other aspects (art, sculpture, sound, visuals...). The first enables us to think about and experiment with the recycling of electronic wastes with the Motherboard. In this context, the concept of the "mother" assumes a central and symbolic role, representing the origin from which life emanates. This archetype of the mother extends beyond the human realm to encompass broader notions such as Mother Earth and Mother Plant, all of which evoke contemplation of the nurturing and life-giving aspects associated with motherhood. By integrating the concept of the "mother" into the project's narrative, we establish a profound connection between technology, nature, and the cycle of life. Which underscores the intricate interplay between electronic devices and the environment. How can we care for electronic waste management and recycling? Motherplant embraces maternal sustainability and environmental awareness, towards recycling e-waste and engendering a new computation network. Secondly, recycling e-waste costs are very high compared to disposing them in landfills, mainly because it often contains hazardous materials that need to be dismantled and processed properly. Also, the lack of recycling infrastructure poses problems to handle the large volume of e-waste generated each year. Finally, due to the lack of awareness from the public, most people have no idea of the importance of recycling their computers and e-waste. This lack of awareness ends up in huge amounts of illegal dumping or in the wrong landfills and incinerators producing more pollution but also exposing people to hazardous materials.

We decided, as artists working in electronic arts and media, to engage in this protracted undertaking of the project Motherplant, and develop our own way of experimenting with living networks. Rather than adhering to the conventional and often inefficient approach of classical digital art research, which typically involves a relentless cycle of producing more work and tirelessly pursuing galleries and festivals for exhibitions, we have chosen a different path. We consider this process in the broader context of "decroissance" or degrowth, lowering our consumption, decelerating our expectations of results, and embracing the temporality associated with decomposition. The philosophy of degrowth posits that the relentless pursuit of economic growth is inherently unsustainable in the long term. It contends that, in order to safeguard the environment and enhance societal well-being, it is imperative to curtail economic activities and their associated impacts.

The other part of this research centres on the production of electricity. While it's important to note that this production is characterised by low power output, it has nonetheless proven to be a sustainable source of electrical current over the past few years, requiring no additional input other than the periodic watering of the Motherplant. This enduring supply of electricity has so far facilitated a continuous eight-year run of our sound installation.

The motherboard, serving as the fundamental underpinning of our installation, adheres to a decompositional timeframe, thereby providing a robust scaffold for the continuous provision of electrical current. This enduring electrical supply, in a reciprocal fashion, sustains the seamless operation of our sound installation, ensuring its uninterrupted functionality. This approach to both energy generation and artistic expression accentuates the intricate interplay amongst technology, ecology, and the dimension of time.

Figure 3: Motherplant : Close-up 2018. ©APO33-Nantes

Documentation

View the 2015 video on Climate Solutions: MOTHERPLANT here: https://www.youtube.com/watch?v=CSUeSdR6n1Q. View the 2018 video on Motherplants here: https://vimeo.com/301201120

References
Bibliograpy

Andrew Adamatzky, “Towards Fungal Computer.” Interface Focus 8 (6), 2018, 20180029, https://doi.org/10.1098/rsfs.2018.0029.

Peter Beaumont, “Rotten Eggs: E-Waste from Europe Poisons Ghana’s Food Chain.” The Guardian, April 24, 2019. https://www.theguardian.com/global-development/2019/apr/24/rotten-chicken-eggs-e-waste-from-europe-poisons-ghana-food-chain-agbogbloshie-accra.

Vanessa Forti, Peter Baldé, Ruediger Kuehr, Garam Bel, S Adrian, M Drisse, Y Cheng, et al. 2020, “Quantities, Flows, and the Circular Economy Potential the Global E-Waste Monitor, 2020”, https://www.itu.int/en/ITU-D/Environment/Documents/Toolbox/GEM_2020_def.pdf.

Jain, Muskan, Depak Kumar, Jyoti Chaudhary, Sudesh Kumar, Sheetal Sharma, Ajay Singh Verma, “Review on E-Waste Management and Its Impact on the Environment and Society”, Waste Management Bulletin 1 (3), 2023, 34–44, https://doi.org/10.1016/j.wmb.2023.06.004.

Johnson, Ceridwen, “Soaring E-Waste Affects the Health of Millions of Children, WHO Warns”, World Health Organization, World Health Organization, 2021, https://www.who.int/news/item/15-06-2021-soaring-e-waste-affects-the-health-of-millions-of-children-who-warns.

Adam Minter, “The Burning Truth behind an E-Waste Dump in Africa”, Smithsonian, Smithsonian.com, January 13, 2016, https://www.smithsonianmag.com/science-nature/burning-truth-behind-e-waste-dump-africa-180957597/.

Julien Ottavi, “Climate Solutions: MOTHERPLANT”, www.youtube.com, Solution Zone TV, December 6, 2015, https://www.youtube.com/watch?v=CSUeSdR6n1Q.

Julien Ottavi, and Jenny Pickett, “Near-1 [[NEAR] Nantes Electronic Art Rencontre]”, Apo33.org, September 22, 2015, https://apo33.org/near/doku.php?id=near-1.

Julien Ottavi, Jenny Pickett, 2018, “Motherplants - Jenny Pickett & Julien Ottavi”, vimeo.com, 2018, https://vimeo.com/301201120.

Roberts Nic, Andrew Adamatzky, “Mining Logical Circuits in Fungi”, Scientific Reports 12 (1), 2022, https://doi.org/10.1038/s41598-022-20080-3.

Ajay Tomer, Ramji Singh, Saurabh Kumar Singh, S. A. Dwivedi, Chilkuri Udaykiran Reddy, Malavika Ram Amanthra Keloth, and Riya Rachel, “Role of Fungi in Bioremediation and Environmental Sustainability”, Fungal Biology, 2021, 187–200, https://doi.org/10.1007/978-3-030-54422-5_8.

Jürgen Wittsiepe, N. Fobil Julius, Till Holger, Gerd-Dieter Burchard, Michael Wilhelm, and Torsten Feldt, “Levels of Polychlorinated Dibenzo-p-Dioxins, Dibenzofurans (PCDD/Fs) and Biphenyls (PCBs) in Blood of Informal E-Waste Recycling Workers from Agbogbloshie, Ghana, and Controls.” Environment International 79, June 2015, 65–73, https://doi.org/10.1016/j.envint.2015.03.008.

Authors Biographies

Julien Ottavi, holds a Doctor of Arts from the Université de Lorraine. His practice covers sound, intermedia, and digital network arts. He is the founding member and artistic director of association APO33, based in Nantes. APO33 has produced numerous ground-breaking explorations in experimental music and electronic arts, and collaborated with artists and musicians across the world.

Jenny Pickett is a member of the APO33, where she works on projects ranging from interactive installations to experimental music and performance to international collaboration in art and technology. She is currently a PhD Candidate at Cyprus University of Technology where she is attached to the Media Arts and Design Research Lab (MADlab), she is also the Vice Chair for Toolkit of Care a EU COST funded cooperation, as well as an associate lecturer at the Nantes School of Architecture (ENSA). 

Life-as-it-could-be, Symbiosis in Interspecifics’ Codex Virtualis_Genesis

Claudia Costa Pederson

Wichita State University Kansas, U.S.A. ccp9@cornell.edu

Abstract

This paper discusses Codex Virtualis_Genesis (2020-2022) as an artistic engagement of Artificial Life (AL) that explores the nature of informational life as a symbiotic process. Created by Mexico City-based transnational collective Interspecifics (INT), this work follows on from the expanded notions of life central to the field of artificial life, including organic, inorganic, material, and virtual forms. According to its founder, Christopher Langton, “there is nothing... that restricts biology to carbon-based life; it is simply the only kind of life that has been available to study.” From this perspective, Langton proposed in the late 1990s that AL be dedicated, as he put it, to speculating beyond "life-as-we-know-it" into the realm of "life-as-it-could-be.” This discussion examines Codex Virtualis_Genesis in light of Langton’s proposal as a speculative inquiry into a symbiotic view of life, and as well in contrast to notions of artificial life art as a predominantly technophilic practice. Instead connected to the speculative imagination, the synthetic life forms of Codex Virtualis_Genesis offer a glimpse into life otherwise: as an interspecific relation.

Keywords

Artificial Life, Artificial Life Art, Mexico, Symbiosis, Symbiocene.

DOI

10.69564/ISEA2023-22-short-Costa-Pederson-Life-as-it-could-be

Codex Virtualis_Genesis

Interspecifics’ Codex Virtualis_Genesis (2021-2022) was created for the SETI x AI Lab residency, an art and science program started in 2018 by the Ars Electronica Futurelab in Linz, Austria in collaboration with the AI Lab (European ARTificial Intelligence Lab) and the SETI Institute in Mountain View, CA.¹ The program calls on international artists working in digital art to create works “that explore the evolution of life on earth and universe and critically reflect on anthropocentric world views.”²

Interspecifics won the 2021 Ars’ AI Lab residency call with their proposal to create Codex Virtualis_Genesis as an evolving taxonomic collection of hybrid AL forms.³ In practice, to create these forms’ shape, color, texture, morphological structures and behavioral dynamics, Interspecifics used a 2D cellular automata system (a planar cell-based computer graphic system that displays emergent global behaviors), and two sets of images of living symbiotic organisms. One set is microscope images (microscopy) of microorganisms living in water bodies that were posted by various scientists on their Instagram accounts.⁴ Another set was digital photographs of colonies of fungi and bacteria (8,000 distinct species) made available online by researchers at the National Agriculture and Food Research Organization, NARO, in Japan. Some of the latter organisms are also known as extremophiles because they are metabolically and biochemically adaptable to harsh environmental conditions such as broad swaths of temperature, pH, pressure, radiation, salinity, energy, and nutrient limitation. They have therefore been historically used in astrobiology experiments, including most recently conducted at the ISS (International Space Station).(1)

The resulting images resemble plant and animal-like microorganisms in a wide range of colors and shapes (Figure 1a and 1b). In a gallery setting, Codex Virtualis_Genesis is shown as a screen-based installation that as well as displaying selected images of said organisms, also displays images of aspects of the selection process and evolutionary procedures (Figure 2).

1a and 1b.Symbionts generated for CodexVirtualis_Genesis, Ars Electronica2021. Courtesy of Interspecifics.
1b. Symbiontsgenerated for CodexVirtualis_Genesis, Ars Electronica2021. Courtesy of Interspecifics.
2. CodexVirtualis_Genesis, Ars Electronica2021. Photo credit: Robert Bauernhansl.

Additionally, in the printed materials provided to visitors at exhibitions, Interspecifics describes Codex Virtualis_Genesis as titled in a loose nod to Mesoamerican codices and based on “the symbiotic narrative of evolution.”⁵ Specifically this refers to endosymbiosis, a process of symbiotic life evolution proposed by the biologist Lynn Margulis in the late 1960s. Endosymbiosis foregrounds symbiotic mergers (pre-nucleic gene transfer between bacteria and other organisms) as the main sources of acquisition of new genomes and thus also of speciation (i.e., the formation of new and distinct species in the course of evolution). From this perspective, random mutation is of marginal consequence as the driver of evolution and so consequently Margulis argued that Darwinian evolutionary theory based solely on competition (i.e., the survival of the fittest) is incomplete. According to Margulis, “Life did not take over the globe by combat, but by networking. Life forms multiplied and complexified by co-opting others, not just by killing them.”⁶

Interspecifics’ Codex Virtualis_Genesis brings Langton and Margulis in dialogue with an eye to complement their respective perspectives on the nature of life. In this interest, it extends Margulis’ restricted notion of life (as carbon-based) to as well, as proposed by Langton, include synthetic forms. Conversely, by linking AL and symbiosis it also clarifies Langton’s speculative vision of AL (i.e., as “life-as-it-could-be”). (2) According to a member of Interspecifics, Leslie Garcia, this dialogue speaks to the project’s overarching concept: namely to simultaneously call attention to our “entanglements” and to stimulate speculative imagings and imaginaries of “non-anthropocentric and non-anthropomorphic but more organicist, metabolic models of life.”⁷

Cooperate!

Codex Virtualis_Genesis not only demonstrates life as a mutualistic relation on screen but is itself created with an eye for developing art and science as mutualistic practices: in Interspecifics’ own words, “to produce in terms of social inclusion, cross-disciplinary practices, and open knowledge.”⁸ To this concern, the project contributes to broadening access to artificial life techniques and consonantly increasing diversity in artificial life art as an area of the global digital arts. As aforementioned, Interspecifics drew on freely available images shared by scientists and scientific communities online and social media to create Codex Virtualis_Genesis’ symbiont organisms. Additionally, Lenia, a readily available system of cellular automata or artificial life, is integral to the project. Created by the Chinese software engineer Bert Wang-Chak Chan in 2015, this system is a variation on The Game of Life (GoL) devised by the British mathematician John Horton Conway in 1970, which was originally played on a simple square grid without the aid of computers. GoL is a class of mathematical phenomena termed cellular automata by John von Neumann in his discussion of self-reproducing machines in the mid-1940s.(3) Both GoL and Lenia are AL systems that focus on life not as a material substrate but as a form of organization and behavior that is akin to the distributed processes characteristic of the functioning of living organisms. Alternatively, these systems can also be conceived as no-player games; meaning that after the initial configuration their evolution does not require further input. The point of cellular automata like GoL and Lenia is in short to demonstrate the emergence of the variety and complexity of behaviors from a few simple rules. These systems differ insofar the sophistication of their graphics. In comparison to GoL’s square graphics, which now look outdated, Lenia’s fuzzy, smooth, and colorful patterns look state of the art. More significantly for artists broadly, as art funding becomes ever more restricted, and in particular artists that as Interspecifics puts it, work via-á-vis the context of “precarity” in the global South, Lenia is as an open-source art and science project. Still requiring some technological expertise, it nevertheless is in theory accessible to a global public.

Already, in an updated version of his 1989 article on Artificial Life, Langton speculated that the future evolution of AL would depend on scientists’ imitating nature’s cooperative behaviors (so he urged biologists and computer technicians to follow its example). (4)Artificial life art, as A-life artist Simon Penny notes, is similarly an area of digital arts with a high degree of interdisciplinarians and/or collaborative partnerships between artists and technicians because artificial life techniques are technically demanding.(5) Because of the emphasis on technical expertise and state-of-the-art technology in the field (what Penny calls “high nerd quotient” and the art historian María Fernández calls digital art’s “aesthetic technofetishism”), artificial life art has been dominated by artists working in the global North.(5)(6) Artists like Interspecifics are at the forefront of developing artificial life art as an emerging area of the digital arts in Mexico along with a new generation of artists in Latin America, including the Argentinian artists Leo Nuñez and Sofia Crespo and the Peruvian artist Paola Torres Nuñez del Prado. Projects like Linea are significant because they broaden access to AL, both beyond the sciences and the global North.¹⁰ It is to this concern that Interspecifics, just as with all their projects, additionally made some of the research, techniques, and coding created for Codex Virtualis_Genesis freely available online.¹¹

Codex Virtualis

Codex Virtualis_Genesis is yet but the first of the four parts of the total project proposed by Interspecifics under the title Codex Virtualis. The collective is currently developing the second part of this project, which is entitled Habitat involves testing the extremophile capabilities of the organisms created in Genesis by exposing them to inputs that simulate the harsh conditions of extraterrestrial environments. A third part, which is titled Emergence proposes testing the possibility of intra-actions (transfers) between artificial and biological organisms. Lastly, Codex Virtualis: Life involves testing the resilience of resulting hybrid organisms and accordingly selecting one organism to live virtually in Codex Virtualis.¹²

As a contemporary artificial life artwork, Codex Virtualis follows on a longstanding quest for mimesis or life-likeness that spans the histories of art, science, and technology, and as well includes social, spiritual, commercial, intellectual, and military histories. Langton himself traced the origins of AL to this pursuit in the history of art, starting with paintings and “statuettes” that “capture the static forms of living things.” He goes on to discuss hydraulic technology such as Egyptian water clocks or clepsydra as the next step towards imitating nature’s dynamic behavior. This trajectory developed with the invention of the mechanical escapement in AD 850, ushering in the age of clockwork technology. Refined throughout the European Middle Ages and the Renaissance, eventually, clockwork regulation of mechanical devices would not only be used to power them but also to sequence their motions or behaviors.¹³

Langton refers to this development as technology of process-control, which in the first instance involved interchangeable cams or drums with movable pegs. A variety of programmable automata are examples, including lifelike mechanical figures and animals such as the writing and picture-drawing automata built by the Jaquet-Droz family in Switzerland in the 18th century. In the early part of the twentieth century, during WWII and the Cold War, as Langton explains, physical programmable controllers gave away to abstract control structures, or sets of rules or programs. In effect, as he puts it, this is the moment when the “’logical form’ of the machine was separated from its material basis of construction.” Today’s computers are a technology of process-control and in this sense the equivalent to an “algorithm: the logic underlying the dynamics of an automaton.” (7)¹⁴

Artificial life art, which in the contemporary context is facilitated by computer technology, is consequently heir to these legacies.(8) According to Penny, artificial life art in digital environments spans three decades, beginning in the late 1980s, as artists turned to “a new type of interactivity” in search of a “new order of mimesis in which ‘nature’ as a generative system, not as an appearance, is being represented”. (5) To put it otherwise, artists became interested in artificial life in pursuit of developing art contrary to ingrained notions of artistic practice. That is, to create art not as an object but as a process, and in extension, to explore the role of the artist not as one about control over materials, but as an initiator of an open-ended process. This initial focus on unpredictability and novelty has since, however, been debated as an illusory pursuit, given that, as Penny adds, “the mechanism of simulated evolution” appear in comparison to the limitations of biological evolution more restricted; that is, as bound to the “conditions... established by logic-theoretic enframing” of mathematical models. (5)

Yet, while Codex Virtualis_Genesis and its larger project, Codex Virtualis shares interest in interactive aesthetics, this is not the project’s main focus. Its focus is rather apropos of the curator and cultural critic Edwina Bartlem’s noting as part of her recent discussion of artificial life art that “A-life discourse contains a prophecy of futuristic and imaginary posthuman or post-organic life.” (9) Codex Virtualis speaks above all to this imaginary, as according to Interspecifics, “’Codex Virtualis’ imagines ways of living together, of deep interspecific relations that may enhance our possibilities of survival.”¹⁵ Garcia further clarified the project’s concept on another occasion, as a proposal to imagine life and a new world beyond the Anthropocene, consonant to “the Symbiocene.”¹⁶

Garcia’s reference is to a term coined by the eco-anarchist philosopher, Glenn Albrecht, the Symbiocene. Symbiocene refutes the fatalistic implications of the Anthropocene, and, as largely inspired by Margulis’ notion of endosymbiosis affirms the possibility of creating a more sustainable world based on non-anthropocentric mutualism. (10) In this light then Codex Virtualis engages symbiosis as a metaphor and as a form of dialogue, debate, and exchange (i.e., between art, science, and politics) for conceptualizing life in a sustainable posthuman age. Because of its ecological focus, it as well in extension broadens the existing e imaginaries and themes engaged in artificial life art.

Notes

1 Interspecifics was founded in Mexico City in 2013. Core members are Leslie García, Paloma López Ramírez, Emmanuel Anguiano Hernández, Felipe Rebolledo Carvajal, Carels Tardío Pi, and Maro Pebo.

2 Ars Electronica is a cultural institute devoted to media art founded in Linz, Austria, in 1979. A leading organization in the global media arts, as well as managing a museum, multidisciplinary art research facilities, and an annual festival, it confers the most prestigious award in the genre, the Prix Ars Electronica. Founded in 1984, the SETI Institute stands for the "search for extraterrestrial intelligence." It is a non-profit dedicated to the study of life and intelligence in the universe through multidisciplinary research, education, and partnerships with industry, academia and government agencies, including NASA and NSF.

3 Available at, https://int-lab.cc/codex/?page_id=2.

4 According to Interspecifics, the initial idea was for collaborating (SETI) scientists to share their images with the group for the project. As it happened, the scientists failed to provide these images in the end. Leslie Garcia, “Interspecifics, Codex Virtualis: Genesis,” lecture, “Neurotalk ‘Máquinas y seres vivos: Comunicaciones desde el arte’”, Programa ACT (Arte, Ciencia y Tech- nologías), UNAM, March 2, 2022, available at, https://www.youtube.com/watch?v=NFE_F6HzakY.

5 Interspecifics, Codex Vitualis_Genesis, exhibition booklet, 2021, 4, available at, https://int-lab.cc/codex/?page_id=183.

6 Lynn Margulis and Dorion Sagan, Microcosmos: Four Billion Years of Microbial Evolution (Berkeley, CA: University of California Press, 1997), 29. Margulis’ speculations have been corroborated in the last decade as for example by scientists discovering that in ecosystems across the world, there are immense, mutually beneficial associations of macrofungi with flowering plants in complex, positive, metabolic, symbiotic relationship to each other. Findings such as these have scientifically overturned the view that evolution and life are solely founded on competitive struggle between species.

7 Leslie Garcia, “Interspecifics, Codex Virtualis: Genesis,” lecture, “Neurotalk ‘Máquinas y seres vivos: Comunicaciones desde el arte’.” It is worth to note here that Margulis’ notion of endosymbiosis implies a critique of heterosexist articulations of life, (i.e., the singularly privileged place of heterosexual reproduction in evolutionary biology), which presently resonates with Karen Barad’s notion of “nature’s queer performativity”, which likewise denotes multispecies co-involvements as the drivers of life’s heterogeneity. See, Karen Barad, “Nature’s Queer Performativity” in Kvinder, Køn og forskning / Women, Gender and Research nos. 1-2 (2012): 25-53.

8 See, http://interspecifics.cc/.

9 Artists working with digital A-life systems mostly reside in North America, Europe, and Australia, and include Karl Sims, Tom Ray, William Latham, Ken Rinaldo, Bill Vorn, Louise-Philippe Demers, Troy Innocent, Jon McCormack, Robb Lovell and John Mitchell, Christa Sommerer and Laurent Mignonneau, Jane Prophet and Gordon Selley, Paul Brown, Richard Brown and Mauro Annunzianto. See also, Fernández, “’Life-like’: Historicizing Process and Responsiveness in Digital Art”; Sarah Kember, Cyberfeminism and Artificial Life (London and New York: Routledge, 2003); “; Michael Whitelaw, Metacreation: Art and Artificial Life (Cambridge, MA: MIT Press, 2004); and Simon Penny, “Twenty Years of Artificial Life Art”, in Digital Creativity 21, no. 3, Dec. 2010: 197-204.

10 To date, along with scientists, game designers, filmmakers and artists across the world, including among them, Interspecifics, have used Lenia to develop their respective projects and in the process added to the evolving diversity and complexity of behaviours of its life forms. According to Chan, to date, more than 400 species in 18 families have been identified in Lenia. B. W.-C. Chan, “Lenia: Biology of Artificial Life,” Complex Systems, 28(3), 2019, pp. 251–286. See also, Siobhan Roberts, “The Lasting Lessons of John Conway’s Game of Life”, in The New York Times, Dec. 28, 2020, available at, https://www.ny- times.com/2020/12/28/science/math-conway- game-of-life.html.

11 See, https://int-lab.cc/codex/.

12 The duration, location, and conditions of this last stage are yet to be clearly specified as at the time of writing, Interspecifics is still developing the second stage in the project, Habitat.

13 The Muslim inventor Ismail al-Jazari’s The Book of Knowledge of Ingenious Mechanical Devices (1206) is currently considered by historians of science to be a significant groundwork for modern engineering, hydraulics, and even robotics. The Book builds on science and wisdom from ancient Greek, Indian, Persian, Chinese and other cultures and offers a how-to manual of sorts that includes alongside richly detailed illustrations as well as instructions on how to build al-Jazari’s inventions spanning playful robots to practical contraptions.

14 Likewise, the Latine artist Micha Cárdenas has traced algorithmic culture to pre-digital technology, citing recipes and rituals as examples. Additionally, Cárdenas reminds us that the word algorithm is a derivation of the name of the scholar Muhammad ibn Musa al-Khwarizmi (780-850 A.D.), credited with inventing algebra in his book Dixit Algorismus. Micha Cárdenas, Poetic Operations, Trans of Color Art in Digital Media (Durham, Duke University Press, 2022), 7.

15 Interspecifics, Codex Vitualis_Genesis, exhibition booklet.

16 Garcia, “Interspecifics, Codex Virtualis: Genesis,” lecture “Neurotalk ‘Máquinas y seres vivos: Comunicaciones desde el arte’”

References

(1) Cortesão See M., T. Schütze, R. Marx, R. Moeller, V. Meyer, “Fungal Biotechnology in Space: Why and How?” in H. Nevalinen, ed., Grand Challenges in Fungal Biotechnology. Grand Challenges in Biology and Biotechnology, New York, Springer, 2020, 501–535.

(2) Christopher Langton, “Artificial Life,” in Ars Electronica: Facing the Future, edited by Timothy Druckrey, with Ars Electronica, Cambridge, MA, MIT Press, 1999, 126-268.

(3) John von Neumann See, Theory of Self-Reproducing Automata, Urbana, University of Illinois Press, 1966.

(4) Christopher G. Langton, “Artificial Life,” in The Philosophy of Artificial Life, edited by Margaret A. Boden, London, UK, Oxford University Press, 1996, 92-93.

(5) Simon Penny, “Art and Artificial Life-A Primer,” in the Proceedings of DAC09, Digital Art and Culture 2009 conference, University of California, Irvine, 12–15 December 2009, unpaged, available at, https://escholarship.org/uc/item/1z07j77x.

(6) María Fernández, “Postcolonial Media Theory,” in Art Journal 58, no. 3, Autumn, 1999, 66-69.

(7) Langton, “Artificial Life,” 41-46.

(8) María Fernández See, “’Life-like’: Historicizing Process and Responsiveness in Digital Art,” in A Companion to Contemporary Art since 1945, edited by Amelia Jones, Malden, MA, Blackwell Publishing, 2006, 557-581.

(9) Edwina Bartlem, “Immersive Artificial Life (A-Life) Art,” in Backburning: Journal of Australian Studies 84, edited by Helen Addison-Smith, An Nguyen and Denise Tallis, Perth, API Network, 2005.

(10) See, Glenn Albrecht, Earth Emotions: New Words for a New World, Ithaca, NY, Cornell University Press, 2019.

Author Biography

Claudia Costa Pederson is Associate Professor of Art History, author of Gaming Utopia: Ludic Worlds in Art, Design, and Media (Indiana, 2021); and co-curator of FLEFF’s new media exhibitions at Ithaca College, NY, since 2017.

Disruptive Avant-Garde Art of Today: Shaping Post-Growth Imaginaries for Symbiotic Futures

Kristina Pranjić, Magdalena Germek, Peter Purg

University of Nova Gorica
Nova Gorica, Slovenia
kristina.pranjic@ung.si, magdalena.germek@gmail.com, peter.purg@ung.si

Abstract

In order to develop new symbiotic relationships and different imaginaries, it is first necessary to critically restructure the representations of forms of cooperation, which in their positive, desired version usually represent a certain romantic idea of nature and human, and the possibilities for a harmonious model and holistic structure of reality. This can be seen in both eco art and activist ecological agendas, which often play on feelings of harmony and mutual reciprocity, and actually further contribute to a distorted and extremely one-dimensional image of reality. Using the concepts of conviviality and cosmpolitics, the article aims to offer new concepts of symbiosis and symbiotic futures that face today's process of defuturing. The second point of the article is to develop a convincing and solid alternative to the neoliberal view of market-driven models based on competencies and the logic of growth. Therefore, the actual task for disruptive avant-garde art of today should be understood as the decolonization of our imaginaries that perceive nature through the logic of growth and the harmonious model in the direction of shaping post-growth imaginaries for symbiotic futures.

Keywords

Imaginary, conviviality, symbiosis, cosmpolitics, defuturing, (de)growth, posthumanities, intermedia art, artistic disruption.

DOI

10.69564/ISEA2023-23-short-Pranjic-et-al-Disruptive-Avant-Garde-Art

Introduction

At the end of the epidemic, faced with another European war, and shattered by climate disasters, we seem to be facing a similar if not even worse zeitgeist to the one of the twenties that called for the avant-garde art and design school Bauhaus to bring a new social, cultural and creative impetus. However, back then there was no experience of the failures of the 20th century artistic and social utopias and movements.

For this reason, we can rightly question any revolution or change through artistic and cultural means. Nevertheless, new artistic expressions that use new technology and scientific methods to conduct true artistic research are developing novel concepts and opening space for another horizon of experiences that can offer images and ideas of a different future, in which it will be possible not only to survive, but also to thrive in symbiosis with other human and non-human beings.

In order to describe the mentioned process through a deep understanding of how current art contributes to the shaping of our imaginaries for the better future in the age of the Anthropocene, the article will first introduce key concepts, such as “conviviality”, “symbiosis”, “defuturing”, “imaginary”, “growth”, and “post-growth/degrowth,” that comprehensively shape the paradigm, criticized by the artistic projects that are addressing the most burning issues of today, and rightfully represent the contemporary avant-garde. In the last part of the article, we will show three distinct examples of current artworks that seek to participate in transforming the human imagination in the direction of symbiotic futures with a different system of relationship not dependent of industrial and consumerist systems, rooted in the idea of continuous growth.

Living together: Conviviality and Symbiosis

The term conviviality was incorporated to the humanities vocabular by Ivan Illich, who in his work criticized the industrial capitalism and productivity, to instead describe a path of human emancipation and autonomy for reaching convivial life.¹ For Illich conviviality means an “autonomous and creative intercourse among persons, and the intercourse of persons with their environment; and this, in contrast, with the conditioned response of persons to the demands made upon them by others, and by a man-made environment.”² Furthermore, because of his instance of the interdependence among living beings, Ilich’s thought is particularly productive in the field of posthumanism to describe interdependence and entanglements between different species, human and non-human agents.

The term “conviviality” comes from Latin convivere, which means “to carouse together, live together” (com “with, together” and vivere “to live”). The more common term “symbiosis” that comes from ancient Greek sýn and bíōsis similarly denotes living together. Today both of these concepts are equally important in the posthumanism research that emphasizes the importance of exploring relations between human and non-human beings and other entities or matter, and insisting on the theoretical break with anthropocentrism and with the dualisms between human/society and nature.

Conviviality does not mean only relationships of cooperation, but it also includes conflict and tensions as constitutive and necessary part of convivial relations; human beings are understood as a part of the network of players living in symbiosis with others. ³ Moreover: “To Illich, the word ‘conviviality’ does not mean joy or light-heartedness; it refers to a society in which modern tools are used by everyone in an integrated and shared manner, without reliance on a body of specialists who control said instruments.”⁴ For Ilich convivial toll can be used by an individual for a purpose that they choose, and it needs to bring about more freedom, autonomy and creativity (for all beings).

Perceptions of harmonious interactions of symbiosis were critiqued by Isabelle Stengers’ research in thermodynamics to include complexity, processes of disturbance and friction that cause a state of crisis and lead to change.⁵ In her seminal work on “cosmopolitics” Stengers described a “symbiotic agreement” as a part of an “immanent process of ‘reciprocal capture’;” she describes “the event” of symbiosis as “the production of new, immanent modes of existence, and not the recognition of a more powerful interest before which divergent particular interests would have to bow down. Nor is it the consequence of a harmonization that would transcend the egoism of those interests.”⁶

We can further find productive thoughts on this topic in Anna Tsing work on rare matsutake mushroom, in which she presents the complex network of cooperation between human and non-human participants that rejects stability as a central goal of relationships, and even stability of the term “naturalness”.⁷ Stengers’ and Tsing’s work not only opens to question the foundations of objective (natural and humanities) science, but also shows a new way of connecting and generating knowledge and possible new imaginaries that can arise from different forms of human activity, which is not necessarily bound to the rationality and authority. Knowledge and science can be, through specific practices and processes, something that we not only discover, but also shape.

Facing Defuturing

To recognize what is, we first need to establish active knowledge about how our current dysfunctional relationships of un-sustainment are sustained. One could assume it is truly difficult to be an avant-gardist in the time of defuturing, as Tony Fry named our condition in his book Defuturing. A New Design Philosophy.⁸ Fry introduced the concept of defuturing as the destruction of the future by design; it means that we live in a world that is taking away futures for ourselves and non-human others. He criticizes Eurocentric, anthropocentric and productivist structures of unsustainability by exploring the history of design that so importantly yet often subliminally coordinates our activities, and influences our thought.

Because our conditions of existence drastically changed, Fry stresses that we also need to drastically change our philosophy of existence and of acting. The new direction for making anything in the world is not a direction within design as-is; within the already established paradigms; it is a direction beyond where design now is. It is also a direction beyond where thought now is. The alternative to this process of “change so that nothing changes” and covering of crisis, would be “informed futuring”, which basically refers to finding the alternative and acquiring the agency in order to create something truly different, an actual change.⁹

For this kind of futuring, a new avant-garde in doing and thinking is needed. Therefore, it is central to look into the past to deconstruct and surpass it. The most significant signifiers of how we do and think things, and how we ascribe value – is growth. Therefore, besides offering a complex image of symbiosis and establishing new and different relationships, it is important to decolonize our imaginary of growth.

Decolonizing our Imaginaries

We can understand “growth” as a centuries-old matrix of thinking that historically constructs Western-civilizational models of meaning-making, collectively and individually, from various institutional macro and microforms of society. The logic of growth is thus not only rooted in the rational categories that shape our concepts but also in the way we look at the world, how we describe, perceive, and represent it. The stubbornness to fight growth by reformulating capitalist parameters – for example, by establishing concepts like social, human, local, and sustainable “development” – can only lead to new ecological, economic, and social failures.

Agendas such as the New European Bauhaus for example, and the UN 2030 Agenda for Sustainable development do seem to be signs of a significant shift toward an awareness about social and environmental health, equality, and justice. However, we need to be critical of the very idea of sustainable development, which remains part of the Western economic imaginary of growth. We can find this critique especially in the works of Serge Latouche¹⁰, who claims that the idea of sustainable development is a mystified and ideologized one that does not bring about any radical turn in thinking but stems from the logic of a compromise.

Consequently, it is unavoidable to start “decolonizing our imaginaries,” a syntagma borrowed from Latouche¹¹, who refers, on the one hand, to the concept of the imaginary by Cornelius Castoriadis¹², and on the other hand, to the anti-imperialist concept of decolonization introduced by anthropologists (e.g., Serge Gruzinski).¹³ If we follow Latouche’s insight, radical change needs another economy; another view of science that would go beyond the concept of Promethean technoscience; another conception of life and death; a different conception of wealth and poverty; another notion of time that would no longer be linear, cumulative, continuous; other conceptions of space; other intergenerational and gender relations; a different concept of work (placing social relations in the center, instead of, e.g., efficiency or value accumulation)¹⁴. Latouche integrates this into an eight-point program with listed imperatives: to re-evaluate, reconceptualize, restructure, redistribute, relocalize, reduce, re-use, and recycle.¹⁵

Constructing the Imaginary of Post-Growth

One solid set of alternatives to the logics of growth was offered by the introduction of concepts such as post-growth and degrowth. And even though the concept of degrowth as the most optimal future direction of our global society is becoming more and more mainstream, it is clear we will not achieve any drastic changes in human behavior if we do not address its phenomenological and epistemological implications that could provide us with new models for everyday life.

Neutralizing the disastrous impact of our growth-based society would therefore mean actually using radical avant-garde methods as a unique combination of chance and plan, happening and controlled work in all aspects of thinking and creating for uncertain futures. We see the legacy of the historical avant-garde movements, such as Dadaism and Surrealism, as a plethora of valuable concepts for thinking degrowth that is yet to be researched and applied from the field of art exclusively to the area of social and environmental care, and precisely to the way we make and do things. The highly experimental avant-garde presented the most radical possibilities for changing the logic of thinking and perceiving reality, thus showing the opportunities to let new forms and relationships emerge.

And although the avant-garde is all too often perceived as art that sought to demolish and destroy everything that existed before it, it carries within it an extremely affirmative and constructive character: avant-gardes invented a whole series of new formal languages and strategies that are still in use today as fundamental techniques of our modern visual or multimodal communication (e.g., photomontage, collage, avant-garde cinema, advertisements, graphic design typography, new architecture). They sought to formalize the coincidence and freedom of expression and erase any automatization and pre-given cultural definitions.

The avant-gardists’ proposition appears to have been precisely what we need today for a symbiotic “metamorphosis in being” as stated by Pasi Haikkurinen, connected with the “practice of releasement” understood as the “new ethos” for degrowth society.¹⁶,¹⁷

Artistic Disruption for Radical Innovation

We finally come to the acknowledgment of the importance of artistic disruption and art-thinking as the key innovative methodology that may provoke radical change in order to create a meaningful difference from what was (wrong). Which are those new (inter)media and investigative artistic practices involving cutting-edge technologies, critically confronting the European social and cultural values with the aesthetics-, ethics- and tech-related legacies of the historical avant-garde?

By way of conclusion, let us try to briefly discuss three of such topical cases selected among the newest (intermedia) artistic practices:

Vladan Joler's recent hybrid animated info-graphic work entitled New Extractivism (2020) visualizes the position of the individual as a user of contemporary information technologies, who is subject to systemic and systematic corporate extraction. Thus, it reveals and critically discusses some of the most pressing issues of modern platform capitalism. The printed materials, which the author considers to be a kind of sci-artistic “assemblage”, are presented in the format of a poster and a brochure, and are accompanied by an animation distributed online, supported by the author's own narrative, which he occasionally also delivers as a lecture (performance). By combining the elements of a map and a guidebook, and accompanied by abundant and extremely analytical notes, the work attempts to produce a symbiotic and fractal-like “blueprint of a machine-like superstructure, or a super allegory.”¹⁸

Often relying on the approaches of speculative design and advanced computer graphics, as well as info-animations or even physical spatial installations, the works of the interdisciplinary group Disnovation.org not only vividly show the paradoxes of the contemporary consumer and hyper-informatized society, but also try to contribute suggestions and tools for resolving them. In the Post Growth Toolkit (2020), a set of so-called critical games, they offer a mix of scientific and speculative literary-narrative tools, which should offer the players of this game (or the users of the toolkit) a set of more sustainable solutions for everyday life. Post Growth Prototypes (2021) complement the mentioned toolbox with critical (animated video) essays functioning as case studies of advanced and symbiotic concepts such as solar income, radical energy transition, or transcending the Anthropocene via the post-growth paradigm shift, introducing a new responsibility of man towards the biosphere.

As the last truly comprehensive example of an artistic practice that radically questions the human role within ecosystems, we wish to discuss the set of terra0 projects. Since 2018 various prototype environments have been built on the decentralized (peer-to-peer) blockchain platform Ethereum, which aims to provide automated frameworks for the resilience of a given ecosystem. By establishing a “Decentralized Autonomous Organization” on the upper layers of the earth to govern them, terra0 research team aims to create technologically augmented ecosystems that are both more resilient and more capable of operating within a predetermined set of rules in the economic sphere, as independent agents: e.g., the forest independently mines cryptocurrency and decides how it will change its material base. The group believes that modern technologies such as remote sensing and machine learning provide an opportunity to rethink existing inefficient governance and regulatory structures. Moreover, they also seem to suggest how, with appropriate art-thinking and speculative-design based assumptions and interventions, these could play a key role in creating a sustainable, resilient, symbiotic and biodiverse future.

There are more such artistic practices and they bear witness to the engagement of art in order to answer the most pressing questions of our time not only by representing what does not work, but above all to construct, in an avant-garde manner, new symbiotic relationships with human and non- human agents that would contain the full complexity of the symbiotic organization of different entities jointly working to sustain a functional environment, without falling back into the old paradigms shaped by the imaginaries and logics of growth, duality, harmony and stability.

References

1 Ivan Ilich, Tools for Conviviality, New York, Harper and Row, 1973.

2 Ivan Ilich, Tools for Conviviality, 11.

3 Michael Given, “Conviviality and the Life of Soil,” Cambridge Archaeological Journal 28, 2017, 127–143.

4 Marco Deriu. “Conviviality,” in Degrowth – a new vocabulary for a new era, ed. Giacomo D’Alisa, Federico Demaria and Giorgos Kallis, London, Routledge, 2015.

5 Ilya Prigogine, Isabelle Stengers, Order Out of Chaos. Man's New Dialogue with Nature, London, Flamingo, 1984.

6 Isabelle Stengers, Cosmpolitics I, Minneapolis/London, University of Minneapolis Press, 2010, 35-36.

7 Anna L. Tsing, The Mushroom at the End of the World: On the Possibility of Life in Capitalist Ruins, Princeton/Oxford, Princeton University Press, 2015.

8 Tony Fry, Defuturing, A New Design Philosophy, 2nd Edition, Bloomsbury, Bloomsbury Publishing, 2020.

9 Tony Fry, Defuturing, A New Design Philosophy, 239.

10 Serge Latouche, Petit traité de la décroissance sereine, Paris, Librairie Arthème Fayard, 2007.

11 Serge Latouche, “Imaginary, Decolonization of,” in Degrowth – a new vocabulary for a new era, ed. Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, London, Routledge, 2015.

12 Cornelius Castoriadis, L'institution imaginaire de la société, Paris, Éditions du Seuil, 1975.

13 Serge Gruzinski, La colonisation de l’imaginaire. Sociétés indigènes et occidentalisation dans le Mexique espagnol (XVIe- XVIIIe siècle) Collection Bibliothèque des Histoires, Paris, Éditions Gallimard, 1988.

14 Serge Latouche, Survivre au développement, Paris, Librairie Arthème Fayard, 2004.

15 Serge Latouche, Petit traité de la décroissance sereine.

16 Pasi Heikkurinen, “Degrowth: A metamorphosis in being,” Environment and Planning E: Nature and Space Vol. 2, issue 3, 2019, 528–547.

17 Pasi Heikkurinen, “Degrowth by means of technology? A treatise for an ethos of releasement,”Journal of Cleaner Production 197, 2018, 1654–1665.

18 Vladan Joler, “New Extractivism (2020)”, extractivism.online, accessed December 6, 2022, https://extractivism.online/. 

Bibliography

1 Corneius Castoriadis, L'institution imaginaire de la société, Paris, Éditions du Seuil, 1975.

2 Marco Deriu, “Conviviality,” in Degrowth – a new vocabulary for a new era, ed. Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, London, Routledge, 2015.

3 “Disnovation.org”, accessed December 6, 2022, https://disnovation.org/index.php.

4 Tony Fry, Defuturing, A New Design Philosophy. 2nd Edition, Bloomsbury: Bloomsbury Publishing, 2020.

5 Michael Given, “Conviviality and the Life of Soil,” Cambridge Archaeological Journal 28, 2017, 127–143.

6 Serge Gruzinski, La colonisation de l’imaginaire. Sociétés indigènes et occidentalisation dans le Mexique espagnol (XVIe- XVIIIe siècle) Collection Bibliothèque des Histoires, Paris, Éditions Gallimard, 1988.

7 Donna Haraway, “Cyborgs and Symbionts: Living Together in the New World Order,” in: The cyborg handbook, ed. Chris Gray, New York, Routledge, 1995, xi–xx.

8 Donna Haraway, Staying with the trouble: Making kin in the Chthulucene, Durham, Duke University Press, 2016.

9 Pasi Heikkurinen, “Degrowth: A metamorphosis in being,” Environment and Planning E: Nature and Space Vol. 2, issue 3, 2019, 528–547.

10 Pasi Heikkurinen, “Degrowth by means of technology? A treatise for an ethos of releasement,”Journal of Cleaner Production 197, 2018, 1654–1665.

11 Ivan Ilich, Tools for Conviviality, New York, Harper and Row, 1973.

12 Vladan Joler, “New Extractivism (2020)”, extractivism.online, accessed December 6, 2022, https://extractivism.online/.

13 Serge Latouche, “Imaginary, Decolonization of,” in Degrowth – a new vocabulary for a new era, ed. Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, London, Routledge, 2015.

14 Serge Latouche, Petit traité de la décroissance sereine, Paris, Librairie Arthème Fayard, 2007.

15 Serge Latouche, Survivre au développement, Paris, Librairie Arthème Fayard, 2004.

16 Ilya Prigogine and Isabelle Stengers, Order Out of Chaos. Man's New Dialogue with Nature, London, Flamingo, 1984.

17 Isabelle Stengers, Cosmpolitics I, Minneapolis/London, University of Minneapolis Press, 2010.

18 “terra0,” accessed December 6, 2022, https://terra0.org/.

19 Anna L. Tsing, The Mushroom at the End of the World: On the Possibility of Life in Capitalist Ruins, Princeton/Oxford, Princeton University Press, 2015. 

Drivers for Resilience in Cultural Organizations: lessons from the Montreal festivals in the face of the COVID-19 pandemic.

Nicolas Ricci, Marine Agogué

Center for Management Science, Mines - Paris
HEC - Montréal nicolas.ricci@mines-paristech.fr, marine.agogue@hec.ca

Abstract

Crises such as the COVID-19 pandemic represent appropriate moments to innovate. Many organizations in the cultural sector have thus proposed numerous changes in their activities trying to develop new forms of symbiosis, bringing back the notion of resilience. Beyond its buzz word aspect, resilience has essentially been associated with a set of organizational capacities to adapt and innovate in the face of a disruption in the cultural environment, leaving little consideration to question the main drivers of resilience in cultural organizations.

We propose then to study the adaptation of the Montreal festivals offer, building on primary data from 8 interviews with festival directors or managers and secondary data from internal and external documentation. We therefore mobilize the concept of the business model to identify and discuss the drivers for resilience in cultural organizations. We show a trend for festivals to come back to their formal business model despite the deployment of different innovations and identify role and purpose as the two main drivers for the resilience of festivals.

Finally, we call for a comparison with other cultural organizations to discuss the preserving and reconfiguring aspects of their resilience.

Keywords

Resilience, COVID-19 Pandemic, Festivals, Cultural organizations, Business models, Purpose.

DOI

10.69564/ISEA2023-24-short-Ricci-et-al-Drivers-for-Resilience

Introduction

The COVID-19 pandemic and the associated restrictions have generated a disruption in the offer and consumption patterns of cultural products⁴. The transition to online has allowed a favorable development for cultural sectors that did not depend on the physical presence of their public. On the other hand, very dependent sectors such as cultural events in a broad sense, have faced significant difficulties despite various innovations in the format and distribution of their cultural offer.⁶

Internally, the teams—with a high turnover rate in some cultural events—have been working intensively since the start of the pandemic to reinvent themselves while preserving their identity. This has led to the emergence of a tension between identity and constrained innovation.¹⁰ We offer with this paper to clarify how this conflict was expressed within cultural organizations by questioning a well-known concept in management when dealing with adaptation under constraints: organizational resilience.

Literature review

Organizational Resilience as a development of capacities to adapt and innovate in crisis period

From its original definition in material science to the first ones in psychology, resilience has been defined through the idea of a capacity of adaptation to a shock.¹ In management science, the development of the concept of Organizational Resilience followed the same logic. For instance, Gibson and Tarrant define resilience as an “adaptive capacity and how we better understand and address uncertainty in our internal and external environments.”³ This observation can be confirmed by systematic review on the concept of organizational resilience.¹¹

The essential of the literature thus focus on the description of this capacity. Begin and Chabaud¹ propose a typology to describe the dimensions of organizational resilience:

Organizational Resilience as a reasoning moment for organizations

However, building on Kraemer definition of resilience in social work studies⁷, resilience can also be defined with a phenomenal lens rather than the notion of capacity. According to the author, an acceptable description of resilience depends on the context of your study (cultures, time, people, etc.) Thus, the fact that resilience involves a phenomenon and a cognitive process from the actors to tackle it, constitutes the common point between every form of resilience.

Organizational resilience can then be described through a succession of moments² illustrating successive cognitive processes. 

By considering a psychodynamic approach of resilience, Winkler¹² investigate actors thinking and reasoning during a crisis period, leading to the consideration of individual drivers for resilience.

Yet, we lack such an approach of organizational resilience allowing us to identify the phenomenon of resilience⁵ and discuss the main drivers for the reasoning of actors from cultural organizations in crisis periods.

Methodology

Field of study: Why considering festivals to reflect the phenomenon of resilience within cultural organizations?

Among the variety of cultural organizations, we propose to study festivals. Their activities deeply depend on the physical gathering of publics and artists in a same place. Since the start of the pandemic, festivals have faced deep changes in their business models and modified their formats in digital forms or cancelled their editions and constitute then an interesting panel of forms of resilience. Moreover, festivals play a central role in the exhibition of several artworks and represent a real platform in the life of cultural ecosystems in general.

Data collection

We started by building a database registering most of the different festivals of Montreal (n=71) and information found on websites and social media such as attendance, followers, period of the year, duration, type of festival (i.e., music, cinema, visual arts, etc.), format of 2020/2021/2022 editions.

We contacted 68 of these festivals through direct mail to directors, contact mail, LinkedIn.

We interviewed 8 directors and managers between December 2021 and June 2022, with a variety within the festivals in terms of size, duration, and types. The semi-structured interviews lasted between 30 minutes and 1 hour, we developed themes such as the adaptation of the festival team, the format before, during and after the pandemic, their relations with the different stakeholders.

We completed this primary data with internal documents (audience studies, annual reports, internal communication) provided by the directors and managers to complete our study. We also collected information from websites and local press articles.

Data analysis

This study follows the principles of a grounded theory: we use data collected directly in our field of study to develop intermediate theorization. To do so, we proposed an open code of the interviews content.⁸ 

We identified two major categories in the content: one part referring to the business model of the festival (format, stakeholders, funding, etc.), a second one relative to remarks on the role of the festival, its purpose. Then, we proposed to encode the information on the business model in canvas format for business models.⁹ We then identified several codes for the drivers of reasoning that we gathered into two concepts: “role” and “purpose.”

Results

Comparison of the Montreal festivals business models before, during and after the COVID19 pandemic: a trend for a “back to normal” Gathering our coding from each festival interviewed, we propose three general business model of Montreal festivals before, during and after the pandemic.

Pre-pandemic business model.

We built the pre-pandemic business model (Figure 1 (1)) around a value proposal common to all the festivals we interviewed: offer a moment and a place for exhibit of specific artworks to a public. We identified a general trend for the revenue streams (public fundings, private sponsors, event sales) and the cost structure (rent of a physical place, equipment and furniture for the event, salaries). Both interviews and secondary data helped us to define the public of festivals, channels, and public relationship management practices. We finally highlighted recurrent public and private partners, key activities (artistic program, event organizing, public management) and key resources (artists, representation place, festival team and volunteers).

Pandemic business model.

The pandemic business model evolved on almost every aspect (Figure 2). Most festivals turned into a digital format, requiring a reconfiguration of resources and skills around online broadcast technologies. The value proposal remains unchanged, illustrating the lack of will to change the concept of festival and their identity. Nevertheless, the business model is only sustainable thanks to public support.

Post-pandemic business model.

The post-pandemic business model regains almost all its pre-pandemic properties (Figure 3). The digital turn is hardly preserved. However, there is a desire to keep the public engagement tools that proved to be rather effective during the 2020 and 2021 editions, such as additional content or contests. 

Role and purpose as a driver for the resilience process

The comparison of the different business models reveals a clear absence of evolution in the value proposition of festivals. The festival directors insisted on their wish to preserve the identity of the festival, directly linked to the idea of “festival as a platform for artworks and artists promotion.”

Some directors admitted that they think a festival mainly for artists rather than for the public. The willingness to keep strong links with the artists illustrates the role of festivals as a service for artists in both exhibition and socialization and its associated benefits (collaboration, inspiration, artistic movement evolution, etc.). As a result, the key point for the shift to online services delivered during the pandemic was to preserve both moments of exhibition for public and moments between artists and practitioners.

Nevertheless, online festivals remained an unsatisfactory solution (not only for economic reasons), revealing purpose as the second driver of the resilience of festivals. The directors associated the digital format to the lack of informal moments and places for both festival teams, artists and public. The significant part of informality and materiality in these events leaves little adhesion for remote formats, making the physical format part of the festival purpose.

Discussion and conclusion

Our results show that role and purpose are fundamental drivers of resilience within festival teams involving that:

In sum, we were able to highlight a preservative dimension of organizational resilience in the case of festivals. This leads us to question the different forms of preserving resilience that the COVID-19 pandemic has brought to the forefront in the cultural and artistic communities. Conversely, it is quite possible to imagine forms of reconfiguring resilience in other cultural sectors, the study of which could allow for an interesting theoretical confrontation.

Finally, we have seen that the business model is a suitable tool for diagnosing and discussing the phenomenon of resilience, and that rather than an approach based on adaptive capacities, we can think of resilience in terms of the drivers of the actors whose role and purpose constitute.

(1) Figures of the business models are located at the end of this document
References

1 Bégin, Lucie, Didier Chabaud, "La résilience des organisations. Le cas d’une entreprise familiale," Revue française de gestion 200 (1), 2010, 127-42.

2 Conz, Elisa, Giovanna Magnani, "A Dynamic Perspective on the Resilience of Firms: A Systematic Literature Review and a Framework for Future Research," European Management Journal 38 (3), 2020, 400-412, https://doi.org/10.1016/j.emj.2019.12.004.

3 Carl A. Gibson, Michael Tarrant. s. d. "A “Conceptual Models” Approach to Organisational Resilience," The Australian Journal of Emergency Management 25 (2), 6-12, https://doi.org/10.3316/informit.084520139241216.

4 Julia V. Gnezdova, Vladimir S. Osipov, et Igor V. Hrip- tulov, "Creative Industries: A Review of the Effects of the COVID-19 Pandemic," In Post-COVID Economic Revival, Volume II: Sectors, Institutions, and Policy, édité par Vladimir S. Osipov, 2022, 159-71, Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-83566-8_10.

5 Julia Hillmann, et Guenther Edeltraud, "Organizational Resilience: A Valuable Construct for Management Research?" International Journal of Management Reviews 23 (1), 2021, 7-44, https://doi.org/10.1111/ijmr.12239.

6 Olena Khlystova, Yelena Kalyuzhnova, Maksim Belitski, "The Impact of the COVID-19 Pandemic on the Creative Industries: A Literature Review and Future Research Agenda," Journal of Business Research 139, février, 2022, 1192-1210, https://doi.org/10.1016/j.jbusres.2021.09.062.

7 Sebastian Kraemer, "Promoting Resilience. Changing Concepts of Parenting and Child Care," International Journal of Child & Family Welfare 4 (3), 2000, 273-87.

8 Ann Langley, "Strategies for Theorizing from Process Data," Academy of Management Review 24 (4): 1999, 691-710, https://doi.org/10.5465/amr.1999.2553248.

9 Alexander Osterwalder, Yves Pigneur, Business model generation: a handbook for visionaries, game changers, and challengers, Vol. 1, John Wiley & Sons, 2010.

10 Michael G Pratt, Majken Schultz, Blake E. Ashforth, Davide Ravasi, The Oxford Handbook of Organizational Identity. Oxford University Press, 2016.

11 Tennakoon Niranjala, Nadira Janadari, "Organizational Resilience: What it is and what it isn’t? A Conceptual Review," Wayamba Journal of Management, 12 juillet, 2021, 171, https://doi.org/10.4038/wjm.v12i1.7520.

12 Astrid Winkler, "Resilience as Reflexivity: A New Understanding for Work with Looked-After Children," Journal of Social Work Practice 28 (4), 2014, 461-78, https://doi.org/10.1080/02650533.2014.896784. 

Figure 1: Pre-pandemic business model
Figure 2: Pandemic business model
Figure 3: Post-pandemic business model

Metabolism and Art

Hannah Rogers, Adam Bencard

University of Copenhagen, Denmark
hannahstarrogers@gmail.com, adam@sund.ku.dk

Abstract

Metabolic arts should be added to the emerging interdiscipline of metabolic humanities and this paper will discuss ways of defining metabolism that might be productive in helping to produce tools and touchstones for metabolic readings of contemporary art before presenting examples of artworks which might be interestingly illuminated by light of this sign, taking time to relish the process of these materially oriented internal analysis coupled with how the work might be considered in terms of its broader implications for the concept of metabolism.

DOI

10.69564/ISEA2023-25-short-Rogers-et-al-Metabolism-and-Art

Introduction

Among the many challenges of our Anthropocene moment, is the new attention that must be paid to how humans and other species are embedded within, exposed to, and even composed of the very materiality of a planet that is rapidly changing. By focusing on metabolism, understood as “the chemical processes that occur within a living organism in order to maintain life; the interconnected sequences of mostly enzyme catalyzed chemical reactions by which a cell, tissue, organ, etc., sustains energy production, and synthesizes and breaks down complex molecules” (Landecker 2011). As a fundamental biochemical process of life, metabolisms are ubiquitous and multilayered, ranging from distinct multispecies bodily processes, to a variety of chemical transformations across many organisms on a planetary scale, atmospheric and respiration relations between human and plant metabolisms, the body as an environment, to notions of metabolism in a more purely metaphorical sense in aesthetics. Tracing the workings of metabolism is a method for connecting change and dysfunction on a bodily, social, and earth-wide scale. As John Bellamy Foster (1999) explains, Marx conceived of a dysfunction in the form of a metabolic rift as a separation “between humanity and the soil, reflected in the antagonism of town and country” (399).¹

Metabolic humanities appear to be a rising sign under which medical humanities, environmental humanities, and agricultural corners might be united, and metabolicarts may be a productive stage for the three-legged stool of ASTS to balance in tension around the subject.²

Metabolism might be taken to mean many things from distinct multi-species bodily processes, to a range of chemical transformations across many organisms on a planetary scale, atmospheric and respiration relations between human and plant metabolisms, the body as an environment, to notions of metabolism in a more purely metaphorical sense in aesthetics. This is not to suggest that metaphor is absent in what is generally understood to be the scientific notion of metabolism, indeed sociologist Hannah Landecker has shown how very entangled social conditions and metabolic science are and have been.³ Landecker suggests that metabolism was foreclosed by understanding the body as a machine, with a ledger, but this could also point to understanding the body as a business, with immediate implications about capital and a relationship to Foster's Marx.

Metabolisms are ubiquitous but may be most noticeable in states of dysfunction. These dysfunctions are the basis of medical sciences, the source of new ways of conceiving of ecological relations in a climate-changed world, and are a longstanding way of diagnosing philosophical indigestions. Metabolism can be the constant that brings methodological tendencies to the fore as a shared subject, and yet the variable historical understandings of metabolism alone give rise to ages possible metabolic unfoldings so that our studies need not be bounded by the current state of metabolic science and indeed, as ASTS scholars would argue, this is best understood in its social, political, philosophical, and historical contexts. Metabolism serves so many possible needs for scientists, scholars, and artists by providing metaphors, models, puzzles, solutions and balances. Metabolic science is built on a stack of ever-modified metaphors,⁵ including the metabolism as an engine or motor (fast/slow and often an emphasis on the notion of fuel/energy sources), furnace (hot/cold), “chain reactions,” “chemical cascades,” the “chemical carnival,” and many others.⁶ Science is implicated in metaphor thinking, as even the classic experiment asks us to make correspondence between the specific findings on the bench and the broader world, and this extends out to our public understandings of science, often with further analogies which both simplify and make more culturally complex these concepts. We suggest a focus on Landecker’s etymology for the term Stoffwechsel, translated as “total metabolism,” with a further emphasis around the "stoff'' or in English “stuff” of the process to emphasize the insistence bioartists have shown in exhibiting the stuff of living things and parts.⁷ This emphasis on stuff has been important in bioart as such work has avoided distance and representation except by a part of the actual living tissue, bacteria thing and at the same time it has insisted on pulling against conceptual art to attempt to produce in material those ideas. Being in the presence of stuff has been a hallmark of bioart and an emphasis on process may well be the hallmark of the metabolic arts.

The dual use Marx and Landecker make of, on one hand, critiquing soil chemistry or medical genetics, and on the other hand, using those subjects as the basis for their thought, is a situation shared by bioartists who are often using the very biological and biotechnical materials they are critiquing. Enter metabolic arts: drawing on metabolism from the sciences, embodied experience of metabolism, and the potential of laboratory and home metabolism practices to create encounters with life and life processes, many artists and art-aligned practitioners have created art and art-science works that relate to metabolism. We observe and anticipate that many artists may return to Marx's original thinking about metabolism, particularly, his interest in the way that capitalist systems attempt to disentangle plants from animals including people, one natural and functional system was separated into two sets of problems with planetary implications: urban/rural waste and food insecurity/soil depletion.

Metabolic arts

The metabolic gaze can enrich understandings or experiences of a range of contemporary life-science engaged artworks. Tissue Culture & Art (TC&A), made up of Oron Catts and Ionat Zurr, along with collaborators specific to individual works, have offered a trajectory of artworks spanning from 2016 to the present, including iterations of COMPOSTUBATOR and Sunlight Soil and Shit (De-)Cycle (3SDC).⁸ While each work is distinct, they represent the formation of the artists’ thinking on these subjects over a number of instantiations. In the COMPOSTUBATOR series, a compost heap powers an incubator for sustaining a group of cells, generally through microbes heating water in conduit which flows past the incubator chamber and regulates the temperature. 3SDC was designed as a series of engagements around a set of agricultural tools with an emphasis on considering their implications and the philosophies behind these and other proposed system changes. 3SDC was launched with the COMPOSTUBA TOR in Freemantle, Australia, positioned by the artists on their project website in relation to agricultural systems: “The heated incubator sustains the growth of cells in a tissue culture flask to create what is today known as “lab-grown meat.” This type of “meat” is the cornerstone of what is called Cellular Agriculture – growing animal products without the animal.” A second portion of the project is the use of Alkaline Hydrolysis (also Aquamation or Biochemiation), based on an 1888 process for turning farm animals into crop fertilizer. The artists created their own version by hacking brewing equipment to break down much smaller animal bodies (meat and fishing waste) to fertilize the Hydroponic Garden, which is ironically supported by artificial lights driven by solar panels when possible. All of these elements are connected by the Control Room which gathers data from sensors across the project (thermometers, CO2 levels, pH monitors, cameras, etc.) The project aims to highlight the problem of increasing metabolic rifts in order to solve agricultural issues, something that seems implicit in many of the lab-based food systems proposed in the public sphere today. The artists write: “SymbioticA’s 3SDC builds resources to enable the community to accelerate metabolic rifts in agricultural innovation. This project considers whether the precursor to sustainable food systems will be the creation of a metabolic rift – where the means of production will grow ever distant from nature.” The artists directly invoke Marx’s metabolic rift as the target of their investigations by explaining that the exhibition is durational and will be changed over the course of the exhibition. Their aim at “maintaining the utmost clarity and transparency of our process is the key topromote understanding of the impact of metabolic rifts.” TC&A are working with metabolisms, in its nuanced form at the microbial level and at the broader level of our food system. 

Figure 1. Baum and Leahy's Cometabolise: A Holobiont Dinner, 2021. Detail installation photograph from The World is in You, Medicinsk Museion and Kunsthal Charlottenborg, 2021. Photo: David Stjernholm.

Baum & Leahy’s Cometabolise: A Holobiont Dinner (2021) was a living sculpture and an exploration in making the idea of the holobiont more familiar for viewers (Figure 1).⁹ A holobiont is an assemblage of a host and other species living in or around it that together form a discrete ecological unit.

From a holobiont perspective, our bodies are permeable living environments for our cells and the cells (and whole bodies) of other living things. It expands and blurs the notion of the host as a unit which could ever be extracted from this entangled set of relations. The artwork insists on the overlapping metabolic processes of the multispecies beings which help to metabolize our food. The artwork emphasizes the idea that bodies are porous, multispecies entities, highlighting the fact that humans and microbes eat and drink together. This bespoke dining set contains a sourdough culture held in a spherical glass carafe reminiscent of a bioreactor. A closed container for a starter culture is, of course, one of our most familiar bioreactors. The piece reminds us of the domestic nature of metabolism and invites thoughts of kitchens and laboratories. The artists emphasize the performative nature of the work as the microbes are constantly metabolizing and are fed while visitors are offered bread baked from the starter at the communal dining set.

George Gessert and Violet Ray’s BREATHE (2022) is a video piece investigating plant metabolism and its circular processing with our human breathing. Given his history with plants as a primary subject and medium for his artwork, Gessert’s work with plant metabolism is an obvious extension of those concerns. The yet unexhibited filmic work which was created with Gessert’s longtime interlocutor but new collaborator media artist Violet Ray, poetically explores the metabolism of plants through photosynthesis by exhibiting the process at a cellular level and using text to invite audiences to connect their own breath to the cycle of the plant’s photosynthesis and respiration. The artists ask us to pace ourselves with plants. As Gessert puts it, as we think of metabolism, “Why stick to humans and animals? Photosynthesis creates the air we breathe and is a key part of the planetary metabolism that supports most life on earth.”¹⁰ This focus on the larger cycles that metabolism is implicated in overlaps with the concerns of scale that appear across the metabolic arts.

Tagny Duff’s Wastelands (2015-2018) explores shit as an energy source in a speculative future without fossil fuels. Duff explores a deep future 500 years away when humans, through collaboration with bacteria and viruses, use their own feces as an energy source in small, portable bioreactors.¹¹ The Wastelands Project relies on Duff's many years of experience working with biotechnologies in its artistic practice, with a particular focus on viruses and on White Heather Hunter’s co-invention of a new bioplastic with art conservator Courtney Books. The latter provided the basis for the biomaterial development used to construct the bioreactor bags for the project. Metabolic ubiquity can create complications since living things always involve metabolisms but all art with living things may not query those processes or engage them directly. Yet, the search for metabolisms may provide new insights and new places of tension for contemporary arts about life and the Anthropocene.

References

1 J. Foster, “Marx’s Theory of Metabolic Rift: Classical Foundations for Environmental Sociology,” American Journal of Sociology 105, no. 2, 1999, 366–405.

2 H. Rogers, M. Halpern, Hannah D., K. Ridder-Vignone, The Routledge Handbook of Art, Science, and Technology Studies, London, Routledge, 2021.

3 H. Landcker, “Food as exposure: Nutritional epigenetics and the new metabolism,” BioSocieties 6 No. 2, Palgrave MacmillanUK, 2016, 167-194.

4 H. Landecker, “The Metabolism of Philosophy, in Three Parts,” In: Cooper, I. & Malkmus, B., (eds.), Dialectic and Paradox: Configurations of the Third in Modernity, New York, Peter Lang, 2013.

5 M. Black, Models and Metaphors, Ithaca, Cornell UniversityPress, 1962.

6 Taylor C See, B. M. Dewsbury, “On the Problem and Promise of Metaphor Use in Science and Science Communication,” Journal of Microbiology & Biology Education, 19(1), 19.1.46 (2018), Rabinbach, A. The Human Motor: Energy, Fatigue, and the Origins of Modernity, Basic Books, 1990.

7 A. Bencard, Stofsk(r)ifter: Metabolic Machines by Thomas Feuerstein. Hauser, J. (ed.), A. Bencard (ed.), M. Grünfeld, L. Whiteley, 1 ed. Copenhagen, Medicinsk Museion, Jun 2020.

8 See the Tissue Culture & Art’s pages at Compostcubator 0.1 & 0.2 – The Tissue Culture & Art Project (tcaproject.net) and 3SDC Sunlight Soil and Shit, sunlightsoilshit.systems.

9 For more, see Baum & Leahy’s art pages at Baum & Leahy, baumleahy.com.

10 Personal correspondence with the artists, February-April 2022.

11 For more, see Tagny Duff’s project page at Wastelands – Thoughts, images and experiments considering human-microbial relations on Earth in 2517, wordpress.com, and Whitefather Hunter’s documentation page at biotechnofeminism: laboratory craft | whitefeatherhunter 

Ephemera: Bubble Representations as Metaphors for Endangered Species

Harpreet Sareen1, Yibo Fu2, Yasuaki Kakehi3

1The University of Tokyo, 2,3 Parsons School of Design
Bunkyo City, Tokyo, Japan, New York, NY USA
sareen@xlab.iii.u-tokyo.ac.jp, fuy644@newschool.edu, kakehi@iii.u-tokyo.ac.jp

Abstract

The effects of a hierarchical relationship of humans with non-humans are now more pronounced than ever. Anthropogenic ecological stressors, including high levels of carbon dioxide, water scarcity, habitat fragmentation have led to disruption of climate systems, in turn endangering many local and global species. ephemera is an installation composed of glass vessels that show bubble images representing animals from all continents and ecologies currently under threat as per the IUCN Red list. These self-assembling bubble pictures, formed by nucleation of CO2 bubbles in water, are in a homeostasis at the beginning of the installation and shrink each hour to eventually disappear in a few days. The tension between the present endangerment and the urgency of the future action, manifests in the shrinking of these bubbles, invoking unnatural ephemerality due to the human effect. The fauna pictures in this installation, composed of carbon dioxide bubbles, symbolize the transitoriness of now threatened species.

Keywords

climate change, biological functions, endangered species, iucn, ephemeral art.

DOI

10.69564/ISEA2023-26-short-Sareen-et-al-Ephemera

Introduction

In 2015, Hawaiʻi was called as the extinction capital of the world.¹ Occupying a small land mass (0.25%), the islands had 25% of the endangered species in the US.² In intertwined ecosystems, direct and indirect effects human activities—monoculture, greenhouse emissions, extraction, and urbanization—on the biological diversity in a different location are significant³ but often hidden from sheltered lives of humans in another place.⁴, ⁵ Separated from nature in this manner, our thinking of endangered species is cultural, and public engagement with endangered species going forward depends on new structures of imagination. How then should we invest culturally in the fate of endangered species? What emotions do we collectively engender?

This work, an installation composed of ephemerality and melancholy, focuses not particularly on the beauty of imperiled fauna but on their current transience. Glucksmann’s philosophy of the ephemeral⁶ is particularly relevant where the moment is not static, but modulated and resonant. Drawing parallels with flux-images, where the process takes precedence, Glucksmann calls out the aesthetics of the ephemeral as fluid and polysensorial. In the context of this installation, images of various endangered species form with carbon dioxide bubbles—bubbles that shrink in a few days gradually waning the entire image. Materials within this installation’s cultural context are designed not for absolute control but for its relegation. 

Concept

Jean-Louis Boissier called out the roles of images as an interface,⁷ which we can act upon, manipulate, and transform—an image that vectorizes a relationship. We focus on such a relationship through evocative material with embedded cultural context of greenhouse gases and their effect on fauna. Specifically, the focus is on the ephemerality of the images. In prior transient works ⁸,,¹⁰,¹¹,¹² durability is determined by the intrinsic material properties in combination with the surrounding ecosystem, often deliberately designed to offer only partial or imperfect control. Material semantics - original meaning of a material - shape the perception and overall experience. The ambiguous ontological status of the ephemeral also makes it a “powerful metaphor for expressing nuances of memory, time and knowledge.”¹³ Interpretation of ephemeral meanings and consequences is thus beyond the object itself.¹⁵ 

1. Glass surface (13” x 13”) with image of a panda made through ~6000 nucleated and controlled bubbles in a carbonated liquid. © Harpreet Sareen, Yibo Fu, Yasuaki Kakehi.

In this work, images of endangered animals form on glass vessels through thousands of carbon dioxide bubbles. These bubble formations shrink by the hour to eventually disappear in a few days. The bubbles are formed by nucleation of CO2 bubbles in water and are representational images of animals from continents and ecologies currently under threat.

The installation is composed of ten glass vessels that are hydrophobically treated for controlled micro/nanostructures on surfaces. When carbonated liquid media is poured onto the surface, bubble generation is activated and bubbles of carbon dioxide of various sizes stick at defined points on the surface. Inspired by Sylvester et al., who mentioned bubbles as a material¹⁶ between “neither real nor fully virtual,” we extend the temporality of bubbles in this work. The process from initial nucleation to stabilization takes an average of twenty minutes, depending on the saturation of carbon dioxide in the liquids. Bubble images and patterns are stable for one day before they start to gradually shrink, completely disappearing in five days.

System Design

We created a novel fabrication technique to selectively modify hydrophobicity properties of surfaces (glass/plastic) such that bubbles in carbonated liquids nucleate at specific positions on surfaces.

2. Custom p5.js based design tool that converts images into bubble patterns based on original contrast to be then used for fabrication on a vinyl cutter. © Harpreet Sareen, Yibo Fu, Yasuaki Kakehi
Figure 3. Sample bubble pixelation. Left: Grayscale photo of a sea turtle. Right: Bubble-pixelated photo with visible features © Harpreet Sareen, Yibo Fu, Yasuaki Kakehi

This turns stochastic nucleation into controlled nucleation of bubbles for images, patterns, and text on various surfaces. Such surface modifications are initially invisible to the human eye. On pouring carbonated water in containers of various shapes (horizontal/vertical) and sizes, bubbles nucleate, coalesce, and grow to accurate sizes (1.0 mm – 5.5 mm) thus creating patterns, images, or text on surfaces. Such bubbles are highly stable and can keep sticking to the surfaces for longer than a week without significant disturbance.

Figure 4. a) Small bubbles nucleating and merging to predefined sizes, b) Penguin representation through bubbles. © Harpreet Sareen, Yibo Fu, Yasuaki Kakehi.

To fabricate the glass vessels, we use Regular (Home Depot, 599047) glass as a substrate which is initially hydrophilic (wettable) in nature. An illustration software (Adobe Illustrator) or our custom design tool is used to create desired patterns and print out an image mask using a vinyl cutter (Cricut Maker 3). The sticker mask is pasted on the surface and a ceramic hydrophobic coating is applied manually on the glass surface using a zigzag technique before drying for 24 hours. This creates a surface with distinct wettability regions that are invisible to the human eye.

Figure 5. a) Arrangement of ten glass vessels on the floor during the exhibition, b)-d) Photos of animals after stabilized bubble patterns were formed on the surface. © Harpreet Sareen, Yibo Fu, Yasuaki Kakehi
Figure 6. Viewers looking at bubble representations of animals in a kneeling position. © Harpreet Sareen, Yibo Fu, Yasuaki Kakehi

When carbonated water is poured into the containers, small bubbles nucleate first coalescing into each other becoming large in size corresponding with the size of the coating on the surface. These large bubbles may become buoyant at first due to the saturation level of CO2 in the liquid. As levels of gas normalize, bubbles stabilize at nucleation sites adhering to the hydrophobic regions with an attractive force¹⁷. Such stabilized configurations of bubbles last for five to seven days.

To design the bubbles patterns for this installation, we create a custom tool based on p5.JS that allows automatically generated wettability patterns by consolidating image import (color or grayscale), grayscale conversion and corresponding bubbles pixelation in a single pipeline. User imported images are analyzed against the background. Our algorithm analyzes the input image to create a comparative brightness map, following which darker areas of images are tagged as proportionately dense in bubble density and lighter areas are sparse in bubble density. For easy accessibility, the tool may be used online and outputs a .svg directly compatible with the fabrication machine for cutting the masks.

Exhibition and Experience Walkthrough

Purpura et al. have previously called ephemeral art been as “good to think with.”¹⁴ Their impermanence is a constitutive part of their aesthetic, and of the ways in which they come to act on the world. In the context of the exhibition shown at a major venue, our intention was to generate emotions of sadness or mourning among the viewers, over the endangerment of species. We reviewed the IUCN Red List of imperiled species at various threatened levels and chose ten species from marine, land and amphibious domains across all continents. The vessels were fabricated as per techniques described before, bottom lit for contrast of bubbles in liquid and setup in an L-shape. These were specifically separated from each other and laid on the floor for two key purposes: a) For viewers to bow their heads down or kneel as if paying homage, and b) To portray the vessels as final mementos of threatened species.

Such an exhibition design was shaped with a perspective of higher-levels of abstraction, focusing on carbon dioxide bubbles as material from a cultural perspective. This work is thus not temporary, rather has a directive intent to survive in the memory.

Conclusion

We presented ephemera, an installation where ephemeral images of endangered species are composed through bubbles of carbon dioxide. Bubbles with extended lifetimes slowly disappear in five to seven days, representing the images of endangered species as final mementos for humans. Through our exhibition, we intended the viewers to observe images in a drifting state and to be involved in the relationship depicted by the image. Glass vessels arranged onto the floor and viewed by the audience while kneeling are meant to invoke mourning or sadness among viewers—emotions that were listed over aspects of nature during the rapid modernization progress. Through this work, we explore a new ephemeral material and its aesthetic of affect, and believe that the ephemerality in this work represents a shift from the art object to a communicative act.

Acknowledgements

This work was initiated and primarily supported by JSPS KAKENHI (Grant Number 20H05960). We also thank the Provost Office Fund at The New School, and Labex Digi-Cosme (ANR11LABEX0045DIGICOSME) operated by ANR as part of Idex Paris-Saclay program (ANR11IDEX000302) for their partial supports.

References

1 Washington Post, (n.d.), Is Hawaii ‘the extinction capital of the world’? Exhibit A: The alala bird. Retrieved December 8, 2022, from https://www.washingtonpost.com/national/health-science/is-hawaii-the-extinction-capital-of-the-world-exhibit-a-the-alala-bird/2016/04/25/3f45c6ac-f210-11e5-89c3-a647fcce95e0_story.html

2 Jason D. Baker, Charles L. Littnan, David W. Johnston, "Potential effects of sea level rise on the terrestrial habitats of endangered and endemic megafauna in the Northwestern Hawaiian Islands," Endangered Species Research 2, 2006, 21-30.

3 Chen Xinli, Han YH Chen, Chen Chen, Ma Zilong, Eric B. Searle, Yu Zaipeng, Huang Zhiqun, "Effects of plant diversity on soil carbon in diverse ecosystems: A global meta‐analysis," Biological Reviews 95, no. 1, 2020, 167-183.

4 Judith Littleton, Gina McFarlane, Melinda S. Allen, "Human–animal entanglements and environmental change: Multispecies approaches in Remote Oceania," In The Routledge Handbook of the Bioarchaeology of Climate and Environmental Change, 493-510, Routledge, 2020.

5 K. Anggi Hapsari, Tim C. Jennerjahn, C. Lukas Martin, Volker Karius, and Hermann Behling, "Intertwined effects of climate and land use change on environmental dynamics and carbon accumulation in a mangrove‐fringed coastal lagoon in Java, Indonesia," Global change biology 26, no. 3, 2020, 1414-1431.

6 Christine Buci-Glucksmann, Emanuele Quinz, "For an Esthetics of the Ephemeral. Interview with Christine Buci-Glucksmann," Hybrid, Revue des arts et médiations humaines 1, 2014.

7 Margaret C. Flinn, "Jean-Louis's Moments of Jean-Jacques." Studies in French Cinema 10, no. 2, 2010, 141-154.

8 Daisuke Uriu, Naohito Okude, "Thanato Fenestra: photographic family altar supporting a ritual to pray for the deceased," In Proceedings of the 8th ACM conference on designing interactive systems, 2010, 422-425.

9 Virolainen Antti, Arto Puikkonen, Tuula Kärkkäinen, Jonna Häkkilä, "Cool interaction with calm technologies: experimenting with ice as a multitouch surface," In ACM International Conference on Interactive Tabletops and Surfaces, 2020, 15-18.

10 Elizabeth Diller, Diana Murphy, Ricardo Scofidio, Blur: the making of nothing, Harry N Abrams Incorporated, 2002.

11 Suzaan Boettger, "34. Within and Beyond the Art World: Environmentalist Criticism of Visual Art," Handbook of Ecocriticism and Cultural Ecology, 2016, 664-682.

12 Parés Narcís, Jaume Durany, Anna Carreras, "Massive flux design for an interactive water installation: WATER GAMES," In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, 2005, 266-269.

13 Allyson Purpura, "Framing the ephemeral," African Arts 42, no. 3, 2009, 11-16.

14 Marilyn Ekdahl Ravicz,"Ephemeralart: A Casefor the Functions of Aesthetic stimuli," 1980, 115-134.

15 Daniel Miller, "Artefacts and the meaning of things." In Companion encyclopedia of anthropology, 430- 453, Routledge, 2002.

16 Axel Sylvester, Tanja Döring, Albrecht Schmidt, "Liquids, smoke, and soap bubbles: reflections on materials for ephemeral user interfaces," In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction, 2010, 269-270.

17 Chen Shi, Xin Cui, Xurui Zhang, Plamen Tchoukov, Liu Qingxia, Noemi Encinas, Maxime Paven, et al. "Interaction between air bubbles and superhydrophobic surfaces in aqueous solutions," Langmuir 31, no. 26, 2015, 7317-7327. 

Jellyeyes – Symbiosis, Evolution and Vision

Jill Scott

ZHdK, Zurich, Switzerland info@jillscott.org

Abstract

Jellyeyes is an augmented reality installation especially created for the viewer based on symbiosis, touch, and biomimicry. Jellyeyes is inspired by narratives from the theories of evolution, and these stories involve the characters of a hunter-diver-tourist and a marine biologist. These narratives are inspired by the personal diving experiences of the writer and residencies in three different science research centers that focus on evolution in relation to symbiosis, marine biology, and neuro-visual systems.

Keywords

Augmented reality, symbiosis, evolution, eyes, climate change, barrier reefs, empathy.

DOI

10.69564/ISEA2023-27-short-Scott-Jellyeyes

Research

In 1967, Lynn Margulis created the concept of endosymbiosis. In this scientifically renowned theory, she claimed that bacteria was the agent of change.¹ Specifically symbiotic bacteria invaded the primary cells to create mitochondria and chloroplasts, the powerhouse parts of the cell as we know it today. Jellyeyes is about the focus of this theory on the evolution of light cells or fluorescent chloroplasts. They are found in fluorescent bacteria that live in the symbiont algae or Zooxanthellae inside any coral reef. These bacteria called Cyanobacterial Endosymbiont evolved to create fluorescent light that acts as a sun block for the coral and is sensitive to temperature change. These chloroplasts are also found in the pigmented cells of our eyes called Rhodopsin, that move through the photoreceptors of our retinas in reaction to incoming light. (Figure 1) 

1. Theory of Endosymbiosis, Zooxanthellae Coral Symbionts and Rhodopsin (photoreceptor pigment). ©Jill Scott

This research in evolution and symbiosis led to discussions at conferences with marine biologists in Australia.² In the Great Barrier Reef, symbiotic relationships between fluorescent algae and modern corals are currently being affected by human impacts on the lives of species in this environment. In fact, the health of these symbiotic relationships between the algae and corals, can shift the very roles they play in aquatic ecosystems, for example, as helpers or guardians against pathogens or enemies. The death of symbiont algae can be seen in the latest sad map about coral bleaching in Australia. (Figure 2) Unfortunately, a warmer climate not only affects chloroplasts in algae but symbiont levels of co-habitation as well as some species levels of visual acuity.

Reef
2. Barrier Reef Map and the symbionts. Zooxanthellae Algae. ©Jill Scott

Neuroscientists study the effects of excess light on photoreceptors, and their pigmented chloroplasts.³ Human photoreceptors have evolved through adaptation and are very similar to the retinas of the box jellyfish and the squid. Classified in evolution under the label of camera-based eyes, all three species share two jelly-like substances; aqueous humor or the moving fluid in the eye that is affected by temperature and vitreous humor: the jellylike shapes with fibers that are attached to their retinas. Hence the title: Jellyeyes!

3. The whole setup of Jellyeyes with 8 photographic panels and Augmented Reality interface on an iPad. ©Jill Scott

Because liquids in these eyes are affected by UV exposure, an increase in bad conditions cause cellular damage and very dry eye problems.³ Photoreceptors need healthy chloroplasts to function accurately. In humans they control low and high light levels and the photoreceptors are formed from neural tissues and directly attached to the brain (optic nerve). In squid: they control low light levels in photoreceptors that are also attached to their brains, but in jellyfish, they use basic pigmented cells to assess light and these are attached to each other by fibrous tissues. This last species has no brain but what is called a distributed brain. (Figure 4) But how can such information on evolution in neurocience, marine biology and symbiont behaviour be interpreted in an artwork? Could Augmented Reality and biomimicry help us “see” and understand the reef environment through the light receptors of the eyes of other species?

4. Eyes of the Australian Box Jellyfish and photoreceptors of the Squid and the Human. ©Jill Scott

Artwork

In summary, Jellyeyes is such an augmented reality investigation. It consists of a large photograph of a dead barrier reef and an iPad built into a sculpture of the human optic nerve. (Figure 3) The image on the iPad brings the photo alive! Sounds of water and bubbles immerse the viewer into the environment. The aim is to explore the delicate balance of this aquatic ecosystem through the 24 eyes of the box jellyfish, our own human eyes and eyes of squid or calamari. But the viewer can choose a temperature gauge to warm the water, and this changes the environmental conditions! Ocean warming affects the predator-prey relationship of these species and the symbionts algae leave the coral to die. The viewer can select the magnifying glass to discover how the loss of this symbiont caused a loss of a food for the coral to eat, a weakened immune system, bleaching, disease, and death. Warming also creates other food-algae and more jellyfish breed in the reef. In another scenario they can choose to have an empathetic and parabolic look through the eyes of the Australian box jellyfish at the changing predator-prey relationships of species in the reef. The viewer can choose the black and white view of the squid’s eye to discover stories about the human hunter or they can use the human eye to see different narratives.

The whole artwork is also available on an iPhone with a downloadable screen-based photograph. Here the aim is to help tourists understand the ecology and the climate change problems on the reef.

But how exactly are these narratives chosen by the viewers?

Evolution and Roleplaying

In Jellyeyes, these narratives are divided into three sections or menus that the viewer can explore: Co-Evolution, Structural Evolution and Comparative Evolution.

Co-Evolution:Here bacteria are seen as the agents of change.⁴ In this tribute to Lynn Margulis, Jennifer Margulis (her daughter) is turned into the character of a marine biologist, who collects evidence about the state of symbiont death in the coral and refers to her mother’s theories. She collects fluorescent bacteria from the algae (zooxanthellae) and measures the corals level of nutrition and growth, how they block excessive sunlight, control toxic compounds, stress levels and even ward off pathogens who attack the algae. The viewer’s interaction causes acidity and symbiotic destruction in this environment. (Figure 5)

Structural Evolution:This menu takes Charles Darwin’s theory of adaptation⁵ and applies it to the camera-based eye. How has this eye evolved in relation to its lineal ancestors. The result is a simplified tree of the evolution of visual perception. (Figure 8) For example, the jellyfish is an early example of our eye’s development and the squid another stage in the evolution of the light receptors of the camera-based eye. In this narrative, an ignorant, destructive tourist steals too much food. Sometimes, even digs into the coral to the horror of the scientist who tries to teach respect for this environment. (Figure 7)

Comparative Evolution:James Lovelock once said that life has created the conditions for its own existence.⁶ In this menu, the viewer can choose either the eye of the box jellyfish, the squid, or their own eye to see the relationship between the behavior of these three species in the barrier reef environment. The ignorant diver tourist plays around with the poisonous Australian box jellyfish and every day more of these very stingers swim in the reef. Ocean warming shifts the ecology of the reef’s predator-prey relations and behaviors like these will have a backlash and harm our own existence. (Figure 7)

As in previous artworks,⁷ Jellyeyes uses the sensorial strategies of touch and biomimicry, to create performative roleplaying to augment the above narratives and their repercussions. In Co-Evolution the role of the participant becomes the CO2 emitter and through touch he or she or they, learn about the reactions of symbiont bacteria, the survival of the coral, depreciation of the symbiont algae, squid, and reproduction of the box jellyfish, as well as the interdependencies between these species. In Structural Evolution, the participant becomes the investigator and learns through touch about the similarities of vision between species and about the evolution and health of the light receptors in the photoreceptors. In Comparative Evolution, the viewer becomes an empathizer, they see through the eyes of these species and can change these views with touch to identify what they see. Here, the biomimicry of sight is used to witness the human environmental impacts on the reef itself. 

Experience1
5. Co-Evolution: Scientist collecting symbiont samples from Alge on the Barrier Reef. ©Jill Scott
6. Structural Evolution: The black and white parabolic view from the eyes of the jellyfish and the menu to choose different eyes. ©Jill Scott
7. Comparative Evolution: The hunter tourist interacts with the jellyfish and the scientist. ©Jill Scott

Conclusion

Perhaps, augmented reality art might be an effective catalyst for climate activism and education. Here the aims are to reveal the influence of evolution on species survival in relation to the health of their habitats, to the need for inherited variations and to the mutual benefits of keeping symbiosis alive. It is dangerous for us and them not to mitigate climate change. Jellyeyes clearly shows the effects of our fossil fuel emissions on the survival of the species in the reef. It offers a novel creative interpretation of structural evolution and ecological interconnections. Here, the viewer becomes immersed in the symbiotic sensory relationship between algae, vision and modern corals. Viewers say that Jellyeyes encouraged them to think about variations and reproduction problems as well as symbiotic mutualism. They empathized with non-humans by “seeing” through the eyes of other species. Afterall, the future of the barrier reef on this planet is in the eyes of many species, but it is now in our hands!

References

1 L Fester, R. Margulis, Symbiosis as a Source of Evolutionary Innovation Speciation and Morphogenesis, MIT Press, 1991.

2 ACRS Australian Coral Ref Society Conferences, https://australiancoralreefsociety.org/conference/.

3 Neuroscience chloroplast research, https://www.sciencedirect.com/science/article/pii/S2666469020300154, Accessed 2023.

4 L Margulis, Symbiotic Planet: A New Look at Evolution Basic BVooks, 1999.

5 Charles Darwin, On Origin of Species, Pub John Murry, 1859.

6 James Lovelock, GAIA: A new look at Life on Earth, Oxford Uni Press, WW Norton, 1988.

7 Neuro_Eco _Media, www.jillscott.org

8. The Evolutionary tree of the.Camera Based Eye. ©Jill
Figure 9. A viewer meets the Marine Biologist. ©Jill Scott
10. The AR interface case based on the Human Optic Nerve. ©Jill Scott
Partners

HFF, The Munich Film Academy. Munich, Germany
Dr. Lisa-Ann Girshwin (CSIRO Australian Marine Stinger Advisory Services, Australia)

Scientific Consultants

Prof. Dr. Stephan Neuhauss (Institute for Molecular and Cellular Research, Neurobiology, University of Zurich, Switzerland) Prof. Christopher Robinson (Aquatic Research EAWAG, ETHZ, Switzerland)

Funding

Pro Helvetia, The Swiss Arts Council
HFF, The Munich Film Academy. Munich, Germany

Production Credits

Concept & Directing: Jill Scott Production: Marille Hahne Camera: Julia Daschner
Light: Georg Nikolaus Animation: Natascha Jankovski Postproduction: Moritz Huber Programming: Nikolaus Völzow

Biography

Jill Scott is a media artist, a writer and art and science researcher. She is professor emerita at the Zurich University of the Arts (ZHdK) in Zürich and founded their Artists-in-Labs Program in 2000. Her own artwork spans 44 years of production about the human body and body politics. In the last 20 years she has focused human health based on research into molecular biology, neuroscience, and ecology. She has had many international exhibitions in both art and science venues. She also directs LASER Salon in Zurich for the Leonardo Society USA and writes books on art and science (Springer and de Gruyter).

www.jillscott.org

Technoshamanism: Symbiotic Techniques of Art and Healing

Edward Shanken

University of California, Santa Cruz United States eshanken@ucsc.edu 

Abstract

Technoshamanism combines traditional shamanic technologies with emerging technologies based in silicon (dry), biology (wet) and hybrid (moist), in the service of healing and sustaining life. This paper explores how contemporary artists pursue expanded forms of consciousness by symbiotically joining technoscientific tools and shamanic techniques.

Keywords

Shamanism, Technoshamanism, Art, Consciousness.

DOI

10.69564/ISEA2023-28-short-Shanken-Technoshamanism

More than ever before, contemporary artists, theorists, scientists, and activists need to pay more attention to socalled indigenous knowledge. – Guillermo Gomez-Peña 

Introduction

Mircea Eliade defined shamanism as “techniques of ecstasy.” He noted that, across cultures, it is fundamentally a technology of healing that is "at once mysticism, magic, and religion."¹ Anthropologist Michael Harner claimed that shamanic traditions around the world have developed a broad range of technologies - from sonic drivers (drums) to plant medicine rituals—in order to achieve trance states that offer insights beyond those available to typical waking consciousness.²

Shamans are both of this world and of the world(s) beyond. They communicate with spirits and ancestors in other dimensions, learn from them, guide members of the community to them, and harness their power to heal and protect individuals and the community. The shaman can embody the consciousness of other beings, including other animals. A shaman also can release an errant spirit that has stricken a member of their community with illness, sending the uninvited “hitchhiker” on its way. This can be dangerous, so shamans must be very strong, capable of self-healing, and masters of their practice.

Shamanism may heal at scales ranging from the individual to the global. Indeed, following the Huni Kuin concept of Xinã bena–“New Time”—some shamans/pajés of the Amazon have decided that the preservation of the Earth demands sharing their cultural insights and healing technologies more widely than in the past. This strategy has even led to the creation of Huni Kuin—Beya Sina Bena, a videogame about the ancient stories of these indigenous people, scheduled for release in 2024. The developer, Philosophical School of Games, claims that the game was produced in collaboration with the Huni Kuin communities of the Jordão River, in Acre, Brazil.

Brazilian cultural critic Fabiane Borges notes that “apart from any possible encounter between technology and shamanism, shamanism is itself a technology for the production of knowledge. ³ Similarly, I take technoshamanism to join a combination of traditional shamanic technologies with emerging technologies based in silicon (dry), biology (wet) and hybrid (moist), all in the service of healing and sustaining life.

The same can be said of art, historically interwound with techne, the Ancient Greek word for art and the etymological root of technology. To risk stating the obvious, the use of shamanism for healing vastly predates the application of science to allopathic medicine. The recent decriminalization of plant medicines and the legalization of psychedelic-assisted therapies in some jurisdictions demonstrate increasing acceptance of ancient indigenous technologies and marks a merger of diverse healing methodologies.

In a parallel manner, the engagement of contemporary art with shamanic traditions bridges knowledge domains and augments art’s potentials to heal and to envision the future. These goals characterize a wide range of artistic practices, from Ernesto Neto’s A Sacred Space to Roy Ascott’s “shamantic web” of dual consciousness, to Anandha Ray’s Covenant VR virtual reality ceremony/performance. A shaman was even part of Refik Anadol’s cohort of artists and software engineers at the Google Artists and Machine Intelligence program in 2016.

Nonetheless, there is a dearth of literature that addresses the nexus of contemporary art and shamanism. Until recently, the work of Joseph Beuys dominated these discourses, with a few notable exceptions. Jack Burnham’s “Artist as Shaman” (1974) applied a structuralist method to interpret shamanic aspects of Dennis Oppenheim’s work. In “Art in the Dark” (1983) Thomas McEvilley interpreted performance by artists from Gunter Brus to Kim Jones as shamanic acts of expression and catharsis. “Weaving the Shamantic Web...” (Ascott, 1998) claimed that “this ancient ritual mirrors our contemporary artistic aspirations using digital technologies.” ⁴ Further essays by Ascott and his circle, especially Lila Moore and myself, have followed in the 2010s and 2020s.⁵

In 2017, Christine Macel curated The Pavilion of Shamans, one of nine “trans-pavilions”, bringing shamanism into the center of mainstream contemporary art (MCA) discourses at the 2017 Venice Biennale. Notable works included Juan Downey’s Circle of Fires Vive (1979), Jeremy Shaw’s Liminals (2017), and Neto’s A Sacred Place (2017), the centerpiece of the pavilion. Despite scant media attention directed to this pavilion, art critic Tess Thackara (2017) highlighted it and discussed many additional international artists, indicating broad interest in the topic, if not by MCA critics, then by makers. ⁶ Indeed, my 2020 Facebook post about technoshamanism yielded over one hundred comments, primarily by artists about works of new media art (NMA) that fit the term. More recent MCA scholarship includes critiques of Mircea Eliade’s and Michael Harner’s universalizing of shamanism and Claude Levi-Strauss’ and Burnham’s psychopathology of the shaman, and is informed by Viveiros de Castro’s concept of “equivocation,” offering a more nuanced and theoretical approach to shamanism and contemporary art. ⁷

Borges and others use the term “technoshamanism” to problematize the destructiveness of western technoscience and its incursion on indigenous technologies of shamanism, which, by contrast, honor the sanctity of the Earth. Technoshamanism names the messiness of cultural hybridity and the commodification of shamanic traditions, including ayahuasca tourism and the gamification of Huni Kuin culture, resulting from colonization and globalization. The visual arts of indigenous cultures, including Shipibo textile patterns, Papunya Tula dot paintings, and Vodun rituals, are also being hybridized with, and appropriated by, international contemporary art and visual culture. Such appropriations raise vital questions about Technoshamanism as a form of aesthetic practice: How are contemporary artists engaging with the cosmologies, technologies, and intellectual property of shamanic cultures in ways that honor and benefit the indigenous and mestizo peoples that have cultivated and preserved shamanic traditions? How are indigenous artists engaging with shamanic (and postindustrial technologies) in ways that strengthen their communities and contribute to the discourses of contemporary art?

Mindful of these issues, my research is primarily concerned with the following speculative prospects:

  1. How can artists embrace visionary consciousness?
  2. How can art support entheogenesis (becoming divine together) by joining ancient shamanic techniques and contemporary technoscientific tools?
  3. How can art catalyze greater awareness of what Thich Nhat Hanh calls interbeing (the unity of all things) to help heal the Earth?

Early Technoshamans

In 1997, British artist Roy Ascott participated in ayahuasca ceremonies in Brazil, which has had a profound impact on his praxis. He theorized parallels between the dual consciousness that emerges in shamanic ceremonies and the expanded field of consciousness afforded by emerging technologies. “In many respects,” he wrote in 1998, “this ancient ritual mirrors our contemporary artistic aspirations using digital technologies.” Claudia Jacques claims that this dual consciousness can only be manifested through “indirection, shared participation, and metaphor,” core tenets of the telematic art that he pioneered. This is the realm, she adds, that “visionary thinkers, creative artists, and shamans alike aspire to experience and explore.” ⁸

Ascott (1998) describes the shaman as “the one who ‘cares’ for consciousness, for whom the navigation of consciousness for purposes of spiritual and physical wholeness is the subject and object of living.” He witnessed the pajé “passing through different layers of reality, through different realities...” The shaman inhabits a state of dual consciousness, of “seeing at once both inward realities and the outward surfaces of the world.”

Although it predates Ascott’s participation in shamanic plant-medicine ceremonies, his 1989 telematic artwork, Aspects of Gaia: Digital Pathways Across the Whole Earth, embodies a technoshamanic, dual consciousness. This installation integrates the global relationality of digital technologies with the shamanic commitment to healing the Earth. Aspects of Gaia took its inspiration from atmospheric scientist James Lovelock’s holistic Gaia Hypothesis (1979), which proposes that the Earth (Gaia) is a living organism, a self-regulating, complex system that maintains the conditions for life on the planet. Artists, scientists, shamans, musicians, visionaries, and indigenous artists were invited to participate by sharing their feelings and beliefs about Gaia, which became work’s content. Ascott likened the participants to healers who access the meridians of the earth’s nodes and creatively interact with the flow of data to perform a type of “global acupuncture.” Their contributions suggested, for Ascott, a telematic “noosphere,” an emergent field of consciousness generated from interconnected individual consciousnesses, which might help harmonize and heal the planet.” ⁹

For American composer and electronic music pioneer Pauline Oliveros, cultivating expanded forms of consciousness was the primary focus of her career. For this early technoshaman, the wisdom of the body is crucial to accessing expanded states of consciousness. “I have progressed through many changes in music technology from the end of the 1950s to the present,” she wrote. “Along the way I developed a bodily relation to machines for making music. It has always been necessary.... because of the essential knowledge of the body that is preconscious and nonverbal.” ¹⁰ Her experience of performing improvised music parallels shamanic double-consciousness: Oliveros became a channel through with spirits from other dimensions could communicate with and heal us: “This altered state of consciousness in performance is exhilarating and inspiring,” she explained. “The music comes through as if I have nothing to do with it but allow it to emerge through my instrument and voice.” ¹¹ Oliveros’ ideal attributes for a future artificial intelligence “chip”—with which she could make music—are refreshingly expansive. They include abstract psychic abilities, that seem to parallel shamanic intentions of achieving unity and healing on a cosmic level. These include: the ability to understand the relational wisdom that comprehends the nature of musical energy; the ability to perceive and comprehend the spiritual connection and interdependence of all beings and all creation as the basis and privilege of music making; the ability to create community and healing through music making; the ability to sound and perceive the far reaches of the universe much as whales sound and perceive the vastness of the oceans. This could set the stage for inter-dimensional galactic improvisations with yet unknown beings.¹²

Wake me up when ChatGPT can offer these features! For Oliveros, expanded consciousness on a galactic scale was the foundation of healing, a form of service that she performed within the frame of a feminist ethics of care.

Chilean-American artist Juan Downey is another early technoshaman. His 1979 video, The Laughing Alligator, resulted from the many months he and his family lived with the Yanomami people in southern Venezuela, on the border of Brazil. He participated with his Yanomami hosts in rituals using yopo, a plant medicine that contains a variety of DMT alkaloids. The artist came to regard shamanism as one of the most powerful elements in the bond that linked the Yanomami to their surroundings, to the earth, and to each other as part of a larger unity. As he wrote, A white and round place opens up in the front of my brain. Excretions of light that vaguely align into circles, the intensity of a spiral or the infinite peace of a mauve color. [...] I want to enter into the white space of my empty consciousness. ¹³

From spending a lifetime in trance-like states, Downey expanded his consciousness. He tuned into the consciousness of others. And he helped others do the same. As a result, his work plays an important role in stimulating new ways of thinking that are the prerequisite to healing society’s pathologies and to recreating the world in a more sensitive, inclusive, and caring way.

Contemporary Technoshamans

American dancer and choreographer Anandha Ray’s virtual reality screen-dance, Covenant VR (Figure 1), was a revelation. Joining shamanic healing rituals, modern choreography and dance performance, and exquisite VR cinematography by Gary Yost, it created a unique container for embodied catharsis mediated through virtual reality.

After putting on the headset, the next thing I knew, BAM! I was suddenly transported to a dance rehearsal studio, at a virtual distance no more than a meter from a striking, dark-skinned dancer (Linda Steele II.) I felt right there in the action, far closer than front row seats, inhabiting a perspective and proximity typically seen only by other dancers. Because this is a 360-degree VR experience, one’s view is not limited to a frontal perspective.

Figure 1. Anandha Ray, Covenant VR, 2019. ©Anandha Ray.

As viewers enmeshed in the VR environment, we witness the shamanic ceremony from a privileged vantage, so close that we can virtually touch the protagonist, a proximity that heightens our sense of being there, of feeling the healing and of experiencing it ourselves. Yost’s cinematographic wizardry is the ideal match for Ray’s shamanic approach to dance, which “allow[s] movement to open portals of inquiry to better understand the state of being human.” ¹⁴ This tour-de-force of Technoshamanism offers an enthralling and cathartic experience.

Korea has a rich, living tradition of shamanism that continues to inspire artists, including technoshaman Kim Jeong Han. His high-tech multimedia installation BirdMan (2005) is deeply informed by Korea’s shamanic and Buddhist traditions. The artwork asks fundamental questions: How do human beings conceptualize the world? How do birds conceptualize it? Can a hybrid world that joins human and nonhuman qualia (the internal, subjective physiological component of sense perceptions) transform perception beyond the limits of human physiology? If, as Donna Haraway claims, new perceptions create new metaphors, can the experience of another species’ perceptual reality help create hybrid perspectives, marked by greater empathy and ecological sensitivity?

The concept of a hybrid bird-man appeared to Kim in a dream. Due to a traumatic childhood experience, even as an adult, Kim’s fear of birds prevented him from helping a one-winged bird that was suffering and dying. He dreamed that he learned bird language from a monster with a bird head and only one wing. The dream and the artwork can thus be interpreted as an effort by the artist to attain catharsis and heal a trauma through aesthetic and shamanic means. As Kim and his co-authors noted:

“In Korean tradition, some shamans can share their own bodies with the deceased soul. Whenever a shaman is possessed by the spirit of the dead, s/he acts, speaks and senses like another person, as if borrowing the perception of the deceased. This moment looks like a coexistent state of the living body and the dead in which perception and identity of the two is hybridised.” ¹⁵

The Buddhist idea that “the ‘Self” is not different than the “Other” is another prevailing concept in BirdMan. The work offers the audience an opportunity to experience a form of hybrid perception that joins human and avian “qualia.” (internal and subjective physiological component of sense perceptions). Kim’s work does not explicitly represent accoutrements or scenes of shamanic healing. As a result, the audience is not likely to visually identify shamanic elements in it. Rather, BirdMan is driven implicitly by a shamanic perspective. Kim leads his own ritual of self-healing through his artistic practice. By enabling us to metaphorically become one with bird, by offering us an experience of hybrid avian-human perception, his work enables us to expand our consciousness beyond the limits of our embodied human minds by joining self and other. It enables us to create new identities in between humans and non-humans. And, as a result of this, it enables us to create new metaphors to live by and to live with.

Such ideas echo Donna Haraway’s eco-feminist theories of multispecies worlds. She advocates a concept of kinship or “making kin” that joins all beings: “all earthlings are kin in the deepest sense [...]. All critters share a common ‘flesh,’ laterally, semiotically, and genealogically.” She applies the term “sympoietic” to emphasize the collective process of poetic emergence in which all beings are collaborators in the process of the Earth’s becoming. “Who and whatever we are, we need to make-with—become- with, compose-with [...].” Taking care of the Earth, for Haraway, demands caring for the diversity of beings, and “multispecies ecojustice” must be not only a goal but a means to living well, together, as kin. By “staying with the trouble,” she proposes, “[m]aybe, but only maybe, and only with intense commitment and collaborative work and play with other terrans [inhabitants of Earth], flourishing for rich multispecies assemblages that include people will be possible.” ¹⁶

Buddhist luminary Thich Naht Hanh has taught that, “Our own awakened consciousness is what can heal the Earth.” Indeed, the pressing and enduring concerns of global warming and the abuses of technology demand that we expand our perceptual domain in order to heal ourselves, our kin, and our planet. To do so, we must expand our metaphors and our means and channels of communication. We must summon the full power of art and technology, while we equally harness ancient shamanic technologies and other modes of entering trance, expanding consciousness, experiencing ecstatic states, fostering communication among all beings. If we want to have a future, the artists of the future must serve as beacons of hope and as active participants in healing the world and our relationship to it. Art, as a “psychic dress-rehearsal for the future,” to quote Jack Burnham, must embrace kinship and harmonious co-emergence with all beings. We must become divine together. Today there is no greater artistic calling, no greater aesthetic necessity than helping to heal and preserve Earth/Gaia’s biodiversity for posterity. Technoshamanismis a potent strategy to move in that direction. 

References

1 Mircea Eliade, Shamanism: Archaic techniques of ecstasy. c. 1951, Princeton University Press, 2004, xvii–xxvii.

2 Michael Harner, "My Path in Shamanism." International Journal of Transpersonal Studies 31.2, 2012.

3 Fabiane Borges, “Seminal Thoughts for A Possible Tecnoshamanismnd, np, c. 2014, Quote edited for clarity.

4 Roy Ascott, “Weaving the Shamantic Web: Art and Technoetics in the Bio-Telematic Domain.” c. 1998, In Telematic Embrace, ed. Edward Shanken, Berkeley, University of California Press, 2004, p. 356-62, Subsequent references to and quotations from Ascott are from this source.

5 Lila Moore, “The Shaman of Cybernetic Futures: Art, Ritual and Transcendence in Fields of the Networked Mind” Cybernetics and Human Knowing 25, 2018, p. 2-3, 119-41.

6 Tess Thackara, Why Shamanic Practices Are Making a Comeback in Contemporary Art, Artsy, 2017, accessed June 13, 2023, https://bit.ly/48hsvk4

7 This literature includes Robert Wallis, “Art and Shamanism: From Cave Paintings to the White Cube," 2019, Camila Maroja, “Persistence of Primitivism: Equivocation in Ernesto Neto’s A Sacred Place and Critical Practice”, 2019, and Karen Gonzalez-Rice, “Revisiting "Art in the Dark", 2023.

8 Claudia Jacques, “Forward: A Tribute to the Messenger Shaman: Roy Ascott,” Cybernetics and Human Knowing 25, 2018, 5-15.

9 For more on Ascott, art, technology, and shamanism, see Lila Moore and Edward Shanken, “Roy Ascott” in Charlie Gere and Francesca Franco, eds. Bloomsbury Encyclopaedia of New Media Art, London, Bloomsbury Press, forthcoming.

10 Pauline Oliveros, “Improvising Composition: How to Listen in the Time Between,” c. 2012, in Gillian Siddall and Ellen Waterman, eds. Negotiated Moments: Improvisation, Sound, and Subjectivity, Durham, Duke University Press, 2016, 83.

11 Pauline Oliveros, Deep Listening: a Composers Sound Practice, New York, iUniverse Inc. 2005, xix.

12 Pauline Oliveros, “Quantum Improvisation: The Cybernetic Presence,” in "Sounding the Margins: Collected Writing 1992-2009, Kingston NY, Deep Listening Publications, 2009, 53.

13 Juan Downey, Dibujando con los Yanomami, (Caracas: Galería Adler Castillo, 1977): np. Quoted in Edward Shanken, “Pushing the Limits. Surrealism, Possession, and the Multiple Self: Juan Downey and The Laughing Alligator” in Maricris Herrera, ed. Juan Downey, 1940-1993. Mexico City, Ediciones MP, 2019, 527-42.

14 Anandha Ray, Covenant VR ,2019, https://youtu.be/Y-B_KqbrS4w?si=zXeMpmxWCuCNv4QY

15 Kim Jeong Han, Kim Hong-Gee, Lee Hyun Jean, "The BirdMan: hybrid perception," Digital Creativity 26, 2015, 1, 56-64.

16 Donna J. Haraway, "Anthropocene, Capitalocene, Plantationocene, Chthulucene: Making Kin," Environmental Humanities 6, 2015, 1, 159-165.

17 Thich Nhat Hanh, Love Letter to the Earth, Berkeley, Parallax Press, 2013, 56. 

Symbiotic Collaborators: The New Creative Subject in Postdigital Participatory Art

Gyung Jin Shin

Hong Kong Baptist University, School of Creative Arts Hong Kong
gjshin@hkbu.edu.hk

Abstract

As the Internet promotes participatory culture, contemporary network-driven participatory art, which I refer to as “postdigital participatory art” (PPA), has introduced additional revolutionary creative subjects. PPA induces a distinctive type of collective agency beyond mere collaboration among individuals by means of the participatory architecture of the web. These multiple participants distribute the authority power of creation throughout the network, transcending the limitations of time and space. In this paper, I attempt to theorize the attributes of these new creative subjects, which I refer to as “participant-superjects,” with the concept of superject serving here to indicate “power by modulation.” I outline the attributes of these diffuse creative subjects and gauge their radical possibilities in terms of the agenda of experimental art. I argue that, based on the new sense of relationality, materiality, and ontological perception associated with the postdigital environment, these unique creative subjects are able to open up a new dimension of creativity that differs from the modernist model, which emphasizes the creativity of the individual. I hypothesize that the fluid power driven by this new creative subject exerts a latent force in building new social relations outside the logic of the capitalist system.

Keywords

Postdigital participatory art, author discourse, superject, participation, collaboration, participant-superject.

DOI

10.69564/ISEA2023-29-short-Jin-Shin-Symbiotic-Collaborators

Introduction

With the death of the modern subject in the twentieth century, the divinization of single authorship on which modern art relied has faced challenges and slowly collapsed along with the notion of “open work,”¹ the postwar avant-garde of the 1960s, and author discourse.², ³ Artists and theorists of the previous century attempted to undermine the traditional relationship between artist and spectator, arguing instead for open-ended interpretations. These developments spurred a renegotiation of the artist-spectator relationship and, in turn, the invention of new types of creative subjects. From this new perspective, the status of the audience, as a “viewer-turned-participant,” has become almost equal to that of the artist, and recent forms of contemporary art that are co-authored, collaborative, and participatory have inherited the preference for antiauthorship. Such forms of art, including “relational art”⁴ and “participatory art”⁵,, ⁷ have flourished since the 1990s as artists have attempted to distribute the authority power traditionally associated with the singular artist by encouraging the participation of viewers and envisioning new social relations.

In a hyperconnected environment that facilitates participation and collaborative creation, contemporary participatory art has become ubiquitous and heterogeneous, transcending the boundaries of online and offline. Recognizing that Bishop’s concept of “participatory art” overlooks the influence of technology, I seek to help fill the resulting conceptual gap by coining a new term, “postdigital participatory art” (PPA), to describe digitally mediated co-creation that relies on digital networks to encourage audience participation. This new form of digitally mediated co-creation reflects changes in the perceptions of time and space that have been described as “postdigital.”⁸, ⁹ In exploring PPA, I pay particular attention to the emergence of new authority power that is fluid and ephemeral. Though such unique participatory creative subjects are profoundly observable in recent digitally mediated participatory art, they have yet to receive careful analysis in terms of scope, motives, characteristics, and patterns. Accordingly, I consider here the manifestation and radical possibilities of this unique creative subject, which involves anonymous, networked participants mass-produced on networks.

Participant-Superject: The Unique Authors in the Network

As I conceptualize it, network-driven PPA relies heavily on collaboration among participants and often lacks a teleology. Creators, materials, artworks, and spectators, in the absence of a predetermined blueprint for object-making, co-emerge in the processing of inconclusive events. This development gives rise, in turn, to the distinctive characteristics of a new creative subject that is ephemeral and fleeting, untethered as it is to a fixed plan and, often, anonymous and unrecognized despite contributing significantly to the act of creation.

The Jogging (https://thejogging.tumblr.com/), an ongoing, network-driven project, exemplifies the attributes and modalities of these authors. [Figure 1] Utilizing a scroll-down thread on Tumblr that has continued over several years, anonymous volunteers on the network Photoshop, reproduce, and reblog the images associated with the original images uploaded by artists Brad Troemel and Lauren Christensen in 2009. Troemel described his role as “initiating” and said in an interview that “‘Jogging’ refers to a work flow, constantly moving, and not really focusing on any one thing, but rather to just continue forward.”¹⁰ Without an individual artist furnishing a predetermined intention or conclusion, such postdigital participatory projects are observable as the inconclusive, event-embracing agencies, networks, and raw digital materials (e.g., information or data) involved in a work. 

1. Brad Troemel and Lauren Christensen, The Jogging (2009-), Source: The Jogging wesite, https://thejogging.tumblr.com/. 

From Author-Subject to Participant-Superject

This new material variability in postdigital participatory art thus entails a reframing of the traditional notion of author-subject. In a digital environment that strengthens temporality rather than spatiality, digital participants have the potential to become temporally modulated subjects, or superjects, bearing latent power by modulation rather than the power of individuals. As the antithesis of the author-subject status, I coin the term participant-superject to describe the diffuse creative subjects that result from the blurring of the artist-spectator boundary.

The concept of the superject, which refers here to “power by modulation,” has proved useful in describing the special modalities of multiple agents that exist digitally when they are involved in PPA as opposed to the modalities of physical participation. Deleuze adopted Whitehead’s concept of the superject in the 1990s to illustrate the shift in subjectivity accompanying the rapid advance of digital technology at the time.¹¹ According to Deleuze, a subject, when perceiving an object that is continually changing, can be defined as a point of view that is likewise in a state of variation rather than fixed or determined in advance.

The superject implies a “plural” subjectivity that cannot be fully described using the concepts of a constant I and we and, instead, implicates a new status of the subject, a temporal I and we. The superject, in Deleuze’s language, is an inherent multiplicity folded into a collective unity. 

The Three Phases of the Participant-Superject

Before gauging the radical possibilities of the participant-superject, it is useful to consider the concept’s unique characteristics. Following Whitehead, Deleuze, and Savat, I describe the three main phases of the superject as 1) temporal, 2) affective, and 3) condividual. The temporal phase relates to the eventual nature of the superject. Referring to this phase, Deleuze described the superject as an event rather than an essence. Adopting this view, Savat explicated the specifics of the temporal aspect of the superject in the context of his analysis of digitality. According to him, under the new temporal and spatial conditions of the digital environment, a superject exists as what Deleuze called a “dividual,” an event that is ceaselessly modulated as a code, in contrast with an embodied or spatial “individual.”¹² In this context, the manner of existence becomes continuous, fluid, and momentary.

The second phase highlights the affective nature of the superject. Whitehead envisaged this facet of the superject in an attempt to separate the subject per se from its experience of the world.¹³ Thus, he proposed, the superject simultaneously emerges through the intake of “data” in the form of sensory perceptions that precede consciousness, unlike the subject that accumulates data. In this context, feelings mediate superjects and their associated data. This aspect of the superject focuses on the moment of affective experience shared throughout the network.

Third, the superject implies the consolidation of collective agents, that is, condividuals. To be specific, unlike the individual, which retains distinctive characteristics, the dividual tends to connect with other similar components and combine with other dividuals to form condividuals).¹⁴ Existing as metadata or data in the digital milieu, the dividuals can be used to build superjects in the network.

The participants in PPA, or participant-superjects, embody these phases of the superject—again, temporal, affective, and condividual—which overlap and are interrelated. Rather than being constrained to a physical venue for performance or exhibition, the participant-superjects appear temporarily, take part in the creative process, and disappear in an instant. As the sequence of the action does not “take place,” the actors rely heavily on temporality rather than spatiality. Further, the participants in PPA tend to manifest momentary feelings or emotions rather than the conscious experiences inscribed in a work. As The Jogging well shows, the intuitive and improvisational responses of the participants manifest in such functions as liking, retweeting, and sharing on social media. Lastly, these participants, as dividuals, merge easily and unite readily with others thanks to the anonymity and easy access that the Internet affords. 

Symbiotic Collaboration Among the Participant-Superjects

This new creative subject of PPA has the potential to fall into the trap of capitalism or to support resistance to it. Stiegler used the term pharmakon to describe the aspect of technology that intoxicates and cures, and postdigital participation has a similar “pharmakonic” effect, serving as simultaneously toxin and remedy.¹⁵ In the 1990s and 2000s, the expectation was that the participatory environment of the web would promote widespread self-awareness, the formation of grassroots communities, and co-individuation by connecting individuals without respect to age, location, or gender. Since the network has also been viewed as an arena for the fulfilment of capitalist desires, however, the participants in it found that they were treated as consumers and unpaid labourers. Most of the major social networking platforms have reverse-engineered the very structure of the participatory web to produce profits from the databases generated as a by-product of the daily behaviours of online users. Moreover, governments and internet giants alike use the network as a source of big data for analysing superjects and recognizing and forecasting social trends.

Regarding the paradoxical loop of postdigital participation problematized above, I argue that the modulatory power of the participant-superject could radicalize the agenda of experimental works by altering the form that power takes. This new form of power, again as Deleuze pointed out, refers to the modulation of dividuals.¹⁶ For Deleuze, of course, such power deserves criticism, but it also serves as a positive starting point for the arguments presented here. Considering the dividual a latent power of struggle, Raunig claimed that the dividuality emerging as self-division in contemporary social media could promote new forms of dispersed resistance against machinic capitalism.¹⁷ In the same vein, Savat insisted that a new mode of politics involving the dividual, which he called the “politics of fluids,” has become a more critical field of participation than the modern “solid politics” involving individuals.¹⁸ Thus, he affirmed that the potential inherent in “fluid action” can be utilized and realized in constructive ways because dividuality represents individuals’ ways of being in the network.

Following Gaunig and Savat, I affirm that the fluid power of the superjectification seen in PPA—again, in contrast with the solid power of individuals—exerts a latent force in terms of inventing social relationships apart from the logic of the capitalism that finances and reifies human bodies and even social relations. The characteristics of superjectification—being temporal, dispersed, affective, and condividual—are more radical and effective in terms of “pre-empting” the looming problems associated with this logic. Furthermore, the unique sense of affinity helps dispersed cognitions cling together contagiously, in turn affecting real politics (e.g., Ushahidi software, the MeToo and Occupy movements, and the activist group Anonymous).

Thus, PPA projects tend to encourage affinity-driven co-individuation outside the context of the labour system and manipulative relationships on the web.¹⁹ Public_Public_Address: A Nationwide Virtual Protest (2020-), for example, is an ongoing virtual protest for which Jason Lazarus, Stephanie Syjuco, and Siebren Versteeg have been accepting submissions of selfie videos of individuals holding protest pickets in support of the Black Lives Matter movement. [Figure 2] The participants include those who had been marginalized in society and those unable to protest in person because of a disability. The artists integrated the video submissions into live-streaming on YouTube and simultaneously in the window of a New York gallery in 2020. Seemingly marching toward the viewers, the images of these anonymous participants have been incorporated into a bizarre new form of space-time. Rather than isolating, separating, and pixelizing the agency of the participants in the network, this project reveals the aesthetics of fluid power that the superjective participants generate in the postdigital environment by transcending the online-offline binary. 

2. Jason Lazarus, Stephanie Syjuco, and Siebren Versteeg, Public_Public_Address: A Nationwide Virtual Protest (2020-), Source: the bitforms gallery website, https://bitforms.art/public_public_address-2020/.

Conclusions

As the exploitation of participation in the network becomes increasingly sophisticated, PPA has the potential to offer space in which to imagine new social relationships by questioning freely the dominant logic and weaving together digital objects, technological materials, and human agency symbiotically. In this regard, art systems centred on individual authorship fall short when it comes to describing a new dynamism that embraces the energy, matter, force, objectiles, and superjects that digitally mediated co-creation can bring about. In the consistent flow of the network, the conventional binaries of matter and form, subject and object, physical and digital, and artist and spectator co-emerge, co-concretize, and intra-act. Then, the new creative subject of PPA, or participant-superject, having emerged in the context of the shift in digital materiality and subjectivity, can radicalize the author-subject model and re-envision the agendas of open-ended and participatory aspects of art.

Superjective authors, then, with their mutual resonances, cause events and collective experiences to proliferate and add further from the postdigital environment. These authors are omnipresent and already part of humans’ ontological status. As Literat observed regarding many cases of online crowdsourced art, however, “the crowd is still a crowd, not yet a community” because the digital fibres and technological structures that mediate digital encounters can hinder efforts to achieve artistic collaboration or build a sense of community.²⁰ Thus, further studies are needed to assess the potential of PPA to invent meaningful communities apart from the profit-making algorithms of the various online platforms. 

References

1 Umberto Eco, "The poetics of the open work," The Open Work 251, no. 1., 1962.

2 Roland Barthes, "The Death of the Author," Contributions in Philosophy 83, [1967], 2001, 3-8.

3 Michel Foucault, "What is an Author?," in Textual Strategies, ed. J.V. Harari, Ithaca, Cornell University Press, [1969] 1979, 141-60.

4 Nicolas Bourriaud, Relational aesthetics. Trans. Simon Pleasance, Fronza Woods, Mathieu Copeland, Dijon, Les Presses Du Réel, 2002, Originally published as Esthétique Relationnelle, Dijon, Les Presses Du Réel, 1998.

5 Claire Bishop, “Introduction: Viewers as Producers,” in Participation, London, Whitechapel/Cambridge, MIT Press, 2006, 10-17.

6 Claire Bishop, “Social Turn: Collaboration and Its Discontents,” in Artforum 44, no. 6, 2006, 178–83.

7 Claire Bishop, Artificial Hells: Participatory Art and the Politics of Spectatorship, London, New York, Verso Books, 2012.

8 Robert Pepperell, Michael Punt, The Postdigital Membrane: Imagination, Technology and Desire, Bristol, Intellect Books, 2000.

9 Sy Taffel, “Perspectives on the Postdigital,” Convergence: the International Journal of Research Into New Media Technologies 22 (3), 2016, 324–38.

10 Brad Troemel, “Interview with Rob Walker, The Jogging, a Tumblr at the intersection of BuzzFeed, 4chan and weirdo experimental art,” Yahoo News, October 24, 2013.

11 Gilles Deleuze, The Fold: Leibniz and the Baroque, London, The Athlone Press, 1993.

12 David Savat, “(Dis)Connected: Deleuze’s Superject and the Internet,” in International Handbook of Internet Research, 11:423– 36, Dordrecht, Springer Netherlands, 2010, 432.

13 Alfred N. Whitehead, Process and Reality: An Essay in Cosmology. eds Griffin, David Ray and Sherburne, Donald W, New York, Free Press, [1929] 1978.

14 Marco Deseriis, Improper Names: Collective Pseudonyms from the Luddites to Anonymous, Minneapolis, University of Minnesota Press, 2015.

15 Bernard Stiegler, What makes life worth living: On pharmacology, Cambridge, Polity, 2010.

16 Gilles Deleuze, “Postscript on the Societies of Control,” October 59, Winter, 1992, 3–7.

17 Gerald Raunig, Dividuum. Machinic Capitalism and Molecular Revolution, Volume 1 (trans. Aileen Derieg), Los Angeles, Semiotext(e)/MIT Press, 2016.

18 David Savat, Uncoding the Digital, Basingstoke, Palgrave, 2013.

19 Marco Deseriis, “The Politics of Condividuality”, transversal.at., 2018, 7-10, accessed August 31, 2021, https://transversal.at/transversal/0318/deseriis/en

20 Ioana Literat, “The Work of Art in the Age of Mediated Participation: Crowdsourced Art and Collective Creativity”, International Journal of Communication 6, 2012, 2962–84.

Bibliography

Claire Bishop. “Participation and Spectacle: Where Are We Now?,” in Living as form: Socially engaged art from 1991-2011, ed. Nato Thompson (Massachusetts: MIT Press, 2012), 34-45. Kyle Chayka, “Art in the Corporatized Sphere: The Impact of Commercial Social Media on Online Artistic Practice,” in A Companion to Digital Art, ed. Christiane Paul (New York: John Wiley & Sons, 2016), 413–425.
Christian Fuchs, Digital Labour and Karl Marx (London: Routledge, 2014)
Ioana Literat, “Facilitating Creative Participation and Collaboration in Online Spaces: the Impact of Social and Technological Factors in Enabling Sustainable Engagement,” Digital Creativity 28 (2), (2017): 73–88.

Acknowledgements

The overall structure and conception of this paper draw on my PhD dissertation published in June 2022 

Proteus 3.0 - Interacting with the cloud

Maria Smigielska*, CompMonks*

mariasni.com; compmonks.com
Zurich Switzerland maria@mariasni.com, compmonks@compmonks.com *authors contributed equally to the project

Abstract

This paper presents the Proteus 3.0 project as an interactive and generative video installation, which tries to remain in conversation with its audience and learns from its engagement to produce new and potentially meaningful sequences of images. This particular version of a series is focused on digitized material as ferrofluid and its digital surrogate generated with a reaction-diffusion algorithm, projected onto a room-scale oval oculus allowing for a collective/immersive interaction. By employing state-of-the-art custom reinforcement learning models coupled with human intelligence in a symbiotic fashion, it aims to make the interaction more meaningful by understanding better human behaviour during an interaction. Additionally, it intends to maintain the interaction as conversations, rather than playful attractions. 

Keywords

Interactive art, generative art, reinforcement learning, computer vision, conversational interaction, mixed-media installation.

DOI

10.69564/ISEA2023-30-short-Smigielska-et-al-Proteus-3-0

Introduction

This paper presents the Proteus 3.0 project as an interactive and generative video installation, which tries to remain in conversation with its audience and learns from its engagement to produce new and potentially meaningful sequences of images. Proteus, as a series (Figure 1), has systematically evolved with its ferrofluid displays and modes of magnetic activation at multiple resolution ranges, particular sequences of digital and analogue layers as well as types of interaction. This version is particularly focused on moving the ferromaterial to the digital realm with its digital surrogate generated from a reaction-diffusion algorithm, scaling up from an object-size analogue display to an architectural oculus of oval shape at room scale, and which, at the same time allows to shift from an intimate interaction to a collective/immersive one. By employing state-of-the-art reinforcement learning models coupled with human intelligence in a symbiotic fashion, it also intends to build an interaction that is based on conversations, rather than playful attractions. Most frequent and straightforward approaches towards interaction design are based on implicit principles and clear instructions built in a closed loop to promote immediate engagement. These oversimplified views on interaction neglect its long-term impact¹ or the ones stemming from more traditional discourses, and which define any interaction with an artwork purely as a mental act.² In that lens, the project revisits the Conversation Theory of Gordon Pask³ in its scope of human-machine interaction as a form of open and emulative conversation. A dynamic process in which both ends learn and adapt from each other. As such, the project aims to construct an interaction that goes beyond the boundaries of predefined scenarios that follow the strict categories of an audience’s behavioral patterns and tends towards individualized and dynamic responses.

1. The evolution of the Proteus series between 2018 and 2022. ©various credits: Maria Smigielska, CompMonks, Ars Electronica 

Background Conversational interaction

What most generally describes successfully an interaction is the degrees of responsiveness and control an artwork and an audience are communicating to each other. An interactive artwork is essentially a composite made of an artificial component (the artwork itself as an object) and a human component (the audience) that both define its nature and behavior⁴. These degrees of communication are then located somewhat between total randomness and total predictability, without reaching any of these extremums. A total random response could only qualify as part of a reactive process, and a totally predictable one as part of a controlled manipulation. This does not necessarily involve a stable exchange and most frequently concurrent dynamics are at play from both ends as human intentions and artificially designed systems may have different and evolving goals over time. While this constitutes a great challenge for an artist to frame and produce such ideas, it also represents the richness and openness that an interaction may provide in qualifying an object. Interactivity is, by design, the relational property of an object and not an intrinsic one. It results that the value of such artwork is consequently constantly in a state of volatility and becoming, evolving throughout interactions. One might argue that this is necessarily the fate of every artwork, and more generally every object. But this becomes fundamental when interaction becomes itself an object of design in the artwork. Although, the most frequent and straightforward approaches towards interaction design are based on implicit principles and clear instructions built in a closed loop to promote immediate engagement and playful attractions. Similarly, most theoretical intent to define interactivity in art has been partially treating aspects of interactions that revolve around the topic of controllability. In general, the ability to improve control makes novelty the source of most types of interactivity⁵. In the general perspective of control, interaction can be seen as a learning stage in order to reach control. Once sufficient learning is achieved, interaction becomes redundant. Interaction becomes endowed with the feature of being instructional. However, even during the early stages of cybernetics, the theory of control was seen as only a chapter in the theory of messages⁶. Interactive art must be dissociated from the grasp of control theory [ref] in order to avoid any pessimistic perspectives⁷ or shortcomings in the envisioned artistic power of such artistic interventions. On the same level, Input/Output theory of interaction with computers⁸ simply defines interactions as a unilateral transactional process of information and therefore fall short on creative content. On another aspect, when interactivity is defined as procedural and participatory⁹, the definition lacks enough abstraction and generalisation to talk about the systemic design of an interaction and the informational novelty that go beyond the sole responsiveness of participations to an artwork. On another end, the evolution of machine perception in human computer interaction can be seen as trivialising the term of interactivity in new media a rts, and eventually rendering the sum of its tentative description as too loose to be useful to qualify an artwork¹⁰. This oversimplified views on the interaction neglects its long-term impact¹¹ or the one stemming from traditional art, which defines interaction with art purely as a “mental act”¹². It also resonates with the cognitive approach of the power of an artwork being located in the beholder’s response ¹³. 

Proteus 3.0 within the series of Protei

Proteus is a series developed as a duo since 2018 driven by the curiosity in exploring aspects of interacting with matter, tensions between analogue and digital, as well as relations between human and machine intelligences¹⁴. The series evolves through the continuous visual and material reference of a colloid compound called ‘ferrofluid’ as a main character ¹⁵. When exposed to magnetic fields and interacting with another liquid carrier, it reveals its intricate and lively nature by presenting strongly contrasted and ever-changing complex patterns ranging from discrete patches of dots, meandering streams, to larger coagulated blobs. The volatility and organicity of this material behaviour in the series is used as an allegory of the Greek mythology of a deity called Prōteus and his ability to assume many forms and provoke uncanny encounters ¹⁶ that remain evasive to the rational mind but open-ended to the visual imagination. Over the course of its serial declination, Proteus has iteratively evolved with varied organic and deep black patterns to serve as an emulator for visual human intelligence. That of seeking meaning in a constantly changing flow of images and unpredictable symbolic relations which can only be found in the human mind, just like gazing at a cloud. But the historical problem in such a romantic contemplation is often emphasized through the notion of distance and the lack of direct communication between a cloud and its beholders. If the cloud could talk back in response to one’s efforts to find visual meaning, would the conversation stabilize to a consensus, or would it constantly move to explore and seek novelty? This problem of distance and communication reveals itself when reframed in the artwork as a generative/interactive installation and its architectural embodiment. 

2. Proteus 3.0 installation during BioMedia exhibition at the ZKM Karlsruhe, 2022. © Photo: Thierry Serbeto.
3. 5 predefined interaction scenarios comprising of generative material visuals and behavioural patterns.

Methods

3. Diagram of the interactive loop. ©Image: CompMonks

Proteus 3.0 is an interactive generative 4K video installation, rear-projected onto a black screen of oval shape. The screen, of dimensions approximating 3.5 x 2.5 meters, is rotated 45 degrees and integrates an HD wide angle and infrared camera to estimate body and head poses from the audience and to detect regions of visual interest located on the screen. This information is continuously tracked and serves to build markers of attention to modify the generated visual frames (Figure 3).

Material surrogate

Because to this day, no digital model has been able to accurately simulate the physical behaviour of ferrofluid material in real time ¹⁷, a specific GLSL shader has been developed for the procedural generation of a digital surrogate. Based on a Gray-Scott reaction-diffusion model between two substances, a series of image filters have been designed as an overlay to mimic the dynamics of the ferrofluid and its pattern formations.

Our experiment was conducted in two following steps: (1) initiate predefined interaction scenarios run in the exhibition context to collect the visitors’ data, (2) building adaptive and evolving interaction models with reinforcement learning. The details of those two phases are described below.

Predefined interaction scenarios

At the first launch, the initial interaction consisted of 5 predefined interaction scenarios running in the 80 seconds loop. Interaction scenarios consisted of generative visuals coupled with the most common human behavioural patterns. The visitors' data responding to these scenarios were collected during eight months of the BioMedia exhibition at ZKM Karlsruhe, starting in December 2021.

In order to build Regions of Interests (ROI) of each interaction, skeleton data of max 6 visitors simultaneously were captured with a Kinect 2 camera to extract their torso and face orientation projected back on the oculus canvas. Each interaction has been recorded into a large dataset organized by timestamp with raw numerical information (visitors' ROI, skeleton data, location in relation to the screen) as well as low-resolution images (like RD noise map and visitors' ROI) for further training. The construction of datasets considers that any relevant personal information is decoupled from the recorded data.

Generative interaction scenarios with reinforcement learning (RL)

The scenarios further evolved into generative characters with the support of a machine learning model programmed in Python, specifically a reinforcement learning model, to learn from collected data and extracted markers of attention. It is being trained discretely offline and concurrently for a real-time generation to support the progressive evolution of the artwork that can adapt to different exhibition contexts and visitors' behaviours. The custom model has been developed in python with a PyTorch library.

The real-time performance of the work and the offline training are orchestrated by two combined processes. For real-time performance, a TouchDesigner file manages both visual inputs coming from the camera and generated outputs going to the display. The generated images are sent to two video projectors, mapped and blended for a smooth projection onto the screen. Daily, the recorded data is updated to a remote server and feeds new offline training sessions for the reinforcement learning model. The updated model is then sent back to the local computer controlling the installation to update Proteus' behaviour. 

Engagement

Engagement can be measured with simple metrics like the amount of time spent looking at art¹⁸ or more complex summative indexes, like Sweep Rate (SRI) and Diligent Visitors (DV) representing exhibitions “Thorough Use”¹⁹ to improve audience engagement in museums at large. However, those static metrics allow us to look at visitors’ response only through a pinhole and we do not know the exact impact of the interaction on the visitor (long-term, imaginary, etc). Therefore, we need more data collected during the interaction itself and more sophisticated tools to analyze them. “The Plant” project²⁰ proves that reinforcement learning can improve some of measures of engagement, like visitors touch count during interaction, but not average duration of the interaction. This project allows us to draw a hypothesis that AI-based tools combined with generative models can increase the quality of engagement and not just its quantitative factors like duration.

In the next developmental step of Proteus project, we aim to utilize the developed RL model in the future exhibition and conduct comparative studies to evaluate this hypothesis. 

Conclusions

Proteus 3.0 is an interactive and generative video installation, which, once in place within the walls of its exhibition and switched on, tries to remain in conversation with its audience and learns from the visitors' engagement to produce new and potentially meaningful sequences of images (Figure 2). Until the end of its exhibition, when it is switched off and set frozen in time. By then, its last learned state is saved and becomes the new generative beginning for its next exhibition. The learning and interactive process continues endlessly during the artwork’s lifetime for each exhibition and conditions its existence. Its generative imagery remains open-ended and its content to be modulated by its beholders. This project is an attempt to utilize AI-supported generative systems, which facilitate an interaction that goes beyond predefined behavioral models directly controlled by a few, easily comprehensible parameters and therefore might increase the engagement in the audience. 

References

1 Edmonds E, Candy L, “Interacting: Art, Research and the Creative Practitioner,” 2011.

2 Paul C, “Digital Art” Third edition, London, Thames & Hudson, 2015.

3 Pask G, “Conversation Theory Applications in Education and Epistemology” Amsterdam, Elsevier, 1976.

4 Margaret A. Boden, Ernest A. Edmonds, From Fingers to Digits: An Artificial Aesthetic, Illustrated edition, Cambridge, Massachusetts: The MIT Press, 2019.

5 “Aaron Smuts, What Is Interactivity?—PhilPapers”, 2023, Accessed September 15.

6 Norbert Wiener, The Human Use Of Human Beings: Cybernetics And Society. Revised ed. edition. New York, N.Y: Da Capo Press, 1950.

7 Terrence Rafferty, “Everybody Gets a Cut,” The New YorkTimes, May 4, sec. Magazine, 2003, https://www.nytimes.com/2003/05/04/magazine/everybody-gets-a-cut.html.

8 David Z. Saltz, “The Art of Interaction: Interactivity, Performativity, and Computers,” The Journal of Aesthetics and Art Criticism 55 (2), 1997, 117–27, doi:10.2307/431258.

9 Janet H. Murray, Hamlet on the Holodeck, Updated Edition: The Future of Narrative in Cyberspace, Updated ed. edition, Cambridge, Massachusetts: The MIT Press, 2017. 

10 Manovich Lev, The Language of New Media, MIT Press, 2002.

11 Ernest Edmonds, “Art, Interaction and Engagement,” In 2011 15th International Conference on Information Visualisation, London, United Kingdom, IEEE, 2011, 451–56, doi:10.1109/IV.2011.73.

12 Christiane Paul, Digital Art. Third edition. London, Thames & Hudson, 2015.

13 Eric Kandel, The Age of Insight: The Quest to Understand the Unconscious in Art, Mind, and Brain, from Vienna 1900 to the Present, 1st edition, New York, Random House, 2012.

14 https://proteusproject.ch/

15 https://en.m.wikipedia.org/wiki/Ferrofluid

16 https://en.m.wikipedia.org/wiki/Proteus

17 Han Shao, Hang Libo, Dominik L. Michels, “A Current Loop Model for the Fast Simulation of Ferrofluids,” IEEE Transactions on Visualization and Computer Graphics, 2022, 1-12, doi:10.1109/TVCG.2022.3211414.

18 Jeffrey K. Smith, Lisa F. Smith, “Spending Time on Art.” Empirical Studies of the Arts 19 (2), 2001, 229–36.

19 Beverly Serrell, “The Aggregation of Tracking-and-Timing Visitor-Use Data of Museum Exhibitions for Benchmarks of ‘Thorough Use.’” Visitor Studies 23 (1), 2020, 1–17, doi:10.1080/10645578.2020.1750830.

20 Zoe Tong, Dana Kulic, “Learning to Engage in Interactive Digital Art,” In 26th International Conference on Intelligent User Interfaces, College Station TX USA, ACM, 2021, 275–79, doi:10.1145/3397481.3450691 

Acknowledgments

Artwork by: Maria Smigielska, Compmonks
Created as part of the intelligent.museum project at the ZKM | Center for Art and Media Karlsruhe and at the Deutsches Museum. Funded by the Digital Culture Programme of the Kulturstiftung des Bundes (German Federal Cultural Foundation). Funded by the Beauftragte der Bundesregierung für Kultur und Medien (Federal Government Commissioner für Culture and Media). Funded by BioMedia ZKM | Center for Art and Media Karlsruhe.

Authors’ Biographies

Maria Smigielska is an architect and researcher based in Zurich, currently working at DBT, ETH. She is interested in enhancement of potentials for creation of architectural elements, design objects and mixed media installations, by using digital and interactive technologies for encoding and modulating materials properties, custom design and fabrication methods. Mariasni.com

Compmonks (PhD) is an architect and a researcher in technology for architecture based in Switzerland. He currently leads research on interactive and generative design with brain-computer interfaces at ETH Zürich. His artistic work follows a series of design objects and mixed-media installations focused on the power of combining humans with computers.

compmonks.com

Co-creation Towards the Post-Anthropocene

Pieter Steyaert1,2,7, Angelo Vermeulen3,7, Amy Holt7, Diego Maranan4, 7, Giusy Checola5,7, Mona Nasser6,7, Jeroen Verschuren7, Pim Tournaye7, Jesper Bruun2, Katrien Kolenberg1, 7

1 University of Antwerp, 2 University of Copenhagen, 3 Technical University of Delft, 4 University of the Philippines, 5 EDESTA-University Paris 8, 6 University of Plymouth, 7 SEADS (Space Ecologies Art and Design)
Antwerpen, Belgium pieter.steyaert@uantwerpen.be

Abstract

Co-creative practices including non-human actors give rise to a series of challenges and critical issues. Ēngines of Ēternity is an ongoing artwork which seeks to attribute agency to microscopic animals called rotifers and to the unique environment of outer space. Scientific experiments involving these animals, as well as the evolving artwork that accompanies them, have gone through multiple iterations having been flown to space and consequently returned. This article reflects on how transdisciplinary approaches can provide a vehicle to connect knowledge and enquiries from art and sciences in the context of real-world problems. We then focus on the case study of Ēngines of Ēternity and how these interactions played out during this process. We conclude the article with a critical reflection on non-human agency, using the aforementioned project as a case study.

Keywords

nonhuman agency, post-Anthropocene, co-creation, transdisciplinarity, ArtScience.

DOI

10.69564/ISEA2023-31-short-Steyaert-et-al-Co-creation

Introduction

The bidirectional exchange of knowledge, ideas and methodologies can foster co-creative practices and allow for a genuine blurring between disciplinary boundaries to occur.¹ Such holistic conditions may act as a catalyst for the synthesis of a deeper understanding with regard to the nature of the universe we inhabit and with it the emergence of a new post-Anthropocene era.² In order to achieve a sustainable future, we need to understand the mechanics and challenges of human and non-human interactions. This article explores and assesses the process of ArtScience co-creation, through a transdisciplinary project entitled Ēngines of Ēternity, acting as a case study. The work's narrative incorporates questions regarding manmade power structures and cultural immortality, through the use of non-human organisms, whilst highlighting the benefits and issues that can occur as a consequence. The author of this work and of this article is SEADS, a transdisciplinary and cross-cultural collective of artists, scientists, engineers and activists. The collective’s mission statement notes that transdisciplinary approaches are key to unlocking collective intelligence, which is a prerequisite for generating more diversified and inclusive futures.³ They incorporate co-creation methodologies in a range of their projects, amongst others are community art projects such as Bio-modd and Seeker.⁴, ⁵ The Ēngines of Ēternity project explores new forms of co-creation between different entities: humans, biological organisms and algorithms within the context of outer space. Before introducing the project, we outline the concept of creativity in the post-Anthropocene, which has been key to functioning as an underlying theoretical framework for this case study.

Creativity and Agency in the post-Anthropocene

If the Anthropocene is understood to be the geological epoch in which human activities are the dominant influence on the environment, the post-Anthropocene can be understood to be the epoch that dawns when human activities and their effects are no longer the dominant influence on the environment.⁶, ² The exact nature that the post-Anthropocene will take is difficult to predict because it depends on many factors, including how humanity responds to the challenges currently faced in the Anthropocene itself. On a global scale, we are not able to predict when and how the post-Anthropocene concept will become a dominant global framework, or indeed if this in fact will take place. However, there may be triggers that are capable of driving societies (both human and non-human) towards it, such as the ongoing global warming crisis and deterioration of the environment, societal cultural shifts, or technological advancements that may even allow humanity to live in greater harmony with Earth's ecosystems, but this is currently speculative. Merging disciplines can contribute to identifying and solving complex problems through the collaborative exchange of ideas and methodologies.⁷ Indeed, it has become apparent that many of the major problems facing the world today are hugely complex in nature and will not be solved by any one single discipline alone.⁸ Addressing complex societal and scientific problems requires involving multiple disciplines. However, there is a danger of oversimplifying the collaborative benefits between these disciplines. We argue that commonalities or filling up gaps can ignore or overlook the need to find new creative ways of dialogue between the disciplines, thus creating new knowledge or lines of inquiry that can transform the conversation on various global and scientific issues.

Differing from alternative approaches to merging disciplines, transdisciplinarity concerns both what exists, and is between and beyond all disciplines, blending into an empirical and theoretical exploration of co-creation.¹,, ¹⁰ One form of transdisciplinarity involves combining art and science (ArtScience) to create new co-creation practices that utilize methods and methodologies from both fields. Such art interventions can take place in present and future shared spaces and may contribute to the constitution of new symbolic orders and organisations of human and non-human connections, staging the mise en forme of post-Anthropocene co-existence. Seen in this way, ArtScience co-creation can be seen as an integral part of rethinking humanity’s relations to nature.

Bruno Latour argues that non-humans, such as animals, plants and objects, are active participants in social life and should receive equal attention in the study of society.¹¹ In this way of thinking non-humans should be seen as actors in their own right, rather than simply as objects that humans act upon; non-humans can have agency, and influence the course of events in the world. Latour’s conceptualization of agency does not strictly correspond with notions of sentience nor free will; neither does it have to be confined within physical boundaries. Agency can be an emergent property of multi-actant configurations.¹¹ According to Latour, social-cultural practices are enunciative and performative actions through which the new “subjects” can test different semiotic, sociological and alterity modes of existence, that lie between “being-as-being and being-as-others.”¹² In a similar vein, we refer to the notion of a “multispecies roundtable” for co-creating systems. This involves bringing together plants, computers, and people in mutually beneficial ways and placing them in a situation where they must arrive at a consensus.⁴, ¹³

Speculative Realism, Posthumanism, New Materialism, and the non-human are just a few examples of concepts that challenge the limits of what Donna Haraway calls the “fantasy of human exceptionalism.” These ideas reject the notion that humans are somehow separate, beyond, or more advanced than our earthly co-habitants.¹⁴ The theorists working within these fields help to illuminate the limits of human experience and thought, highlighting the capacities, experiences, and potentials of other living, non-living, and non-human entities. Their examples embrace a radically holistic and inclusive approach to the future of humanity and are established concepts in critical cultural thinking.

The Engines of Eternity project puts these ideas into practice by directly interacting with microscopic organisms, computational algorithms, and the physicality of outer space. These practical engagements hint at new opportunities and insights but also confront the limitations of an extended co-creation practice. Continuous hands-on approaches and their analysis allow us to move beyond established thinking and gain new critical insights through hands-on artistic research. 

Figure 1. Evolution and translation of the “code” and the application on the experiment label. © SEADS

Ēngines of Ēternity

Ēngines of Ēternity is a transdisciplinary project that explores how creativity and agency might function in a post-Anthropocene context. It is a work in progress and is intended to be a series of mixed-media installations. The project is a joint effort between SEADS and scientists from Karine Van Doninck's laboratory at UNamur. The foundational idea of the project is to take the biological phenomena of cloning and DNA repair as metaphorical departure points for an art installation about humanity’s obsession with cultural immortality.¹⁵ By metaphorical departure point, we mean that we use the fascinating biological characteristics of the rotifer animal to question creating a perfect society. At the core of the project is an artwork that evolves through complex systems of interplay and co-creation between biological organisms, humans, and technology. The project explores dominating ideologies about evolution and cultural identity within the space environment. The discussion on cultural immortality is discussed in a separate article³, as this one focuses on the methods and their implications that were crucial to developing the artwork. Here, we instead describe the background of our ArtScience co-creation activities: space biology experiments with the Bdelloid rotifer.

The Bdelloid rotifer or ‘wheel animal’ typically lives in freshwater environments and is part of the greater ecosystem that exists there. On the surface, rotifers seem to represent a biological culture perfected through evolution, cloning themselves in endless repetition. However, during this cloning process, genetic material actually gets reshuffled and diversity is generated through the process of horizontal gene transfer. This method of generating diversity can be defined as the non-sexual movement of genetic information between organisms, and genetic material from organisms as diverse as fungi, bacteria and plants have been discovered lurking inside the rotifer’s genome.¹⁵

Through a series of space biology experiments, scientists from UNamur sent rotifers to the International Space Station (ISS) in both 2019 and 2020. The goal of these experiments is to investigate rotifer DNA repair mechanisms and cloning in the conditions of outer space. In December 2019, the rotifers going to ISS were accompanied by an artistic "seed" in the form of a specially designed visual code that accompanied the rotifers. The code was developed by the Ēngines of Ēternity team and serves as a seed for a series of 3D-printed sculptures. These sculptures show the physical changes to the rotifers in response to environmental conditions and experimental techniques. The sculptures are based on a mapping of recorded experimental data 3D sculptures.

The work is driven by two main objectives: the evolution of the "code" throughout different missions and the translation of these seeds into three-dimensional sculptures. Analysis was conducted on both the bags that went up into space and the bags that served as a control group on Earth in order to reveal the differences between protein expressions. Each individual bag that was sequenced was assessed against the average from the control group bags, resulting in discrete sets per bag of large quantities of raw data. These sets were then grouped and ordered per function, accumulating the differences compared to the control group. The result was a visualization per bag showing the impact of the space environment on rotifer animals. These visualizations were then used to evolve the original visual "code" that accompanied these bags by using it as an attractor to manipulate the pixels of the original seed.

The second objective was to translate these seeds, both the original and the evolved version, into a three-dimensional sculpture. The glyphs that make up the seeds were transformed into a circle and a linear regression algorithm was applied to layer multiple versions of the original visual on top of each other and connect them. This resulted in a three-dimensional sculpture where the base visualises the overall shape and every layer shifts towards a higher level of detail. In this way, the final sculpture shows both the impact of the environment on the rotifer animals at a higher order and on the smaller, individual protein clusters.

The “code” and sculptures will gradually evolve into the final artwork over several missions based on the changes in the rotifers. Hence, the context of outer space is not merely treated as a passive gallery space to exhibit art in, but rather as a dynamic environment that can actively shape the evolution of the work.

Co-creation revisited Co-creation in Art and Science

The historical development of scientific and artistic disciplines has resulted in the development of organisations and structures in which these two areas do not organically engage or have dialogue. The organisational and procedural infrastructure that exists within these two fields is therefore sometimes seen as opposites. Science is often associated with the use of strict methodologies and systems to produce empirical data, unbiased information, and facts that help us better understand the physical and natural world around us. On the other hand, art is often viewed as a more subjective practice that allows individuals or groups to express their creativity and imagination.¹⁶ While these definitions are not entirely inaccurate, such reductionist viewpoints are unhelpful when beginning to ask deeper questions. ²³ Indeed, in recent times a variety of Art-Science networks have emerged that act as places where such communities can interact, and begin to create imaginaries for developing questions regarding our place in the wider world, whilst reaching large audiences. Interactions between the fields of science and art can be regarded as a one, or two-way, form of communication between the two. When considered as a one-way form of interaction, there are two possibilities for one to influence the other, either art provides science with some form of benefit, or vice versa.

The SEADS team developed tools using techniques and knowledge that differ from those of the pure scientific team. The goal was to gain insights into the impact of the alien environment of space on living creatures in an exploratory context. Initially met with skepticism, the collaboration with the pure science team evolved throughout the project. By communicating about our results, the team was able to establish new ways of trust and collaboration. The SEADS collective comprises members with backgrounds in both art and science tapping into collective intelligence. This allows them to push boundaries when collaborating within and outside of the network. For instance, in the Ēngines of Ēternity project, the artists received hard drives containing all the raw transcriptomic data to explore. This not only resulted in the aforementioned sculptures but also allowed the team to analyze their findings and compare them to those of the pure science team. It appears that new insights emerged from this transdisciplinary approach, which has yet to be validated and might lead to a future publication. In some ways, it is also possible to view the process of horizontal gene transfer as a biological counterpart to transdisciplinarity itself. The generation of novel genetic material through the incorporation of foreign DNA into the rotifers, from such phylogenetically distant organisms, echoes the process of numerous distinct disciplines joining together to create a more efficient system. Much like transdisciplinarity itself, it is not merely the merging of disciplines (or in this case genetic material) that makes this comparison applicable, but the fact that the process has wider implications on a much deeper level as well.

If transdisciplinary practices are considered to be a combination of, translational and methodological approaches from various disciplines, utilised to address a problem, then the horizontal integration of genomic elements from cross-species sources can be said to function in a similar manner. DNA segments once integrated into the rotifer’s genome can be translated into new useful proteins, that are capable of carrying out a variety of functions, and in turn, contribute to the evolutionary adaptation of the organism. The whole is greater than the sum of its parts, in both the case of transdisciplinary inquiry and biological evolution.

Figure 2. Sculptures and Bdelloid Rotifer organisms © SEADS and Boris Hespeels

Posthuman co-creation

A fundamental question in this project is whether true co-creation between humans and non-humans has been established, more specifically between the art-science team and the rotifers. How much agency did the rotifers actually have in participating and contributing to the artwork? In addition to exploring human-human interactions, we also investigate the agency of non-human entities, specifically the Rotifer animal. The evolution of the artwork is driven by changes in protein expression. However, this approach raises the question: are these non-human entities truly being given agency?

There are two ways to approach this question. One way is instrumentalist, which would argue that regardless of the artists' intentions, the rotifer animals are being exploited for both science and art. In fact, there is even an element of cruelty, as the animals are taken from their natural habitat and subjected to experimentation in an extreme environment without any choice in the matter. From a pragmatic standpoint, it is difficult to refute this way of framing the issue.

We present a second approach as follows: The artist team explores the intrinsic attributes of the rotifer animal and co-creates the artwork based on their findings. In this instance, horizontal gene transfer is viewed as a form of creation. By incorporating the protein expressions from these animals as the main driving force, and using complex bio-informatics algorithms, some level of agency over the artwork is given to these non-human entities. Extreme environments have been fundamental driving forces for the evolution of rotifers. However, in the case of the artwork discussed here, the driving force of its evolution is not solely an attempt to depict the intrinsic attributes of these animals. Instead, it also considers the impact of the alien environment of near-zero gravity in space on these extremophiles. This marks the first time in the history of the species that they have been exposed to the environment of space.

In this sense, the two levels of co-creation within this case study can be extended to include "The alien" as an environmental contributing factor. Additionally, the extensive use of algorithms and predictive computational methods has played a significant role in the visual outcome of the project. As a result, this project explores new forms and levels of co-creation between humans, biological organisms, algorithms, and outer space. Horizontal gene transfer and parthenogenesis are used not only as a metaphor, as was discussed in the previous section, but as an act of self-creation and self-transformation which instruments served as contributing factors in the formation of the artwork.

Conclusion

Co-creative practices including those involving non-human actors were explored through the creation of an ongoing artwork project entitled Ēngines of Ēternity. The nature of such transdisciplinary methodologies was investigated by a small community of artists and scientists involved in the project, whose evolving discourse during the process of developing and reflecting on the project has helped to provide a framework to inform conversations regarding the future of humanity, and whether and how we might move towards a more post-Anthropocene stage in the future. Establishing an ethical post-Anthropocene requires approaching the Other from a humble and empathic perspective. This case study provides insight into a practical approach towards this objective. For example, by focusing on the intrinsic qualities of creation present in the non-human agents. Nonetheless, it is evident that there are challenges that need to be addressed to successfully accomplish this goal. 

References

1 Erich Jantsch, "Inter-and transdisciplinary university: A systems approach to education and innovation", Higher education 1, no. 1, 1972, 7-37.

2 Erle C. Ellis, Anthropocene: a very short introduction, Vol. 558, Oxford University Press, 2018.

3 Angelo CJ Vermeulen, Diego Maranan, Pieter Steyaert, Nassim Versbraegen, Ann Peeters, Jeroen Verschuren, Arise Wan et al. "Ēngines of Ēternity: An Artistic Inquiry into Space Settlement Ideology Using Rotifer Experiments on Board the ISS," In 73rd International Astronautical Congress, 2022.

4 Pamela G. Cajilig, Diego S. Maranan, Arlene Sy, Oliver Salva, Angelo Vermeulen, "Multispecies Roundtable for Climate Impact: A Speculative Proposal," 2017.

5 Diego S. Maranan, Angelo CJ Vermeulen, "When ideas migrate: A postcolonial perspective on Biomodd [lba2]," 2015.

6 A. C. J. Vermeulen, Caroline Nevejan, Frances Brazier, "Seeker: Co-Creating Diversified Futures," Studio Time: Future Thinking in Art and Design, Black Dog Press, London, 2018, 172-182.

7 Van der Bijl-Brouwer Mieke, Giedre Kligyte, Tyler Key, "A co-evolutionary, transdisciplinary approach to innovation in complex contexts: Improving university well-being, a case study," She Ji: The Journal of Design, Economics, and Innovation 7, no. 4, 2021, 565-588.

8 Solomon Bililign, "The need for interdisciplinary research and education for sustainable human development to deal with global challenges," International Journal of African Development 1, no. 1, 2013, p.8.

9 Farzam Ranjbaran, Cristina Marras, "European Peer Review Guide: Integrating Policies and Practices into Coherent Procedures," ESF Member Organisation Forum on Evaluation of Publicly Funded Research, 2011.

10 Bruno Latour, Reassembling the social: An introduction to actor-network-theory, Oup Oxford, 2007.

11 Bruno Latour, "On technical mediation," Common Knowledge 3, no. 2, 1994, 29-64.

12 Bruno Latour, Enquête sur les modes d'existence: une anthropologie des modernes, Paris, La découverte, 2012.

13 A. C. J. Vermeulen, "Living computers, Mars simulations and DIY Starships: Advancing cross-disciplinary and cross-cultural collaboration," Link 3 Workshop Space, Plymouth University, 2016.

14 Donna J. Haraway, When species meet, Vol. 3. U of Minnesota Press, 2013.

15 Matthieu Terwagne, Emilien Nicolas, Boris Hespeels, Ludovic Herter, Julie Virgo, Catherine Demazy, Anne- Catherine Heuskin, Bernard Hallet, Karine Van Doninck," DNA repair during nonreductional meiosis in the asexual rotifer Adineta vaga", Science advances 8, no. 48, 2022, eadc8829.

16 Norman G. Lederman, "Nature of science: Past, present, and future," In Handbook of research on science education, Routledge, 2013, 831-879.

17 Jonathan Osborne, Collins Sue, Mary Ratcliffe, Robin Millar, Rick Duschl, "What “ideas-about-science” should be taught in school science? A Delphi study of the expert community," Journal of research in science teaching 40, no. 7, 2003, 692-720.

Acknowledgements

This project is funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant No 860470.

Author Biography

SEADS (Space Ecologies Art and Design) is a transdisciplinary and cross-cultural collective of artists, scientists, engineers and activists. Its members come from all corners of the world. SEADS is actively engaged in deconstructing dominant paradigms about the future and develops alternative models through a combination of critical inquiry and hands-on experimentation.

SEADS employs its own signature methodology which is centered around community building, co-creation and bottom-up design. SEADS believes that these approaches are key to unlocking collective intelligence, a prerequisite for generating more diversified and inclusive futures. Furthermore, SEADS also embraces a hacking and open-source ethos, with the goal of engaging as many people as possible in the activities and ideas that they initiate. Since 2009, the collective has co-created more than 40 art projects, together with local communities all over the world. 

Spectral Plain: A case study for exploring the world-building potential of co-creative systems that combine text generation models with game mechanics.

Guillemette Legrand1*, Vincent Thornhill2*, Isaac Clarke3

1École des Arts Décoratifs – PSL (EnsadLab), University of Applied Sciences and Arts Northwestern Switzerland (IXDM) 2KU Leuven, LUCA School of Arts, 3Hong Kong University of Science and Technology
1
guillemette.legrand@ensad.fr, guillemette.legrand@fhnw.ch, 2vincentsimon.thornhill@kuleuven.be, 3ijclarke590@connect.hkust-gz.edu.cn
*These Authors contributed equally.

Abstract

This paper describes the game installation ‘Spectral Plain’; introducing a case study of an interactive artwork that intersects algorithmic, sensing, and gaming technologies to explore new forms of generative world-building, which are different from the world-imaginations created by commercial and political narratives. Through this case study, we aim to explore how information technologies might embed specific socio-cultural beliefs. Here, we investigate the imaginaries created and disseminated through dominant representations of information technologies, while simultaneously searching for ways to create new multiscalar imaginaries of the earth through the mobilisation of these technologies. This case study describes how physical and digital interactions can be combined with AI and randomising technologies to foster forms of co-creation with and between humans. It also explores the capacity for AI-integrated game environments to decenter anthropogenic perspectives by creating new symbiotic interrelations with the non-human and the non-living that can impact participants’ perception of their understanding of the world.

Keywords

World-building, cosmologies, game mechanics, interactive installation, autoregressive language model, co-creative system.

DOI

10.69564/ISEA2023-32-short-Legrand-et-al-Spectral-Plain

Introduction

This project started with an intention to examine, and communicate the capacity of information technologies to construct imaginaries of the "world." The initial technological context for this investigation was the commercial and political representation of 5G wireless technology; the latest in a lineage of "network" imaginations of a world coinciding with the development of communication technologies. Professor of Anthropology Shannon Mattern argues for a need to acknowledge how marketing fantasies and policy scenarios have limited our imaginations, not just of what the technology is, but also of how we organise and live in the world through these imaginations.¹ Philosopher of Science John Tresch scales Mattern’s concerns to the imagination of the planet, stating that “when modern technology is linked to a ‘cosmology’ or a ‘world(s) view,’ we are offered a much thinner picture than that suggested by anthropology’s presentations of the ways humans have organized nature and society—a ‘mechanization of the world picture’ simplifying all experience to utilitarian building blocks of masses and forces, functionalist means and ends.”² Between the marketing and political narratives of 5G, and the technical operations of this infrastructure, we saw the potential to build a new reading of wireless technology.

This rereading was pursued through decentering wireless technology from the regulatory tradition of allocating and auctioning ‘bandwidth’ of electromagnetic frequency to commercial and geopolitical actors. Instead, we wanted to reposition this technology by placing it within a wider spectrum of frequencies that human and non-human entities are intertwined with. Imagining electromagnetic energy in new scales and alignments prompts us to think at a cosmological dimension, a concept derived from Timothy Morton’s description of a "Spectral Plain;" an infinite surface where “you can’t really distinguish very easily between alive and not alive, between sentient and non-sentient, between conscious and non-conscious.”³ Through enriching and diversifying the comprehension and representation of electromagnetic frequencies, the project aims to open up new imaginaries of the earth.

 

Research Process

The research process centred on testing ways to produce narratives about electromagnetic energy that cross between human and nonhuman scales, from the atomic, to the terrestrial, to the cosmic. To do this, we created a series of working sessions through which the project was developed through participation with varied audiences and formats, while iterating the development of technological systems for generating narratives. These experimentations took the shape of online and physical workshops and interactive performances that were set up to test different forms of collective world-building; methods that were later implemented in the final artwork.

We used the game environment as a site for experimentation, which enabled us to test ways that various modes of representation and points of view could coexist or create friction. It also allowed us to play with multi-scalar experiences; from micro to macro, from the local (storing of individual data about the player locally) with the planetary (data processed through large distributed algorithmic models). Through recombining different information technologies, we created an experimental ground for a collective to engage with world-building. We use the exhibition periods as a moment to present the work to a public, but also a moment to extend our research through further testing of the system we created, while collecting additional feedback on the players' experiences.

Figure 1. Screen capture of the research process that led to the development of Spectral Plain. Here you can see a working session where participants were guided through a game environment, while collectively creating stories that reflected their impressions of the journey. Credit: the artists.

Of the different technologies involved, our research into the use of GPT-3 had the most impact on how we were able to reconsider the ways collective world-building processes could be constructed in co-creation with AI technology. Several games have explored the use of GPT-3 to generate open-ended narratives, such as AI Dungeon.⁴ However, rather than using this technology to create divergent storylines, we sought to use the intrinsic properties of this linguistic model to establish connections and patterns between different bodies of knowledge about electromagnetic energy.

Project Description

Our artistic research into co-creative systems through the medium of the game environments led us to develop the artwork as an interactive game-installation, called Spectral Plain (figure 2).

In this interactive installation, three players can navigate three levels of an environment (the skies, the terrestrial and the underground). The game story takes place in a period following a fictive geomagnetic reversal (a change in the planetary magnetic field), where ‘spectral energy’ is unleashed, and players are engaged in discovering new perspectives on electromagnetic energies.

Figure 2. Image of the physical installation of Spectral Plain. The three layers of the game are projected on top of each other. A diagram representing the ‘back-end’ of the installation is legible on the ground. Three chairs were specifically designed to position player’s view towards one of three projections. Credit: the artists.

The installation uses a real-time data capture and a text-generation algorithm (GPT-3) via an integrated blueprint in the game engine (Unreal Engine) to log in various datum (time stamps, players' statement selection), which is then used to inform the final audio-visual cosmogram.

The three levels of the game are developed in a systematic way with each environment being constructed around five “spectra” of electromagnetic frequencies (figure 3) that are named: aqua, light, geology, matter, and device. These frequencies manifest as different entities within the game environment, for example, the light spectra manifests as chlorophyll in the terrestrial level.

Figure 3. Visual symbols for the 5 different typologies of energetic entities. Credit: the artists.

The presence of the three players in the physical environment of the installation creates a unique disturbance in the gamma radiation that is present around the installation. In the game infrastructure, this disturbance triggers the Geiger counter to produce a "true random number." This number is then used within the game engine blueprint to randomise the graphics that the players will interact with and the parameters within GPT-3.

The game environment is developed to give agency over the creation of a new world-imaginary of spectral energy that is different from the dominant 5G narratives. Here, we align with philosopher Federico Campagna’s position towards the concept of "world;" not using the word to describe an ontological reality, but rather seeing the world as an artificial construct of the imagination, “a metaphysical process that creates discontinuities, separations between things, individual identities, ‘somethings’ out of a plane of pure existence where no clear divisions are already inscribed.”⁵

At the end of the players’ journey, a cosmogram is automatically generated (figure 4). This cosmological diagram is constructed by gathering data about the players’ movement and interactions within the levels of the game. From this cosmogram, unique cosmological readings of the unified journey are then created through fine-tuning the GPT-3 language model. These readings are then "spoken" through the use of speech synthesis software. In this process of co-creation between player and algorithms, we are interested in how automated AI technology can be used to generate connections between diverse perspectives and narratives of energetic entities (scientific, mythologic, commercial narratives). The reading is not an attempt to accurately describe relationships between these perspectives, but rather, to create an artistic and lyrical intervention.

Figure 4. A screen capture of the final visual cosmogram representing the intersecting journeys of the three players. Credit: the artists.

Technical Realisation

In the Spectral Plain interactive game-installation, we combine physical and virtual player interaction, together with a combination of predictive (GPT-3) and randomising information technologies (Geiger counter radiation sensor as true random number generator (TRNG). The input and output of these different interactions and technologies are linked and moderated through Unreal Engine’s Blueprint Visual Scripting system.⁶ The node-based interface of this system allowed us to map and program interactions in the back-end of game creation software in a visual manner.

At the intersection of cosmology, and generative technologies, we also became interested in logic and structure of different divination techniques from different cultures. In Spectral Plain, we develop an analogy to techniques such as birth chart readings, and Yi Ching, by approaching their logic as a form of programming. In our technical realization of the project, processes of symbolisation, inclusion of chance, and geometric calculation can be seen as referential to ancient divination techniques that link the individual to the planetary.

Figure 5. Flow chart of the step-by-step interactions with the game installation. Credit: the artists.

The section attempts to walk through the different steps taken in the process of playing Spectral Plain by annotating the diagram (figure 5 below):

The text is then processed by a voice synthesiser to produce an audio file that is played when the players reach the visual cosmogram that is automatically inserted into the concluding environment of the game.

Conclusion

Spectral Plain is an interactive game-installation that builds upon concepts present within ancestral technologies (birth chart construction, Yi Ching reading instruments, etc.) through the use of current technologies in order to enable a space for co-creation, where symbiotic interrelations between disciplines (art, geoscience); the non-human (e.g., glowing micro-algae, chlorophyll photosynthesis); and the non-living (e.g., AI, geological elements) can occur. In this technical and conceptual description of the work and its intended effects, we aim to contribute to the discourse surrounding the ontologies of information technologies. This paper also aims to create a case study to allow for continued questioning of how artistic research can challenge, mobilise, and enrich the imaginary potential of these technologies.

By advancing the possibilities to engage with and through narrative-based technologies within game-environments, we follow Alenda Y. Chang’s proposition to explore "the untapped potential of games to create meaningful interaction within artificially intelligent environments, to model ecological dynamics based on interdependence and limitation, and to allow players to explore manifold ecological futures—not all of them dystopian.”⁷

The project requires further technical and conceptual development in order to explore ways to balance and represent generative-based components, chance factors, and collective agency within the game-narrative. It is also important to question the ability of large language models like GPT-3 to generate a truly novel story, and find ways to negotiate with its capacity to homogenise culture and beliefs through language parsing and recomposing.

References

1 Shannon Mattern, “Networked Dream Worlds,” Real Life , accessed December 7, 2022, https://reallifemag.com/networked-dream-worlds/

2 John Tresch, “Technological World-Pictures: Cosmic Things and Cosmograms,” Isis Vol. 98, No. 01, accessed December 7, 2022, http://dx.doi. org/10.1086/512833 (online)

3 Timothy Morton, Being Ecological, Cambridge Massachusetts, MIT Press, 2018, 126.

4 Latitude “AI Dungeon (2019),” accessed December 7, 2022, https://play.aidungeon.io

5 Federico Campagna, Prophetic Cultures, Recreation for Post-Future Adolescents, Bloomsbury Publishing, 2021, 13.

6 Unreal Engine, “Anatomy of a blueprint (2022),” accessed December 7, 2022, https://docs.unrealengine.com/4.27/en-US/ProgrammingAndScripting/Blueprints/Anatomy/

7 Alenda Y. Chang, Playing Nature: Ecology in Video Games, University of Minnesota Press, 2019, 16. 

Ecotechnologies of Practice: in-forming changing climates

Gisèle Trudel

École des arts visuels et médiatiques, Université du Québec à Montréal. MÉDIANE Canada Research Chair in Art, Ecotechnologies of Practice and Climate Change. Montréal, Canada.
trudel.gisele@uqam.ca

Abstract

How do ecotechnologies of practice actualize? This paper traces the material/theoretical operations of an ongoing research-creation concerned with changing climates. It mixes information in experimental approaches from collectivities of trees, media arts and forest sciences. Through individuations of symbiotic modulations, the paper is a thinking-with Balsam Fir, Diana Beresford-Kroeger, Camera, Domingo Cisneros, Dendrometer, Erin Manning, Isabelle Stengers, Jack Pine, Gilbert Simondon, Light Emitting Diodes, Numbers, Microphone, Recorder, Sapflow, Sensings, Speakers, Sugar Maple, Temperature, Yellow Birch.

Keywords

Research-creation, changing climates, trees, ecotechnologies of practice, information, individuation, art/sci/phi.

DOI

10.69564/ISEA2023-33-short-Trudel-Ecotechnologies-of-Practice

Figure 1. November 2020. Out in the field at the Smartforests Canada research site (Sainte-Émélie-de-l'Énergie, Québec). L to R: Ælab (Gisèle Trudel and Stéphane Claude) and forest engineer Christoforos Pappas. Photo: Zoé Fauvel.

Differential interdependence

An interdisciplinary sharing is at work. Ælab joins forest engineer Christoforos Pappas who is explaining to them how to attach a sapflow sensor to a tree in the boreal forest (Figure 1). The tree already senses with the surroundings, the sapflow sensor brings the tree's operations into human understandings, as sociologist Jennifer Gabrys has discussed.¹ Inter-cross-transdisciplinary approaches activate philosopher Isabelle Stengers' concept of “ecologies of practice” in modern science.² To move outward of disciplinary silos, she states openness is required while still maintaining tension, to question how practices produce knowledges (plural added; savoirs in French). Stengers argues for knowledges production in interdependence, within the practices of each scientific field, and in-between them. Each field can contribute a differential worldview to one another, a distinguishing plurality.

Scientific fields work actively to try to solve climate changes. Yet, what else can occur in-between the sciences, trees, art and machines? There is an important increase of sensual and affective artistic explorations with trees, among others by Jane Tingley, Domingo Cisneros, Susan Turcot, Rasa Smite and Raitis Smits, Agnes Meyer-Brandis. In this research-creation project, scientific and tree knowledges are expressed in outdoor media art installations. How can knowledges and practices resonate together, within and through a multitude of fields, given the concerns for the “more-than of life-living” of the blue planet?³

Figure 2. May 2021. Tree and machine sensings at Station biologique des Laurentides, Saint-Hippolyte, Québec. Oxygen and CO2 in-form the living for millennia. Photo: Gisèle Trudel.

Experimental in-forming

The concepts of information and individuation offered by Gilbert Simondon provide insight for the encounter of ecologies of practices.⁴ In his philosophy, information is more-than what is generally considered as factual data. Information is operation. It activates change through perturbation, by provoking or inducing an encounter between disparate elements/factors, as event. Potential is propelled into actualization through a dynamic play of forces. This is of particular importance in the context of climate change where fearful and helpless affects predominantly circulate. How else does change occur?

Figure 3. Ælab, bois eau métal, July 2021. Outdoor media art installation with sound, LED tiles, scaffolding, weather kit. Espace pour la vie - Jardin botanique de Montréal. Photo: Richard-Max Tremblay.

The first outdoor media installation by MÉDIANE to explore the potential of changing climates is bois eau métal (2021), an artwork concerned with the ecophysiology of three trees, with numerical data from Smartforests Canada research site, including immediate weather sensing. Presented in the Arboretum, the scaffolding structures are a site of temporary co-constructions (chantier), mini pavilions without walls for experiencing three types of immersive sounds setups, video imagery and graphic visualizations. Tree data from a forest is co-composed with colors from a custom-made weather kit. The whole manifests as rhythms of physical processes in feedback loops. This installation acts as information, pushing the exploratory differential encounters between fields of knowledges and experiences in a shared reality with trees. Stengers analyzed science domains which often perpetuate their own methods of safeguarding knowledges production. Contemporary art often relies on the unwavering codes of modernist heritage to produce and present artwork as autopoiesis, production of self “in and of itself”. To promote changes, in Stengers' words, “this is the reason why, through the exploration of knowledges, there is also a type of ethical experimentation” (our translation).⁵ Respecting and challenging, one and other. Shifts occur, along with the tree's own experience; despite the tree being rooted in one location, seemingly immobile, the sensor data shows the tree is constantly moving and changing at microscales.

Figure 4. Ælab, bois eau métal, July 2021. Outdoor media art installation with sound, LED tiles, scaffolding, weather kit. Espace pour la vie - Jardin botanique de Montréal. Publics are invited to participate in semi-directed interviews, engaging with their experience of the artwork. The analysis of these interviews influences the production of the following installations in the iterative series. Photo: Caroline Pierret. 

Changing climates warrant to be addressed as a “problem” in and of change, within research fields and in their differential gathering, and in dialogue with diverse publics. How can ecotechnologies of practice address the problem of changing climates, understood here as an obstacle for thought in and into action, rooted generally in fear, anxiety and helplessness? This is an obstacle needing new thought, a coproduction of knowledges in practice, in experience. Stengers and Simondon provide compelling offerings to think-with and act-with problems, in experimentation. In Simondon's concept of individuation, no individual (human, instrument or otherwise) preexists to the encounter. Individuation occurs through the meeting of forces, garnered little-by-little and in proximity, gatherings that build and change in time. Propelled into change by the perturbation of information, triggered by the tree's mode of living, change happens. For better or for worse, without moral imperative, it is a propositional ongoing, a doing, building with each tiny difference.⁶ This approach is core to MÉDIANE's ongoing research-creation process, whereby each installation proposes change, building with the tree, data gathering, analysis, and by the gradual spreading of knowledges production and circulation in-between trees, forest science, out-of-doors art and publics.

An orientation, not a beginning

The slogans of the FridaysForFuture global strike movement (FFF) repeat that governments, corporations and publics need “to listen to the science, to listen to the scientists.”⁷ Scientists are directly convened to address climate change, outside of the silos of fundamental knowledges production. Undeniably, scientists are needed and respected to do so. However, it points to the domination of science as holding exclusivity to resolve climate emergencies. Peoples (plural) have perhaps given over climate change issues to science and to government to solve, forfeiting distributed agencies. How can a combination of practices contribute to new actions? With Stengers, it becomes possible to extend the ecologies of practices in scientific realms to a mixing of approaches by ways of practice, in-between collectivities. A new knowledges circulation, a thinking-with trees, machines, materialities, artists, scientists and publics, even if only temporarily. This problem for thought in/to action, perturbed by climate changes, is addressed by ecotechnologies of practice. In Figures 5 and 6, the 2022 media installation entitled Orée des bois addressed these questions through the study of the birch tree's phenology, the seasonal changes from Spring to Fall which affect growth and connections with flora and fauna. 

Figure 5. Ælab, orée des bois, June 2022. Outdoor media art installation with sound, LED tiles, scaffolding, weather kit. Presented at Jardin du Coeur des sciences, UQAM. In this instance of the installation, an arch was created between triangular plots of land, community city gardens. The scaffolding is a physical drawing vibrating in spacetime with a multitude of experiences. Publics circulated between and under the structures, traversing the site to another zone with scaffolding tables of different heights. Additional activities onsite included workshops and presentations about medicinal plants, sensory ethnography, cartography, performances, conversations with drawings on birch bark, etc. Photo: Alexis Bellavance. 

A problem of & with & in practice

For Stengers, ecology is always double: scientific and political (italics added).⁸ This doubling can be activated in contemporary art, design and well as social sciences, charting the trajectories of actants. In Greek, technè is art, skill, and craft or the method employed resulting in an object.⁹ Logos tries to lay claim to (human) language, to “collect, gather, in derivatives of word, speech, statement, discourse, computation, account.”¹⁰ To consider technology only in a prepotent human way provides humans with vector to dominate and control, to justify the means to an end, often neglecting entangled elements/lives/factors. Technology understood in this dominant way severely limits approaches, because if it is seated only in the human as an exclusive semiotic rationality, it prevents or even opposes new informations to coalesce or collide with more-than and other-than. It is pertinent to note that in sustainable architecture, a definition for ecotechnologies is holistic systems or machines with the environment, including reuse, passive housing, renewable energies, crade-to-cradle designs.¹¹

Figure 6. (detail). Ælab, orée des bois, June 2022. Outdoor media art installation with sound, LED tiles, scaffolding, weather kit. The project focused on the Spring to Fall seasons of a birch tree through alterations of time-lapse sequences of photographs of the tree from camera placed directly on the tree, mixed with visualizations of inner workings (on left). The slits between the LED tiles accentuates embeddedness in physical site's activities, along with rain, wind, city noise, etc. Photo: Alexis Bellavance. 

Simondon proposes another technology, neither thing nor method fashioned by human will. In his philosophy, individuation is actualized through “phases” of relations between physical environments, instruments and humans. Technology is in-forming, changing, extending another expression of logos, the cosmos expressing its manifold in movement with animals, minerals, plants¹². A “life-living.”¹³ It is also the combined strengths of trees, scientists, artists and publics in changing climates.

This ongoing research-creation proposes ecotechnologies of practice in a crafting of “response-abilities.”¹⁴ The tree crafts problem-solving operations with the surroundings: tackling water preservation or evapotranspiration as needed, regulating sap flow, making clouds, coordinating burgeoning and senescence timings, going dormant in northern winters, turning sapflow into antifreeze to prevent embolisms.¹⁵ Furthermore, trees produce photosynthesis in a cosmic relation of sun and soil, they yield oxygen and even grow stronger and bigger with the extraction of carbon dioxide from the atmosphere.¹⁶ They communicate through VOCs (volatile organic compounds) and mycelium networks.¹⁷ These are but a few of the tree's wondrous techniques, parts of a larger continuum to coevolve their technologies over 350 million years.¹⁸ To partake gently in their knowledges, Indigenous artist Domingo Cisneros proposes eating and drinking parts of coniferous trees without harming the plant, a celebration to share in the strength of the longevity of their problem-solving techniques.¹⁹

Figure 7. Ælab and CREAF. Les trois frères (2020). Documentary concerned with forest gastronomy of the “three brothers”: spruce, fir and cedar. L to R: Stéphane Claude, Domingo Cisneros, Antoinette de Robien. Photo: Gisèle Trudel.

In a similar celebratory fashion, medical biochemist, botanist, climate activist and author Diana Beresford-Kroeger is as adamant as she is hopeful in times of changing climates. Her “Bioplan” makes simple statements. She advocates to protect trees in the immediate vicinity as well as native forests in local communities.²⁰ So efficient, it's alarming. Her approach changes the habitual urgent call associated with resiliency and adaptation of diverse communities that are foregrounded through catastrophic discourse. Change—with life-living trees—is summoned as operation. As Stenger states, consensus is stasis, refusing (to) change. Beresford-Kroger's bioplan extends symbiotic intimacy with tree technologies.

Ongoing Doings

Ecotechnologies of practice entail a perturbation in the habitual conception of the relation of ecology and technology, explored here as operative process. The tree's ecological practice, in conjunction with that of arts and sciences, investigates changing climates with creative problem-solving in dialogue with publics. The differential fields of knowledges bring new expression by their combination, a strength emerging in-between, pushing forth modular alliances. Crafting changes is ecotechnologies of practice: even if ephemeral, potential is actualized through encounters, soliciting variation and individuating collectivities. 

Acknowledgements

The research-creation is conducted on the unceded lands of the Kanienʼkehá ka Nation. The MÉDIANE team gratefully acknowledges funding from SSHRC, CFI, FRQSC and support from Hexagram-UQAM.

References

1 Jennifer Gabrys, Program Earth. Environmental Sensing Technology and the Making of a Computational Planet, Minnesota, Minnesota University Press, 2016.

2 Isabelle Stengers, La guerre des sciences, Cosmopolitiques 1, Paris, La Découverte, 1997.

3 Erin Manning, Out of the clear, Colchester and New York, Minor Compositions, 2022.

4 Gilbert Simondon, L'individuation à la lumière de notions de forme et d'information, Grenoble, Jérôme Millon, 2005.

5 Isabelle Stengers, La guerre des sciences, 8.

6 Gilbert Simondon, L'individuation à la lumière de notions de forme et d'information.

7 Accessed November 4, 2022, https://fridaysforfuture.org.

8 Isabelle Stengers, La guerre des sciences, 58.

9 Accessed November 6, 2022, https://www.oxfordlearnersdictionaries.com/definition/english/technology.

10 Accessed November 6, 2022, https://www.etymonline.com/word/-logy?ref=etymonline_crossreference.

11 Accessed November 6, 2022, https://leverageedu.com/blog/ecotechnology/.

12 Carl Sagan, “Cosmos (1980),” IMDb website, accessed November 1, 2022, https://www.imdb.com/title/tt0081846/characters/nm0755981#quotes.

13 Erin Manning, Out of the clear.

14 Donna J. Haraway, Staying with the Trouble: Making Kin in the Chthulucene, Durham, Duke University Press, 2016.

15 Jehová Lourenço Junior, Community Ecology - Wood Anatomy - Plant Physiolog, Accessed November 3, 2022, https://jehovajr.wixsite.com/science/post/the-hydraulic-architecture-of-trees-in-a-climate-changing-scenario

16 As Carbon Dioxide Grows More Abundant, Trees Are Growing Bigger, Study Finds, Accesssed October 10, 2022, https://e360.yale.edu/digest/carbon-dioxide-climate-change-bigger-trees

17 Suzanne Simard, Finding the Mother Tree, Allen Lane, Penguin Random House Canada, 2021.

18 Serge Muller, Germinal Rouhan, “Comment les arbres sont apparus sur Terre,” Museum, November 30, 2021, accessed November 1, 2022, https://www.mnhn.fr/fr/actualites/comment-les-arbres-sont-apparus-sur-terre.

19 Ælab and CREAF (Centre de recherche et d'expérimentation des arts forestiers), “Les trois frères (2020),” https://vimeo.com/channels/472281/452356412).

20 Accessed November 15, 2022, https://dianaberesford-kroeger.com/mission/. 

Author Biography

Gisèle Trudel and Stéphane Claude founded the experimental documentary artist research unit Ælab in 1996 (aelab.com). Collectivities actualize the potential of MÉDIANE (2020-2025, mediane.uqam.ca).

Water Stories: Visual Poetics and Collective Voices

Andrea Wollensak, Brett Terry, Bridget Baird

Ammerman Center for Arts & Technology, Connecticut College Sonalysts
Emerita Computer ScienceDepartment, Connecticut College New London [authors 1 and 2] and Waterford [author 3] CT,USA
ajwol@conncoll.edu brettcarrollterry@gmail.com bbbai@conncoll.edu

Abstract

Water Stories: Visual Poetics and Collective Voices is a two-part project that brings together multiple points of view from local youth, community, and poets in Alaska to share what water means in their life. Visual Poetics combines a live poetry reading by Alaskan poets and interactive video in which the poets’ voices trigger generative visual elements. Collective Voices is a sound work featuring excerpts of community voices sharing water-based memories against a backdrop of processed environmental sounds of Alaskan waterways. Water Stories is part of a year-longartist residency (2021-2022) with the Anchorage Museum culminating in a series of listening sessions broadcast at the Anchorage Museum and Out North Radio, live interactive poetry readings at the museum,and video projections on the museum façade from November 2022 through January 2023.

Keywords

Generative, digital poetics, audio-reactive software, community engagement, collaborative, ecosystems,resilient societies, multiple scales.

DOI

10.69564/ISEA2023-34-short-Wollensak-et-al-Water-Stories

Introduction

Alaska’s climate is projected to warm several degrees by the end of the century, more than twice the warming compared to the rest of the country. The record-breaking temperatures experienced by Alaska communities have resulted in thawing permafrost, thinning sea ice, and more wildfires. The Water Stories: Visual Poetics and Collective Voices project engages the Alaskan public to consider how climate change alters their experiences of the land, and to consider the role of water in their lives.

Background

Water Stories is the most recent artistic collaboration between the authors, and is part of aseries of works focused on climate change and the environment. Reading the Wrack Lines (2021),an outdoor audio video projection on a lighthouse, featured community-sourced textual reflections on climate change. Open Waters (2020) was a multiple-media installation exhibited at museums in Rhode Island, and New York, connecting the five-hundred-year imaginative history of the open polar sea and northwest passage with current geopolitics and arctic climate change. Another previous audiovisual collaboration, Ice Core Modulations: Performative Digital Poetics (2017) included generative imagery and poetic texts controlled via historical CO2 data taken from ice core samples made available from the National Snow and Ice Data Center (NSIDC). All three of these works have been presented at past ISEA Conferences.

These collaborations have sought to create engaging works that address complex environmental issues through a plurality of artistic, poetic, and scientific perspectives. The collaborators have brought to bear individual expertise in audiovisual generative computer programming, visual art media, poetry, and electroacoustic sound composition to synergistically create works unified by shared source materials that include place-based personal narratives, historical source material, raw scientific data, local audiovisual media collected on site, and original poetic texts.

Goals, Objectives and Artistic Dimensions

Water Stories seeks to amplify voices of Alaska residents through storytelling, poetry, and local landscape imagery. The overarching project goals and objectives include:

Figure 1. Videostill of Ruckle’s poem “Ferry” in audio-reactive projection

Ecosystems of Collaboration

There are two synergistic types of collaboration taking place within the WaterStories work. In the first, the collaboration between the creators of the work allows artists to bring their own unique perspectives and disciplinary skill sets together. As the collaborators have previously worked together on site-based work exploring themes of climate change, several methods already developed to facilitate this collaboration were leveraged for this effort, in particular the use of original poetic phrases to provide asemantic glue unifying different visual, sonic, and textual elements. In this case, the performative nature of the collaboration was underscored by having a live poetry reading be a central element of the work. With the poet present, hearing their words while seeing the work respond to their speech connected the underlying collaborative synergy between text and image on the surface, and at a deeper level, with the connections implicit in the semantic connotations and subject matter of the poetry itself.

The second level of collaboration in the work connects the audience and creative producers,capturing community narratives through interviews and recorded reflections. CollectiveVoices empowers individual voices through community collaboration, allowing them to interplay and be heard in a public creative context, fostering a reflective audio-based ecosystem through which the listener experiences a plurality of narrative and poetic perspectives. Through its use of intertwined recorded personal reflections, the work bridges the private/public gap, and was shared publicly via broadcast on radio, internet, and as part of an outdoor audio installation at the Anchorage Museum.

Visual Poetics

Alaskan poets Erin Coughlin Hollowell and Jen Stever each composed ten poems as part of this collaboration. Framed by her Iñupiaq culture and history, Stever’s poems focus on the local environment, family, and the changing climate. Hollowell’s poems explore careful observations of the coastline and reflections of visual language and form. Distinct phrases from each poem were incorporated into the Processing software in several different visual representations. Randomly selected phrases fade in and subsequently fade out within the audiovisual work, with some phrases coalescing from and dispersing into granulated pixels.

Figure 2. Poetry Live Reading Performance

During several trips to Alaska, photographic and video source materials were collected and used as background imagery within the work (Figure 2). Adding to the text and shifting background imagery, different visual elements and behaviors were developed in Processing for each poem, using a vocabulary of moving generative forms include triangular networks, shards, and ellipses, each with distinctive color palettes tied to the background imagery (Figure 3). A reactive audio system was developed to listen to the poet’s voice, using amplitude and frequency to trigger and modulate the generative forms, with different behaviors for each poem.

Figure 3. Video still of Hollowell's poem “Choreographic” in audio-reactive projection.
Figure 4. Video still from Stever’s poem “Swan Lake Fire” on the Anchorage Museum façade from November 2022-January 2023.

Two versions of the work were developed. The first version was a live performance in which both poets were present reading their works, accompanied by the audio reactive Processing software running in real-time to generate visuals projected behind the poet. The second version (Figure 4) was made to project nightly on the façade of the Anchorage Museum for several months, and was made by recording a performance of the work as a video file. The two versions thus exhibit differences of scale, accessibility, and connection to the spoken voice.

Collective Voices

To create the eight-minute audio composition Collective Voices, several recording sessions took place at the Anchorage Museum and OutNorth Radio in which community participants shared personal narratives about place and local waterways. The authors would like to gratefully acknowledge community partners Indra Arriaga Delgado and OutNorth, a progressive arts organization with a mission to advance contemporary art in Anchorage, support underrepresented voices, and promote cultural dialogue by amplifying the alternative voice. In order to promote a diversity of voices, many different community groups were invited to participate including high school students enrolled in the Anchorage Museum’s afterschool Teen Climate Communicators program and members of the general public invited to record their stories at the recording session venues. Selected excerpts from these narratives were edited into an audio work using Logic software and combined with multiple layers of local processed environmental sounds from the Anchorage Museum’s audio collection. Collective Voices was presented at a listening station at the Anchorage Museum during their November First Friday community event, broadcast on Out North Radio, and made available for internet listening via SoundCloud.

Conclusion

Water Stories: Visual Poetics and Collective Voices engages multiple points of view from local youth, community, and poets in Alaska to share what water means in their life. Grounded in a year-long artist residency at the Anchorage Museum, this interdisciplinary project explores the changing environment through place-based ecosystems of collaboration.

Authors Biographies

Andrea Wollensak is a multimedia artist, designer and educator. Her work spans media from traditional and digital fabrication to generative-interactive systems and includes frequent collaborations across disciplines. Wollensak’s work has been exhibited internationally, most notably at the Göteborg International Biennial of Contemporary Art and the Brno Biennial of Design. Her work has been supported by the Rockefeller Foundation, the International Artist Studio Program in Sweden, Anchorage Museum, Banff Centre for the Arts, and the National Science Foundation. She has presented her artwork at numerous venues including ISEA, CAiiA, Generative Art, and College Art Association. At Connecticut College, she is on the faculty in the Studio Art Department, and an Associate Fellow at the Ammerman Center for Arts and Technology where she served as Director from 2012 to 2020.

Brett Terry is a composer and sound artist, when not busy with his daily life as a software engineer. His electro-acoustic, choral and chamber compositions have been performed at venues such as SEAMUS, ICMC, ISEA, CAiiA, and Sound Culture in addition to collaborating with visual artists on numerous audiovisual works. As an associate editor of Computer Music Journal (MIT Press), he has curated a special issue on Visual Music.

Bridget Baird is a Professor Emerita in Computer Science and Mathematics at Connecticut College and a past Director of the Ammerman Center for Arts and Technology. Her research examines the intersections among the arts and various technologies. Projects include investigating an archaeological site in Ecuador through virtual reality, exploring music and dance through motion capture and multiple modalities, using digital techniques and algorithms to better understand and mine historical documents, and more recently, addressing climate change and environmental concerns by using generative art. Baird collaborated, as a Fulbright scholar, with colleagues in both Mexico and Ecuador. Involvement with the local community has also been important to her and a constant interest has been to increase the number of women in the sciences.

Nga manawataki o te koiora: biological rhythms, posthuman design and decolonial thought.

Rewa Wright, Simon Howden

Queensland University of Technology Independent Artist Meanjin/Brisbane, Australia rewa.wright@qut.edu.au

Abstract

Western science, in fields such as computational ecology, has grown to accept the truths that Indigenous culture have long known: that computational ecology accepts that ecological models are too complex to be summarised in computational form. Since this complexity evades the codification of mere indexing, how then, should we work with computational companions (code, algorithms, programs, platforms). What new ways of intra-acting can we develop alongside computational frameworks, which bring us one more step closer to sentient machines? Most importantly, how can ethical ways of thinking and doing motivate transformations in the computational space, in areas such as machine learning where extreme problems of bias are now embedded? This research does not answer these complex questions, for they are genuinely ‘wicked problems’ that reach toward wider issues of equity, sustainability, and economy. Our aim is to use creative practice to generate gestures and markings that tentatively trace a way forward. This research contributes to new modalities of human computer interaction that attempt to restore the dynamic pathways developed by Indigenous thinking, challenging artificial boundaries such as nature/culture, instead giving respect to concepts of interconnection. Examining some of the differences between Western epistemology and Indigenous thinking opens a pathway toward Indigenous Futures that are crafted in support of a decolonial ecology.

Keywords

Decolonial art, Touch Designer, Plant signals, ‘More-than-human’ Design, Intra-action, mātauranga Māori.

DOI

10.69564/ISEA2023-35-short-Wright-et-al-Nga-manawataki-o-te-koiora

Introduction

Through imaging technologies such as electron microscopes, quantum physics has been slowly proving to itself that all particles in the universe are deeply entangled. Now, the curious situation exists where Western science has ‘validated’ what mātauranga Māori (Māori knowledge) has known and preserved for thousands of years. A close analogy for the unseen movements of quanta is mauri (transitory and shifting states of being), while the structure these quanta form with other particles, emerges in wairua (felt interconnections between all living things). In the Māori world view, people and nature are inseparable and share genealogy. This resonates with current scientific opinion which has only come around too lately to the perspective that plants are to be as valued as humans. Indigenous thinking is in fact far ahead of Western science, in that we never lost the intuitive ways of knowing and being, and we never blindly accepted the artificial separation of mind and body of Descartes’ Enlightenment edict. Through quantum imaging in the 1990s, physicists finally saw that the whole world was connected, every nanoparticle and atom is entangled with every other. This was not new to mātauranga Māori, which for thousands of years had preserved that intuitive knowledge of deep entanglement in our concepts of wairua (life force, shared between all living things) and mauri (transitory and shifting states of being). Underscoring the radical cosmological contingency that materially binds humans to all other entities, the concept we know as mauri is expressed to a degree in quantum physics. That is, the principle that nano level particles indeed bind together very entity in the universe. Our work is about the entangled connections that are intuitively felt, and join humans, nonhuman/more-than-human kin and algorithms in a constant state of becoming. Our research thinks alongside evolutionary biologists such as Monica Gagliano and Stefano Mancuso, who argue that plants are intelligent [2,4]. Situated agency emerges as a tangled network of augmented reality infrared vision, gestural and plant bio-electrical data, audible through a bespoke sensing network of hardware devices and custom software. Understanding the invisible world humans cannot see is important, therefore, to weave an understanding of interconnection back into the process of everyday life. In our work, plants signal to one another as part of their growth process, and we captured these signals as MIDI, an audible expression of their invisible movements. The arhythmic and micro-temporal signals were then incorporated as a foundation to co-create a sound scape with our plant companions. Contact/Sense explores the entangled connections that are intuitively felt between humans and other life forms, sentient or not. Framing the eco-digital within our artistic practice is the scientific context of evolutionary plant biology, on the one hand, and the spiritual Indigenous approach to knowledge as beyond human, assembled in complex networks that are not only empirical but acknowledge unseen unquantifiable forces and the insights it gives as to human behavior and thinking in relation to plants. Agential realist accounts such as those elaborated by Barad [2008], go some way toward destabilizing, at least within posthuman philosophy, the Humanist/Enlightenment duality, opening space for extended speculation on matter and meaning. However, these Western systems occlude the cosmology of knowledge systems like mātauranga Māori. Underscoring the radical cosmological contingency that materially and spiritually binds humans to all other entities, mātauranga Māori traverses both material and spiritual realms, pointing toward a universe that is not entirely knowable, where empirical claims to deep knowledge about the production of matter by material forces are only partially relevant. Permeated with temporal and spatial relations that defy the "arrow of time" and granulize past, present and future into a non-linear series of instances, mātauranga Māori affirms the validity of techniques and methods that trace speculative knowledge.

According to plant neurobiologists Mancuso and Viola, plants are not "passive machines" for processing light, water and food: In fact, they are "intelligent" and show this by constantly signaling to one another and to the world, for example, giving off scents that attract or repel insects and even sharing resources, such as water, amongst their communities [4]. Tracing the connections between quanta and nanoparticles as mauri and wairua as a starting point for a discussion of new media art in a decolonial framework, this paper reflects upon intersectional threads within art, physics, data and posthuman design, to weave an interconnected path between Western and Indigenous science, philosophy and cosmology. We use our practice in media art blending computational networks, human bodies, and plant signals to explore the metaphysical and embodied knowledge manifest through a decolonial symbiosis or science, art and technology, and inclusive to all humans as well as our "more-than-human" interlocutors (plants and algorithms). Concepts of symbiosis may seem attractive since they indicate hybridity, however, as art practitioners we need to be mindful of the power relations in any merger: so that in our writing we approach symbiosis in a decolonial sense, to alleviate the tendency for a more powerful force to overtake a lesser one.

"More-than-human" creativity

"More-than-human" creativity is an emergent thread that reaches toward forming relations with more-than-human kin, organic and silicon, such as plants and algorithms. Through research that bridges plant signaling with augmented and extended reality, machine learning, and interaction design for virtual environments, this project explores the entangled quality of human creativity in the computational networks and living ecologies. This embeddedness, or interdependency, or entanglement, between the organic and the silicon enables new forms of creativity, where artists’ partner with nonhuman kin via iterative acts of co-creation.

This approach destabilizes traditional principles of design, which emanate from the intentionality of the artist or designer. A more-than-human approach allows for an open space of generative creativity where unplanned events and phenomena can unfold, shift, and disrupt the final work. Through practice-based and traditional research, the installation project Nga manawataki o te koiora: Biorhythms, has several interconnected aspects. Indigenous art is conventionally seen as manifesting tradition and not technology. However this work overturns that power relation, where embodied cultural knowledge is allowed to flow through computational networks.

Embodied knowledge and algorithmic design

Nga manawataki o te koiora: Biorhythms, is a projection mapping and installation piece that takes you on a journey into a computational transduction of the forest, rivers and oceans of Aotearoa/New Zealand. Visually, an interconnected natural ecology is translated into the real-time world of audio reactive geometries and mesh topologies. The concept was to convey the feeling of these things, without literal interpretation. Traditional kete (woven baskets) inspire fluid movements which become pixel topologies. Animated motion made with noise oscillators, shift from reimagined nets used to catch eel (hiinaki), to seed pods exploding from pixel plants, such as the red pōhutukawa. The soft blue/green of kina (sea eggs) become fluffy vectors transparently overlaid on a fluid mesh of waves. Following and modifying the tradition of naturalism and curvilinear geometry that marks traditional Māori art, this piece visually encapsulates the feeling of the natural world without being a literal representation that vested in Western pictorial traditions of realism. This 20-minute piece, recorded as live audio reactive in Touch Designer, is sonically a composite of human-nonhuman music, alternately interspersed and mixed together. From the "human" side, music consists of three original electronic compositions by Simon Howden. From the plant side, we intersperse original recordings of plant sonics captured in research since 2019. We consider plant to be co-composers of this work. A previous installation, Contact/Sense was performed at the SIGGRAPH Asia Art Gallery in Brisbane 2019, and combined plant sonics with mixed reality. Donna Haraway introduced the notion of "companion species" to describe non-human organic life forms that we cohabit alongside in society and culture. Plants and humans have lived alongside one another for thousands of years, in a co-dependent relationship of care and cultivation. Applying the decolonial philosophy of mātauranga Māori (Māori epistemology), combined with posthuman lens to art and science, our hope is that through this artwork, people will feel a little closer to the hidden bio-electrical processes of plants, and consider plants not as a resource for extraction, but as a ‘companion species’ in a sustainable ecology.

The plant sonics were recorded during earlier live audio-visual performances with an agave attenuate, a series of vines, several palms, and a multitude of grasses, tropical and sub-tropicals since 2018. Rewa Wright has developed a unique mode of mixed reality performance with plants and algorithms networked with the human body. The plant signals are essentially bio-electrical impulses, which we then assign to MIDI and apply sound design to, so that soft wooden drums with loose skins and resonant tapping highlight the micro temporality and asymmetry of plants, whose signals sound unstructured to the human ear. Plant rhythm is phenological, and traces their processes such as photosynthesis and osmosis as they follow patterns of growth. Several plant neurobiologists have noted the signals that plants emit are akin to intentional communication and sentience, and this is now a recognized area of scientific study.¹, ²

Plant neurobiology speaks to the material reality, unraveled by quantum physics, that every particle in the universe is connected to its nearest neighbors, and through those neighbors to all other things. Quantum imaging has provided visual proof of this connection, as discussed extensively by philosopher Karen Barad,³ whose concepts of "situated knowing" and "intra-action", both grounded in quantum (meta-)physics, have resonance with Indigenous notions of deep time/space and interconnectivity. As a foundational concept, the deep interconnection of life forces (i.e., quantum entanglement) has always been known to indigenous people, and in my culture, Māori from Aotearoa/ New Zealand, we understand this unbroken link between vegetal, organic, silicon, geological forms (to name just a few categories) to underpin all material and cosmological reality. Understanding the invisible world humans cannot see is important, therefore, to weave an understanding of interconnection back into the process of everyday life. In our work, plants signal to one another as part of their growth process, and we captured these signals as MIDI, an audible  expression of their invisible movements. The arhythmic and micro-temporal signals were then incorporated as a foundation to co-create a sound scape with our plant companions. Using computer art to translate emotions about place from bush into pixels, this work explores the potential for a symbiosis of data, plants, ecology and algorithms. This work is deeply influenced by Rewa’s cultural background as a First Nations Māori artist, from the Ngai Tawake, Te Kaimaroke, and Te Uri o Hau hapu of Aotearoa/New Zealand. Her ancestry recognizes human and plant relations going back thousands of years, and Rewa is committed to multi-species justice and sustainable living on Planet Earth. Mixing the biorhythms of plants and the calls with human produced electronic music, reveals a co-creative mesh of human and nonhuman kin, an entanglement that is fundamental. For example, in Rewa’s pepeha (personal genealogy), she traces lineage to the maunga (mountain) called Tokatoka, the awa (river) called Wairoa, and the moana (sea) called Kaipara. These connections are at the base of her identity and this is the same for all Māori, since our genealogy is tied to the whenua (land). Transducing this physical connection with the land into computational space is only ever partial. However, hints at the ways Indigenous knowledge might be embodied as data and algorithms, nurturing a framework that advances decolonial thought and gestures toward Indigenous futures. While quantum imaging has provided visual proof of this connection between all vibrating atoms and matter, it is a concept that has always been known to indigenous people, and in my culture, Māori from Aotearoa/ New Zealand, we understand this connection between vegetal, organic, to underpin all material and cosmological reality. 

Figure 1. Screenshot from ‘Nga manawataki o te koiora’.
References
Books

1 Paco Calvo, Monica Gagliano, Gustavo M. Souza, Anthony Trewavas, Plants are intelligent, here’s how, Annals of Botany 125, no. 1, 2020, 11-28.

2 Stefano Mancuso, Alessandra Viola, Brilliant green: the surprising history and science of plant intelligence, Island Press, 2015.

3 Karen Barad, Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning, duke university Press, 2007.

Authors Biographies

Dr. Rewa Wright (Ngai Tawake/Te Uri o Hau/ Te Kaimaroke) is Senior Lecturer in Film, Screen and Animation at the Queensland University of Technology, Australia. Simon Howden is a sound designer and independent music producer. Both humbly reside in Meanjin/Brisbane on the lands of the Turrbal and Yugara, First Nations owners. We recognise that these lands have always been places of teaching, research and learning, and that sovereignty was never ceded. 

The Dark Side of the NFTs: The artists’ need for new systems of collaboration

İpek Yeginsu

Kadir Has University Faculty of Communication, Department of Visual Communication Design Istanbul, Turkey
ipekyeginsu@khas.edu.tr

Abstract

NFTs are often imagined as the initiators of a democratic revolution in the art world, and yet their effects are much more complicated than meets the eye. While the financial independence and the new exposure channels they offer to the artists are undeniable, they also disturb the art world’s current equilibrium, resulting in the emergence of a chaotic art ecosystem that makes the independent artists’ already existing vulnerabilities even more pronounced. This paper defends the position that artists necessitate new systems of collaboration to sidestep these negative effects, overviews some promising examples of such attempts and discusses the potential solutions for the future.

Keywords

Art ecosystem, digital art, NFT, blockchain, crypto art, artist communities, collaboration.

DOI

10.69564/ISEA2023-36-short-Yeginsu-The-Dark-Side-of-the-NFTs

Introduction

In today’s art ecosystem radically reshaped by the conditions of the Covid-19 pandemic, the artists working with digital technologies have more exposure channels available to them compared to those working with traditional media. The former are able to exhibit their works online without facing the criticisms the artists using conventional media often receive for choosing a digital exposure platform, and they are free from the costs associated with transporting, insuring and installing a physical exhibition. Neither do they necessitate the partnership of a gallery or a museum to present their works; social media platforms and NFTs allow them to sidestep the gatekeepers in the mainstream arts and culture institutions, contributing to the emergence of a more democratic and decentralized art world. In fact, Sarah D. McDaniel and Denny Galindo observe that “art galleries and auction houses have been the market makers of the art world”, and “NFTs and their marketplaces disrupt this process by allowing more artists to go straight to market and sell directly to buyers.”¹

The NFTs’ market success empowers the artists in other ways as well. For instance, Bernard Marr draws attention to the individual power the NFTs have bestowed upon Pplpleasr, the artist also known as Emily Yang, who uses her NFT sales’ revenue to support the other artists and especially those of Asian descent.² Artists without a digital background are also able to access this unfamiliar territory thanks to a new line of service providers facilitating migration into the digital realm without being lost in technicalities. A good example for such endeavors is Vive Arts, a company specialized in the sales and exhibition of art “in all digital formats”, assisting its clients starting from the artwork’s creation phase.³ On the other hand, for the digital artists wishing to develop their skills, Visa launched the “Visa Creator Program” to “tutor a curated selection of digital artists through the NFT process, helping them maximize their talent and use the blockchain space as both a creative tool and marketplace.”⁴

Although NFTs emerge as a convenient funding source for independent artists, they are also criticized for their adverse effects on the environment due to their heavy energy consumption and the subsequent carbon emission. But a more subtle phenomenon challenges the artists in other ways that might threaten their career sustainability in the long run unless they succeed at establishing new systems of alliance and collaboration. The following section addresses both the NFTs’ ecological impact and these relatively implicit effects in more detail.

NFTs and Chaos in the Art Ecosystem

NFTs and Climate Change

The NFTs’ impact on the environment has been a source of ardent debate since the beginning. Some artists have been among the leading opponents of their use in the art world due to their adverse effects on the planet, and they have also been expressing their concerns publicly. For instance, when media artist Joanie Lemercier discovered that the sale of his NFT series consumed the same amount of energy needed to run his studio for two years, he decided to cancel his two upcoming NFT releases.⁵ Another media artist, Memo Akten, designed a system to assess Ethereum’s carbon footprint in numeric values and created the website “cryp-toart.wtf” to share the most up-to-date information about the issue.⁶ Nowadays, he continues to produce NFTs on “eco-friendly blockchains.”⁷ However, others like Eric James Beyer claim that “criticism is exaggerated and conflates hype with the truth.” Beyer compares the NFTs’ yearly energy consumption with that of the small mining industries, giant tech’s data storage facilities or residential air conditioning devices, and reports that the former is relatively lower. He adds that new measures to reduce carbon emission rates are also being implemented, such as Ethereum’s migration from a PoW (Proof-of-Work) to a PoS (Proof-of-Stake) system, and mentions various ecofriendly trading platforms as plausible alternatives.⁸ Raisa Bruner is another optimist mentioning the success of the Canadian company CurrencyWorks in “turning oil waste into environmentally friendly energy that powers crypto mining”, and the Israeli company StarkWare that was able to invent a technique to store more information in one block unit, reducing the system’s overall energy consumption.⁹ Even Mike Winkelmann, the artist widely known as Beeple, ensures the public that he will eventually manage to “completely offset emissions from his NFTs by investing in renewable energy, conservation projects, or technology that sucks CO2 out of the atmosphere.”¹⁰ Efforts at building synergy between the NFT min(t)ers and renewable energy sectors are already plenty. But whether the artists are ready to face the challenges brought about by this new technology reshaping the art ecosystem is seldom addressed.

 

The NFTs’ Negative Effects on the Artists

The problems regarding the NFTs are not limited to their ecological impact; in addition to their advantages, they have adverse effects on the artists’ careers as well. In his article on The Atlantic, Anil Dash observes that “the current NFT market is drawing an extraordinary range of grifters and spammers” and the artists’ works are regularly being hijacked on social media and converted to NFTs and sold by other people without their permission, so much so that, when an app designed for blocking user groups on Twitter was released, its leading customers were the artists “using it to block NFT spammers from hijacking their works and monetizing them as NFTs without permission.”¹¹ The artists’ vulnerabilities in the digital environment are also pointed out by Cass Marshall, who explains that the artists exhibiting work on social media and worrying about hijackers ultimately decide to avoid online presence altogether. And if the hijackers are able to generate the right type and amount of speculation around the work, they are able to obtain much higher revenue from the same work compared to the original artist.¹² Thus, as Simon Spichak says, it is no surprise that “much of the pushback against NFTs is coming from the artists that they were supposed to help”, due to the unbridgeable difference between the copyright culture of the latter and those of the newly emerging NFT fans. He adds that only 50% of the artworks in the NFT market are sold for prices above 200 US dollars, and no correlation exists between the work’s artistic quality and financial worth. To the contrary, the majority of the astronomic sales prices belong to works without artistic sophistication.¹³ Andy Storey identifies yet other ways in which NFTs negatively impact the artists’ careers, such as speculative and fluctuating prices hurting the artists’ overall market value, the NFT platforms’ high entry costs, low respect for the works’ artistic value, and the possibility of the work being used in contexts the artist does not wish to be associated with.¹⁴

The NFTs have also become a source of conflict for various actors in the art world. The artists continuing to produce NFTs and the NFT platforms offering their works for sale are often pressurized by the anti-NFT artists to stop doing so. The pressure can be so serious that, for instance, the artist portfolio platform ArtStation had to cancel its upcoming NFT initiative due to intense online criticism by its users based on environmental concerns.¹⁵ The platform was even compelled to release an official statement apologizing for “all the negative emotions” that their actions caused.¹⁶ Similarly, the artists participating in the Art Wars project declared that the project’s initiator, artist and curator Ben Moore did not ask for their permission when producing their works’ NFT versions. Consequently, the NFT marketplace OpenSea decided to remove the project from their inventory. And complaints do not only come from the artists either; they target the artists instead. Hermès, a giant brand, sued the artist Mason Rothschild for releasing “MetaBirkins” or “100 NFTs resembling the iconic Hermès Birkin handbag.” The result of the lawsuit will also impact the collectors who have already bought items from the collection.¹⁷ Rothschild defended himself by arguing that under the First Amendment, he had the right to “create art based on [his] interpretations of the world around [him]” and drew an analogy between his work and Andy Warhol’s Campbell Soup Can, but it was not sufficient for the court to dismiss the case.¹⁸ Put differently, the NFTs emerge as an important source of chaos and conflict in the art world, particularly in relation to issues around ecology and copyright.

In sum, the NFTs constitute an additional income source and an alternative exposure channel especially for the independent artists, allowing them to bypass the gatekeepers in the museums, galleries and auction houses. But they simultaneously give rise to a more chaotic art ecosystem in which artists have to choose between increased visibility and the violation of their intellectual property rights, or producing critical work referencing iconic images and facing corporate lawsuits. Environmental concerns associated with NFT minting and transactions add further complexity to the issue, increasing the tension between pro-and anti-NFT artists. Moreover, due to the medium’s accessibility to everyone which appears to be an advantage, the artists have to compete against the NFT creators without an artistic background who prioritize financial return over artistic quality.

New Systems of Alliance and Collaboration for the Artists

As a response to the challenges at hand, artists necessitate new systems of alliance and collaboration allowing them to navigate this chaotic art ecosystem more safely. Apparently, one of the basic strategies with fruitful results is to join forces with other artists in the form of co-productions. In fact, fine art photographer Gabriel Dean Roberts explains that he finds NFT collaborations with other artists and even cross-disciplinary collaborations with musicians highly beneficial, as each side brings their own audience into the picture while the uniqueness of the collaboration itself is a form of added value. Joint ventures also contribute to the sense of community among NFT artists by highlighting interconnectedness.¹⁹ Such a solidarity project took place in 2021 when a hundred artists jointly created a single NFT composed of a hundred parts that sold out within a few minutes. According to the project’s curator Loopify, the artist list consisted of a few established figures and “one that is new to NFTs”, and the idea emerged in response to “the current limitations for lesser-known artists to mint and sell their own NFTs such as high gas fees, limited understanding of the tech, and little visibility”. Moreover, to ensure that the artists were exempt from paying additional fees, the revenue was first transferred into USD coin, from which each artist received an equal share. ²⁰ These examples illustrate that artist alliances around NFT creation have the potential to enhance the visibility of every individual artist while reducing the costs associated with NFT production in comparison to doing so single-handedly, and they particularly benefit the emerging artists with limited experience in crypto art. These, in turn, could increase diversity in the art world, ultimately contributing to its long-term sustainability. Collective production also reduces the carbon emissions per capita associated with individual NFT minting, which would minimize the NFTs’ negative impact on the environment and further contribute to the sustainability of the NFT art ecosystem.

The problems related to copyright, intellectual property theft and hijacking require even more organized action on the part of the artists and a closer collaboration with the legislative parties as well as with legal professionals specializing in this subject. Jessica Rizzo observes that, ultimately, the authority resides in the hands of the courts whose exemplary decisions will gradually establish the principles dominating the creation, distribution and trading of the NFTs. According to Juliet Moringiello from Widener University, the laws of intellectual property in the physical world are actually valid in the web3 environment as well, and she refers to the 1996 “Declaration of the Independence of Cyberspace” by John Perry Barlow as a similar example where his attempts at justifying the bypassing of the existing laws ultimately failed.²¹ Yet, the legal framework is still insufficient for the artists even in the United States. For instance, on its official website, Thienel Law LLC refers to The Visual Artists Rights Act of 1990 (VARA) as well as The Lanham Act (Trademark Act) and the Copyright Act as only partially protective documents.²² And as the current legal framework around NFTs is still in formation, the artists themselves should be more directly involved in the establishment of these principles. One way in which they have already begun to make a positive difference is to hold the minting and sales platforms accountable for regulating their clients’ activities with regards to intellectual property. A good example for such systems is provided by OpenSea; in 2022, the platform implemented an anti-plagiarism mechanism consisting of both algorithmic detection and review by actual humans.²³ OpenSea also enabled a stolen art reporting system thanks to which the original artist can file a takedown request against the plagiarist, although the system does not guarantee that the request will be accepted.²⁴ Another solution could be to have the NFT market platforms involve the artists themselves in the artwork curation and elimination processes, so that unethical behavior by a member of the community is directly sanctioned by that very community, e.g., by banning the user from the platform following the receipt of a certain number of complaints. In short, collective action on the part of the artists can be effective in pushing all NFT platforms to take the necessary measures, which would also accelerate the establishment of a basic legal, or at least ethical framework for NFT art creation and transaction around the world, and would help reduce the chaos in the NFT art ecosystem.

As illustrated by the MetaBirkin case, brands are another crucial party in the legal disputes over NFTs. One of the most difficult dilemmas emerges around the artists’ right to borrow iconic imagery from popular culture vs. the brands’ urge to protect their intellectual property. This is one of the most sensitive territories for the future of artistic creativity. If the courts decide to overprotect the brands, they will also have acknowledged that at least half of the seminal artworks from the 20th century are copyright violations. Such a verdict would condemn all the artistic practices involving appropriation, collage and pastiche, and it would do so because it “can” thanks to the digital technology facilitating surveillance and tracking. This, in turn, would seriously deprive the NFT world from the ability to produce art with a political or critical stance, almost “sterilizing” it into a corporate-friendly environment. The dilemma is further complicated by the fact that brands complain about the NFT artists in the same way that artists complain about their online hijackers, and technically differentiating the two contexts is not always easy. Introducing itself as “the first blockchain ecosystem built by the art community, for the art community,” Arcual is a promising initiative in this regard. Jointly founded by Luma Foundation, Art Basel’s parent company MCH Group and BCG Digital Ventures, the platform aims at creating a clean NFT trading environment and offering the artists additional revenue from secondary sales by using “smart contracts with embedded resale terms.”²⁵ Again, the main dilemma presents itself as the trade-off between creative freedom and financial/legal safety, for companies founding such initiatives also have vested interests in the corporate world especially in the form of sponsorships, and whether in the case of a dispute between an artist and a brand they would come in defense of the former instead of the latter leaves a big question mark.

Conclusions and Ideas for Future Research

As illustrated by the overview above, the NFT market’s global outreach and independent environment is extremely beneficial for the artists, but it also intensifies their existing vulnerabilities especially in terms of intellectual property rights by destabilizing the art ecosystem as well as contributing to the deterioration of the ecological one. As the ethical framework of this newly emerging territory has not yet fully developed, the artists might still seize the opportunity to directly shape its future if they manage to act in solidarity, hold the NFT trading platforms accountable and insist on directly participating in the ongoing legislative processes. Further research is necessary to deepen our understanding of the dynamics dominating the NFT art world, the existing legislation at the national, regional and/or global level and how artist communities do and can operate in the face of the existing challenges. Most importantly, further study is necessary to explore how broader artist communities can be mobilized for collective action.

References

1 Sarah D. McDaniel, Denny Galindo, “Democratizing Art: How NFTs Are Reshaping the Art World”, Morgan Stanley website, accessed December 1, 2022, https://www.morganstanley.com/articles/nft-art-market-nft-collectibles#:~:text=NFTs%20and%20their%20marketplaces%20disrupt,galleries%20and%20artists%20get%20paid

2 Bernard Marr, “Web3, NFTs, And The Future Of Art”, Forbes, August 19, 2022, accessed December 2, 2022, https://www.forbes.com/sites/bernardmarr/2022/08/19/web3-nfts-and-the-future-of-art/?sh=6731712f1e05

3 Vive Arts, “Combining industry-leading metaverse technologies with bespoke services, Vive Arts marketplace showcases and sells art in all digital formats”, Vive Arts website, accessed December 1, 2022, https://www.vivearts.com/about-platform

4 Mark Hunter, “Visa Launches NFT Artist Support Program”, FullyCrypto, October 14, 2021, accessed December 1, 2022, https://fullycrypto.com/visa-launches-nft-artist-support-program

5 Gregory Barber, “NFTs Are Hot. So Is Their Effect on the Earth’s Climate”, Wired, March 6, 2021, accessed November 30, 2022, https://www.wired.com/story/nfts-hot-effect-earth-climate

6 Memo Akten, “The Unreasonable Ecological Cost of #CryptoArt (Part 1)”, Medium, December 14, 2020 (updated December 2021), accessed November 30, 2022, https://memoakten.medium.com/the-unreasonable-ecological-cost-of-cryptoart-2221d3eb2053

7 Memo Akten, “NFT”, www.memo.tv, accessed June 16, 2023, https://www.memo.tv/works/nft/

8 Eric James Beyer, “NFTs and the Environment: Why the Anger Is Unjustified”, nftnow, September 16, 2022, accessed November 30, 2022,

9 Raisa Bruner, “Environmental Concerns Have Cast Doubt on NFTs—But That’s Changing”, Time, November 18, 2021, accessed November 30, 2022, https://time.com/6120237/nfts-environmental-impact/

10 Justine Calma, “The climate controversy swirling around NFTs”, The Verge, May 15, 2021, accessed November 29, 2022, https://www.theverge.com/2021/3/15/22328203/nft-cryptoart-ethereum-blockchain-climate-change

11 Anil Dash, “NFTs Weren’t Supposed to End Like This”, The Atlantic, April 2, 2021, accessed December 2, 2022, https://www.theatlantic.com/ideas/archive/2021/04/nfts-werent-supposed-end-like/618488/

12, 15 Cass Marshall, “NFTs are generating huge paydays for some artists, others feel under siege”, Polygon, March 12, 2021, accessed December 1, 2022, https://www.polygon.com/22327806/nft-artists-online-theft-non-fungible-token

13 Simon Spichak, “Independent Artists Say NFTs Are the Bane of Their Existence”, Futurism, January 24, 2022, accessed December 2, 2022,
https://futurism.com/why-artists-hate-nfts

14 Andy Storey, “9 Reasons Why NFTs Are Bad For Artists”, Postergrind, May 7, 2022, accessed December 2, 2022, https://postergrind.com/9-reasons-why-nfts-are-bad-for-art-ists/#:~:text=Artists%20claim%20that%20their%20art-work,is%20a%20lot%20of%20effort

16 ArtStation Team, “A Statement from ArtStation”, ArtStation, March 8, 2021, accessed December 1, 2022, https://magazine.artstation.com/2021/03/a-statement-from-artstation

17 Simon Fitzpatrick, Rosie Adcock and Sophie Mellor, “February NFT Litigation Roundup: Art Wars, Hermes "MetaBirkins", and more...”, Lexology, February 23, 2022, accessed December 3, 2022, https://www.lexology.com/library/detail.aspx?g=666c41a0-7815-4d21-a0f6-6da07ec1388e

18 Andrew Rossow, “The Hermès Lawsuit May Dictate the Future of NFTs”, nftnow, May 19, 2022, accessed December 4, 2022, https://nftnow.com/guides/how-the-hermes-lawsuit-could-deter-mine-the-future-of-trademark-rights-in-nfts

19 Gabriel Roberts, “NFT Collabs: Why You Should Work with Other Artists”, YouTube video, 3:32. April 5, 2021, https://www.youtube.com/watch?v=AeW3dm8xU4Q

20 Joshua Mapperson, “100-artist NFT collaboration sells out in minutes, increases 7X in price in 24 hours”, Coin Telegraph, March 4, 2021, accessed December 3, 2022, https://cointelegraph.com/news/100-artist-nft-collaboration-sells-out-in-minutes-increases-7x-in-price-in-24-hours

21 Jessica Rizzo, “The Future of NFTs Lies with the Courts”, Wired, April 3, 2022, accessed December 4, 2022, https://www.wired.com/story/nfts-cryptocurrency-law-copyright

22 Thienel Law LLC, “NFTs and Artists: Understanding Intellectual Property Laws in the Metaverse”, Thienel Law LLC web- site, July 1, 2022, accessed December 5, 2022, https://www.thienel-law.com/blog/2022/6/23/nfts-and-artists-un-derstanding-intellectual-property-laws-in-the-metaverse

23 Protos Staff, “Humans and computers to fight NFT plagiarism on OpenSea”, Protos, May 12, 2022, accessed June 16, 2023, https://protos.com/humans-and-computers-to-fight-nft-plagiarism-on-opensea/

24 Jack Morse, “How to report plagiarized NFTs as stolen art”, Mashable, April 13, 2022, accessed June 16, 2023, https://mashable.com/article/how-to-report-plagiarized-nft-stolen-art

25 Arcual Art website, accessed December 6, 2022, https://www.arcual.art 

STAND BY/ME

Yu Zhang

STUDIO Ü Eindhoven, the Netherlands
yuzhang.nl@gmail.com

Abstract

This paper describes initial intention, concept, concerns in design process, production, and technical details of an artwork named “STAND BY/ME”. STAND BY/ME is an interactive installation that uses machine-learning models to generate speeches for Xi Jinping and Donald J. Trump, and uses randomness to build virtual conversations for the political spectrum, while connecting the information flow with the lived reality of the everyday. Visitors are exposed to narratives of both digital communism and digital capitalism that is randomly controlled. Next to the visual flow of randomly generated speeches, the work involves seemingly mundane, yet “super-charged” electrical household items, i.e., power sockets, as the actuality of human-technology confrontation. This work (1) allows visitors to feel like they are on “stand by” and (2) triggers questions about how technologism impacts individuals' views and information consumption, while (3) people face randomly generated political speech as a quiet, mundane confrontation. In sum, visitors are invited to be bystanders of randomized political speech, visualized through streams of text ad made tangible through everyday technology.

Keywords

Information representation, information flow, digitals and physicals, social media, technologism, politicism.

DOI

10.69564/ISEA2023-37-short-Zhang-et-al-STAND-BY-ME

Introduction

With the advent of smart and learning systems, connected technology is no more a simple luminous torch to shine upon our wishes or serve our needs. It senses a multitude of signals from our environment, and shapes our thoughts, behavior and social interactions. We find ourselves at the moment when technologism becomes an almost unavoidable aspect of the everyday. Most often our true realization is overshadowed by technological wonder or even deeply influenced by a form of techno-mysticism. When we place the finger on the screen and swipe it, an “informative” world starts and soon it is filled with bits and glimpses of ever-faster news cycles. As individuals, we consume the abundance of news that produced to capture our attention, move us emotionally and sometimes even take action. Over time, we reach a state of being alert and at the same time wholly incapable of action, especially when consuming news about politics. The initial intention of the STAND BY/ME installation (Figure 1) is to synthesize the experience of “stand by,” in the context of technologism and politicism, where each individual can view cycles of randomized and virtual information from two political “speakers” as a bystander. “Stand by” expresses a human inability to act socially, as an inhibition of the mind, a perversion of mindfulness. It is a deeply tiring state. The deeper intention behind this art scene is to state: when an individual is incapable of action, digital data still changes its meaning and form just depending on its medium of distribution, communication and interface.

Figure 1. Exhibition view from STAND BY/ME at Albert van Abbehuis (Eindhoven, the Netherlands).

“Stand by” and STAND BY/ME

STAND BY/ME focuses on the concept of “stand by” and transforms feelings of confusion, in-betweenness and randomness into an immersive audio-visual experience. This experience is designed to trigger visitors’ reaction to human and technological expressions of “stand by”, and visitors’ feeling when encountering a situation gripped by a general sense of “stand by.”

The physical part of this installation is composed of 30 linked power sockets (Figure 2), which are technologically extended and enhanced with light and sound actuation. Apart from the layer of connected physical sockets, the installation incorporates all public speeches by Chinese president Xi Jinping since 2014¹, ², ³ and thousands of tweets by U.S. former president Donald J. Trump since 2016⁴ until January 2021. A recurrent neural network (RNN) was trained with the speeches and tweet corpus, and we fine-tuned the parameter space to obtain synthetic speeches that express the current acceleration of digital communism and digital capitalism. The two “speakers” are then visually juxtaposed with a simulation of randomized draws of numbers (balls) from the National Lottery.⁵ The lottery serves as the metaphorical random actor in this installation: conceptual and technical randomness controls relations between recurring concepts in each politician’s generated speeches. The content and the processing of approaching information data are displayed as a digital information interface (Figure 3).

This work projects the patterns of information flow from the machine-generated speeches first, and connects with the lived reality of the everyday. The ordinary power sockets, “super-charged” household items, visualize a connected data system that enacts the randomness in the actuality of human-technology confrontation. Throughout the design process, this work touches on four symbolic and logic assumptions in the concept “stand by.”

Two Politicians

As two “speakers,” Mr. Xi and Mr. Trump have different ways of approaching the public, quite oppositely.⁶ In 2020, we have read a lot about “truth” by analyzing and interpreting the information and actions from Mr. Xi and Mr. Trump. The citizens have been (still) confused about information and have perceived them as a strongly regional particularity. STAND BY/ME uses a format of public stances and, at the same time, is filled with random content—formal posing as a facade. The concept of STAND BY/ME was accomplished around March of 2020. At the moment of writing this paper, the new conflicts in the world started to develop —presidential election result for the 2020 US election became a turning point for American foreign policy regarding Beijing, followed by Xi has secured a precedent-breaking third term as the Communist Party’s leader and Donald Trump announced a White House bid for 2024. This work is filled with future projections including the efforts to establish dialogues between different cultures and regions. The result of such efforts seems very “virtual,” either in this work or in the reality.

National Lottery

National lottery symbolizes the daydream of picking easy money, yet participating in a random gamble. The format of National lottery used in this work contains three main contents: jackpot size, six ball numbers and one bonus ball number. STAND BY/ME uses this format but creates new feeds that are machine-generated random numbers.

In STAND BY/ME, the lottery is a metaphor for randomness and offers each visitor only a passive role who are not able to take action in the process—intelligent agents submitting to luck.

Figure 2. 30 linked power sockets that are extended with light and sound output in the day light.

Power Socket

The power socket in its most ordinary form is a distribution hub for electricity. Here, its function is transcended: it is a physical connection hub which is available all the time to connect devices that can send and receive all information packages. Power sockets often embody the most common notion of indicating “stand by,” a glowing LED. The use of power sockets in this work metaphorically refers to the action “to be on charge” and similarly as an active participant in propagating information and visualizes the information flow in the audio-visual patterns. Through multiple power sockets, STAND BY/ME embodies a sense of “something somehow happening,” and visitors are incapable of actions.

Interaction

Interaction in this installation went through different stages: from a means to engage visitors in exploring and discovering the emptiness behind the façade to the final implementation where interaction is deliberately withheld from the visitor. The interaction becomes that visitors only remain passive in the installation to simplify the dynamics of light and sound (reacting only to the data processing of the National Lottery). By observing the data processing instead of being part of it, visitors are drawn into the processes of how information spreads or hits a levee or how light and sound trigger the translation of the digital speech data. As visitors realize that they are bystanders, in the back of their minds, they will feel the pressure and obligation to act, without being empowered. 

Figure 3. Information interface including from left to right Xi Jinping, Donald J. Trump and National Lottery for real-time generated text by a machine-learning model and generated “lucky” randomness.

Technical Details

In this installation, a central single-board computer (SBC) runs a Processing sketch that (1) displays the information interface with virtual speeches and conversations from two politicians and projects the results as a live “dashboard” of speeches, highlights and lottery drawings (Figure 3), and (2) translates into 30 linked power sockets that are extended with light and sound output (Figure 2). Each of the 30 power extension sockets is internally altered, while keeping its original appearance. Inside each socket, WIFI-connected electronics (based on ESP32 boards) allow for decentralized control of each two LED lights and one speaker module. The installation generates its own data stochastically modeled after patterns of information flows from social networks. The patterns are mapped and distributed to all sockets via a message bus running on the SBC. The dynamics of light and sound of each altered socket are driven by each sockets’ internal programming and designed to be responsive to the data in the system.⁷ Let’s dive in a bit deeper. The information display in figure 3 shows three columns. The rightmost column is the data display from National Lottery. It contains four main parts—(1) data source and its introduction, (2) real-time lottery information (jackpot size, six ball numbers and one bonus ball number) in technical randomness, (3) the latest coming six ball numbers, (4) the keywords from speeches by each “speaker” that generated from the latest coming bonus number. In the same figure 3, the two “speakers” are shown in the left and middle columns with each four parts of content: (1) data source introduction, (2) real-time RNN generated speeches, (3) up to six highlighted keywords from speeches selected by the latest draw from the National Lottery, and (4) the statement selected from the speeches with shared keywords selected by the latest draw of the bonus ball. The speeches differ in content and important repeating keywords, which are extracted in a ranked list per speaker. These lists are used to draw keywords from by means of random numbers from the Lottery draws.⁸

Conclusion

STAND BY/ME creates an immersive experience set up as interplay between humans, machine and artificial entitles in a cognitively overwhelming way using text, light and sound. Although STAND BY/ME seems calm in general, it puts visitors in an almost passive position when facing the complexity of the machine-generated progress. At the same time, visitors soon discover the need to process each incoming scenario of an entirely modified and virtual information for both digital communism and digital capitalism under the control of gambling randomness. The installation unpacks the concept “stand by” in a collaboration of data sources, data system, technologies, multiple electrical household items, and the digital interface. The design process of this work dives into the process of refining audiovisual complexity with special attention to the details of information representations, metaphorical or not. This work provides the prompts for different perceptions and enables visitors to have a felt experience of the connection, distraction and confusion when digital data changes its meaning and form at every point, depending on its medium and the current “mood” of randomness.

Acknowledgments

The grant of work was awarded by ARGEkultur in the frame of DIGITAL SPRING FESTIVAL. STAND BY/ME was shown in the group exhibition “POLATECH 1” in the period of 27th of August until 6th of September, 2020 at Albert van Abbehuis, Eindhoven. The work was also invited and exhibited at Athens Digital Arts Festival 2022. This installation was built with much inspiration and support from Mathias Funk from the Industrial Design department at Technology University of Eindhoven.

References

1 China.org.cn, Bilingual Texts, accessed October 30, 2022, http://www.china.org.cn/english/china_key_words/node_7216079.html

2 Mission of the People’s Republic of China to the European Union, Speeches (Resources), accessed October 30, 2022, http://www.chinamission.be/eng/zywj/ZYYJ/

3 Ministry of Foreign Affairs of the People’s Republic of China, Speeches (Policies and Activities), accessed October 30, 2022, https://www.fmprc.gov.cn/mfa_eng/wjdt_665385/zyjh_665391/

4 Donald J. Trump, Tweet (text), accessed December 30, 2020, https://twitter.com/realdonaldtrump

5 THE NATIONAL LOTTERY, Lotto results, accessed October 30, 2022, https://www.nationallottery.co.uk/results/lotto/drawhistory

6 Chris Buckley, “Trump’s and Xi’s Differences Magnify Uncertainties Between U.S. and China (2016),” accessed October 30, 2022, https://www.nytimes.com/2016/12/19/world/asia/china-donald-trump-power.html

7 Mathias Funk, OOCSI, Zenodo, 2019, DOI: https://doi.org/10.5281/zenodp.1321220

8 Yu Zhang, “STAND BY/ME information interface (2020)”, accessed October 30, 2022, https://vimeo.com/395705214

9 Yu Zhang, “STAND BY/ME electrical items (2020)”, accessed October 30, 2022, https://vimeo.com/395734433 

Author Biography

Dr. Yu Zhang has a background in fine arts and design. In her Ph.D. research she investigates the theory and artistic practice of interactive technologies for public, large-scale installations. Over the past years, she has designed and researched interactive systems that respond to everyday phenomena, environmental concerns, child-system interaction, online collaboration platforms, and uncertainty in data visualization. Yu has participated in various international Art Residencies of the last years, and her work has been exhibited at galleries, museums, and festivals world-wide. Besides, Yu’s teaching experience covers a broad range from traditional classrooms and workshops to designled project-based learning activities. Her book “Coding Art,” co-authored with Mathias Funk, was published by Apress/Springer in 2021. https://yuzhang.nl/

Long Papers

Ag Tanya Ravn
Art’s Intratemporal Relation to the Future

Amato Etienne Armand, Pereny Etienne
For a more symbiotic co-individuation with our technological avatars: how to go, with the Sciences and the Arts, beyond hybridizations? 

Ammar-Khodja Brice
Symphony of the Stones: A Research-Creation Exploration on the Animation of Heavy Metal Residues in Contaminated Urban Landscapes

Badani Pat
The “Bichi” Project Symbiotic Food Networks & the Alchemist Kitchen

Bae Joonhyung
Thief of Truth: VR comics about the relationship between AI and humans

Batsis Dimitris, Grigoriadou Magdalini, El Raheb Katerina, Politis Archontis
Disembodiment in VR: Immersed in 3D Audio Experiences

Breuleux Yan, Thibault Alain, Lapierre Remi (invité)
The Enigma A/V performance & the concept of Agnostic Media Environment (AME)

Brueckner Sophia
White Cube / Black Box: Investigating Bias in Museums and Algorithms

Brunner Christoph, Fritsch Jonas
Human Energetics in an Era of Post-Humanism

Cîrcu Sorina-Silvia, Chen Chu-Yin
How Digital Anthropomorphism Enhances Creativity in Human-to-Robot Dance Interactivity

Contreras Paul Rosero
The Revolution Will Launch in the Garden: Politics of representation and vegetal intellig(senti)ence

Correia Nuno, Souza Debora, Nêves Inês, Lobato Jaime
Bio Elektron - A Multisensory Approach to Augmenting Dance, Combining: Biosignals, Drawing, Sound and Electrical Feedback

Costa Pedro, Ribas Luísa, Carvalhais Miguel
“I’m a virtual assistant so I don’t have pronouns the way people do, thanks for asking”: gender neutrality, diversification and fluidity in AI

Cunin Dominique, Durand Emmanuel, Seta Michał
Immersive environments, video tracking and collective interactivity on smartphone: a generic “dispositif”

Družetić-Vogel Ivana, Fuchte Alina, Bauernfeind Marina
ARt chat - A Museum App combining AR, Art and Communication

Duarte Regino Juan Carlos
A hybrid listening to atmospheric processes

Foo Louise Lind, Fritsch Jonas
Eleven Movements of the Cryoscape – Ecological Explorations in Sonification for Affectively Engaging with Climate Change

Gatz Sebastian
Cosmo-Techno-Poiesis: Architecture of Environmental Control

Gaylor Brett, Hennessy Kate
Welcome to the Metaverse: Hacking Affect in Immersive Documentary to Increase Critical Big Data Literacy

Gemeinboeck Petra, Saunders Rob
Hybrid Entanglements: a posthuman dramaturgy for human-robot relationships

Georgakopoulou Nefeli, Kosti Makrina Viola, Diplaris Sotiris, Benayoun Maurice, Camurri Antonio, de Gelder Beatrice, Wald-Fuhrmann Melanie, Koulali Panagiota, Valsamidou Kalliopi, Bronner Kai, Wanner Leo, Gaki Froso, Vrochidis Stefanos, Kompatsiaris Ioannis
ReSilence: Retune the Soundscape of future cities through art and science collaboration

Georgakopoulou Nefeli, Zamplaras Dionysios, Kourkoulakou Sofia, Chen Chu-Yin
Towards a Sympoietic Relation with Materials in Interactive Artworks

Gnecchi Ruscone Gerolamo
Thinking with Tides: Engaging with Embodied Technical Processes within the Tidal Ranges of the Thames Estuary

Gould Charlotte
Chthulucene Hekateris

Guljajeva Varvara, Canet Sola Mar, Joseph Clarke Isaac
Artistic Strategies to Guide Neural Networks

Haute Lucile
Kombucha as a Guide. Serendipitous Journey through Taste, Feminism, Free and Open Source Culture, and Ritual

Hofmann Yannick
Towards an Intelligent IoT System for the Data-Informed Museum of the Future

Huang Yu-Hsiung, Hsu Su-Chu
The Creative Design and Social Service Practice of zen_Farm

Irrgang Daniel
Thought Exhibition. On critical zones, cosmograms, and the impossible outside

Jones Maurice, Wester Meaghan, Blottiere Marek
Curation as Research-Creation: Speculating on the Future of Art and Technology Festivals

Kobryn Olga, Couteau Matthieu, Sagot-Duvauroux Rémi, Balcon-Fourmaux Sophie, Garnier François, Ronfard Rémi, Soulez Guillaume
Feeling One’s Way: In Search of a Symbiotic Vocabulary of the Virtual

Lautenschlaeger Graziele
Post-Human Motherhoods: Reflections on mother-offspring bonding as symbiotic individuation in Contemporary Art

Lengelé Christophe, Gauthier Philippe-Aubert
Live 4 Life: A dream for a free and open spatial performance tool towards symbiosis or death?

Mariategui Jose-Carlos
A Latin America Network for Art and Cybernetics: The Centre for Art and Communication (CAyC)

McFarlane David
Towards a Methodology for Co-creating Artistic Acoustic Ecologies with the Great Lakes

Meshi Avital, Chiaravalloti Treyden
Structures of Emotion: Speculating on an AI-Human Symbiosis

Nuñez del Prado Paola Torres
The w(e)aves of complexity: Relational ontologies within the Symbios Art Exhibition

Pataquiva-Mateus Alis
E-cellulose: symbiotic cultivation for the production of smart textiles in a framework of sustainable fashion and electronic art

Polydorou Doros
The Fall of R’Thea: Digital Fiction

Purg Peter, Pranjić Kristina
To Know a Tree: Symbiotic Mutualism and Artistic Exploration against Anthropocentric Science

Rashtian Hamed, Aceves-Sepulveda Gabriela
Same Old Story: Agential Realism in the Study of Colonial Histories

Ribeiro Clarissa
Data-Phantoms: Impossible Nests (Memories Post Extinction)

Ridgway Renée
Black Box versus Black Bloc

Roussel Vivien, Haute Lucile, Teyssier Marc, Dhulster Pascal
Processes, Fabrication and Design with Kombucha Bacterial Cellulose: Mapping Practices

Sagot-Duvauroux Rémi, Quaetaert Nils, Garnier François, Ronfard Rémi
The Hitchcock Experience - a Spatial Montage project

Salter Chris, Thomasson Timothy, Uro Pierrick
Animate: A Theatrical Exploration of Climate Transformation through the Medium of Extended Reality (XR)

Schnugg Claudia, Brill Daniela, Stary Christian
Towards Sensemaking in the Meshwork of Technology, Ecology and Society: Symbiosis of Aesthetics, Performance and Digitalization

Scurto Hugo, Chemla-Romeu-Santos Axel
Deeply Listening Through/Out the Deepscape

Teles de Castro E Costa Isadora, Chen Chu-Yin, Hong Hui-Ting
A Performance Co-Created with an Autonomous Virtual System: A Symbiotic Approach

Thorogood Miles, McCulough Kirsteen, Dulic Aleksandra
Light Up Kelowna: Coordination and Development of Networked Community-based Media Art Urban Screen Infrastructure

Thurow Susanne, Del Favero Dennis, Ostwald Michael J., Grehan Helena
Augmenting Creative Symbiosis Using a Cyber-Physical Aesthetic

Timurgalieva Olga, Direito Eva, Moreira Patrícia R.
(Re)imagining human-yeasts relations via art-science collaboration

Weber Rasa
A Sympoietic Ocean. Design research with/in the marine holobiont

Wu Ziwei, Fei Danlu, Ma Xinyu, Fan Mingming, Zhang Kang
NER: Physical-Virtual Multimodal Generative NFT with a Rarity Model

Xu Dan, Lamers Maarten, van der Heide Edwin
Towards a Relational Model of Co-located Interaction in Interactive Art

Yat Siu Shum Pierre, Klein Tobias
VR CALLIGRAPHY Transposing Chinese calligraphy as choreographed movements into whole-body performances in VR

Zhang Weidi, Luo Rodger
LAVIN: An AI-Navigated Art Experience in Virtual Reality

Zhang Weidi, Su Shaoyu
Volume of Voids: Artistic Visualizations of the Disequilibrium

Zolotova Maria
The Worlds of Entanglements: Reflection on Posthumanist Ontologies in Art&Science Projects

Art’s Intratemporal Relation to the Future

Tanya Ravn Ag

Centre for Art as Forum, Department of Arts and Cultural Studies, University of Copenhagen Copenhagen, Denmark
tanyaravnag@gmail.com

Abstract

The world of art has always been occupied with art’s temporal relations to the future. In the current artistic landscape, we see a wealth of exhibition themes and titles concerned with ‘the future’ in responses to the dominant narratives of a contemporary technological world driven by algorithmic systems and prediction models. This momentous future-orientation is my cue to rethink art’s relation to the future, by zooming in on its temporal modes of existence. With a take-off in the notion of art as “time-based,” as conceptually based in the time, duration, and/or the function of a medium and the experience it mediates, I propose a different intratemporal mode of existence for art. This concerns how art co-exists with, evolves through, and co-produces temporal relations in between humans and technology. This proposal of an intratemporal perspective on art might contribute to further investigations into art epistemologies in which art becomes a part of larger narratives in which human beings and communities co-evolve—and have always co-evolved—with technics. It might offer inroads to study art on its new paths of exploration in collaboration with science and technology and when art is occupied with the very making of the future through participation in innovation projects.

Keywords

Time-based art, intratemporality, future, innovation culture. 

DOI

10.69564/ISEA2023-38-full-Ravn-Ag-Arts-Intratemporal-Relation

Introduction

My temporal investigation in this paper takes off in the catalog Alchemists of the Future published for the Ars Electronica Future Lab’s 25 years anniversary in 2021. In the concluding chapter, “Perspectives,” we can read about how the Future Lab’s activities of visionary prototypes and innovative collaborations between art and science in 1996 were initiated to contribute with future narratives to address urgently needed paradigm changes The visions expressed in the Future Lab catalog, about art’s involvement in our greater societal narratives of technological change, echo the bringing together of art, science, and technology with the conception of the New York-based organization EAT—Experiments in Art and Technology—in 1967, which was founded by engineers Billy Klüver and Fred Waldhauer and artists Robert Rauschenberg and Robert Whitman. The visions of creative and experimental research processes between artists and engineers for bringing artists closer to the materials of technology and more in touch with the forces shaping contemporary society, the use of projection and new communications technology to achieve this, which entailed the exploration of new roles for art in the changing ‘technological environment’ of the late 1960s/early 1970s, and the migration of these practices from an art to non-art contexts,² altogether paved an explorational path for art’s evolvement through changing relations to the future.

Today, as we find when searching through numerous recent titles of exhibitions, knowledge forums, and events of art, (1) the future orientation has saturated the broader field of art. This future-orientation should interest us as more than a thematic trend.

The legacy of E.A.T., and the catalyzation of ideas of collaboration between art, science, and technology through the Future Lab, among many more initiatives, informs a fast-growing discourse in art whereby the art is treated, funded, and appropriated as a catalyst for change. For example, when art migrates into cultures and contexts of technological innovation; when artists are invited into residencies, technology and science labs of corporate technology companies; or, where art becomes a protagonist in major creative funding schemes and innovation programs and is granted support as a catalyst for, for example, industrial innovation, urban development, or human rights. For one example amongst many, the call “Art-driven use experiments and design” under the Horizon Europe Framework, which explicitly allocates a strategic role for art in technological innovation culture. These movements in art, whereby art has gained new roles in strategic projects of future-oriented and future-shaping technological innovation, require new approaches to grasp and assess art’s modes of existence, which I propose that we understand through its relations to the future as an epistemological and methodological compass.

My inquiry is guided by the following line of questions: Why, in the context of our contemporary technological environment, is the orientation towards the future in art so momentous? What characterizes art’s relation to the future in our current technological environment? If the occupation with the future in art concerns a temporal orientation towards how everyday lives, cultures and societies will or might evolve with technology, then how does art participate in the temporal processes that will bring us there? Why does art’s relation to the future matter to the roles that art pursues and gains within technological innovation—as a locus for human symbiotic imagination (about the future) and our technocultural making of it?

My overall suggestion is that we need to grant more attention to art’s temporal modes of existence as simultaneously a matter of object functionality and environment, human and intersubjective experience, technocultural context, and cultural evolution. With a point of departure in the conception of art as “time-based,” I engage an alternative, intratemporal mode of existence for art, with which I understand art to be a part of a larger temporal complex: art is not based in time but existing through intratemporal infrastructures and relations with its contemporary technological environment, which in our current age is characterized by and evolving through data-driven algorithmic processes. I unfold the intratemporal perspective on art through three temporal dimensions—object temporality, worldly temporality, and deep temporality—that relate art to epistemologies on how human experience changes with technological culture. 

Art and temporality
- beyond “time-based” media

When art is described and categorized in a temporal perspective, it is commonly referred to as “time-based.” My claim in what follows is, however, that this temporal conception of art and the epistemological framework that it engages is insufficient to grasp art’s behavioural modes of existence and interdisciplinary evolvement today.

The conception of art as “time-based” is broadly used by museums with reference to artworks that rely on technology, such as video, film, audio, slide, installation artworks, as well as artworks that function only for the duration of their time on display, like computer-based and mechanical works of art. The conception of time-based art ties time to the expressive and functional qualities of the medium. Time-based art is conceptually rooted in “time-based media,” a term coined by museum conservators for durational works of art that unfold over a period of time. It is used widely by art institutions to describe art that is ‘dependent on technology and has a durational dimension’ (Tate), that ‘unfold to the viewer over time’ (Guggenheim), and that are ‘dependent on time, duration, or function’ (National Gallery of Australia). Time-based media has a run-time enabled by the form or medium that limits and contains the experience. The medium enables the inscription of the spectator in different experiences of time. By looking at art as “time- based,” we focus on how the art facilitates meetings between different durations. For example, between the durations of human experience and the durations of a rationalized society. This understanding is fueled by a broad theoretical interest in temporary multiplicity in the writings of among others Henri Bergson, Alfred North Whitehead, Gilles Deleuze, Michel Foucault, and Michel de Certeau.

Christine Ross’ examination on art and temporality in The Past Is The Present, It’s The Future Too exemplifies this understanding of art as facilitating meetings between durations. She describes various ‘durational’ temporal strategies in art as aesthetic counter reactions to the forwardness of the modern era. These are temporal strategies of, for example, endlessness, ephemerality, repetition, real-time, contingency, randomness, slowmotion, condensation, acceleration, extension, abbreviation, speeding up, hesitation, disruption, fissuration, extendibility, and interminability—all temporal strategies for suspending linear conceptions of time that confirm one universal temporal logic.³ Such temporal strategies of suspension evoke a tendency emerging in contemporary art of the 1960s, which is described by Pamela M. Lee in terms of “chronophobia”—a sense of unease or maybe even rebellion in art against temporal societal narratives that dominated during the middle of the 20th century and which translated into a critical consciousness in artistic expressions of performance, conceptual art, sound art, installation practices and land art.⁴ The art of the 1960s that both  Ross and Lee write about reacted against a relation between temporality and historicity, namely one dominant narrative about technological progress that characterizes Western modernity, which celebrated technological transformation, automatization, acceleration and standardization. The dominant narrative reflects a universal conception of time as linear, structured around past, present and future, and organized based on classical physics’ ideas about absolute mathematical time and ground principles of natural science about relativity. In this narrative, time and space are compressed by technological and mechanical processes—what David Harvey has named “time-space compression” which refers to how global communications technologies and information economy compress barriers and distances, which is a function of late capitalism.⁵ This global, temporal narrative is structured around a singular temporal scale characterized by rules of regulation, discipline, speed, effectivity, immediacy and progression—as Jonathan Crary describes in the book 24/7.⁶

The time-based conception rests on a philosophical notion rooted in the ideas of Plato and a substantivist and absolutist conception of time, treating time as an empty container with rules and logics, that is, temporal rules and logics that are ready for art to critically engage with. We recognize this conception of contained time when art is accounted for as an aesthetic, conceptual, critical manifestation capable of presenting and representing alternative temporal modes to those driving capitalism by which to inscribe people into different experiences of time. This temporal containment, however, delimits art’s relation to the future as representational or reflective material that eventually becomes confirmative of the future narrative that it speaks to.

I would like to propose a different temporal condition for art. Because, although art is situated in a specific temporal slot and has a particular duration, and although it might depend on the phone or a mobile device that enables specific temporal qualities of the experience, the work is not delimited to a temporal capsule. It is not delimited to exist “based in time,” as if in a form of a temporal container that we can individually step into for a direct experience with represented image or concept. The time-based conception relies on a direct experience between the human and the artwork. This does not correspond to the ways in which we experience and exist with temporalities through the ways in which most of us engage with technology today. Nor does the time-based conception account for the human-perceptual and technocultural effects of these temporal experiences. Time is articulated in technical systems but only in connection with human engagement with technics, as we learned from the writings of Gilbert Simondon.⁷

In the following, I will propose the contours of an alternative temporal conception of art to that of time-based; one that considers an intratemporal mode of art’s existence. This involves the conception of time as something that the art is relationally entangled with, through which it evolves, and which the art contributes to generating.

Art’s Intratemporal Mode of Existence

Art’s intratemporal relation to the future today bears traces of future-orientations in art of the past. Along with the ongoing critical discourse in art continuing the critical occupation with the forwardness of the modern era that Ross locates in art of the 1960s, we recognize a trajectory from the futurist art movement of the early twentieth century that sought to capture in art the dynamism, energy and movement of the modern world and modern life. Preceding future-oriented movements unfolded in effect and response to a technological environment before the internet, social media networks, and data-driven distributions and accumulation of registrations of our behavior. Today, however, the technological environment conditions a different intersubjective condition than that of the 1960s.

An intratemporal perspective on art might immediately evoke Martin Heidegger’s understanding of phenomena and objects in terms of temporal relations rather than substance, in Being and Time. (2) ⁸ Heidegger contrasts intratemporality with authentic temporality, seeing intratemporality as an existential structure to Dasein determined by calculation and measuring instruments. My use of the term “intratemporality” takes a different reference, in Yuk Hui’s connection of intersubjectivity (subject-context relation) and interobjectivity (object-milieu-relation) in the term. In Hui’s theory, intratemporality is a dimension of changing f temporal relations between objects and neurosensory evolution that happens through our networked and synchronous co-evolvement. ⁹ In my adaptation of this understanding of intratemporality to this inquiry on art, I consider art as temporally related with temporalities of technologies and technological cultures well beyond the medium, the art experience, and the discourse of the art environment.

In the following, I will draw some perspectives on how intersubjectivity, as a matter of temporal relations, is conditioned by the temporalities of digital objects and worldly connectivity as well as by deep temporalities of our cognitive and cultural heritage from technocultural pasts. These temporal dimensions combine in art’s intratemporal mode of existence and tie art to the concept of the future in new ways.

Object temporality: In Hui’s account of the conditions of digital objects and extension on Heidegger’s notion of intratemporality in this regard, he notes how data-driven temporal processes mediate between intersubjective and inter-objective relations and influence temporal experiences in our everyday lives. The ways in which things are quickly shared, behavior and ideas are quickly adopted, and experiences are synchronized, effects an organization of consciousness about how things are temporally related to each other. This reorganization of consciousness with object temporality is what N. Katherine Hayles addresses in How We Think: Digital Media and Contemporary Technogenesis.¹⁰ Hayles writes about how the media interface (e.g., the screen), for example, might seem like it correlates directly to human modes of sensory experience and cognitive processing while it only indirectly correlates to these modes of experience, since it involves technical operations to which we lack a direct access. This is because different time scales of human cognition and machine cognition intermesh. As computational processes occur at time frames that are below the threshold constitutive of human perceptual experience and introduce levels of operationality that impact our experience but do not have any perceptual correlate, we are not conscious of their consequences to our actions. The mutual interference between temporalities of machinic systems and human-temporal functioning of consciousness means that our creation of (abstractions, forms, content, systems, meanings) is not rooted in a direct human relation with what we create but depends on unconscious processes. This unconscious aspect of perception with object temporality connects us to worldly temporality.

Worldly temporality: Global effects—from economical dynamics at a macro level to machinic operations at a micro level—reach us through object temporality. The unconscious cognitive processes at work in our engagement with object temporality not only concern an engagement with digital objects (in and beyond art). They condition intersubjective experience as object temporality relates to the temporalities of networked media technologies, which is entangled with human experience. As Mark Hansen notes in Feedforward: On The Future Of Twenty-First-Century Media, with digital media, the external world has become a part of individual experience, while experience has become externalized and environmentalized in contemporary forms of mediation.¹¹ While we physically exist in the phenomenal world, our thinking, behavior, and the effects of our actions are also of a worldly context, conditioned by mediating factors of different temporalities affected by environmental and global connectivity, and experienced across timespaces.

What this meshing of human and machine temporality results in are operational processes that function as a kind of technical “memory,” which becomes a cultural support structure, and which affects how perception and intersubjective imagination are at work. This technical support structure evolves from a long process of evolutionary adaptation of technical tendencies and their logics, whereby art is intratemporally entangled with a sense of deep temporality.

Deep temporality: Art’s experiences can amplify and resonate through volumes of people, connect us to our ancient past and memories of cultural rhythms, rituals, and practices, and throw us into uncertain futures. An intratemporal dimension of deep temporality links art to cultural patterns, which have shaped the ways in which we use and develop tools and technology since our human origin and the perceptual habits we have evolved and enact when experiencing something, including art. Collective memories, cultural programs and imaginations have been transmitted via habit and repetition through communities and historical epochs. With reference to technoanthropological ideas from the philosophical writings on human technogenesis of Bernard Stiegler¹² based on the anthropology of André Leroi-Gourhan that roots human co-evolvement with technics in the origin of human civilizations,¹³ we can consider how art has a part in the shaping of cultural memory, symbols and rituals that we have adapted from ancient pasts. These have formed through civilizations, cultures and generations to manifest in the cultural codes, meanings and logics we navigate by today. These cultural adaptations of technocultural aesthetics and behavior inform how human cognition meets machinic operations today.

With this intratemporal dimension of deep temporality, I wish to emphasize a technicity in the art as having a function with regards to our cultural evolution with technics. Art, as a human aesthetic expression, has evolved with evolutional adaptations that carry the past into the present—and entwine with the future—through technological tools and the cultural and cognitive memory structures they engage. From studies on ancient human pasts, we know that art, and the technicity with which it operates, in the ancient shapes of rituals, ornamentation, craftwork traces of human gatherings, and more, has taken on various roles as a cultural transmitter and as an aesthetic mechanism of societal organization. For example, as a kind of mediator of societal imaginaries; a vehicle for intelligence, memory, language, forms of expression and pattern recognition to travel through generations; as both depiction and facilitation of rituals (practical, cultural and spiritual); as a connector of human beings to their past and origin and a basis for collective consciousness and emotional intelligence; as a connector of humans to the materials and environments of our world and its ecosystems; as embodying conceptions and philosophies of science as a foundation for furthering civilizations, among many others. These are observations from my research on various intersubjective functions art has had in ancient societies.¹⁴

Intratemporality concerns temporalities that are within us, among us, beyond us and preceding us—and which entangle in our tenement towards the future. The intratemporal mode of art’s existence therefore cues a re-examination of art’s relation to the future. 

Art’s intratemporal relation to the future

The current future orientation in art is not either confirming or resisting a utopian desire. What should interest us are also not the future destinations that art offers or is used to test, project, or speculate upon, which are ideas that inform scenario-based design and conceptual attention to "possible futures." It is also not the critical comment on future-driven regimes. Future-oriented art of today does more than make room for reimagining the future.

My proposal here is that art’s intratemporal relation to the future concerns how art and its experiences are entangled with temporalities that relate us to our everyday engagement with (digital) technologies, with the flows and dynamics of worldly (data) processes, and which engage cultural adaptations and intersubjective evolvement through intuitions and perceptions that precede our experience today. This perspective ties art’s relation to the future to contexts of technological cultures beyond that of Western rationalization. The intratemporal perspective on art concerns what kinds of temporalities the art engages and connects in our bodies, objects and surroundings, and in which ways (by the use of which techniques and aesthetic means). It concerns how art intervenes in our experience of those temporalities. This perspective writes art into a larger narrative in which human experience is changing with technology and in which art has always played a role in the ways in which human beings have co-evolved with technics.

This reconception of time-based art can help us to grasp the new routes and roles art pursues through temporal engagements with technological innovation culture. When art collaborates with science and technology in the domain of innovation, it not only envisions, problematizes, or proposes but also co-produces our futures. This involves a change in perspective, from how art represents and responds to the future, to how art has a constitutive relationship to the future. This is because art engages with human intuitions, desires, and aspirations from where our futures emerge. Art becomes a part of larger intratemporal processes of human co-existence and co-evolvement with technology. This calls for further examination of futurity in art, how the art’s techniques and experience is temporally entangled with future-driven systems and processes of human co-evolvement with technics.


(1) Some of the many recent future-oriented exhibition contexts. I’ve come across in my research, which exhaust and understate my argument that the attention to and conceptualization of the future in art is momentous: Possibles (ISEA2022); Futures Implied (Media Architecture Biennale 2020); Writing the History of the Future (ZKM – Center for Art and Media Karlsruhe 2019); Futures (Smithsonian 2022); Future and the Arts: AI, Robotics, Cities, Life - How Humanity Will Live Tomorrow (Mori Art Museum 2019); The Future Starts Here (V&A South Kensington 2018); Possible Spaces (Danish Architecture Center 2018); Future Shock (180 Studios 2022); WHO Futures Art Exhibition: Envisioning the Future of Health in 2050 (World Health Organization 2022); Future World (Art Science Museum Marina Sands 2022); Future U (RMIT Gallery 2021); Hope for the future & meaning of life (Kawaguchi Art Museum 2021); Sampling The Future (National Gallery of Victoria 2022); Edible Futures (The Dutch Institute of Food & Design 2022); Future Food Today (Space10 Gallery 2022); The Future We Create (Art Works for Change 2022); Remembering the Future: 100 Years of Inspiring Art (Heard Museum 2022); Future Perfect (worldwide 20192022); Future Retrieval: Close Parallel (Cincinnatti Art Museum 2021); TECH/KNOW/FUTUREU/ From Slang to Structure (Montclair State University 2021); Past Present Futures: Notions of Time in Twentieth-Century Art (Blanton Museum of Art 2001); Future Is Today (Al-Tiba9 Global 2020); Decriminalised Futures (Institute of Contemporary Arts 2022); Futureritual (Institute of Contemporary Arts 2022); The Future of Now:; Contemporary Art in Our Unstable World (Emmanuel Art Gallery 2022); Designs for Different Futures (Philadelphia Museum of Art 2020); The Future States (Latvian National Museum of Art 2018); Remember the Future Orleans House Gallery 2021); Designs for Different Futures (Walker Art Museum 2021).

(2) Heidegger’s attention to intratemporality concerns an existential structure of Dasein that is inauthentic and measured by technological instruments and by calculation. In Heidegger’s optics, intratemporality denotes an inescapable horizon for Western history of being.

References

1 Horst Hörtner, Roland Haring, Hideaki Ogawa, Andreas Hirsch, Alchemists of the Future: Ars Electronica Futurelab: The First 25 Years and Beyond, Hatje Cantz, 2022.

2 Christophe Leclercq, “The Legacy of Experiments in Art and Technology (E.A.T.): An Environmental Aesthetics,” paper presentation at ISEA2011: 17th International Symposium on Electronic Art, 2011.

3 Christine Ross, The Past is the Present; it's the Future Too: The Temporal Turn in Contemporary Art, New York, Continuum, 2012.

4 Pamela M. Lee, Chronophobia: On Time in the Art of the 1960s, Cambridge, The MIT Press, 2006.

5 David Harvey, The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change, Hoboken, Wiley- Blackwell, 1991.

6 Jonathan Crary, 24/7, London and New York, Verso, 2014.

7 Gilbert Simondon, “Culture and technics, 1965," Radical Philosophy 189, Jan/Feb 2015, 17–23.

8 Martin Heidegger, Being and Time, New York: State University of New York Press, 2010 [1927].

9 Yuk Hui, On the Existence of Digital Objects, Minneapolos, University of Minnesota Press, 2015.

10 N. Katherine Hayles in How We Think: Digital Media and Contemporary Technogenesis.

11 Mark. B. N. Hansen, Feed Forward: On The Future Of Twenty-First-Century Media, Chicago, University of Chicago Press, 2015.

12 Bernard Stiegler, Technics and Time, 1: The Fault of Epimetheus, Stanford, Stanford University Press, 1998.

13 André Leroi-Gourhan, Gesture and Speech. Translated by Anna Bostock, Berger. Cambridge, Mass., MIT Press, 1993.

13 André Leroi-Gourhan, Evolution et techniques I: L'homme et la matiere (Man and Matter), Paris, Albin Michel, 1943.

14 Tanya Ravn Ag, “Media Art in the Hybrid City – Why?” keynote at Art in the Smart City symposium, hosted by Art Republic at Stavanger Art Museum, Norway, November 2018.

Bibliography

Jonathan Crary, 24/7, London, New York: Verso, 2014. Hansen, Mark. B. N. Feed Forward: On The Future Of TwentyFirst-Century Media, Chicago, University of Chicago Press, 2015.

N. Katherine Hayles, How We Think: Digital Media and Contemporary Technogenesis, Chicago, University of Chicago Press, 2012.

Martin Heidegger, Being and Time, New York, State University of New York Press, 2010 [1927].

Horst Hörtner, Roland Haring, Hideaki Ogawa, Andreas Hirsch, Alchemists of the Future: Ars Electronica Futurelab: The First 25 Years and Beyond, Hatje Cantz, 2022.

David Harvey, The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change, Hoboken, Wiley-Blackwell, 1991.

Yuk Hui, On the Existence of Digital Objects. Minneapolos, University of Minnesota Press, 2015.

Christophe Leclercq, “The Legacy of Experiments in Art and Technology (E.A.T.): An Environmental Aesthetics,” Paper presentation at ISEA2011: 17th International Symposium on Electronic Art, 2011.

Pamela M. Lee, Chronophobia: On Time in the Art of the 1960s, Cambridge, The MIT Press, 2006.

André Leroi-Gourhan, Gesture and Speech, Translated by Anna Bostock Berger, Cambridge, Mass., MIT Press, 1993.

Christine Ross, The Past is the Present; it's the Future Too: The Temporal Turn in Contemporary Art, New York, Continuum, 2012.

Bernard Stiegler, Technics and Time, 1: The Fault of Epimetheus, Stanford, Stanford University Press, 1998.

Author Biography

Tanya Ravn Ag is a curator and scholar on art, technology and digital culture, focused on how art changes with contemporary technogenesis and algorithmic culture. She is the editor of Digital

Dynamics in Nordic Contemporary Art (Intellect, 2019) and co-editor of What Urban Media Art Can Do – Why, When, Where, and How? (av edition, 2016) based on which she co-founded the globally networked Urban Media Art Academy in 2017. Her curatorial engagements with urban, media-based art include the Screen City Biennial 2017 in Stavanger, the SP Urban Digital Festival in São Paulo in 2013 and 2014, and Nordic Outbreak presented in New York City and across the Nordic region by the Streaming Museum in 2013-2014. She has served on the ISEA Board since June 2023 after chairing the IIAC (ISEA International Advisory Committee) since 2020.

Acknowledgements

The research informing this paper has been supported by the Carlsberg Foundation’s Reintegration Grant for the project Art of our Times: Deep Temporalities of Art and Algorithmic Culture at the Department of Arts and Cultural Studies, University of Copenhagen. 

For a more symbiotic co-individuation with our technological avatars: how to go, with the Sciences and the Arts, beyond hybridizations?

Etienne Armand Amato & Etienne Péreny

Laboratoire DICEN-IDF, Université Gustave Eiffel
& Université Paris 8 Paris, France
etienne-armand.amato@univ-eiffel.fr ; pereny@gmail.com

Abstract

With interactive computing, the metaphorical use of biological notions of hybridization and symbiosis has become widespread. They refer to the possibilities of mixing as well as to the conditions of emergence of relationships ranging from mutual benefit to instrumentalization between technologies and humans. In order to better understand the relevance of such analogies with the living, this article draws on scientific and artistic research concerning the interactive avatar. These seem particularly instructive on our relationship to technology because this virtual being hybridises the living and the artificial, while constituting the key and the condition of access to digital spaces, to co-evolve with other users, themselves avatarised, or other autonomised agents. The article distinguishes between "cyber" avatars populating persistent universes, video games, virtual realities, i.e., cybermedia environments of simulation and interaction. The "hyper" avatars are those of the Web, online services, 2.0 platforms or socio-numerical networks that are juxtaposed in networked informational and documentary hypermedia. From then on, the challenge is to reinvest the sense and responsibility of their potential for augmentation or simulation, in particular by promoting cooperative interactions through science, art and technology, which are themselves by their very nature synergistic with each other.

Keywords

Avatar, Cybernetics, Symbiosis, Video games, Hybridization, Hypermedia, Cooperation, Responsibility, Arts, Sciences.

DOI

10.69564/ISEA2023-39-full-Amato-et-al-technological-avatars

Introduction: symbiotic design at the origin of interactive computing

Since the great convergence of telecommunications, computer science and miniaturized electronics, the expansion of networked digital environments has been accelerating, which never ceases to raise questions, so much so that it guides the civilizational changes underway. Digital art has been able to contribute to these developments from the outset by appropriating the different waves of inventions, software and tools: network art, code art, interactivity, Web art and Game Art, etc. However, due to a lack of recognition, these influences have remained underground and unofficial, as industrial and commercial creations dominate. After decades of intellectual and artistic, academic and cultural engagement, initiatives such as ISEA or numerous Arts & Sciences projects, have made common the idea that artists contribute to create an alternative technological culture to those stimulated by consumption, market and entertainment, in order to produce new practices and regenerative imaginaries. By choosing the theme of "Symbiosis," ISEA 2023¹ sends a strong signal to the communities in charge of digital cultures and electronic arts, a kind of encouragement to favour a transformation of our relationship with the digital, to move towards more responsibility and reciprocal benefits, while comfortable consensuses and evidences are cracking. However, the matter is not so simple. Admittedly, the symbiotic approach has the advantage of offering a qualitative leap forward and a step back, integrating the latest scientific knowledge on cooperative interdependencies, at all scales of life and at levels previously invisible, and just which have been revealed by technical images.² However, this notion does not have the same cultural and political effects depending on whether we take it from an apparently neutral angle; with the Anglo-Saxon meaning, which considers as symbiotic all forms of interactions between species, including the most aggressive and predatory, parasitic and deadly ones; or whether we take it in its French and European meaning, which favours synergistic, mutualist or commensal symbioses, offering mutual or at least unilateral benefits.

But in a facetious and creative way, let us open our remarks with an enigma offered to the reader of this article, before he or she dives into the heart of digital systems and environments by following the path of co-evolution made possible by avatars.

From whom does this excerpt come? "The potential of the human-machine symbiosis is easily visible in the arts, where computing technologies have enabled the creation of previously unrealizable forms of expression. Computing technology has empowered a new legion of artists working in mediums such as immersive and augmented reality games, animated feature films, and music composition and performance (...) In these areas, we are beginning to see humans and machines as complete partners in artistic creation."³ It is striking that an eminent researcher in computer science, having joined the most demanding military programs of the DARPA in terms of success, recognizes artistic activity as one of the best contexts for the emergence of new partnerships for the benefit of an original creation. To agree with this, it has been widely shown that artists, through their close contact with techniques and their potentialities, in their exploration of the logic and capacities implemented in their tools, which have become software with computers, constitute an avant-garde that clears out the possible and opens up new appropriations. If the works in art history have shown this, it is up to thinkers more in tune with the arts and technologies of the image, such as the Frenchman Edmond Couchot, to show it in the course of his numerous works.

But beyond the man-computer symbiosis and the place of the Arts in its advances, let's go back for a moment to the origin and the evolutions of this new paradigm of microbiological symbiosis that appeared following the work of Lynn Margulis⁴ concerning eukaryotes in the 1960s, co-author of the "Gaia Hypothesis" with James Lovelock.⁵, ⁶ This research eventually produced major scientific advances concerning plants, forest and soil, but also the intestinal microbiota in humans, emphasizing the symbiotic interdependence of living things, both inside and out their bodies.

For our purposes and references concerning the coevolution of man and his technique, we will retain the teachings of Leroi-Gourhan⁷ and Gilbert Simondon⁸. The former analyzed hominization through "gesture and speech," techniques that preform both our morphology and our cognition, the latter insists on the "individuation process" and its "associated environment," which organize the individual and collective mode of existence of man, as well as that of technical objects.

But the difficulty remains in finely articulating the human and the artificial, in respecting proven frameworks of thought while going beyond them, updating them, making their complexity a little more "simplex," following the teachings of Alain Berthoz⁹ supported by the phenomenology of Merleau-Ponty¹⁰ and the enaction of Varela.¹¹ By proposing an emerging concept, that of the symbiotic co-individuation of man and his avatar, that is to say an individuation—not to be confused with individualization—which is made joint by their common process of co-evolution, it is indeed a question of situating its context and exposing the premises of a vast site of studies and creations which remains to be undertaken through an approach seeking to create a new synergy between the Sciences and the Arts. 

I) Symbiosis and hybridization: biological metaphors in vogue in the computer and digital field, from the beginning of computing to the present day.

a) A novel concept of a possible partnership between humans and computers

Anyone interested in the links between symbiosis and computer science will identify the inaugural 1960 article "Man Computer symbiosis"¹² written by one of the acknowledged founders of modern interactive computing, J.C.R. Licklider. Both a psychologist and a computer scientist, he developed a way of thinking about the partnership with the computer, and later helped coordinate, within military programs, the transition from the first computer networks to the Internet. Such an early reference to and mobilization of a concept from the life sciences, and moreover in connection with the military-industrial complex, may surprise and even arouse suspicion. A few points need to be made here. Firstly, it bears witness to a context of technical emergence, that of computers finally operating in real time and becoming sufficiently programmable and accessible thanks to their sensitive interfaces (screen, optical pen, and keyboard) to envisage a genuine cooperative interaction that would bring together the best of the two worlds, living and artificial. Secondly, it also highlights a typically cybernetic vision of putting human and computer in parallel, becoming colleagues and partners, who, far from being reduced to each other, assert their specificities and differences, but pull their strengths and weaknesses. Thus, by associating and communicating according to the right methods, they will constitute a tandem achieving what no one alone would manage to do. Finally, this positive, even idealistic, vision must be understood as a "political move" opening up another perspective at the heart of the technocratic system to confront the promoters of the all-machine approach and of a progress leading eventually to the advent of artificial intelligences superior to humans. However, several controversies could immediately invalidate such an idea and send it back to the side of naive approximations. In fact, symbiosis is observed in the field of the living, not the artificial. Applying it to computers would mean granting this class of logical machines the status of a species endowed with certain properties of the living, such as evolution, adaptation and reproduction. But in our western conception of the world, practising a “naturalistic ontology” as French anthropologist Philippe Descola call it,¹³ the highest faculties are the prerogative of humanity. If we want to continue the analysis with Gilbert Simondon,¹⁴ in the absence of integration with culture, technology is conceived as separate and external to man, who at best uses it, while attributing to it his own misdeeds. When in fact, technology, this human creation, has the role of mediating between nature and culture. However, the metaphorical use of symbiosis has a few merits: better respect for each stakeholder in the tandem; understanding of their interdependencies; and the establishment of a more balanced cognitive and sensitive dynamic than the sole technical and strictly functional approach, that of a mechanically extended human, "mechanically extended man" or "humanly extended machines" (quoted in ¹²). With this approach, enlightened by the living, a perspective is developed that is neither completely anthropocentric nor technocentric. It will indeed influence a whole community of inventors and engineers to seek mutually beneficial relationships. This begins in the field of complex problems, extends to ergonomic interfaces, in connection with the interfaces and work methods invented by Engelbart ¹⁵ with his "mother of all demos,"¹⁶ and will continue with the first technological research involving computer networks, at the origins of the premises of the Internet.

b) Instrumentalization and hybridization: what association between the living and the artificial?

It is clear that more than 60 years later, this cooperative conception has not really governed the deployment of the "digital." The triumphant regime is that of a generalized and excessive instrumentalization of some by others. It is also based on older, naturalistic metaphors that have become dogma: the struggle for survival, selection, competition and domination, all stimulated by a misunderstood and ideologized Darwinism of Anglo-Saxon and capitalist origin that fetishizes the market and competition. This cultural overdetermination tends to reduce both humans and computers to a few utilitarian functions, rather than mutually emancipating them, or just amplifying them (Licklider¹²) or increasing them (Engelbart¹⁵). For example, workers turn out to be replaceable as soon as a task can be automated, while the computer itself is devalued, becomes obsolete, as soon as a new, more powerful and "modern" generation arrives on the market. In other words, technological "progress" is based on the division and specialization of tasks to the detriment of others: models implementing interests, reconducting the existing and thus rigidifying socio-technical systems. (1) Following an instinct of conservation hiding under the imperative of innovation, it is essentially a matter of optimizing and stabilizing rather than solving and changing; of intensifying short-term productivity rather than changing the level of symbolic integration and moving towards "a better ecology of the mind."¹⁷

Another notion, also imported from biology, is proliferating in the field of new info-communication technologies, and continues to be very much in vogue on the cultural level.

It is the notion of hybridization, synonymous, depending on the context, with convergence, fusion, mixing, blending, and association, concerning both the technologies themselves and the interpenetration or interweaving of heterogeneous dimensions: the human and the artificial, or the real and the informational. It carries an enthusiastic conception of the possible crossings between various technical lineages as well as of what results from them, their offspring. It probably has a positive echo in our anthropological collective unconscious because of the reproductive experiments dating back to the Neolithic period, carried out by our species on plants and animals to make them edible, domesticate them, use them, and thus develop ever more new hybrids to improve our living conditions.

As a result, there is a tendency to explore and implement all possible configurations through clever and inspiring mixtures, according to a principle of free experimentation that does not take into account the stakes and consequences from the outset, but rather evaluates them, possibly and only after the fact. Believing himself to be the "master and possessor" of his technique (2), humans let themselves go to "hubris", to this excess of power, to this attractive vertigo that is generated by a too continuous success. The conquest of all possibilities guides our collective strategy. Most forms of exploitation, first and foremost of the living through technology, are justified, pending the next techno-solution that will correct the effects. In this vein, transhumanist conceptions assume that humanity must continue to mutate by itself thanks to Technology and Science, assisted by an Art that would become its official foil. But do they realize that the solutions obtained by a definitive hybridization (the cyborg) or by deep environmental modifications (the geoengineering) remain prisoner of the same paradigm that caused the problems it tries to solve? Although some forms of hybridization may be viable and desirable, with lines that improve through heterosis, which is the alliance of the best of two species, we will try to think according to a higher general principle, that of an evaluation and regulation of a symbiotic nature, in the French sense of the term, and therefore mutualistic, which could frame, anticipate and therefore limit any possible deleterious effects.

II) "Cyber" and "hyper" avatarizations: two main ways of entering the digital world

a) The cybernetic avatar: a body situated to experience virtual technologies

The fundamental phenomenon that networked microcomputing has made possible, but which was not immediately grasped or understood as such, concerns the "existential" entry of the human into the machine and into the heart of its software. If this is realized in many ways, the avatar is the most widespread and concrete emblematic figure, especially the one that networked video games have made known to us. Indeed, since its baptism,¹⁸ the avatar has made it possible to inhabit an immersive environment with other human persons and programmed entities, and even to cooperate with artificial beings or with other "semi-living" beings, which have yet to be better qualified and understood, both scientifically and artistically.

The original symbiotic vision already mentioned had stopped at the face-to-face meeting of the human being and the computer, forming an interdependent and winning team. With the integration of networked computers, an informational, quasi-corporeal vector was needed, if only to manifest itself to other distant people, caught in the same technological constraints. In a shared place, benefiting from a system of exchanges and actions, the human invests his avatar to interact in a more complex way than with conversational interactivity based on dialogue and instruction. Collectively, the first interactivity (reactive, sequential, asymmetrical) is overcome and gives rise to the second interactivity (proactive, simultaneous, mutual). The first occurrences of avatars (3), retrospectively identified by semiologists, computer scientists and other analysts,²⁰ result from the marginal appropriations of the very first transistor computers by new adepts of creative subcultures, looking for a way to divert and explore the capacities of new electronumerical machines, at a more human size and finally becoming more accessible.

Since then, despite their improvement and growing sophistication, playable avatars, those of the simulated worlds of video games, can still be analyzed in terms of "living-artificial complexes." Indeed, the technical part, with the programmed functioning, and the living part, with the human behaviours adjusting in real time to the action in progress, are mixed in the same virtual being. In this respect, avatars are similar to techno-human or bio-technical "hybrids," depending on the point of view adopted. In this respect, field studies in the human and social sciences show that online participants who use avatars know how to consider any "player character" they encounter in a persistent universe from both sides. They evaluate both their operational capacities and the living intentionality that drives them. In addition, the hybrid avatar has several remarkable characteristics that relate to its mode of operation and attendance. On the one hand, it results from a "soft" hybridization, because it is temporary and reversible (unlike the cyborg), which preserves the integrity of both halves, the human and the artificial. On the other hand, this humanized technical hybrid is unified and maintained by two parallel processes that can be analyzed through the symbiotic metaphor. On one side of the screen, in the ordinary world, there is a human-machine-computer coupling which, for the time being, joins the Lickliderian vision of a dynamic symbiosis. Except that here, it is even more marked because it is based on a total interdependence: the human needs a formalized avatar to evolve in a simulated world while, without its player, the avatar is only an empty shell, identifiable by its looped attitudes, as if waiting for a soul to re-function. Both interface and vehicle, it is an entity that articulates the real and the virtual. On the other side of the screen, this time in the simulated world, the avatar must compose, negotiate and build its partnerships, if possible symbiotic to last and grow, both with its fellow avatarized, and with other inhabitants fully programmed and more or less autonomous. Its survival and development often depend on relationships of co-construction and cooperation, even in a competitive context. By associating as a group, by combining their talents, by carrying out collective strategies, while benefiting from the support of artificial beings, the human beings hybridized by the avatar adapt to a place lived in common. They develop a powerful intersubjectivity, a feeling of living together that is confirmed by their decisions and actions. Unified by a particular spatio-temporality and physicality, this virtual but concrete place mobilizes their situated cognition; that is, sufficiently spatialized to transform the environment and its components or actants into resources. As for the corporeality of their avatar, it engages a cognition here virtually embodied, but otherwise instantiated according to principles partly analogous to our human condition. Finally, through their avatars, humans enter into a very strange trade with certain autonomized agents, typical of their adventure worlds: pets, helpers and assistants, drones or robots, supernatural allies... With these, a distributed cognition develops, i.e., distributing the processing of information over these autonomized entities, for example, the tracking, surveillance or management of this or that process. All of this is linked to the ongoing cooperation with the other comrades in the game, the other avatars. In this way, many intimate and vital relationships are established and unraveled with programmed or hybrid technical beings, whose radical otherness is forgotten, by dint of being dressed up in the colors of the narrative, of the game or of simple co-existence, as within Second Life-type metaverse based on collective creativity.²¹

Figure 1: "Cyber" hybridization process doubly framed by symbiotic relations leading to co-individuation

This diagram visualizes the way in which an avatarial hybridization takes place, framed and framed by a double relationship that can potentially be interpreted in symbiotic terms: on the left, that of the human coupled to the nested triad computer/software/avatar; on the right, that of a cybernetic "avatar" comparable to a symbiont, as soon as it is engaged in relationships of interdependence with mutual benefit with its other congeners, or even with various entities entirely programmed and eligible for a relationship of interest. It is in these virtual worlds that symbiotic co-individuation could take place not only between the human and his avatar, but also with the empowered virtual beings that also inhabit them.

b) The hypermedia avatar: an aggregate of data in the service of an identity model in hybrid spaces in two ways

In contrast, we must address, even briefly, the other immeasurable domain of hypermedia, because the way in which humans must avatarize themselves in order to exchange and cooperate with their fellow human beings is quite different from the previous one.²² This involves the establishment of a "profile," with oriented functionalities. This is in fact a particular identity model, a sort of explicit information sheet specifying the aspects useful for the activities specifically allowed by the platform. By filling in fields and forms, by providing qualities and quantities, by uploading media (photos, thumbnails, videos, visuals, sounds), the user explicitly creates his profile, which will then be fed more or less invisibly by the "datas," "metrics" and other "analytics" that are generated according to his activities. Because the activities are more disembodied, more informational than simulational, the participant feeds the platform where he acts by his punctual acts, by his simple consultations, requests, clicks and orders, which have become his editorial and media contributions. There are certainly some services that focus on cooperation, such as Wikipedia for collaborative writing, or GitHub for collective programming. However, most platforms, especially commercial or social ones, juxtapose hyper avatars with no common space other than a customizable interface. They offer specific functionalities, for example, for shopping or conversation, which require a simple interactivity, of command and impulse: summoning contents, pages or profiles; clicking to act; typing and sending asynchronous messages to exchange; just exciting by notes, scores or likes, or on the contrary, inhibiting by blocking and unsubscribing.

In the inaugural vision, it is within this informational, documentary and media hyperdimensionality; from Vanevar Buch's Memex to Ted Nelson's precursor "hypertext"²³ ; that positive symbiotic relationships were supposed to unfold. But we are struggling to find them, so much so that the algorithms that model and influence behaviour encourage compulsive buying, isolation in filtered bubbles, the excitement of passions and cognitive biases. Thus, these hyperspaces have tended to reduce the human to a set of models and data,²⁴ made of functions and behaviors that integrate with software. User-consumers are thus influenced and made predictable by traceability, profiling and probabilistic anticipations. The mainly utilitarian relationships that are currently developing within the "hyper" digital environment have shown their danger, even toxicity for democracies, potentially influencing crucial voting processes. At issue is the modelling of stakeholders, which is often done without their knowledge, and the monitoring and stimulation of behaviour guided and motivated by forms of economic, cultural and political predation that go beyond the logic of advertising or the attention economy alone.

Such extreme pressure and overkill can be explained by the doubly hybrid nature of the Web and its variations. On the one hand, "hyper" informational environments have a strong potential for intrinsic hybridization, linked to the ease of combining media and documents that are hyperlinked, enriched with information, recombined, declined, merged, diverted and replicated. They are also able to accommodate all other old (radio, television, press) and present (platforms, services, applications) media, including cybernetic virtual worlds, which a simple, well-programmed web page can display in some cases.

On the other hand, these "hyper" platforms are hybrid in the sense of devices (educational, professional) that mix real and informational spheres. Most of the time, the infosphere constitutes a duplication of the real world, which generates relationships of reciprocal influence. Human avatars, "hyper profiles" point, despite their pseudonymity, to civil identities, while online businesses or connected data visualisations drain real flows of goods and data. The result is numerous and strong power stakes and possible holdings attracting the will to power, mobilizing all sorts of stakeholders: states, hackers, companies, collectives. Conversely, cyber worlds and avatars seem to escape this extrinsic hybridity, because they remain more disconnected from the real world, at least until the promises of metaverses claim to encompass them all and link them to our real values through NFTs and other speculative crypto assets. Conversely, "hyper" universes are hybridizing both internally and externally, the latter aspect having been intensified by the multiplication of mobile computers (phones, tablets) and other connected and "smart" objects that link infosphere and biosphere, artificial and living.

III) Discussions and critical hindsight

a) Cyber and hyper polarizations: towards which symbiotic co-individuations through the avatar?

At this stage, we should recognize the limits of this opposition between a symbiosis that is more possible in simulated cyberworlds (immersive environments) and an unbridled hybridization in informational hyper-spaces (distributed hypermedia and other Web platforms).

In the first place, there are still some "hyper" logics within cyberworlds, for example instant teleportation, or interface parameter layers. Symmetrically, this time in hyper-spaces, some "cyber" systems act and function: predictive models, recommendation engines and other underlying simulations ²⁵. Well hidden, in the background, there is also this implicit avatar made without our knowledge, this identity avatar calculated and manipulated by the 2.0 platforms whose existence has been recognized by the repentant Silicon Valleyers and who have been described as "Digital Voodoo Dolls"²⁶ evoking the spell. This verifies the excesses that many have denounced. All the more reason to take up one of our collective challenges, which would be to endow hyperspaces with more cooperative and symbiotic properties/characteristics,²⁵ in particular by interacting through hyperspace avatars that are more transparent, temporized, situated, regulated and better controlled. As for "cyber" avatars, which are themselves driven by basic modelling or AI that may also contain obscure biases (known as dark patterns), let us beware of idealizing or valorizing them without detailed investigation. For the challenge is still to highlight, in order to cultivate and preserve them, the modalities and relational conditions of regulation (coupling, embodied, situated and distributed cognition) and of positive integration (benefits and gains, emancipation and empowerment) in order to foster mutualistic partnerships. This can only be done through reciprocal adjustments and constitutive reinforcements towards a co-individuation of all inhabitants of cyberworlds, which relies on all affordances available in these inhabitable virtual spaces.

Secondly, it should be noted that the processes of hybridization have been explored in very positive and constructive ways, and this in the field of Art. Communities of artists, technologists, experimenters, or university researchers, wanted to open other ways of convivial and regenerative co-existence with the new digital technologies of art. The international colloquium "For a digital imaginary with Edmond Couchot. Hybridizations between the arts, sciences, technologies and the human" in November 2022 was able to give an account of how the revolutionary capacities of a computer have been understood for 50 years. The latter have opened up the digital dimension by becoming universal simulators. Their interactive images constitute matrices of hybridization "squared," producing through language and gesture "the image with image power."²⁷ This has made it possible to revisit, invent and materialize our imaginations, to make our fictions or visions tangible. This conception is in line with the idea of a computer structuring a meta-technology²⁸ that can simulate and accommodate all the others. This vast line of creations and works have also paved a way towards partnership symbiosis by endowing technical virtual beings with a certain autonomy, as shown by the interactive installations La Funambule (presented at ISEA 2000 ²⁹) and more recently InterACTE³⁰, in which a humanoid creature learns and individuates itself through contact with a participant who in turn experiments with new relationships with this autonomizing virtual being.

However, there is no spontaneously and naturally good and right approach: the artists in question have been nourished and framed from the very beginning of their "human-computer partnership" by the contributions of science and culture. This has given them the resources, knowledge and principles that enable them to escape the hubris of omnipotence in order to assume the ethical, political and scientific responsibilities related to these new demiurgic powers.

Thirdly, and lastly, the challenge is to offer intelligible and sensitive resources so that the anthropological entry of humanity into its "virtual meta-machines" (connected computers), this exo-somatisation of our own virtual double, can enrich us in return in a way that is both "technaesthetic" and conscious of everything that determines it. The innumerable lives offered by avatars of all kinds—both "hyper" informational profiles and "cyber" simulated corporeality—invite us to experience other real and imaginary worlds, dystopian or desirable, acting as initiatory gateways, instructing our relationship to artefacts, to ourselves, to the living, to our One Earth, and to this virtual sphere, to be envisaged as a "virtual common" to be healed.

b) What synergies between the sciences and the arts to finally assume our current responsibilities?

The productive articulation Arts/Sciences including Technologies is justified by the interest that there would be in the creative activities taking support on the scientific knowledge and investing the big changes of paradigm which currently restructure a number of disciplines, of which precisely that of the symbiosis with the rise of ecology, of the human and social sciences, of the sciences of the complexity and its solutions, biomimetic and simplex derived from the evolution of the living. Certainly, if it should be reaffirmed that, by definition, art is a domain that escapes the principle of responsibility, thanks to the famous artistic license, it nevertheless deserves to take its full share and responsibility in today's "paradigmatic revolutions." These are achieved precisely through changes in the model of understanding of the world,³¹ as is happening today with anthropology, the sociology of science or the philosophy of technology: holistic approach, destitution of the subject individual king, taking into account technical relations and processes and new entities at work (non-human actors, stakeholders, resources and forces), better integrations of interdependencies and a growing consideration for subtle, invisible or microscopic dimensions, inviting temporalities and spaces incommensurable for us mere humans.

As for the problematic of avatars, it links us to the more general one of the exosomatization of the highest cognitive faculties of the human species in virtual techniques and creations. We believe that in this field, art can not only bring us alternatives, shifts, surprises, wonder, but that it constitutes with science one of the most adapted means to approach complexity and uncertainty, to synthesize even paradoxical advances. The Art-Science alliance, which is still to be built, can be based on different ways of coupling them, in order to favour a capitalization, a mutual collaboration. If art is stratified and memorized through culture, scientific advances build knowledge. Now, in view of the emergence of new technologies, there is every reason to encourage a crossing of these two systems of accumulation, so that art is a source of knowledge and science also structures and disseminates culture. In order to better understand this, let's think about how the ludic activity exploring the potentialities of computer science has finally escaped the arts and the instituted sciences for a long time, even though games and arts anticipated together and restructured the bases of the future of interactive computer science, with an intensity and a power recognized today through games studies like the 10ème art.

This is also why recent initiatives such as issue 9 of the journal HYBRID, published at the end of 2022 in English and French,³² organize a cross and cumulative dialogue between scientific studies and artistic research, in this case around the figure of technological avatars offering their practitioners media incarnations to encounter informational environments or immersive environments.

It was during one of the dialogues at the heart of this review that object-oriented transversal approaches were able to discover their complementarity on both sides of the arts and sciences, and that a difference in temporality between artist-researchers preoccupied with the future and scientist-analysts more focused on the present or the past could be better formulated. The interest in tomorrow's artificial creatures, which should be welcomed and respected, to quote Edmond Couchot,³³ justifies many projects and experiments. Let's take up again the parallel with Licklider, when he specified that it was necessary to deal with the symbiotic partnership with the computer in the time interval that separated us from the arrival of "strong" AIs. In these still open times, before the promises of sentient robots or a perfect autonomized simulation of human beings are realized, there is every reason to take care of our partnerships with our avatars, by seeking together the conditions that will make us environmentally and behaviorally responsible. Hence the richness of symbiotic approaches, both in the virtual and in the real, since the two are constantly hybridizing in a way that is now inextricable.

IV) Conclusion in the form of programmatic questions, bias and references

The first question, of general scope, would be to know: "in what way does the avatar belong to a case of human-technological hybridization? For as we have detailed, the technological avatar achieves the integration of the living and the artificial in real time in the same iconic programmed object, in an animated body giving access to a virtual environment and manifesting the human to his fellow creatures of the same species and other quasi-species, virtual beings in the process of empowerment. But does this result in more than a hybridization of techniques between them, a mixing, a specific mixture, producing the interpenetration of stakeholders (subject, machines, programs and human and machine intentionalities)? And how, in return, does the sensitive and embodied internalization of technical logics in the human being take place through "existential" experiences, psychosocial and individualizing experiences (Simondon), with subjective and objectifiable experiences, echoing Edmond Couchot's technesthesia to the “cyberesthesia” proposed by the French artist Yann Minh, or even Macluhan and the media extension of the senses? But also, how to go beyond symbiosis as a metaphor for the living and elaborate its factual declensions, adapted and relevant to these hybrids of flesh and calculation that are the avatars? Consequently, how to configure their habitat, these virtual cyberworlds?

And finally, how can we configure differently, with the virtual, the technical conditioning and the cultural overdetermination, both inevitable, in order to achieve a common world that is equitable, sustainable and desirable in terms of reciprocal emancipation? The construction site is vast and the construction techniques to be implemented remain to be invented and popularized by this synergy between science and the arts that we are calling for. Let us be numerous and determined to invest ourselves in it for the benefit of all, human and non-human.


(1) In our theory, we prefer to distinguish techno-social (when living being are inside artificial construct) and socio-technical level (socialization of technical artefact).

(2) As the famous French philosopher René Descartes explains in his time in “Discourse on the Method of Rightly Conducting One's Reason and of Seeking Truth in the Sciences”, 1637.

(3) Named after the 1986 Habitat network games that referred to the inhabitants of this networked world as Avatars

References

1 Call of communication ISEA, 2023, online.

2 Vilem Flusser and Ann Roth Nancy, Into the Universe of Technical Images, NED-New edition, Vol. 32; University of Minnesota Press, 2011, accessed December 08, 2022, Ac http://www.jstor.org/stable/10.5749/j.cttttcq8.

3 William Regli, "Revisiting the Human-Machine Symbiosis," Computing Research Association Bulletin, April 2016, accessed December 08, 2022, https://cra.org/revisiting-human-machine-symbiosis.

4 Lynn margulis, Symbiotic Planet, Basic Books, 1998.

5 James E. Lovelock, Lynn Margulis, "Atmospheric homeostasis by and for the biosphere: the gaia hypothesis", Tellus, 26:1-2, 2-10, (1974), accessed December 08, 2022 https://www.tandfonline.com/doi/abs/10.3402/tellusa.v26i1-2.9731.

6 James Lovelock, Gaia A New Look at Life on Earth Oxford, Oxford University Press, 1979.

7 André Leroi-Gourhan, Gesture and Speech, The MIT Press, 2018, Translated by Anna Bostock Berger, First French Edition, Albin Michel, Paris, 1965.

8 Gilbert Simondon, Individuation in Light of Notions of Form and Information (Volume 1 & 2, Translated by Taylor Adkins, Univ of Minnesota Press, 2020, original French edition, Paris, Jérôme Millon, Collected papers of 50's, 2005.

9 Alain Berthoz, Simplexity: Simplifying Principles for a Complex World, Yale University Press, 2012. 

10 Maurice Merleau-Ponty, Phenomenology of Perception, Routledge, 2013, Translated by Donald Landes, original French edition, 1945.

11 Francisco J. Varela, Evan Thompson, Eleanor Rosch, The Embodied Mind: Cognitive Science and Human Experience, MIT Press 1991.

12 Licklider, J. C. R., "Man-Computer Symbiosis", Ire Transactions on Human Factors in Electronics 1 ,1960, 4-11. 

13 Philippe Descola, Beyond Nature and Culture, University of Chicago Press, 2014, Translated by Janet Lloyd, original french edition 2005.

14 Gilbert Simondon, On the Mode of Existence of Technical Objects, Translated by Cécile Malaspina, Univ of Minnesota Press, 2017, original French edition, Aubier, 1958.

15 Douglas C. Engelbart, William K. English, 1968, A research center for augmenting human intellect, In Proceedings of the December 9-11, fall joint computer conference, part I (AFIPS '68 (Fall, part I)), Association for Computing Machinery, New York, NY, USA, 1968, 395-410.

16 Claudia Salamanca, "The mother of all demos" in DAC'09, Digital Arts and Culture 2009, accessed December 08, 2022, https://escholarship.org/uc/item/91v563kh.

17 Gregory Bateson, Steps to an Ecology of Mind. St. Albans Paladin, 1972.

18 Chip Morningstar, F. Randall Farmer, The lessons of Lucasfilm's habitat, Cyberspace: first steps, MIT Press, Cambridge, MA, USA, 1991, 273-302.

19 Michel Bret, Marie-Hélène Tramus, Alain Berthoz, "Interacting with an Intelligent Dancing Figure: Artistic Experiments at the Crossroads between Art and Cognitive Science", Leonardo 38, no. 1, 2005, p.47-53, accessed December 08, 2022, http://www.jstor.org/stable/1577645.

20 Bob Rehak, "Playing at Being: Psychoanalysis and the Avatar." In The Video Game Theory Reader, 2003, 103-138, accessed December 08, 2022, https://www.academia.edu/3751080/Playing_at_Being_Psychoan alysis_and_the_Avatar.

21 Tom Boellstorff, Coming of Age in Second Life: An Anthropologist Explores the Virtually Human, Princeton, Princeton University Press, 2008, 316.

22 Louise MERZEAU, " Habiter l'hypersphère", Documentaliste-Sciences de l'Information, 2010/1 (Vol. 47), accessed December 08, 2022, 30- 31, https://www.cairn.info/revue-documentaliste-sciences-de-l-information-2010-1-page-30.htm.

23 Ted Nelson, Computer Lib/Dream Machines, Revised Edition, Tempus Books, 1987 (first edition,1974)
Felix Guattari, Three Ecologies, Translated by Ian Pindar & Paul Sutton, Bloomsbury Academic, 2014 (original French edition 1989).

24 Bernhard Rieder, "Beyond Surveillance: How Do Markets and Algorithms 'Think'?" The Foucaldian 3, no. 1, 2017, 1-20, DOI: https://doi.org/10.16995/lefou.30 [Note: In 2022, Le foucaldien relaunched as Genealogy+Critique.] accessed December 08, 2022 https://www.genealogy-critique.net/article/id/7055/.

25 Bernhard Rieder, Engines of Order: A Mechanology of Algorithmic Techniques, Amsterdam University Press, 2020, Accessed December 08, 2023, https://www.jstor.org/stable/j.ctv12sdvf1.

26 Marija Slavkovik, Clemens Stachl, Caroline Pitman, Jonathan Askonas, "Digital Voodoo Dolls", In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21), Association for Computing Machinery, New York, NY, USA, 2021, 967-977, Accessed December 08, 2023, https://www.researchgate.net/publication/353603454_Digital_Vo odoo_Dolls.

27 Edmond Couchot, Images. From optical to digital. The visual arts and the evolution of technology, Paris, London, Lausanne, 1988.

28 Alan Kay, Adele Goldberg, Personal Dynamic Media. In: Wardrip-Fruin, Noah / Montfort, Nick: The New Media Reader, Cambridge MA, The MIT Press, 2003, 393-404.

29 Marie-Hélène Tramus, "Multisensoriality Interactive Digital Art, Presentation of an Experience: The Virtual Tightrope Walker" (paper submitted in ISEA 2000) Only Proceeding, 2000, 200-204, https://www.isea-archives.org/docs/2000/proceedings/ISEA2000_proceedings.pdf 

30 Dimitrios Batras, Judith Guez, Jean-François Jégo, InterACTE: Improvising with a Virtual Actor, In Proceedings of the 3rd International Symposium on Movement and Computing (MOCO '16), Association for Computing Machinery, New York, NY, USA, Paper 52, 2016, 1-2,
https://doi.org/10.1145/2948910.2955109

31 Bruno Latour, An Inquiry Into Modes of Existence: An Anthropology of the Moderns, Havard University Press, translated by Catherine Porter, 2018.

32 Etienne Armand Amato, Judith Guez, Etienne Perény, Marie-Hélène Tramus, "Prospective and retrospective reflections on avatars and virtual actors in artistic experimental exploration," Hybrid [Online], 9, 2022, online 30 November 2022, accessed 09 December 2022,  http://journals.openedition.org/hybrid/2808 

33 Edmond Couchot, Automata, robots and virtual humans in the performing arts, PUV, 2022

Authors Biographies

Étienne Armand Amato has been a lecturer in information and communication sciences at Gustave Eiffel University and the DICEN-IDF laboratory since 2014 (www.dicen-idf.org). Since the beginning of high-rate networks, he studies interactive audiovisual forms (video games, virtual reality), ludic communication and media avatars. He also contributes to the work of IHEST (Institute of High Studies for Science and Technology), a training organization of the French Ministry of Research.

Étienne Perény is honorary professor of communication and interactive design, a member of the laboratory Paragraphe at the Université Paris 8 de Vincennes and associated with the DICEN- IDF laboratory. Since the 70s, he experiments new technologies: video, architectural simulation, videodiscs, cable networks, interactive video prototypes, offline multimedia, then online, in the early days of the Internet. He has developed an original theory on interactive images and video games, and the research device "becoming avatar" with Alain Berthoz and E.A. Amato. 

Symphony of the Stones: A Research-Creation Exploration on the Animation of Heavy Metal Residues in Contaminated Urban Landscapes

Brice Ammar-Khodja

Concordia University
& École des Arts Décoratifs – PSL (EnsadLab)
Paris, Montreal
brice.ammar-khodja@mail.concordia.ca / brice.ammar@ensad.fr

Abstract

In the late 1980s, the Canadian Pacific Railway abandoned a rail yard on the outskirts of Montreal’s Mile End district. Within a few years, the return of animal and plant species encouraged the citizen community to reinvest this site known as Le Champ des Possibles. Despite community efforts to rehabilitate this site, hydrocarbon and heavy metal pollution persists in the soils and thus requires rethinking the engagement with the imperceptible mutations of ecosystems. Symphony of the Stones was created in response to this context. This research-creation project consists of several urban art installations that activate residual metals in soils by their magnetic characteristics to make these imperceptible pollutants visible. The following paper unfolds the different processes, methodologies and strategies that led to in site interventions blending art installation, collaboration with different communities and associations and leading to a rethinking of art practices in the urban environment.

Keywords

Digital arts, research-creation, art-science practices, situated artistic interventions, environmental installations, contaminated urban soils, active residual materials, material agency, pedological studies, environmental participation, socio-environmental issues.

DOI

10.69564/ISEA2023-40-full-Ammar-Khodja-Symphony-of-the-Stones

Introduction

Although invisible, heavy metals abundant in soils have the potential to contaminate natural environments, thereby increasing the risk of serious diseases. In recent years, the discovery of new polluted sites underlines the extent of this pollution. Over a century of industrial activities has led to this pollution spreading at an alarming rate throughout urban areas. Nowadays, 54 percent of the global population lives in an urban area,¹ and, in Quebec alone, 10 percent of the identified sites are located within Montreal.² In this regard, citizens' exposure to these toxic residues requires a rethinking of engagement with the imperceptible mutations of ecosystems due to anthropogenic activities.

Despite the fact that Karl Marx anticipated ecological urgency and soil loss by what he theorized as the "metabolic rift," soils have only recently gained interest from social and biological/geological sciences and science and technology studies (STS).³ On the other hand, the works of contemporary artists and artist-researchers such as Debra Solomon, Amy Balkyn, Martin Howse, Joana Hadjithomas, Khalil Joreige or Disnovation.org, illustrate this urgency by raising awareness on biodiversity loss and soil degradation. As the push intensifies to transition these discussions out of strictly academic and artistic realms, the pressing challenge becomes: how can we expand the conversation on soil contamination, and more broadly, urban pollution, ensuring it resonates with the general public, especially the communities and stakeholders most affected by it?

Écotones is a collaborative research-creation project which arose from this question. Led with artist Philippe Vandal and in collaboration with the citizen association Les Amis du Champ des Possibles and more precisely with the coordinator Sugir Selliah and the co-president and climate policy analyst Émile Boisseau- Bouvier, Écotones operates as an in situ urban laboratory. It melds experimental exhibitions and artistic interventions, articulating an aesthetic, critical, and social perspective on soil pollution. This research-creation project, blending art-science methodologies, seeks to explore heavy metals and hydrocarbon pollutant agencies to activate new experiences of soil pollution and offer critical visibility to these invisible residues. Two directions were taken: Symphony of the Stones (my own work) which explored heavy metal pollutants and Fluorescent Forensic by Philippe Vandal, which explored residual hydrocarbons. The first iteration of this project took place at Le Champ des Possibles in Montreal on October 14th. Le Champ des Possibles (CDP), is a former marshaling rail yard left as a brownfield by the Canadian Pacific Railway company in the 1980s and rehabilitated as a green space in 2013. Despite community efforts to rejuvenate this land, hydrocarbon and heavy metal pollution still lingers in the soil. The ongoing proliferation of these pollutants presents dire ecological and health risks that cannot be overlooked. As the site was formerly used as a waste disposal site, the municipality and the association les Amis du Champ des Possibles (ACP) have therefore agreed to rehabilitate the site by excavation under section 22 paragraph 9 from the Loi sur la Qualité de l'Environnement.⁴ However, few citizens are aware of these hazardous materials, whose existence deserves to be discussed. 

This article zooms in on Symphony of the Stones, an in situ art installation that seeks to enhance the understanding and perception of heavy metal residues in contaminated soils. The article contextualizes the making of the installation and the way it connects to CDP. First, the article focuses on recontextualizing what CDP is and how the concept "ecotones"—that gave the name to the series of interventions—emerged from exploring this site. The second part of the article delves into the art installation and how it activates soil's heavy metal contaminants to shine a spotlight on the invisible traces of pollution lying under the ground. Finally, the third part delves into the concept of public intervention, drawing inspiration from various contemporary in situ art installation practices.

Exploring the entanglements

Le Champ des Possibles as a space of negotiations

Le Champ des Possibles is located in the Mile-End district of Montreal. The district underwent significant gentrification, initially with the influx of artists in the 1980s and more recently due to the emergence of art galleries, design showrooms, and companies in the innovation and video game industries. Despite this socio-economic context, CDP remains in the state of a brownfield. The site is characterized by the entanglement of various plants, animals, and human communities living together with the traces and ruins of its past industrial activities. The foot paths laid by morning joggers and commuting wanderers are lined with campfire remains and industrial ruins (figure 1). Steel structures, now enveloped by diverse species of trees and plants, provide shaded resting spots for local workers. These makeshift sitting areas are occasionally rearranged by visitors to the site. The northern border of CDP neighbors a railway which traces a separation between the Mile-End and the other districts. For years, these fenced borders have been the subject of debate among the citizens, the municipal authorities and the Canadian Pacific Railway that owns the railroad.⁵

Breaches in the fence allow for easy crossing to the other side of the railroad and the reconstruction of this same fence by the public authorities are thus common things to observe throughout the year. Because of the diversity of uses and communities that make CDP a unique space, it is not easy to give a single definition of the site as it conjures up different imaginary perceptions. Facing this post-industrial biotope and the entanglement of different human and more-than-human figures negotiating the site with each other, it appears that CDP was scientifically reminiscent of a concept in environmental science that would help us formalize and conceptualize a dialogue with the space: the ecotone.

Figure 1. Picture of Le Champ des Possibles. We can observe the biotope of the site as well as the railway and the fence in the background of the picture © Brice Ammar-Khodja

Ecotone as a conceptual framework

Ecotone is a term that emanates from environmental sciences. It describes areas of gradation between different ecological communities, ecosystems or even regions. In other words, places where diverse ecologies are put in tension.⁶ Although ecotones are not restricted to the natural environment, they can also be induced by human activities.⁷ According to Jurek Kolasa and Maciej Zalewski’s book Ecotone attributes and functions,⁸ these contact zones are hard to define.  They point out that ecotones can range in size from a few centimetres to kilometres and can be distributed across spaces in different contexts with varying degrees of porosity and types of boundaries. Moreover, ecotones can be horizontal and extend over planar surfaces or vertical when localized in the atmosphere or water bodies.  This overlapping inspired us to build a conceptual framework that would foster insitu exploration of CDP. This allowed characterizing the area, not only at the ground level but also by understanding the different vertical layers and boundaries on the distribution of natural elements and pollutants in the soils. This notion of ecological boundaries guided the exploration of the site. Discussion with members of the citizen association ACP, highlighted that these ecological boundaries were overlapping with a map from the Quebec government classifying soils by degrees of contamination within CDP. The map shows a satellite view of CDP where zones are divided into polygon sections, each providing information on the name and concentration of pollutants, and the possible effects of these residual matters. In addition to this map providing technical information about the site, the ecotones framework became a tool for understanding the socio-environmental interactions of the site. While the map showed CDP's contaminants split into polygons, the notion of ecotones revealed other boundaries that would appear through the exploration of the site: different paths generated by the users of the site oscillating between gravel roads and tracks where the grass does not grow anymore, the green rehabilitation zones scattered on the site to allow the vegetation to grow back, the campfires where only the waste of late-night rave parties remain, and finally the railroad, which illustrates the most significant transition zone of this unique ecotone. Through this first overview, CDP appears as a space of socio-cultural and biological diversity in constant mutation. From a natural space transformed into a rail yard to a half-industrial, half-green public space, and finally, to a space in the process of decontamination, the transition of this landscape is not only remarkable for the diversity of communities, boundaries, and uses of the space, but also its transformations through space and time.

While a simple walk in CDP would reveal the aforementioned ecological boundaries, the former use of CDP as a waste disposal site left traces of which a map alone cannot give a complete account. As the entanglement of the many elements that compose CDP tends to blur what this site is composed of, the primary approach of the project consisted of developing art installations, unveiling imperceptible elements.

Sensing mutations

Residual invisibilities

Soils are dynamic systems composed of organic matter, living organisms, gases and water. When the amounts of naturally occurring elements, such as heavy metals (i.e., copper, lead, nickel, arsenic), increase abnormally due to external, and usually anthropogenic, factors, a pollution risk arises.⁹ When elements, not naturally occurring in soils, such as hydrocarbons, are found, the phenomenon is termed contamination.¹⁰ As with heavy metals, excessive levels of these external components can lead to pollution risks and disrupt the natural soil cycles. However, most of the dynamic interactions and transformations in the soils, even the most concerning, are not easily perceptible to the naked eye. Human relations with soils are largely limited to what human sensory organs can detect and interpret. Indeed, perception relies on a combination of the senses. Among these, vision, described by Plato as "the noblest of the senses," dominates as the "most informative" and "associated with the highly valued faculty of reason."¹¹ The effectiveness triggered by perception can contribute to increasing awareness, understanding, and care for soils. However, how can one foster these sensations when the soil's mutations are imperceptible? In Thinking with Soils, Material Politics and Social Theory, Anna Krzywoszynska, Manuel Tironi, Matthew Kearnes, Céline Granjou, and Juan Francisco offer socio-environmental and ethico-political ways to approach soils with a care-oriented analysis inspired by the work of Puig de la Bellacasa. In the book's second chapter, the authors point out that soil conservation is challenging because soil ”is commonly presented as invisible, hidden under our feet, and lacking a recognizable 'face.'"¹² Since soils are composed of microscopic elements (living beings, organic matter, minerals, pollutants, etc.), soils' intrinsically animated and living nature is not always evident to those who observe them. Thus, the authors insist that the complex nature of soils and their non-zoomorphic and non-anthropomorphic characteristics make them more vulnerable and, therefore, more at risk of being neglected and damaged. However, this mixture of non-human entities, whether organic or inorganic, animate or inanimate, possesses their agency and the potential to influence and affect systems of which humans are part. This notion relates to what Jeanne Bennett calls "thing-power", defined as “the curious ability of inanimate things to animate, to act, to produce effects dramatic and subtle.”¹³

Soils are discrete entities, and pollution is invisible. In some contexts, and environments, the biotope sometimes contributes to weakening the perception of pollution. Let us recontextualize this assumption: at first glance, CDP looks like a site in the process of green rehabilitation. Since the Canadian Pacific Railway company left the site as a brownfield, species of plants, trees, and animals have begun to reoccupy the site. This reoccupation might have paradoxically contributed to blurring the distinction between the past and present of the site; that is, a place undergoing green rehabilitation versus a place where pollutants still lie and mutate beneath the soil. Discussions with the citizen association ACP have confirmed these suppositions by revealing that many of the citizens living next to the site were unaware of CDP's pollution. Symphony of the Stones emerged in response to these issues. This research-creation project explores how the activation of heavy metal pollutants can operate on perception and senses to engage in a critical dialogue with the communities bordering these sites. As part of the series of artistic interventions titled Écotones, this installation draws from practices and interests in active materials, material agency, material forensic and art-science practices and focuses on generating critical discussions with the riverside communities on the issues related to CDP discussed in the next section of the article. 

Symphony of the Stones: a research-creation case study

Symphony of the Stones is a series of urban art installations that examine the correlation between the abnormally high presence of heavy metals and magnetic residues in CDP's soils. This first iteration was developed within the framework of the research-creation thesis, titled the Cycles of Attraction and derives its methodology from geomagnetism—a geosciences subfield specializing in identifying soil pollutants via magnetic fields.¹⁴ The installation reinvestigates this technique to activate heavy metal and magnetic contaminants in CDP's soils that present high responsivity to magnetic fields. The installation features two geometric aluminum sculptures, designed to harmoniously integrate with the landscape, reminiscent of the metallic remnants of CDP's past industrial activities (figure 2 and 3). 

Figure 2. First interactive sculpture exhibited during Écotones at Le Champ des Possibles ©Etienne Massicotte 
Figure 2. Second interactive sculpture exhibited during Écotones at Le Champ des Possibles ©Etienne Massicotte 

The sculptures are flush with the ground and strong enough to be walked on. Using the Quebec Government map indicating soil classification by degrees of contamination, these ephemeral sculptures are strategically placed over highly polluted polygons and topped with CDP's soils. Concealed within and below the sculptures is a kinetic system comprising N45-grade permanent magnets. These magnets, driven by DC motors and controlled by a microcontroller (figure 3), generate magnetic fields that create material tensions and attractions, activating the magneto- responsive constituents of the soils. This includes ferromagnetic residues, construction debris like nails, discarded municipal items like beer caps, and variously sized earth clumps. In a metaphorical sense, the installation seeks to visualize the invisible soil transformations. The movement of the soil creates a subtle yet captivating animation aimed at drawing attention to the ground. Concurrently, the installation delves into the acoustic properties of soils and residues. One sculpture integrates a piezo microphone, connected to a TDA2822M mini audio amplifier and a Bluetooth transmission system, relaying the audio signal to a discreetly placed Bluetooth speaker (figure 4). This system captures and amplifies sounds produced from the collisions of various materials. By amplifying these subtle soil movements, the installation adds an auditory dimension to the ambient sounds of the CDP environment. By animating both organic and inorganic residues and materials from CDP's soils—those seemingly inert to casual observation—Symphony of the Stones aspires to foster a renewed, sensory connection between humans and soils. The project is informed by the concept of "active materials" or “kinetic materials.” These materials can react and adapt by changing colours, textures or events” moving in stimuli such as heat, magnetic fields, or moisture. In Materials That Move: Smart Materials, Intelligent Design, Murat Bengisu and Marinella Ferrara explore the potential of several active materials features and how their application in design changes users' relations with everyday objects. Drawing inspiration from Donald Norman, an expert in the application of cognitive science to design and the theorist behind emotional design, Bengisu and Ferrara highlight the potential of active materials to "materialize intangible information that is imperceptible to the human senses in daily life."¹⁵

Figure 3. Kinetic system hidden in the sculpture’s shell. The horizontal rods containing the magnets are activated by motors and driving belts. © Brice Ammar-Khodja
Figure 4. Close up of the piezo microphone in contact with magnetic soils in motion © Brice Ammar-Khodja

The dynamic created by these objects, and materials, made possible by organic and organic-like movements, might signify the expression of something that is animated and living, triggering different senses and emotions. Exploring discarded materials, residues and contaminated soils coming "alive" by their activation, Symphony of the Stones aims to encourage the public to not only further observe soils but also get closer to the ground, carefully observe the different patterns drawn by the materials in motion and haptically engage. The project's first iteration showed promising results during the Écotones exhibition, where visitors would actually get closer to the installation and eventually touch the soil. The future versions of the installation will put more emphasis on the haptic aspect. As Puig de la Bellacasa points out in Matters of Care, haptic promises to develop new forms of knowledge based on attentiveness through the body's engagement.¹⁶ The knowledge that the vision alone, by its distant and detached character, could not reach.

Intrusions, furtivity, and symbiosis

Making an in situ urban art installation involves intervening in the environment. The first part of this paper explained why CDP is a space in constant negotiation between human and non-human communities. In this context, Symphony of the Stones explores new ways to engage with the environment and its communities, rather than merely subjecting the site to a single artist's vision. This approach required working closely with ACP for a deeper understanding of the site (see section "Exploring the entanglements"). However, as illustrated in the first part of this article, there is no consensus on what CDP is or should be. Out of respect for the site, Symphony of the Stones was designed to neither disrupt nor damage the site's environment. In doing so, the installation diverges from the traditions of environmental art, especially Land Art, an art movement from the 1960s that showcased open-air artwork—often large-scale installations using natural elements found on site. Land Art sought to foster a dialogue between art and natural environments, underscoring the localized aspects of works now displayed beyond the confines of museums and galleries. Prominent examples include Robert Smithson's Spiral Jetty (1970), a 457-meter- long basalt stone dike spiraling counterclockwise over slightly reddish water at the Rozel Point peninsula on the northeastern shore of the Great Salt Lake, or Walter De Maria's The Lightning Field (1977), a permanent installation of 400 steel poles spread over a mile, designed to attract lightning in a desert plain in Quemado, New Mexico. While sometimes classified as a form of ecological art, Land Art is not without controversy. Sometimes equated as a form of ecological art, Land Art is also controversial. Some interventions were indeed drastic, leading to artworks that altered their environments ¹⁷. Since then, myriad art forms have emerged, influenced by the practices of environmental and land art. Some extend their focus beyond the natural to encompass the built environment, with a growing emphasis on addressing socio-environmental and political challenges linked to the sites under consideration. Regrettably, a recurring issue with in situ urban or environmental art installations is the gap between the discourse of these practices and the methodologies they employ. In her article Le Lourd Bilan Carbone de l'Art Contemporain, Gill Gasparina argues that many contemporary urban and environmental installations, while raising awareness of ecological concerns, still rely on processes detrimental to the environment, such as pollutant technologies, excessive energy use, and waste generation.¹⁸ Given the increasing integration of new technologies and the creation of custom artworks, the carbon footprint of contemporary—and particularly digital—art is undeniable. This highlights the pressing need to further evaluate energy consumption and material repurposing in the pursuit of more sustainable artistic practices. 

Symphony of the Stones was conceived with the dual intention of minimizing its ecological and aesthetic impact on CDP. In pursuing this, Symphony of the Stones is influenced by what art critic and art historian Patrice Loubier terms “furtive practices.” Furtive practices are described as “multiple art practices that infiltrate the urban landscape through works without signs, not necessarily recognizable as artworks, serendipitously surprising the pedestrian, transforming them into an incidental observer” (My translation).¹⁹ In contrast to Land Art, furtive practices do not dominate due to their scale; instead, they subtly alter details within these environments. Underlying these practices are a range of artworks whose subtlety setting does not diminish their political, social, or environmental significance. Drawing inspiration from Danae Stratou’s Concentric (2019), a piece that produces continuous concentric ripples in Nirox Sculpture Park’s Lake, Symphony of the Stones integrates seamlessly with its surroundings, invoking elements already present. The aluminum's polished surfaces help to camouflage the sculptures by mirroring the environment around them. Moreover, the installation is largely enveloped by CDP’s soils, which are retrieved from the topmost layer using DIY (Do It Yourself) permanent magnet soil collectors, thus avoiding any digging or excavation. Neither of the two sculptures, nor their concealed kinetic systems, needed anchoring or attachment. This adaptability ensures the installation's portability and ease of removal. Such features position Symphony of the Stones as an artwork that may elude immediate notice, nudging site visitors to be more observant of their surroundings. Yet, the installation's auditory aspect offers clues about where and what to focus on. The initial display of the artwork yielded varying reactions based on CDP's ambience and foot traffic. On certain occasions, passersby might overlook the sculptures entirely, even walking over them without pausing. However, when the contours of these metal forms caught their eye, some would halt, scrutinize the sculpture more closely, and approach upon observing the animated soil atop the structures. During the Écotones outdoor exhibition, the influx of visitors meant that some remained oblivious to the sculptures. Nonetheless, those who became aware of them through the animated soil often ventured closer, with a few even interacting directly with the residues by touching them (figure 5 and 6). 

Figure 5. One of the Symphony of the Stone’s kinetic sculpture exhibited during the exhibition Écotones ©Etienne Massicotte 
Figure 6. Close up to the residues activated by the kinetic magnetic system © Etienne Massicotte

After the opening, we organized a reception in a space adjacent to CDP, and allowed the installation to remain on site into the evening. Upon our return to disassemble the installation, the darkness had enveloped CDP, and a group of night-time explorers had kindled a campfire approximately a meter away from the sculptures. In spite of the resonating sounds and the movement of the residues, this group remained oblivious to the installation. By this attempt of creating an installation furtively within the site, it is hard to evaluate what are the best conditions for it to be perceived and understood in its message but also as a work of art. A consideration for future iterations of this installation, and more generally within the domain of furtive practices, might involve implementing strategies for artistic and cultural mediation for artworks displayed in urban settings.

Conclusion

This article sketches the premises of a methodological framework for research-creation practices that operates at the scale of urban spaces and more specifically spaces in the process of rehabilitation. Although contextualized on a specific site in Québec, I hope that this contribution will foster new ways of collaboration with outdoors spaces and hopefully participate in the debate on urban soil pollution. The CDP’s case study has demonstrated that wild urban spaces, brownfields, and any other grey area are places of entanglements and negotiation between different communities. Amongst these communities that range from human wanderers and species of plants to toxic residues, different types of agencies whether positive or negative, might emerge or persist. Against this backdrop, a question emerges: how can artistic endeavors be realized in harmony with these intricate interconnections? By highlighting presence of heavy metal pollutants in the soil, Symphony of the Stones opted to harmonize with the CDP. By suggesting new forms of non-intrusive collaborations with the site and its communities, this research-creation project seeks to activate and increase the perception of contaminants. In this regard, the project does not only seek to redraw borders on the site, but rather amplify its already existing aspects. As part of the project Écotones, this first iteration is perhaps not immune from mutating towards a more radical but still furtive kind of intervention in the urban landscape.

References

1 Food and Agriculture Organization of the United Nations, International Fund for Agricultural Development, International Organization for Migration, and World Food Programme, The Linkages between Migration, Agriculture, Food Security and Rural Development: Technical Report, 2018, 5.

2  Gérald Dermont, “Sols Pollués Par Les Métaux Lourds Résultant De L'enfouissement De Déchets Industriels (montréal, Canada) : Géochimie, Spéciation Des Métaux, Et Décontamination Par Flottation,” Dissertation, Université du Québec, Institut national de la recherche scientifique, 2008.

3 Juan Francisco Salazar, Céline Granjou, Matthew Kearnes, Anna Krzywoszynska,Manuel Tironi, eds, Thinking with Soils: Material Politics and Social Theory, London, UK, Bloomsbury Academic, 2020, 4-32.

4 Les Amis du Champ des Possibles, Interview by Brice Ammar-Khodja, December 6th, 2022 and October 14th, 2022.

5 Ibid.

6  Robert Leo Smith, Ecology and Field Biology, 4th ed, New York, Harper and Row, 1990, 251.

7 Salit Kark, Rik Leemans, Environmental Systems Analysis Group, Wageningen University, Wageningen, 6700 AA, Netherlands, 2013. 
“Ecological Systems: Selected Entries from the Encyclopedia of Sustainability Science and Technology,” Essay, In Ecotones and Ecological Gradients, New York, NY, Springer, 147–60.

8 Jurek Kolasa, Maciej Zalewski, “Notes on Ecotone Attributes and Functions,” Hydrobiologia: The International Journal of Aquatic Sciences 303, no. 1-3, 1995, 1–7.

9 Peter M Chapman, and Sixth Iberian and 3rd latinoamerican Congress on Contamination and Environmental Toxicology Cadiz (Spain) 20050925-20050928. “Determining When Contamination Is Pollution — Weight of Evidence Determinations for Sediments and Effluents.” Environment International 33, no. 4, 2007, 492– 501.

10  Ibid.

11  David Howes, “Sensory Craft,” Craft Forward, California College of the Arts, San Francisco, 2 April 2011.

12 Francisco Salazar, Céline Granjou, Matthew Kearne, Anna Krzywoszynska, Manuel Tironi, 17.

13 Jane Bennett, Vibrant Matter: A Political Ecology of Things, Durham, Duke University Press, 2010, 6.

14  Xue-Feng Hu, Li Mei, Zichen He, Lei Cui, Liu Rui, Xin-Dong Wang, Zhi-Han Wang, “Magnetic Responses to Heavy Metal Pollution of the Industrial Soils in Shanghai: Implying the Influences of Anthropogenic Magnetic Dustfall on Urban Environment,” Journal of Applied Geophysics, 2022197.

15 M. Bengisu, Marinella Ferrara, Materials That Move: Smart Materials, Intelligent Design, Springerbriefs in Applied Sciences and Technology, 1, 2019.

16 Puig de la Bellacasa María, Matters of Care : Speculative Ethics in More Than Human Worlds, Posthumanities, 41, Minneapolis, University of Minnesota Press.

17  Virginie Luc, De La Nature À L'œuvre, Paris, Ulmer, 2014.

18  Jill Gasparina, Le lourd bilan carbone· de l'art contemporain, Le Temps, 8 janvier 2019, https://www.letemps.ch/culture/lourd-bilancarbone-lart-contemporain

19  Patrice Loubier, “Un Art À Fleur De Réel : Considérations Sur L’Action Furtive,” Inter 81, no. 81, 2002, 12–17.

Bibliography

M. Bengisu, Marinella Ferrara, Materials That Move: Smart Materials, Intelligent Design, Springerbriefs in Applied Sciences and Technology, 2019.

Jane Bennett, Vibrant Matter: A Political Ecology of Things, Durham, Duke University Press, 2010.

Peter M. Chapman, and Sixth Iberian and 3rd latino american Congress on Contamination and Environmental Toxicology Cadiz (Spain). “Determining When Contamination Is Pollution — Weight of Evidence Determinations for Sediments and Effluents,” Environment International 33, no. 4, 2007, 492–501.

Gérald Dermont, Sols Pollués Par Les Métaux Lourds Résultant De L'enfouissement De Déchets Industriels (montréal, Canada): Géochimie, Spéciation Des Métaux, Et Décontamination Par Flottation, Dissertation, Université du Québec, Institut national de la recherche scientifique, 2008.

Food and Agriculture Organization of the United Nations, International Fund for Agricultural Development, International Organization for Migration, and World Food Programme, The Linkages between Migration, Agriculture, Food Security and Rural Development: Technical Report, 2018.

Jill Gasparina, “Le lourd bilan carbone· de l'art contemporain,” Le Temps, January 8, 2019, https://www.letemps.ch/culture/lourd-bilancarbone-lart-contemporain.

David Howes, “Sensory Craft”, Craft Forward, California College of the Arts, San Francisco, April 2, 2011.

Xue-Feng Hu, Li Mei, Zi-Chen He, Lei Cui, Rui Liu, Xin-Dong Wang, Zhi-Han Wang, “Magnetic Responses to Heavy Metal Pollution of the Industrial Soils in Shanghai: Implying the Influences of Anthropogenic Magnetic Dustfall on Urban Environment,” Journal of Applied Geophysics, 2022, 197.

Kark Salit, Rik Leemans, “Ecological Systems: Selected Entries from the Encyclopedia of Sustainability Science and Technology,” Essay, In Ecotones and Ecological Gradients, New York, NY, Springer, 2013, 147– 60.

Jurek Kolasa, Maciej Zalewski, “Notes on Ecotone Attributes and Functions.” Hydrobiologia: The International Journal of Aquatic Sciences 303, no. 1-3, 1995, p.1–7.

Les Amis du Champ des Possibles, Interview by Brice Ammar-Khodja, December 6, 2022, and October 14, 2022.

Virginie Luc, De La Nature À L'œuvre, Paris, Ulmer, 2014.

Patrice Loubier, “Un Art À Fleur De Réel: Considérations Sur L’Action Furtive.” Inter 81, no. 81, 2002, 12–17.

María Puig de la Bellacasa, Matters of Care: Speculative Ethics in More Than Human Worlds. Posthumanities, Minneapolis: University of Minnesota Press, 41.

Juan Francisco Salazar, Céline Granjou, Matthew Kearnes, Anna Krzywoszynska, Manuel Tironi, eds., Thinking with Soils: Material Politics and Social Theory, London, UK, Bloomsbury Academic, 2020.

Robert Leo Smith, Ecology and Field Biology, 4th ed., New York, Harper and Row, 1990.

Aknowledgements

I would like to thank the Canada Excellence Research Chair (CERC) on Smart, Sustainable and Resilient Communities and Cities at Concordia University, the Concordia Sustainability Action Fund (SAF), the research-creation international network Hexagram, and the Haute École des Arts du Rhin (France) and the Arrondissement Le Plateau-Mont-Royal for their support and funding. Also, I would like to thank the Concordia University Research Chair in Critical Practices in Materials and Materiality, the Milieux Institute, the Topological Media Lab, and the EnsAD, EnsadLab’s Reflective Interaction Research Group for providing conceptual and technical support. Finally, I am thankful to the citizen association Les Amis du Champ des Possibles, the Arrondissement Le Plateau-Mont-Royal and artist Philippe Vandal for making this collaborative project possible.

Author’s Biography

Brice Ammar Khodja is an artist, graphic designer, and Ph.D. student based in Montreal and Paris. His work examines active materials, residual matter, and low-technologies to explore the socio-environmental and political interconnections pertaining to materiality and visual information.

He is currently pursuing a thesis jointly supervised in Concordia University – Montreal (Individualized Program) and EnsAD, EnsadLab – Paris (Reflective Interaction research group, SACRe program).

He is a member of Chaire Arts & Sciences, Concordia University Research Chair in Critical Practices in Materials and Materiality, Speculative Life Research Cluster (Milieux Institute), Centre for Sensory Studies, and Concordia’s Canada Excellence Research Chair in Smart, Sustainable and Resilient Communities and Cities. Brice is a current member of the international research-creation network Hexagram.

Co-director of the typography magazine Pied de Mouche, Brice Ammar-Khodja, creates workshops and educational tools for the general public.

His works have been exhibited at Ars Electronica (Linz, Austria), MUTEK (Montréal, Canada), Centre Pompidou (Paris, France), Biennale internationale du Design, la Cité internationale des Arts (Saint-Étienne, France), V2_Institute for Unstable Media (Rotterdam, Netherlands), Musée historique de la Ville de Strasbourg (Strasbourg, France) and Second Internatinal Print Biennale (Yerevan, Armenia). 

The “Bichi” Project Symbiotic Food Networks & the Alchemist Kitchen

Pat Badani

Independent Artist/Researcher @ ISEA International Board Argentina/Canada/USA pat.badani@gmail.com

Abstract

The author uses symbiosis as metaphor and tool to create food-related projects that critically examine the anthropogenically induced impact of global warming on food chains. She argues that "art-through-food" projects promote an alternative worldview informed by the utopian premise that art can facilitate reflexivity and influence behavior to prevent future massive starvation by safeguarding the future of food. Three projects reveal the workings of energy flows in ecosystems to reimagine hybrid relations with living matter and with systems that are biological, technological, social, and political. They establish sustenance networks in which symbiotic interactions between several actors form the basis for a reevaluation of our human relationships to the planet and the fungal, plant, and animal agents within it. Based on assemblages with multiplicities that cofunction via a viral logic of contagion, projects explore aspects of symbiosis in human/plant/bacterial consortiums (the Bichi project), as well as symbiotic intelligence combining computing with neural networks (@Comestiblemealplan). Informed by research linking various scientific fields with art and technology, projects explore networks of relations between entities tangled in interdependence and involving parasitism, mutualism, adaptation, and resilience. Congruent with a larger systems approach in which symbiosis is the core principle replacing an essentialist conception of individuality, hybrid works contribute research and knowledge production with the potential to assess and generate affective insights and acts in the world.

Keywords

Energy flows, ecosystems, art-through-food, future food, sustenance networks, symbiotic consortium, consilience, speculative fictions, utopia, climate emergencies, practice-as-research.

DOI

10.69564/ISEA2023-41-full-Badani-The-Bichi-Project

Art-through-Food - Symbiotic consortiums

The world is made up of complex and sometimes unmappable landscapes of interacting biological, climatic, social, economic, and political forces. Interconnected global systems have been created to produce, reproduce, and consume our material environments—systems in which not all partners have deriv