The letters falling down stop at the contours of your silhouette, forming words captured by the speech recognition system built into the browser.

The work refers to the Text Rain installation by Romy Achituv and Camille Utterback, but instead of a pre-written poem, it uses the user’s language – words spoken here and now.

Rain turns language into sensual contact – words fall on the body like drops, often disappearing before they can be read.

“Rain”

The second part of “Triptych on the Weight of Words,” “Rain,” references Romy Achituva and Camille Utterback’s 1999 interactive installation “Text Rain”. In Achituva and Utterback’s installation, falling letters come to rest on the edges of the bodies of those present in the monitored space, forming the words of Evan Zimroth’s poem “Talk, You” and, in a sense, embodying the reading process. However, “Rain,” like the first part of “Triptych,” does not use a pre-written text, relying instead on AI and a speech recognition system built into browsers that tracks the utterances of people using Miniatura. The rudimentary body contour detection system used in the original installation (based on a simple mechanism assuming that everything in the space monitored by the camera that is darker than the brightly lit wall in the background is an obstacle stopping the letters from falling) has also been replaced by AI and a basic image segmentation mechanism that extracts body contours from the background, but does not require special arrangement and lighting of the space, or careful selection of the background.

our book on interactive installations includes my text discussing “Text Rain” – we can use fragments or “reprint” the whole thing

“Triptych on the Weight of Words” combines three Miniatures into a larger whole that examines and plays with the titular issue, and is based on two sources of inspiration.

The first is the (meta?) metaphor of speaking about language, and particularly about words, which, when described, are given physical properties, especially weight and mass. Such metaphors are produced by both literature and everyday language (perhaps the tendency to carve words considered important into heavy, massive objects—tombstones, columns, monument pedestals, etc.—and other similar activities are also an emanation of the same phenomenon). Giving words physical attributes is a cross-cultural practice and gives the impression of a natural human predisposition that does not require special training, coming with the development of linguistic competence. The power of such a practice seems to correspond to the importance of the brain system of intuitive physics with which evolution has endowed humans (and with concepts of evolutionary aesthetics in general)—in this case, it would be a beautiful example of the culture-forming potential of the side effects of natural selection mechanisms. Falling and sinking words, subjected to gravity and the resistance of dense media in which this falling takes place, or simply encountering hard obstacles, anchor the reading process in the parameters of the physical world.

The second source of inspiration concerns the physical nature of information (and thus the “content” of language and words) and the mass/energy/information (M/E/I) equivalence hypothesis. According to the M/E/I hypothesis, information is a physical quantity that can be associated with the energy necessary to change or erase a single bit of information. In very simple terms, the hypothesis can be explained by invoking the so-called Landauer’s rule (a physical principle stating that erasing a single bit of information requires at least a certain minimum energy, which is proportional to the temperature of the system—erasing information therefore entails an increase in entropy, e.g., at room temperature, erasing a bit “costs” 3 x 10 to the power of -21 joules). Landauer’s rule determines the theoretical efficiency of every conceivable computer, but above all, it relates information to the energy values ​​required to change it. By employing a few more formulas from thermodynamics, Shannon’s information theory, and remembering the matter-energy equivalence rule (E = mc2), we can calculate the mass of a bit of information at room temperature (300°K, or 26.85°C), which is 3.19 x 10 to the power of -38 kg. This is not a large mass, and experimental confirmation (or refutation) of the mass/energy/information equivalence hypothesis is currently not possible. The M/E/I hypothesis is widely discussed among researchers exploring the intersection of physics and biology, for example, those attempting to define the characteristics of life—there are concepts that emphasize the exceptional energy efficiency of some life processes, approaching the barrier defined by Landauer. James Clerk Maxwell’s famous thought experiment (“Maxwell’s demon”), in its modern version, also expands on the demon’s task of acquiring and processing information about the world and the associated energy expenditure.