This project comprises a series of six interactive exhibits developed for the travelling exhibition Alles im Angebot (Everything on Offer). The exhibition explores how hate and disinformation spread online and affect society. Each exhibit focuses on one specific aspect of this dynamic and communicates it through hands-on interaction, physical hardware, and different sensory modalities.

The exhibition is designed to look like a supermarket — a metaphor for the digital economy: the data generated through everyday use of digital services is itself a product, bought and sold across many industries.

My role covered the development and implementation of all six media stations, both on the software and the hardware side.

1 — Deepfake It

Photo via Kay Herschelmann.

This installation lets visitors experience face swapping in real time and then confronts them with a second step: a simulated deepfake scenario. Deepfaking here means inserting a person's face into intimate or sexualized content without consent.

For privacy, the station is placed inside a changing-room-like booth with a curtain. Once the experience starts, the webcam activates and enters live face-swap mode. Visitors can choose from several well-known celebrities and see the swap directly on the camera feed. After this playful entry point, a call-to-action invites them to continue. The system captures a frame of their face and inserts it into a prepared video sequence, showing how quickly this technology can be used against someone. The result is a strong emotional contrast: the same technology feels harmless and entertaining at first, then immediately intimidating once the context changes.

Photo via my iPhone

Hardware

• PC with AMD Ryzen 5 5600
• NVIDIA RTX 5060

Software

Custom face-swapping engine inspired by roop-unleashed.

A word on the technical stack

Choosing the right approach was the biggest challenge. The first iteration of the concept was to swap the visitor's face into two matching target images: a clothed version and a nude version of the same person for the "shocker" reveal. In practice, this quickly failed. Face swapping is most convincing when source and target are visually similar, so the system would have needed a massive image dataset with broad coverage across age, skin tone, body type, hair, and facial structure.

synthetic dataset matrix

As an early exploration step, I built a ComfyUI workflow that automatically varied its prompts and generated more than 2,000 synthetic target images. I then used smaller matching models and face embeddings to select the closest target for each visitor. Even at that scale the result was not robust enough: the variation of real visitors is simply too high.

A second iteration improved diversity by using a neutral scene prompt combined with random identity vectors injected via a FaceID adapter. This produced much better latent diversity, but the scaling problem remained. To reach meaningful coverage I would have needed roughly 5,000+ images per scenario; with clothed and nude variants across multiple scenes that grows to 20,000+ generated images, with no reliable way to benchmark real-world quality.

I also explored the possibility of using the live webcam feed for the "shocker" reveal by computationally "undressing" the visitor in real time. For this, I experimented with real-time censoring: processing the webcam video to detect and bound the visitor's clothing, segmenting it via lightweight AI models, and trying various ways to blur, pixelate, and recolor the region to achieve the look of censored nudity.

realtime censoring examples - screenshots taken from a fast preview video

Even though the results were promising in some areas, the limitations quickly became apparent. Masking out the clothes often resulted in unnatural, non-human shapes, particularly when visitors would wear bulky items like winter jackets. Even when attempting to use the background to cover parts of the clothing, anything extending beyond the natural body shape—such as a hood—would immediately break the illusion, regardless of whether it was censored or left untouched.

So I changed course to the final concept: use live face swapping with familiar celebrity faces for the playful part, where exact visual matching is not required. For the "shocker" use a pre-recorded video (just as real deepfakes do) and swap the visitor's face into it.

The source video shows a woman in a bikini emerging from a pool. Applying the segmentation and censoring techniques from the earlier experiments, I wrote a Python script that processes the frames, isolates the bikini area, and pixelates it to evoke perceived nudity without actually generating explicit content. This manipulation was intentionally kept visible. Using AI to fully remove the clothing would have reproduced the exact behavior this installation is meant to critique.

2 — Algorithmische Verstärkung

Photo via Landesbibliothekszentrum Rheinland-Pfalz.

This exhibit shows in a simple but effective way how recommendation algorithms shape what we see online. A screen displays a shelf of supermarket products — at first calm, unremarkable, slightly desaturated. Next to the screen sits a large physical dial. Turning it amplifies the algorithm: as the dial increases, healthy products like organic apples are desaturated further, while unhealthy ones — vapes, energy drinks, convenience food — move to the foreground. Their colors become more vivid, animations draw the eye to them relentlessly, until only the promoted products are visible.

Hardware

• Raspberry Pi
• ADS1115 ADC
• 10k potentiometer

Software

A custom Python application reads the potentiometer through the ADS1115 ADC and uses its value to scrub through a pre-rendered video in real time, displaying raw decoded frames directly from memory. OpenCV handles video decoding and frame display. A tkinter polling loop reads the ADC continuously without blocking the OpenCV window. The raw ADC values are filtered and stabilized through a median filter, an adaptive exponential moving average, and time-based debouncing.

A word on video codecs and realtime scrubbing

Modern video codecs are optimized for size and quality — and they excel at that. But this comes at a cost: formats like H.264 or H.265 compress video by storing only the differences between frames, anchored to periodic keyframes called I-frames. Scrubbing to an arbitrary frame means decoding everything since the last keyframe first. Scrubbing backwards is even more expensive. For a standard video player this is fine — nobody expects instant frame-perfect random access. But for an interactive installation where a physical knob should map 1:1 to a video frame in real time, this model simply doesn't work.

There are codecs that address this by encoding every frame as a keyframe — ProRes, HAP, NotchLC, and MJPEG — but each comes with trade-offs: restrictive licensing (ProRes, NotchLC), GPU requirements (HAP), enormous file sizes, or noticeably lower quality (MJPEG).

The solution was to sidestep the codec at runtime entirely. On startup, all frames are decoded once and held in RAM as raw image data. At runtime there is no decoding, no seeking, no buffering — the potentiometer value becomes an index, the index retrieves a single frame from the array, and it is displayed immediately.

The only real constraint is the memory budget, but it is straightforward to calculate:

1920 x 1080, 200 frames (8-bit BGR)

Raw frame size:
  1920 × 1080 × 3 = 6,220,800 bytes/frame

In MiB:
  6,220,800 / 1,048,576 ≈ 5.93 MiB/frame

For 200 frames:
  5.93 × 200 ≈ 1,186 MiB ≈ 1.16 GiB

A 200-frame full-HD video occupies roughly 1.16 GiB in memory. On a Raspberry Pi 5 with 8 GB RAM, that leaves enough headroom for the OS, services, and application. Even a 4K version of the same video would fit at around 4.64 GiB.

3 — Das Erleben der Anderen

Photo via Kay Herschelmann.

This installation shows video testimonials of people whose lives were directly affected by digital violence. Next to the screen stands a clothes rack with several T-shirts, each printed with a statement that mirrors the subject of one of the videos. The visitor picks a shirt and hangs it on a hook built into the installation — the corresponding video starts playing automatically.

Photo via my iPhone.

Hardware

• Raspberry Pi
• Webcam

Software

A custom Python application continuously monitors the webcam feed for QR codes. When a code is detected it is matched to the corresponding video, which then starts playing.

A word on identifying the T-shirts

The initial approach was NFC or RFID — small chips embedded in the hangers, read by a sensor in the hook. Both technologies turned out to be unsuitable for this context: they are highly sensitive to the distance and rotational alignment between chip and reader, have a very short reliable read range, and are easily disrupted by the metal hook and surrounding hardware. Any misalignment, even just a slightly crooked hanger, would cause a missed read.

A 1-Wire ID chip was considered as a fallback: a small chip in the hanger making direct electrical contact with a pin in the hook. The problem is that the contact point is exposed and under constant mechanical stress from visitors hanging and removing shirts repeatedly, making it fragile and prone to failure in an exhibition context.

The solution was to mount a small webcam inside the hook and print a QR code on each hanger. OpenCV allows setting a fixed manual focus, so the depth of field and code size could be tuned once and locked in — making reads fast, direction-independent, and very solid regardless of how a shirt is hung.

The software also handles edge cases gracefully: detection is debounced so a few missed frames do not interrupt playback. If the code briefly disappears and reappears within a short window, the video resumes from where it was rather than restarting — keeping the experience smooth even when something goes slightly wrong.

4 — Die Wahren Kosten

Photo via Landesbibliothekszentrum Rheinland-Pfalz.

Photo via Landesbibliothekszentrum Rheinland-Pfalz.

This installation reframes the supermarket theme to show that data and attention are the real currency of the digital economy. The station is built around a modified shop-counter scale with three custom weights. Each weight represents a different amount of daily social media consumption. When placed on the scale, the display shows how much revenue the platform generates from that level of engagement — making concrete what is usually invisible: the visitor's attention, and the data trail it leaves behind, are the actual product being sold.

Hardware

• Raspberry Pi
• Modified consumer scale
• HX711 ADC

Software

Custom Python application that reads the scale input and serves the corresponding information via a web frontend displayed on a screen.

A word on getting data out of a consumer scale

Scales that expose their weight data over USB barely exist in the consumer price range — anything with a proper data output quickly enters industrial territory and is priced accordingly. Every consumer scale on the market that has a USB port uses it purely for charging. The options were: buy an expensive industrial unit, or find a consumer scale with the right size and appearance and read the sensor directly.

Fortunately, a few weeks before this project I had been working with load cells, which made the approach straightforward. Most consumer scales use four load cells arranged in a Wheatstone bridge — one in each corner of the platform. Each load cell is a strain gauge: a thin resistive element whose resistance changes slightly as it bends under weight. In a four-cell bridge, the cells are wired so their signals add together and cancel out the noise caused by uneven load placement, giving a stable and accurate total weight regardless of where something sits on the platform.

Wheatstone bridge - Image via ANYLOAD.

The bridge produces a very small differential voltage in the millivolt range, far too small to read with a standard ADC. Scales normally include a dedicated amplifier chip — typically an HX711 — that amplifies the bridge signal and outputs a clean 24-bit digital reading. Consumer scales include this chip on their internal PCB but route the result only to their own microcontroller.

The solution was to open the scale, cut the traces between the load cell bridge and the built-in microcontroller, and wire the four load cell leads directly to an external HX711 breakout board connected to the Raspberry Pi. The original scale electronics are bypassed entirely. The Pi reads the weight via the HX711's simple two-wire interface and maps the measured value to one of the three weight thresholds.

5 — Silencing, eine Erfahrung

Photo via Landesbibliothekszentrum Rheinland-Pfalz.

This station lets visitors directly experience online silencing — the process by which individual voices are gradually suppressed, directly or indirectly, in digital spaces. The installation consists of a screen, a microphone, and headphones. The screen prompts the visitor to choose a statement and read it aloud, karaoke-style. They can hear their own voice in the headphones. As the statements progress, their voice becomes quieter and quieter while hateful comments and hostile voices grow louder, eventually drowning them out entirely — mirroring the lived experience of being silenced online.

Hardware

• Raspberry Pi
• Behringer X-Air 12 mixer
• CM-503 Low Profile Microphone
• Headphones

Software

A custom Python application manages the UI, drives the experience flow, and communicates with the audio mixer via OSC over the local network. When a statement is selected, a video with karaoke-style text begins playing. A simple configuration file defines which audio track to layer in, the timing and rate at which the visitor's voice fades out, and the timing, rate, and final volume of the hateful voices.

A word on realtime audio processing

Routing audio through software adds latency at every step — USB transfer, OS scheduling, application processing, and output buffering. For a typical USB audio interface chain, the total round-trip latency (microphone in → headphones out) sits somewhere between 20–60 ms. If you have ever heard your own voice played back with a slight delay, you know how disorienting it is: the brain expects to hear itself instantly, and even delays as short as 15–25 ms are clearly perceptible and feel deeply wrong.

Achieving sub-15 ms round-trip latency purely in software on a Raspberry Pi is not reliably possible — the OS scheduler, the USB protocol, and audio buffering all work against it.

The solution was to not process the microphone signal in software at all, but to handle it in dedicated hardware. Professional audio mixers and interfaces include an onboard DSP that runs the entire monitoring chain internally: the microphone signal is routed directly to the headphone output at near-zero latency (~1–2 ms), without ever touching the Raspberry Pi. The Pi only sends OSC commands over the network to control fader levels and mute states — the audio data itself never reaches the Pi. This sidesteps the latency problem entirely by keeping the time-critical signal path in hardware, while the application can still fade the visitor's voice in and out in software-controlled steps.

For this single-input, single-output use case the Behringer X-Air 12 with its 12 channels is significantly oversized, but it is by far the cheapest reliable option for hardware DSP on the market. Comparable DSP-based interfaces like the Universal Audio Apollo start at around 700 € and go upward from there.

6 — Wer bist du

Photo via Kay Herschelmann.

The final station continues the thread of the previous one: your data is the product. Here the installation takes the form of a supermarket checkout counter. Next to it stands a shopping cart filled with 16 mock products. Visitors pick up the hand scanner and scan the products one by one, just like a cashier would. As each item is scanned it is added to the digital receipt on the display — and the register responds with a brief, pointed comment about the choice. The effect builds gradually: with every scan the system appears to know a little more about who you are, mirroring how data collected across many small interactions accumulates into a surprisingly detailed profile.

Hardware

• Raspberry Pi
• USB barcode scanner

Software

Custom Python application that receives barcode input and passes it to a web frontend, which renders the shopping list and displays the associated comment for each product.

A word on barcode scanners

I had not worked with barcode scanners before this project, and they turned out to be much simpler than expected: a USB barcode scanner presents itself to the operating system as a standard keyboard. When a code is scanned, the device simply types the encoded string followed by Enter — no drivers, no special protocol, no library needed. The application just listens for keyboard input.

Each of the 16 mock products was assigned a unique Code 128 barcode. Scanning one sends its identifier to the application, which maps it to the product name and an associated comment. The comment is either a specific remark about that particular product or a more general observation about consumption patterns drawn from a predefined pool.

A pleasant surprise during development: barcode scanners are themselves configured by scanning barcodes. The device ships with a printed manual full of special configuration codes — scan one to toggle the beep, restrict which symbologies are accepted, change the output format, and so on. The scanner effectively programs itself through the same mechanism it uses to read data.