science outreach – software and interactive kiosks

I design interactive kiosks for geology dissemination, focusing on concept ideation, software coding, and graphic design. The proven effectiveness of interactive media in science communication has shifted visitor expectations toward hands-on experiences and active engagement, transforming them from passive observers to active participants. This is why I am passionate about utilizing computer vision approaches!

A notable example of this approach is the exhibit “Quando Pramollo Stava all’Equatore,” co-organized with Corrado Venturini in Pontebba, Italy. For this exhibit, I coded software that employs RGBD cameras, combining depth (D) and color (RGB) data for real-time motion sensing. This technology enables visitors to directly interact with the exhibit, such as engaging with a 300-million-year-old amphibian projected on a wall simply by moving within the space.

In this installation I designed, the room becomes an immersive world set in the Carboniferous. Prehistoric organisms, from the giant amphibian Eryops to the griffinfly Meganeura and a eurypterid, react to the visitor position within the room. 

Specifically, the visitor moves within a room where an animated scene reconstructing a Carboniferous river environment is projected onto the walls and floor. Based on the visitor position, prehistoric organisms react: a large amphibian (Eryops) opens its jaws as the visitor approaches, a dragonfly-like insect (Meganeura) takes flight when physically touched, and a eurypterid assumes a defensive stance if the visitor’s feet get too close.

Similarly, the software I have developed allows to dive into seas from 300 million years ago. An animated underwater scene is projected onto a large wall, where visitors see their silhouette superimposed on the ancient marine environment. This enables them to “swim” in the Carboniferous seas, interacting with organisms such as trilobites (which roll up when touched!), crinoids, gastropods, and nautiloids.  

The software I have developed allows visitors to see themselves projected into a Carboniferous sea, interacting with trilobites, nautiloids, crinoids, and other organisms from 300 million years ago. 

The origin of my computer vision approach dates back to 2007, marked by the organization of the project “From Trilobites to Man: 500 Million Years through Geopark Naturtejo Meseta Meridional.” The exhibition, co-organized with Carlos Neto de Carvalho, was hosted at the Natural History Museum of Lesvos in Greece, where it achieved remarkable success, attracting 35,000 visitors in just three months! It aimed to capture the active tourism spirit and geologic heritage of UNESCO Geopark Naturtejo, which traditional media struggle to convey. Sites like Penha Garcia and Portas do Rodão, known for outdoor activities and trace fossils, require a more immersive approach to engage visitors.

To achieve this, I developed a computer vision system integrating a motion sensing device and monitor/projector outputs to create an augmented reality experience. The visitor body served as the input device, allowing real-time interaction with virtual geologic landscapes and activities, enhancing visitor engagement significantly.

< The visitor faces a challenge: he has to eat as much as possible…but to do it he must behave like a trilobite!

In the “Cruziana Maker” kiosk visitors stood on a platform, facing a display of the Ordovician seafloor, optionally wearing gloves mimicking trilobite limbs. The kiosk engaged visitors with the task: “Eat like a trilobite!” By moving their arms, visitors simulated trilobite feeding and burrowing, with real-time scratch marks appearing on the screen. This interaction educated visitors on trilobite behaviors and the formation of the trace fossil Cruziana.

Additionally, a competitive aspect encouraged visitors to increase their scores by “eating” more. Scores were depicted through a trilobite avatar, evolving from a protaspid (larval) stage to holaspid (adult) stage as scores rise. This kiosk met great success in the “Trilobites to Man” exhibition. People of all ages were swinging the limbs like trilobites!

When visitors move their arms, scratch marks appear on the screen. As the visitor “eats” more, the trilobite grows. If you’re fast enough, you’ll witness its transformation from protaspid to meraspid.

The “Kayaking River Tejo” kiosk featured a real kayak and paddle. The software I coded detected paddle movements, allowing visitors to virtually kayak in the Geopark. The screen displayed a first-person view that responded to paddling, encouraging active exploration of the river.

As visitors concluded the exhibit, they encountered a final interactive experience. Entering the target area triggers a projection of a giant trilobite on a wall, producing Diplichnites-like trackways as it moved. The trilobite movements were synchronized with the visitor actions, enabling real-time interaction. Visitors can engage with the trilobite by moving their hands or by following it, enhancing their connection to ancient life.