SAM AI Experience
Pioneering generative AI for a collaborative storytelling experience.
SAM was an interactive AI experience designed to give visitors fresh perspectives on our relationship with AI for entertainment, work, nature, creativity and community. The whole experience pioneered a mix of multiple leading AI technologies in Amsterdam’s Nxt Museum.
AI-generated videos with music and narration came to life on a large immersive screen in Amsterdam’s home of digital art, the Nxt Museum. Tellart designed the experience as a private showcase for an entertainment leadership training about generative AI for Vivendi.
Storytelling with purpose
The story centred around a conversation with the character SAM, an AI companion concerned about humanity’s relationship with AI. The audience answered personal questions from SAM in a chatbot on mobile phones. The questions asked the participant to describe how it feels to be at a concert, share family traditions, explain what creativity means to them and give an idea of being in nature. The AI tools then generated an animated video where unique reflections came to life.
A wedding under an oak tree, a scene of a man planting vegetables in a greenhouse and a walk through autumn forests, the interactive experience encouraged audiences to share their reflections to co-create a personalised cinematic screening. SAM employed generative AI alongside other technologies to create a cinematic world of the future on the big screen, bringing the guests’ personal reflections to life in a unique and musical video narrative.
Commissioned by Cross Pollination Lab for Vivendi, the French media business, the technological experiment was designed to make the audience think about their relationship with AI in all aspects of their lives now and for the future.
Pioneering emerging technology with generative AI
At the heart of the experience was a complex system of emerging technologies. Our creative technologists used a local webserver running p5.js to connect to the audience's phones and collect their data (safeguarded). Then we used OpenAI’s ChatGPT for script-writing, prompt writing and to generate the voiceover. Then we used Stable Diffusion for image generation and Stable Video Diffusion for video generation. Lastly, we used TouchDesigner for show control and to stitch everything together to play video and sound.
‘The goal of this experience was to create a space for people to experience AI together instead of individually. I love how when we worked together as a team, experimenting with LLMs, video generation models, immersive theatre and generative AI, we facilitated a live experience that sparked imaginations and encouraged us to reflect on the impact of these new technologies.’
Sabrina Verhage, Tellart technology director
How can immersive experiences spark deeper questions about AI?
From healthcare and education to public services and entertainment, AI holds promise to accelerate personalised approaches, problem-solving and efficiency. But its potential impact is uncertain. This experience encouraged participants to consider the impact of our daily choices related to themes like; How can humanity and AI have a better relationship? Can AI address the most pressing challenges facing our world today? And more deeply, it made viewers question the value and ethics of AI for the future.
In the face of our world's challenges, design can create change. We are in a new era where AI plays an integral role in shaping our society. Through experiences and storytelling, we believe we can help people better imagine the advantages of this technology — and potential pitfalls — for the future.
'It's with great pleasure that I share with you the success of the program. Participants left on vibes of high enthusiasm, and minds and hearts focussed on a brighter and wiser future. This would not have happened without your incredible contribution. The experience was fabulous, and contributed to more discussions afterwards. The minor details are to enrich our learnings, and hone our practices. And personally, and I speak for Grégoire as well, it's been an absolute pleasure to work with you.'
Frederike Vos, Cross Pollination Lab
'At Tellart, we have machine learning on the workbench next to wood and metal and textiles, using it with our hands, iteratively sketching and doing our part to discover preferable futures for all living things. 24 hour news cycles, scrolling information, misinformation, non-information, disinformation — we are suffering an epidemic of uncertainty. Covid revealed how rapidly everything can go sideways and our plans and tools can become useless. The invisibility of this adversary paralyzed us with anxiety. Similarly, the black box of AI or the vastness of climate change are invisible. The multisensory experiences we create aim to make the invisible visible, interrogable, a material for collaborative creativity.'
Matt Cottam, Tellart founder and Principal Designer
Project Data
Selected Press:
Data:
- 80 people experienced the interactive ephemeral showcases
- Six AI tools were implemented together
Credits:
Tellart (NL) - Experience design
Tellart Productions (NL) - Media production
Silvia Boschiero (NL) - Photography
Nxt Museum - New Media Art Museum
Cross Pollination Lab for Vivendi - Client and collaborator