한국어 | English

8PM and the Cat: AI-driven VR film

C# | Unity Engine | Gen AI
Playthrough Video
GIF 1
GIF 2
GIF 3

Overview

Funded by: AT Lab at KArts
Targeted Submission: Venice Immersive Film Festival 2025

Synopsis

At 8PM, when time halts after the loss of a loved one, a quiet room, a sleeping cat, and lingering memories unfold like panels in a dreamlike comic.

After losing his partner Mina in the Itaewon crowd crush, cartoonist Haru’s world freezes at 8PM. He drifts between two realities—one where she still arrives home, and one where she never will. He draws her endlessly, speaks to her endlessly. In this generative VR experience, monologues and visuals emerge in real time, each moment fleeting, each encounter singular. The audience steps into Haru’s stillness, gently bearing witness to the enduring weight of her absence.

Project Overview

8PM and the Cat is a 13-minute, AI-driven VR narrative offering a personalized and emotionally adaptive experience. Through the interwoven perspectives of Haru and Mina, users navigate the tender space between longing and healing—where time stands still, but memories continue to breathe.

My Role

Development Environments and Tools

Engine: Unity (C#)
Platform: Meta Quest
Collaboration: Plastic SCM
Gen AI: GPT-4o (OpenAI API), ElevenLabs (custom voice cloning)
Plugins: Meta SDK
Languages Supported: English & Korean

Prototype

💡 How can AI shape interactive storytelling in XR?

This question guided the early development of 8PM and the Cat, a 13-minute AI-driven VR narrative about a man named Haru and his partner Mina. After losing Mina, Haru’s world freezes at 8PM—the hour they always spent together.

The story is inspired by the idea that just as ants live in a two-dimensional world, unaware of the third, we humans may not fully perceive the fourth dimension—time. In this view, past, present, and future may coexist, and those we’ve lost might still be with us in ways we can’t yet perceive. We wanted this narrative to go beyond one man’s grief and speak to a collective human experience of loss, memory, and connection. To bring this idea to life in an immersive and deeply personal way, we turned to AI.

As both prototyper and developer, I spent several months experimenting with how generative AI could help bring this vision to life—crafting a story that responds and adapts to each user. I explored ways to generate voice narration, images, and room textures in real time, and tested how the story could change depending on what the user types or chooses.

Two key prototypes emerged from this phase: Gaze to Voice and StyleShift. Each explored a different technique for real-time AI storytelling. Together, they not only helped refine the final system used in 8PM and the Cat, but also revealed how AI can unlock new emotional and spatial dimensions in XR narrative design.

Gaze to Voice

#AI

Gaze to Voice:
Real-Time Story
StyleShift

#AI

StyleShift: Real-Time Texture Generation
with Stable Diffusion

System Structure

Each viewer’s story is different—images and monologues shift dynamically based on user input, making the experience deeply personal and reflective.

🧑‍💻 User Input ➝ 🎨 GPT-4o Image Generation

Via a virtual keyboard, users are prompted with reflective questions such as:
“What do you usually see outside your window at 8PM?”
The response is used to generate real-time imagery that reflects the user’s emotional world.

🗂️ Pre-defined Prompts ➝ 🎨 GPT-4o Image Generation

In the experience, both the 2D visuals and the room’s textures change in real time based on predefined emotional prompts. Since Haru is a comic artist, the images are styled like illustrated panels. At the same time, the entire room environment transforms to match the mood expressed in the prompt. For example, if the prompt describes a tense or uneasy emotional state, both the visual content and the space around the user shift to reflect that atmosphere—creating a fully immersive emotional experience.

🗂️ Pre-defined Prompts ➝ 🧠 GPT Narration ➝ 🔊 ElevenLabs Voice ➝ 🎧 Immersive Playback

Emotional voice-over monologues are dynamically generated by GPT and instantly synthesized into natural speech using ElevenLabs—based on a custom-trained voice model of the main characters.

Core Features