AI

AI workflow

  • Dialogue Proto repository

    • linear sample
    • BTree sample

  • merge ↑ to kwik

  • audio services

  • voice overs - [ ] ElevenLabs - [ ] Play.ht - [ ] Amazon Polly - [ ] Google Wavenet

  • SFX

    • Freesound.org
    • Unity Asset Store
    • Humble Bundle SFX packs
    • Audo.ai

  • Music

    • YouTube Audio Library
    • Pixabay Music
    • AIVA
    • Soundraw
    • Mubert

  • fountain script

    1. conversion to model

      local model = {
  name = "cabinScene",
  components = {
    layers = {
      "bg_cabin.png",
      "elara_neutral.png",
      "elara_scared.png",
      "elara_determined.png",
      "elara_happy.png",
      "lumin_seed_normal.png",
      "lumin_seed_glowing.png",
      "lumin_seed_dim.png",
      "lumin_seed_pulsing.png",
      "door_open.png",
      "door_closed.png",
      "door_broken.png",
      "chest_locked.png",
      "chest_unlocked.png",
      "chest_open.png",
      "chest_empty.png"
    },
    audios = {
      long = {},
      short = {
        "door_rattle",
        "door_creak",
        "key_turn",
        "chest_open",
        "door_slam"
      }
    },
    groups = {},
    timers = {},
    variables = {
      "hasKey",
      "hasMagicKey",
      "chestLooted"
    },
    joints = {},
    page = {}
  },
  -- Actions only
  commands = {
    "setHasKey",
  },
}

    2. conversion to dialogue

      local sceneDialogue = {
    { type = "narration", text = "You stand before an old cabin in the woods." },
    { type = "show", what = "cabin_door" },
    { type = "object_state", object = "cabin_door", state = "closed" },

    { type = "narration", text = "The door is firmly closed. You try the handle..." },
    { type = "sfx", sound = "door_rattle" },

    -- Conditional: Door opens only if player has a key (string reference)
    { type = "object_state_conditional", object = "cabin_door", state = "open", condition = "hasKey" },
    { type = "sfx", sound = "door_creak" },

    { type = "narration", text = "The door creaks open, revealing a dark interior." },
    { type = "show", what = "elara" },
    { type = "emotion", character = "elara", state = "determined" },

    { type = "narration", text = "On a dusty table, you spot the Lumin Seed." },
    { type = "show", what = "lumin_seed" },
    { type = "object_state", object = "lumin_seed", state = "glowing" },

    { type = "narration", text = "In the corner, an old chest catches your eye." },
    { type = "show", what = "chest" },
    { type = "object_state", object = "chest", state = "locked" },

    { type = "narration", text = "The chest is locked. You search for a key..." },

    -- Set flag when key is found (string reference)
    { type = "custom", action = "setHasKey" },

    { type = "sfx", sound = "key_turn" },
    { type = "object_state", object = "chest", state = "unlocked" },
    { type = "sfx", sound = "chest_open" },

    -- Conditional: chest state depends on whether it was already looted (string reference)
    { type = "object_state_conditional", object = "chest", state = "open", condition = "chestNotLooted" },

    { type = "narration", text = "The chest is empty! Someone got here first." },
    { type = "emotion", character = "elara", state = "scared" },
    { type = "object_state", object = "chest", state = "empty" },

    { type = "sfx", sound = "door_slam" },
    { type = "object_state", object = "cabin_door", state = "closed" },
    { type = "narration", text = "The door slams shut behind you! You're trapped!" },

    { type = "choice", options = {
        "Try to force the door open",
        "Search for another way out",
        "Investigate the strange markings"
    }}
}
     ```

Spec1 - assets generation

images/audio/videos are created by the model, the prompts for generative AI to create files are also generated from the fountain script

  • Photoshop
  • generative AI
    • firefly
    • elventhlab
    • nano banana/sara etc

Spec2 - composition

  • inial states

    • App/{BOOk}/components/index.lua

      • object states are implemented as animations or can be as spritehseet
      • sfx is to play audio

    • App/{BOOk}/components/{PAGE}/layers/*.lua

    • App/{BOOk}/commands/{PAGE}/*.lua

  • sceneDialog triggers animations or sprites:play() or play audio etc

Spec3 - more interactions/animaions/replacements or physics?

should we add a type: button to dialogue?

{ type = "object_buttone", object = "chest", action = "onTap" },

it means index.lua ’s layer have a button class

 {chest={class={"button, "animation"}}}

each entry of dialogue can be matched to one of kwik components

  • object(image) + class(animation, button, spritehseet, physics body)

so a class with props is a pattern?

Spec4 narration flows as a key frame

each narration brings new states to objects or sfx(audio) etc

object states can be changed by an action which calls objett's class function ex animation:play(), spriets:play()

Spec5 model.md

create a model document that extracts all the necessary technical elements from the Fountain script for implementation purposes.

Key Sections:

  1. Backgrounds - All scene locations needed

  2. Characters - Elara with all her emotional states and required sprite images

  3. Props/Objects - Detailed specifications for:

    • Lumin Seed (with glowing state)
    • Cabin Door (locked, open, sealed states)
    • Iron Key & Brass Key
    • Old Chest (locked, unlocked, open states)
    • Boarded Window
    • Arcane Markings

  4. Sound Effects - All SFX with file names and triggers

  5. Music/Ambience - Background audio tracks

  6. Game Flags/Variables - Boolean flags for game logic

  7. Dialogue/Voice-Over Lines - All of Elara’s lines with file names and triggers

  8. Scene Flow - Step-by-step event sequence

  9. Player Choices - Three ending options with next scene references

  10. Implementation Notes - Technical suggestions for effects and transitions

This document bridges the creative screenplay (Fountain) with the technical implementation (Lua), providing everything a developer or artist needs to build the scene while keeping the narrative separate from the technical specifications.

Spec5 behavior tree

https://github.com/0xabad/behavior_tree/

->
|    [Show cabin exterior]
|    [Elara approaches]
|    ->
|    |    [Try cabin door]
|    |    ?
|    |    |    ->
|    |    |    |    (has iron key)
|    |    |    |    [Use iron key on door]
|    |    |    |    [Play door creak SFX]
|    |    |    |    [Open door]
|    |    |    ->
|    |    |    |    !(has iron key)
|    |    |    |    [Play door rattle SFX]
|    |    |    |    ?
|    |    |    |    |    ->
|    |    |    |    |    |    (key visible near doorstep)
|    |    |    |    |    |    [Highlight iron key]
|    |    |    |    |    |    [Find iron key]
|    |    |    |    |    |    [Set has iron key]
|    |    |    |    |    ->
|    |    |    |    |    |    !(key visible near doorstep)
|    |    |    |    |    |    [Search near door]
|    |    |    |    |    |    [Reveal iron key]
|    ->
|    |    (is door open)
|    |    [Transition to cabin interior]
|    |    [Play ambient cabin music]
|    |    [Show lumin seed glowing]
|    |    [Elara: "The Lumin Seed... I found it."]
|    ->
|    |    [Show chest in corner]
|    |    [Try chest]
|    |    ?
|    |    |    ->
|    |    |    |    (has brass key)
|    |    |    |    [Use brass key on chest]
|    |    |    |    [Play lock click SFX]
|    |    |    |    [Play chest creak SFX]
|    |    |    |    [Open chest]
|    |    |    ->
|    |    |    |    !(has brass key)
|    |    |    |    [Elara: "There has to be a key somewhere around here..."]
|    |    |    |    ?
|    |    |    |    |    ->
|    |    |    |    |    |    (searched floorboard)
|    |    |    |    |    |    [Highlight loose floorboard]
|    |    |    |    |    ->
|    |    |    |    |    |    !(searched floorboard)
|    |    |    |    |    |    [Search room]
|    |    |    |    |    |    [Find loose floorboard]
|    |    |    |    |    |    [Reveal brass key]
|    |    |    |    |    |    [Pick up brass key]
|    |    |    |    |    |    [Set has brass key]
|    ->
|    |    (is chest open)
|    |    [Reveal empty chest]
|    |    [Elara change to shocked state]
|    |    [Elara: "No... it can't be!"]
|    |    [Elara: "Someone got here first."]
|    ->
|    |    [Play door slam SFX]
|    |    [Door slams shut]
|    |    [Seal door]
|    |    [Set is trapped]
|    |    [Screen shake effect]
|    |    [Play tension music]
|    |    [Elara change to panicking state]
|    |    [Elara: "What?! No!"]
|    ->
|    |    [Elara tries door]
|    |    [Door won't open]
|    |    [Elara surveys room]
|    |    ?
|    |    |    ->
|    |    |    |    (player choice force door)
|    |    |    |    [Focus on cabin door]
|    |    |    |    [Present choice: "Try to force the door open"]
|    |    |    |    [Go to force door scene]
|    |    |    ->
|    |    |    |    (player choice window)
|    |    |    |    [Focus on boarded window]
|    |    |    |    [Present choice: "Search for another way out"]
|    |    |    |    [Go to window escape scene]
|    |    |    ->
|    |    |    |    (player choice markings)
|    |    |    |    [Focus on arcane markings]
|    |    |    |    [Present choice: "Investigate the strange markings"]
|    |    |    |    [Go to arcane markings scene]

Creating assets

https://deepai.org/api-docs

Adobe

PS generative fill

Express Add On ⭐️

https://firefly.adobe.com/

Pr

Podcast

Audio

Misc

Text to audio

Free Sound Effects

Adobe Audition

How to organize

Whisper

Video

RunwayML Gen2 ⭐️

Stable Diffusion

Automatic1111

DreamStudio

ControlNet

https://github.com/Mikubill/sd-webui-controlnet

Cloud Providers

Machine Specs