AI Generated Storytelling Aides
- jcdabney
- Mar 18
- 2 min read
In 2020 I started dream journaling as a way to keep track of some of the trends and consistencies that I was starting to notice in the worlds I would visit. It started out as simply logging what had happened, and eventually I began converting these experiences into these short narratives describing the dreams. This would give inspire me to be creative first thing in the morning (afternoon) when it was still fresh in my mind. Using a different part of the brain for processing also had the effect of helping me to remember what had happened even better. This process would help me to eventually build enough of a base to start writing the fiction novel I am very slowly working on.
For a long time I figured I would possibly publish these narratives as a collection of short stories if I ever felt that they were interesting enough and if I could built up a large enough database of good ones. With the advent of modern generative AI video, however, that has changed. Now I can more easily create a visual aide for these stories in a way that would have been impossible in the past.
My general workflow is to hand write the stories out, read them aloud into google docs, edit, and then narrate them to create a wav file voiceover. I then divide that total time of the narration into 8 second clips to get the total number of clips I must generate. I then use a custom prompt that I am still developing to have an LLM model create me that number of scene descriptions that I can then feed into a text to video generative AI program.
This cost would add up with any of the paid AI generators, so I use ComfyUI and render them on my home computer. It takes about 45 minutes of processing for each minute of rendered content. My favorite so far is LTX2, but I would use some of the more powerful modern ones if I had a fast enough computer.
I figured the combination of the surreality of the dreamstate with the strange generations of AI video would make a nice combination, and so far I am pleased with the results. I'm not spending a tremendous amount of time on each one, but it is a process of refinement. I kind of like the fact that I don't know how it's going to turn out until I look at my computer on the next workday and see what has been rendered. Perhaps once I get enough of these clips made, I will make a video tutorial on how I do it. But for now I have to perfect the process a little more, before I feel like I can teach others.
Comments