Can you imagine watching yourself in your favorite show even though you have never acted in real life? Or Is it possible to change the beginning and the end as you like?
Fable Studios, a San Francisco startup, just released its SHOW-1 AI tech that is able to write, produce, direct, animate, and even voice entirely new episodes of TV shows. Fable Studios did it using different diffusion models. These work on a simple method of adding and removing random noise from the data over time which can generate and reconstruct the output. One can start with an image as a random noise and gradually transform it into the required output.
Fable Studios trained their diffusion models through a dataset comprising 1200 characters and 600 background images from the TV show South Park. Their first model task was to generate single characters against a background color. Autonomous characters can be generated in the show based on one’s characteristic look, writing style, and voice. The character diffusion models allow one to create South Park characters based on their own looks through image-to-image stable diffusion.
The second model’s task was to generate a clean background that can act as a stage to let the characters interact, allowing one to design multiple scenes and scenarios. The only limitation of this model was that these produced low-resolution images. The team tackled it by using AI upscaling techniques which enhanced the image quality. Producing vector-based graphics as they don’t lose their resolution upon rescaling.
Fable Studios redefined an episode of the TV show by changing the sequence of dialogues in specific locations and runtime in order to match the original length of the episode. By using simulation data as a prompt chain, they built a story system that runs parallel to the showrunner’s system to monitor the sequence of actions and dialogues to keep the audience engaged. The voice of each character has been cloned in advance, and voice clips are generated for every new dialogue.
The data produced by the simulation acts as a creative dictionary for both the individual who is writing the initial prompt and the generating story system. It is common that even experienced story writers get stuck while writing dialogues; such problems can be overcome as the simulation provides context and data points before starting the prompt chain.
The story generation process is shared in proportions between the user, simulation, and GPT-4. The simulation produces foundational context as history, emotions, and events for the characters. It acts as an initial creative context. GPT 4 acts as the main generative engine, which merges the scenes and dialogue based on prompts it received from the user as well as the simulation.
Finally, integrating the strengths of the simulation, the user, and the AI model creates a richer interactive and engaging storytelling experience. On the contrary, personalizing the shows will lead to the loss of jobs. As AI-powered tools can now perform tasks like video editing and music composition, which were once done by human professionals, this will lead to concerns about the future of jobs in the entertainment industry.
Check out the Project and Reddit Post. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 26k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
🚀 Check Out 900+ AI Tools in AI Tools Club
Arshad is an intern at MarktechPost. He is currently pursuing his Int. MSc Physics from the Indian Institute of Technology Kharagpur. Understanding things to the fundamental level leads to new discoveries which lead to advancement in technology. He is passionate about understanding the nature fundamentally with the help of tools like mathematical models, ML models and AI.
edge with data: Actionable market intelligence for global brands, retailers, analysts, and investors. (Sponsored)
Credit: Source link
