Stanford scholars train AI to better augment human creativity

Stanford researchers are studying how people communicate during visual-creation tasks and are building open-source tools — including ControlNet, FramePack and a neuro-symbolic scene-coding system — to help generative AI collaborate with human creators.

Stanford researchers are developing methods to help generative AI and people share a common set of concepts when creating visual content. The work is funded by a Hoffman-Yee Research Grant from the Stanford Institute for Human-Centered AI. The team studies how people collaborate on creative tasks by analyzing chat logs and sketches to learn how humans establish conceptual ground. They are also building open-source tools that apply those lessons to text-to-image and 3D generation. Key developments: - The project received support from a Hoffman-Yee Research Grant at Stanford HAI and aims to create a shared conceptual grounding for human–AI collaboration. - Researchers run experiments analyzing how people communicate while making images, using chat logs and sketches to map collaborative processes. - ControlNet is being used to teach diffusion models about spatial composition via separate blocking and detailing stages that mirror an artist’s rough-to-finish workflow. - FramePack is a tool to generate 3D videos from text prompts that helps models prioritize important scenes for multi-scene storytelling. - A neuro-symbolic visual scene coding approach converts natural-language prompts into editable code that is executed to render 3D scenes, keeping humans in the loop to inspect or modify output. - The team is working with Roblox to let players generate 3D objects from text while enforcing game restrictions (for example, preventing weapon creation in a nonviolent game). Summary: The researchers say a shared conceptual grounding could expand how creators of different skill levels express ideas with AI across design, simulation, animation, robotics and education. The team is continuing tool development and partner work such as the Roblox collaboration; Undetermined at this time.