Unreal Engine meets ComfyUI: an asset generator, camera mapping and style transfer, built in the Bootcamp

Type a prompt inside Unreal Engine and get a 3D asset in your scene. Frame a shot, send it to ComfyUI, and get it back as a watercolour or a comic book style, projected onto the geometry. Once the projection is applied, you move around the scene in real time, as in any Unreal scene. Kevin Bertelli built both during cohort 1 of our Generative AI Bootcamp for Film and TV, as two Unreal Engine plugins that talk to ComfyUI.

Kevin Bertelli is a CG Supervisor and a real-time specialist. Most of his work is in Unreal Engine, and he wanted the tools where he works, in the same window. He presented the two plugins to the final jury on 11 September 2026.

🧱 Plugin 1: a 3D asset from a prompt, inside Unreal

The first plugin is for 3D modelling without leaving Unreal. You type a prompt in the plugin's panel. It goes to a ComfyUI workflow that generates a concept image with Z-Image Turbo. The panel keeps a history of your prompts, and you can check the concept, generate again, or send it on. When you accept it, a second workflow turns the image into a 3D asset with Pixal3D, an image-to-3D model that keeps the mesh aligned with the input image, and the asset loads into the scene.

The goal is speed, not final quality: quick assets to fill a scene for previz. One asset takes about 200 seconds to generate.

Unreal Engine needs the memory of the graphics card (VRAM) to keep playing back in real time, and the generation models need a lot of that same memory. To keep both, the ComfyUI server runs on a second computer next to Kevin's, and the plugin sends the generation there. Unreal keeps its full performance while the asset is being made.

🎨 Plugin 2: restyle a shot and project it back

The second plugin is camera mapping, also called camera projection: you take an image from the camera's point of view and project it onto the 3D geometry, so it sticks to the surfaces when the camera moves. Kevin's version adds a style transfer in the middle. You capture the camera view in Unreal, send it to ComfyUI, and Qwen Image Edit restyles it from one line of text, "convert the image to a watercolour painting" for example. The plugin projects the result back onto the geometry from the same camera.

Projection has a classic problem: everything behind an object gets the same image, which is wrong once you move. Kevin masks the projection so that nothing lands behind an object or in its shadow, with a feather on the mask edge to keep the transition soft, and a control on the field of view. When you capture again from another angle, you can include or exclude the earlier projections, so several captures cover an object without polluting each other: his truck carries three projections, one per side. When the model gets something wrong, you export the capture, paint over it in Photoshop, and bring it back to project only the painted part on top.

Each style takes one generation. Then you change the line of text and try another: comic book, watercolour, and whatever comes next.

🎥 What it is for

Both plugins are made for previz, the early stage where a shot is sketched in 3D before it is built for real. The first plugin fills the scene with quick assets. The second gives the scene a look, and lets you try another look by changing one line of text, without rebuilding anything.

Next on Kevin's list: finish the multi-camera projection, so that one asset gets covered from several angles in one pass, and automate the clean-up between projections, which he still does by hand.

🎓 Built in the Bootcamp

Cohort 1 of the Generative AI Bootcamp for Film and TV finished on 11 September 2026 with a group of six participants, each presenting their own project, built in ComfyUI across the sessions, with a midpoint review and a final review by a jury of working AI professionals. Kevin's two plugins are one of them.

Cohorts 2 (from 2 October 2026) and 3 (from 6 January 2027) are open for enrollment. You bring one production case, and you build it. Learn more about the Bootcamp.

🔗 Sources & Links

🎬 The project:

🛠️ Tools:

🎓 The Bootcamp:

📄 Related Posts:

Join the conversation

Have thoughts on this article? We'd love to hear from you!

Let's work together

Inquire now