Case study · Generative AI and agentic systems
AI level design from a written prompt
A US game development studio needed walkable graybox environments generated automatically from written prompts instead of blockouts built by hand. Brainy Neurals built a multi-agent AI level design pipeline that turns one text prompt into a structured 3D scene. Every object in the level file is generated, placed, and sized automatically by the pipeline.
Mitesh PatelComputer Vision Expert
Viral TrivediAI Agents Expert
Published August 2026 · Last updated August 2026 · 11 min read
02Quick answers
At a glance
Q·01What problem did this solve?
Building graybox 3D levels meant placing every object by hand before anyone could walk the space. Iteration was slow, and each new idea cost the same manual effort.
Q·02What did Brainy Neurals build?
Brainy Neurals built a multi-agent AI level design pipeline for a US game development studio. It turns a written prompt into a structured 3D scene and a walkable graybox file.
Q·03What changed after it went live?
A level idea now goes from text to a walkable graybox automatically. Designers spend their time judging layouts instead of building them, and revisions take a prompt edit.
Q·04Who else could use this?
Prompt-to-3D generation fits any team that plans space before building it. Game studios, architecture firms, warehouse planners, and training-simulation builders all sketch layouts the same way.
| Industry | Sub-vertical | Client | Engagement | Timeline | Capabilities | Delivery |
|---|---|---|---|---|---|---|
| Gaming | 3D level design tooling | US game development studio | Prompt-to-level generation pipeline | Not disclosed | Generative AI, agentic AI | Project-based delivery |
03Problem
Why was level blockout so slow?
A US game development studio builds 3D titles, level by level. Every level starts life as a graybox: plain blocks a player can walk before any art exists. Blockout is the stage where a layout either earns its art pass or gets cut from the game.
The studio wanted far more layout ideas tested each cycle than its designers could block out by hand. And generative AI development had so far only given the team images and text, never a space it could walk.
None of this is unusual, in games or anywhere else. A 2013 survey of procedural content generation said it plainly: hand-made game content no longer scales against rising production costs[1]. That gap is exactly what this AI level design build closed.
04Alternatives
What do studios usually reach for?
Four routes, each reasonable.
| Approach | What it gets right | Where it stops | Who it still suits |
|---|---|---|---|
| Hand blockout | Full control, every block | Hours per layout, per revision | Hero levels and final passes |
| Rule-based generation | Endless variation, fast | Cannot follow a written brief | Roguelike runs and terrain |
| Learned generators | Pick up a game’s style | Need existing levels to learn | Games with levels to spare |
| Prompt-to-level pipeline, our route | A brief becomes a walkable file | Schema and validation up front | Teams testing many spatial ideas |
Machine-learned generation names the third route’s catch[2]. The models are trained on existing levels, so a working library has to exist before anything can be generated.
Tried one of these and hit the same wall? Tell us where it stopped.
05Solution
How we designed the prompt-to-level pipeline
Brainy Neurals built the pipeline as a chain of small agents, the central AI agent development call on this build. A planning agent reads the incoming brief and splits it three ways. Specialist agents then own one concern each: the layout, the objects, and the constraints between them.
Their answers merge into one structured scene, and one strict schema defines exactly what that scene may contain.
The model decides what goes where, and deterministic code turns that decision into geometry.
The first decision was what the model would be allowed to produce. We rejected free-form output early, because geometry code can’t negotiate with prose. So every agent answers inside the schema, and anything else counts as a failed attempt.
The second decision was where generation stops and code takes over. The model never emits a vertex or a mesh. It only emits a description of the level, and an open-source mesh library builds real geometry from it.
The clever part isn’t any one agent. It’s the contract every agent answers to.
The model decides what goes where, and deterministic code turns that decision into geometry.
06Stack
The technology stack we used
Every layer on this stack exists to keep language and geometry apart. The technology selection questions were the usual ones: what the model decides, what code guarantees, and where the line sits. The list is short on purpose. Fewer moving parts means fewer places where a scene can quietly go wrong.
| Layer | What we used | Why | What we ruled out |
|---|---|---|---|
| Language model | A frontier model family | Read spatial briefs most reliably in our tests | Two other families we benchmarked |
| Orchestration | One agent per concern | Small focused prompts hold detail | A single do-everything prompt |
| Scene schema | One strict schema per scene | Geometry code needs guarantees, not prose | Free-form model output |
| Validation | Schema and spatial checks | Bad scenes bounce back before becoming geometry | Trusting the first answer |
| Mesh generation | An open-source mesh library | Deterministic geometry from validated numbers | Asking the model for meshes |
| Output | One standard 3D file | Opens in the engines the team already uses | Engine-locked exports |
07How it works
How does one prompt become a level?
One prompt, start to finish, in the exact order the pipeline runs it.
- A designer writes the level in plain language: the spaces, the paths, the obstacles, and the overall mood.
- A planning agent reads the brief and splits it into layout, objects, and constraints for the specialist agents downstream.
- Specialist agents turn each part into entries in one structured scene, with a shape, position, and size for every object.
- A validation pass checks the scene against the schema and against the spatial rules, and sends failures back for repair.
- The mesh stage reads the validated scene and builds real 3D geometry from it, one primitive for each entry.
- The pipeline then writes one standard 3D file, and the team opens it in the engine and walks the level.
Six steps, and a person appears only at the two ends of the chain.
CTAWalkthrough
Want this walked through for your content? Book 30 minutes with Mitesh Patel.
No pitch. If it is not a fit, you will know in five minutes.
08Failure
The three walls we hit first
Three things broke, in this order.
The first scenes read well and parsed badly. A field went missing, a type changed, a list arrived as prose. And the geometry code crashed on input it had every right to trust.
The second failure was spatial rather than structural. Scenes started validating and still made no sense: objects overlapped, floated above the floor, or sealed off the only path.
Valid JSON. Unplayable level.
The third was overload. One prompt handling layout, objects, and constraints together dropped requirements as briefs grew longer. The model wasn’t wrong so much as stretched.
Work on language-guided 3D scene generation names object positioning as its own challenge[3]. The published fix is explicit spatial constraints and a solver, not a model’s raw coordinates. These are the weeks when clients bring in specialist engineers rather than learning every lesson the slow way.
09Repair
How we got past each one
Each of the three fixes reads as one line, and none was found in one.
Structure
We locked every agent to the schema and made every answer validate on arrival. A failed check goes back with the exact error, and the agent repairs its own output. And nothing malformed has reached the geometry code since that change landed.
Space
We added a spatial pass behind the schema pass. It checks overlap, support, and reachability, then returns violations with the offending coordinates. A scene becomes geometry only after it clears both gates.
Split
We broke the one big prompt into the agent chain the pipeline runs today. Each agent holds one concern, so detail stopped falling out of long briefs.
This is the work a proof of concept exists to surface, before anything gets promised.
10Results
What changed after go-live?
| What | Before | After |
|---|---|---|
| Getting to a walkable layout | Hand placement in the editor | One written prompt |
| Objects placed by hand | Every object in the scene | Zero |
| Revising a layout | Rebuild the blockout | Edit the prompt, run again |
| Trying variants | One at a time, by hand | One prompt each |
| What a designer does | Building the space | Judging the space |
We have not published a generation-time or an hours-saved figure, because this build measured neither. A number we have not measured is a number we will not print.
Day to day, AI level design changes where attention goes: a layout idea gets typed out, not dragged into place. And because every variant costs only one prompt, exploring more ideas no longer costs the team more building.
The graybox itself did not change. What changed is how it gets made.
What should a prompt build for you?
Levels, layouts, massing, racking, scenario spaces. If your team plans it before building it, a prompt can probably draft it.
11In production
What is running today
The AI level design pipeline is in use at the studio today, on the agent chain the first version proved. A designer writes the brief, runs the pipeline, and opens the finished file in the engine. The schema is still the contract between language and geometry, and nothing becomes a mesh without passing it.
Every level the pipeline produces still starts as one prompt, and that hasn’t changed since the first working run. Judging layouts is the real design work now, and typing is the whole build step.
12Lessons
What would we do differently?
Design the schema before the prompts
We wrote the prompts first and reworked them when the schema landed. The schema is the product.
Put a check at every handoff
Our first validator sat at the very end, so one early mistake surfaced three stages late. Errors got cheap the day checks started running at every handoff.
Walk every candidate before judging it
A generated level that cannot be walked is a picture, not a design. Early reviews leaned on top-down views, and clean layouts read wrong at player height. Now nothing gets judged unwalked.
Keep the model away from numbers
It’s tempting to let the model fix coordinates when a scene is close. Every nudge invites drift. So repairs go back through the schema like everything else.
A generated level that cannot be walked is a picture, not a design.
13Porting
Where else does prompt-to-3D fit?
Prompt-to-3D generation turns a written description into a walkable 3D layout, used wherever space gets planned before it gets built.
| Industry | The equivalent problem | What changes in the build |
|---|---|---|
| Construction | Clients want walkable massing before drawings exist | Real dimensions, code-driven constraint rules |
| Logistics | Warehouse rack layouts sketched before steel is ordered | Racking sizes, aisle-width rules in the schema |
| Retail | Store planners test fixture layouts per site | A fixture vocabulary, footfall constraints |
| Training simulation | Safety teams need many scenario spaces cheaply | Hazard placement rules, scenario variants |
| Virtual production | Scenes blocked before any set is built | Camera-aware layouts, per-shot variants |
Porting takes a new object vocabulary in the schema, that domain’s constraint rules, and a validation pass tuned to both.
14FAQ
Questions buyers usually ask
How it works
Q·01Can AI generate a 3D level from a text prompt?
Yes, when the model describes the level and code builds the geometry. A language model turns the prompt into a structured scene, and a mesh stage turns that scene into geometry. Asking a model for raw 3D geometry directly does not hold up.
Q·02What is a graybox level, and why start there?
A graybox is a level built from plain untextured blocks, made to test layout and flow before any art exists. Teams start there because moving a gray block is cheap and moving finished art is not.
Q·03How is this different from procedural generation?
Classic procedural generation follows fixed rules or noise, so it cannot read a designer’s brief. A prompt-driven pipeline starts from written intent: a designer describes the level and gets that level, not a random one.
Time, cost, and rollout
Q·04How long does it take to build a prompt-to-level pipeline?
A working proof of concept on your own content usually takes a few weeks. The scene schema, the agent split, and the validation rules each need a dedicated pass. A production pipeline follows once generated levels start surviving review unedited.
Q·05How much does an AI level design build cost?
The cost depends on the object variety, the constraint rules, and how much content already exists. Brainy Neurals scopes it from a short call and a close look at your content, then quotes a fixed price. An AI readiness assessment tells you first whether your briefs and assets can actually drive one.
Q·06Does AI level generation replace level designers?
No. AI level design replaces the placement labor, never the judgment about what plays well. Designers still decide what makes a space worth playing, and the pipeline gets them to that decision sooner.
Sources cited on this page
- [1]Hendrikx M, Meijer S, Van der Velden J, Iosup A. Procedural Content Generation for Games: A Survey. ACM Transactions on Multimedia Computing, Communications, and Applications. 2013;9(1):1-22. DOI 10.1145/2422956.2422957.
- [2]Summerville A, Snodgrass S, Guzdial M, Holmgård C, Hoover AK, Isaksen A, Nealen A, Togelius J. Procedural Content Generation via Machine Learning (PCGML). IEEE Transactions on Games. 2018;10(3):257-270. DOI 10.1109/TG.2018.2846639.
- [3]Yang Y, Sun FY, Weihs L, VanderBilt E, Herrasti A, Han W, Wu J, Haber N, Krishna R, Liu L, Callison-Burch C, Yatskar M, Kembhavi A, Clark C. Holodeck: Language Guided Generation of 3D Embodied AI Environments. CVPR 2024. arXiv 2312.09067.
CTAClosing
If your team plans space before building it, tell us what a prompt should build.
15Services
The services this was built from
A proof of concept is the fastest way to test this on your own content. AI consulting helps choose the model family and the language-to-geometry boundary first. An AI readiness assessment tells you up front whether your briefs can drive a schema. And the industries hub shows where else this generation pattern already runs.
16More proof
Other builds with this shape
Three more Brainy Neurals builds, every one of them shipped into a real environment rather than a slide deck.
Cite this case study
Trivedi, Viral and Patel, Mitesh. AI level design from a written prompt. Brainy Neurals, August 2026. https://brainyneurals.com/case-studies/ai-3d-level-design/