Vi3W
Skip to content

Guides

How does text-to-3D generation turn a written description into a usable 3D model?

The stages of text-to-3D generation, from prompt encoding to shape, mesh and textures, and the cleanup that makes the result usable in a project.

6 min read

A ribbon of light flowing out of a glass speech bubble and condensing into a small clay treasure chest on a turntable

Quick answer

How does text-to-3D generation turn a written description into a usable 3D model?

The prompt is encoded into numbers a neural network understands. A shape model trained on hundreds of thousands of 3D objects then generates geometry that fits those numbers, which is extracted as a triangle mesh. A second model paints textures. Making it usable takes cleanup: retopology, UVs, scale, pivot and export to GLB, FBX or OBJ.

From words to a model, in five stages

Text-to-3D generation turns a written description into a usable 3D model in five stages: it encodes the prompt, generates a 3D shape that matches it, converts that shape into a mesh, paints textures onto the mesh, and then (this is the part demos skip) cleans the result up so it works in your tool.

  1. Encode the prompt. Your sentence becomes a vector of numbers.
  2. Generate a shape. A model trained on 3D data produces geometry that fits that vector.
  3. Extract a mesh. The shape, held in the model's own internal format, is turned into triangles.
  4. Texture it. A separate step paints base colour and often PBR maps (roughness, metalness, normals).
  5. Make it usable. Retopology, UVs, scale, pivot and export.

The first four are automatic. The fifth is where a "cool demo" becomes an asset you can actually put in a game, a render or a print.

Stage 1: the prompt becomes numbers

A neural network cannot read "a rusty iron lantern with a ring handle". A text encoder converts it into an embedding, a long list of numbers in which similar meanings sit close together. "Lantern" lands near "lamp" and "lamp post"; "rusty" lands near "weathered" and "corroded".

This is why wording matters, but not in the way it does for chatbots. The model has no memory of what you meant last time and no idea about your game. It only has this vector. How a text-to-3D generator decides shape from a short prompt covers what happens when the prompt is vague.

Stage 2: a shape model generates geometry

This is the core, and there have been two main approaches.

The older way: sculpt against a 2D image model

DreamFusion (Poole et al., 2022) showed you could make 3D from text without any 3D training data. It optimises a 3D representation (a NeRF) from random viewpoints until a pretrained text-to-image model rates every rendered view as a good match for the prompt. Its paper states the method "requires no 3D training data". The catch was speed and consistency: each object was a slow optimisation run, and shapes often grew extra faces.

The current way: models trained on 3D directly

Most tools today use models trained on large 3D datasets. Two well-documented open examples:

  • TRELLIS encodes objects into a "Structured LATent", a sparse 3D grid carrying visual features. Its paper reports models of up to 2 billion parameters trained on about 500K 3D objects, which can decode to radiance fields, 3D Gaussians or meshes, from text or image conditions.
  • Hunyuan3D 2.0 splits the job in two: Hunyuan3D-DiT, a flow-based diffusion transformer for shape, and Hunyuan3D-Paint for textures. Its GitHub page describes "a two-stage generation pipeline, starting with the creation of a bare mesh", and says shape alone needs 6 GB of VRAM, shape plus texture 16 GB.

In both cases the model starts from noise and repeatedly refines it into a shape that fits the condition. Because a picture is a far more specific condition than a sentence, many text-to-3D tools first generate a reference image from your prompt and then run image-to-3D on it. Some hide that step; others show you the image so you can reject it before spending a generation.

A pipeline of floating objects: a speech bubble, a cloud of glowing points, a rough grey mesh and a clean textured lantern
Illustration. Prompt, latent shape, raw mesh, textured asset.

Stage 3: the shape becomes a mesh

The model's internal format is not something Blender or Unity can open. The final step of generation extracts a surface from it and writes out triangles. Surface extraction works on a grid, so the triangles come out dense and evenly sized, following the surface but ignoring the structure of the object. That is why raw AI models look fine in a viewer and are painful to edit. Why AI 3D models have messy topology explains the mechanics.

Stage 4: textures

Texturing usually runs as its own model. Hunyuan3D 2.0's paint model, for instance, is described in its paper as producing texture maps "for either generated or hand-crafted meshes", so the shape and the paint job are separate steps you can rerun independently. Better pipelines output PBR maps, not just colour. Seams and blurry patches tend to appear on the parts no generated view saw clearly, such as the underside or deep crevices.

Stage 5: what "usable" actually takes

A generated model is a draft. To make it usable, work through this list:

  1. Check the silhouette from every side, including the back and bottom. Text gives the generator the least information of any input, so hidden sides are the least reliable.
  2. Reduce and clean the topology. Retopologise anything that animates or is seen up close. Decimate background props. Decimate vs retopology helps you pick.
  3. Check UVs and textures after retopology, because changing the mesh can break the mapping. Bake if needed.
  4. Set real-world scale and the pivot. Generators have no idea how big a lantern is.
  5. Export the right format: GLB/glTF for the web and Godot, FBX for Unity and Unreal, OBJ for general interchange, STL or 3MF for printing.

Three approaches to text-to-3D, compared

  • Optimisation against a 2D model (DreamFusion style): needs no 3D data, but is slow per object and prone to inconsistent views. Mostly historical now.
  • Native 3D generative models (TRELLIS, Hunyuan3D): fast and consistent, with good shapes for common objects. Limited by what is in the 3D training data.
  • Text to image, then image to 3D: gives you a checkpoint, since you approve the image first. Quality depends on how clean that reference image is.

When text-to-3D is the wrong tool

  • Exact dimensions or mechanical fit. Text-to-3D makes meshes, not CAD solids. See AI tools that create solids.
  • A specific real product. Photograph it and use image-to-3D, or scan it.
  • A whole scene. Generators make one object at a time. Make props separately and assemble them.
  • A precise design you have already drawn. A sketch or image gives far more control than words. Text vs image vs sketch to 3D compares the inputs.

Where Vi3W fits

Vi3W covers the generation stages and part of stage five in the browser. You can start from text, a sketch or an image, get a model with AI PBR textures, run automatic retopology from high-poly to low-poly, and export GLB/glTF, FBX or OBJ. It does not rig or animate, and it does not build scenes, so plan those steps in Blender or your engine. For better first results, start with how to write prompts for AI 3D models.

Frequently asked questions

how does text-to-3d generation actually turn a written description into a usable 3d model?

In stages. A text encoder turns your words into a vector, a shape model generates 3D geometry conditioned on it, the geometry is converted into a triangle mesh, and a texture model paints colour and material maps onto it. The raw result is a dense mesh, so "usable" also means retopology, checking UVs and scale, and exporting to the format your engine or printer expects.

Last reviewed