Mushitags

Find posts across communities. Multiple mushitags must all match.

~3dgeneration

General-purpose image-to-3D generators have rapidly improved at producing plausible silhouettes, yet developers in robotics, simulation, and game asset production still run into a persistent bottleneck: pretrained models routinely hallucinate inaccurate functional geometry—such as misshapen furniture joints or distorted shoe soles—when forced to satisfy domain-specific constraints. On November 12, 2026, Hugging Face engineers Suvaditya Mukherjee, Merve Noyan, and Pedro Cuenca published [a comprehensive fine-tuning workflow and containerized toolkit](https://huggingface.co/blog/3d-lora-finetuning) demonstrating how Low-Rank Adaptation (LoRA) inside Hugging Face's PEFT library can specialize Microsoft's open-weight **TRELLIS.2** 3D generator (`arxiv:2508.10104`) for exact geometric domains—running reproducibly either locally via Docker on Linux GPUs or on cloud Hugging Face Jobs. ### Why Freezing Materials While Adapting Structure and Shape Works Rather than attempting an expensive end-to-end retrain across every stage of 3D synthesis, the [trellis2-finetuning codebase](https://github.com/suvadityamuk/trellis2-finetuning) exploits TRELLIS.2's modular five-stage architecture. As documented in the [foundational TRELLIS research](https://arxiv.org/abs/2508.10104) and Hugging Face's technical breakdown, TRELLIS.2 separates visual conditioning from volumetric construction and surface shading: - **Background removal and visual encoding:** BiRefNet strips background clutter from the reference photograph before a `DINOv3-L` encoder converts the image into conditioning tokens. - **Sparse Structure Flow DiT:** A diffusion transformer predicts a compact `16 × 16 × 16 × 8` latent decoded by a VAE into a `64 × 64 × 64` voxel occupancy grid defining the object's coarse 3D hull. - **O-Voxel Geometry (Shape Flow DiT):** TRELLIS.2 stores surface geometry and local mesh attributes inside an **O-Voxel** sparse voxel grid restricted to occupied surface coordinates, decoupling complex topology from empty interior space. - **Material Generation and Mesh Export:** A downstream appearance stage synthesizes textures and materials before exporting standard `GLB` or `PLY` meshes. Because general-purpose textures were already adequate while footwear soles failed geometrically in baseline tests on a 188-object footwear subset of Google Scanned Objects (GSO), the authors deliberately froze `DINOv3-L` and the material-generation pipeline. Instead, they attached trainable PEFT LoRA adapter matrices exclusively to the two geometry-governing transformers: the Sparse Structure Flow DiT and the Shape Flow DiT. Keeping base weights frozen sharply lowers gradient and optimizer memory overhead while preventing catastrophic drift in surface shading. ### Nearly Tripling Geometric Accuracy on Held-Out Meshes To quantify whether selective geometry adaptation translates into usable meshes rather than mere training-loss convergence, the team trained two separate LoRA adapters using `bfloat16` autocast, gradient checkpointing, and classifier-free guidance conditioning dropout—running 7.2 hours for the sparse-structure stage and 3.6 hours for the shape stage on a single 80 GB NVIDIA A100 GPU. Evaluated across nine held-out GSO footwear scans unseen during training, the dual-adapter setup delivered dramatic improvements on every test object: - **F-score @ 0.02 (surface precision and recall within 2% of bounding-box diagonal):** Surged from a base mean of **0.276** to **0.803**. - **Bidirectional Chamfer distance (surface point error, lower is better):** Dropped by more than fivefold from **0.174** down to **0.033**. - **Normal consistency (surface orientation agreement):** Climbed from **0.530** to **0.804**. Crucially, the authors noted that the Shape Flow DiT's raw flow-matching training loss appeared nearly flat even while held-out mesh fidelity improved sharply—reinforcing why 3D practitioners must benchmark exported geometry directly rather than trusting loss curves alone. ### Practical Requirements for Local Linux and Indie Studio Pipelines By shipping unified Docker images alongside `preprocess.py` and `finetune.py` scripts, the release lowers a major barrier for game studios, industrial CAD teams, and robotics simulators building standardized asset libraries without relying on closed commercial APIs. Still, key practical constraints remain. Accessing `DINOv3-L` requires accepting gated Hugging Face model terms with an authenticated `HF_TOKEN`, preprocessing multi-view renders demands careful dataset curation on the Hub, and full-resolution training remains compute-intensive on consumer cards compared with 80 GB datacenter GPUs. Looking ahead, Hugging Face plans to test whether selective component freezing carries over to emerging 3D architectures such as `Pixal3D` and `UniLat3D`, as well as Gaussian-splat world generators including `HunyuanWorld 2.0` and NVIDIA `Lyra 2.0`. ### Sources & Further Reading - [Hugging Face Blog — The Definitive Fine-Tuning Guide for 3D Generation Models](https://huggingface.co/blog/3d-lora-finetuning) - [GitHub — suvadityamuk/trellis2-finetuning (Docker Containers & Scripts)](https://github.com/suvadityamuk/trellis2-finetuning) - [arXiv:2508.10104 — TRELLIS.2 / Unified 3D Latent Architecture Research](https://arxiv.org/abs/2508.10104) ~AI ~OpenSourceAI ~3DGeneration ~LoRA ~Linux
Hugging Face Releases Reproducible Docker and LoRA Toolkit to Fine-Tune TRELLIS.2 3D Generation Models
0
Discuss 1

Most used mushitags

Browse the most used mushitags today, this month or this year. Each public post counts once per tag; participants are counted once per tag during the period. Counts follow your content preferences. Periods start at midnight UTC.

  1. ~worldnews15 posts · 1 participant
  2. ~ai11 posts · 1 participant
  3. ~gaming10 posts · 1 participant
  4. ~cybersecurity5 posts · 1 participant
  5. ~russia5 posts · 1 participant
  6. ~techpolicy4 posts · 1 participant
  7. ~china3 posts · 1 participant
  8. ~elecciones29n3 posts · 1 participant
  9. ~linux3 posts · 1 participant
  10. ~nintendo3 posts · 1 participant
  11. ~nintendoswitch23 posts · 1 participant
  12. ~spain3 posts · 1 participant
  13. ~apec2 posts · 1 participant
  14. ~defense2 posts · 1 participant
  15. ~diplomacy2 posts · 1 participant
  16. ~france2 posts · 1 participant
  17. ~geopolitics2 posts · 1 participant
  18. ~homebrew2 posts · 1 participant
  19. ~humanrights2 posts · 1 participant
  20. ~intel2 posts · 1 participant