Shanghai AI Lab and Nanyang Technological University Unveil 3DTopia 2.0: A Powerful New 3D Object Generation Model
Shanghai, China -The Shanghai AI Laboratory, in collaboration with Nanyang Technological University, has announced the release of 3DTopia 2.0, a groundbreaking 3D objectgeneration model. This advanced AI tool leverages a novel primitive-based 3D representation method called PrimX, enabling the creation of high-resolution 3D assetswith remarkable efficiency and fidelity.
3DTopia 2.0 utilizes the Diffusion Transformer framework, allowing it to generate high-quality 3D assets with physically-based rendering (PBR) properties from either text or image inputs. Themodel’s code has been open-sourced, offering free commercial authorization, making it a powerful tool with the potential to revolutionize 3D content creation workflows across industries like gaming, film, architecture, and design.
Key Features of3DTopia 2.0:
- Multimodal Input for 3D Object Generation: 3DTopia 2.0 can rapidly generate corresponding 3D models based on text descriptions or image inputs.
- Highly Efficient Generation Process: The model can transform input into a 3Dmodel within five seconds, significantly enhancing creative efficiency.
- High Quality and Detailed Textures: The generated 3D objects feature smooth geometric shapes and spatially-varying textures and materials, closely resembling real-world physical materials.
- Direct Integration with Game Engines and Design Software: The generated 3D models can be directlyutilized in game engines and industrial design software, eliminating the need for additional processing.
- Support for High-Resolution Geometry: Based on the PrimX representation, the model can create high-resolution 3D geometric shapes.
Technical Principles Behind 3DTopia 2.0:
- PrimXRepresentation: This innovative primitive-based 3D representation method encodes the shape, albedo, and material information of a 3D object into a compact tensor format. Each primitive is a small voxel parameterized by its 3D position, global scaling factor, and corresponding spatially-varying payload (including SDF, RGB, and material information).
- Primitive Patch Compression: A 3D variational autoencoder (VAE) is used to compress the spatial information of each primitive, resulting in latent primitive tokens. The process utilizes 3D convolutional layers to compress the primitive’s payload from a high-dimensional space into a low-dimensional latent space, providingefficient input for the subsequent generative model.
- Latent Primitive Diffusion: Based on the Diffusion Transformer (DiT) framework, the model learns to progressively remove noise from random noise, generating latent primitive tokens that align with the input conditions. This process simulates the diffusion and denoising processes found in physics, enabling thecreation of 3D objects with high-resolution geometry and PBR materials.
- Differentiable Rendering: The PrimX representation supports differentiable rendering, allowing the model to learn directly from 2D image data, enhancing its ability to learn from existing image resources.
Applications of 3DTopia2.0:
- Game Development: Rapidly generate a wide range of 3D game assets, such as characters, props, and environmental elements, enhancing game development efficiency and richness.
- Film and Animation Production: Create 3D scene and character models for films and animations, reducing manual modeling time andcosts while offering greater creative freedom.
- Virtual Reality (VR) and Augmented Reality (AR): Generate realistic 3D environments and objects for VR and AR applications, improving user experience.
- Architecture and Urban Planning: Rapidly generate 3D building models and cityscapes for architectural design and urban planning, assisting designers and planners in exploring design options and visualizing results.
The release of 3DTopia 2.0 marks a significant advancement in 3D object generation technology. Its open-source nature and free commercial authorization make it accessible to a wide range of users, empowering them to create high-quality3D content with unprecedented ease and speed. As this technology continues to evolve, it has the potential to reshape the landscape of 3D content creation, driving innovation across various industries.
Views: 0