近日，来自南加州大学、哈佛大学等机构的研究团队提出了一种全新的基于提示学习的方法——DreamDistribution，该方法可以学习任意文本提示生成模型的偏好分布，从而实现个性化参照图片生成多样图片。
DreamDistribution 方法通过将图片集反演到语义空间的分布，生成多样个性化图片或3D 渲染。用户只需提供一组参照图片，比如不同的高达玩具图像，DreamDistribution 就可以学习到一个对应于这一组图片的文本提示分布 D*。在推理时，通过从 D* 中采样生成具有足够变化和多样性的分布内输出图像。此外，DreamDistribution 还可以调整分布的方差来控制多样性，并结合多个提示分布生成混合概念图片等操作。
实验结果显示，DreamDistribution 能够生成在颜色、视角、姿态、布局、细节设计等方面产生实质性变化且与参照图片相符的适当视觉属性的图像。与基线方法相比，DreamDistribution 在多样性和质量方面均取得了最佳效果。
这项研究为个性化生成任务提供了一种新的解决方案，使得生成的图像更具多样性和创新性。未来，研究人员希望能够进一步优化 DreamDistribution 方法，提高其在类似3D生成任务上的效果，并将其应用于更多实际场景中。
英语如下：
**Headline:** “Revolutionary New Study Shows Personalized Image Generation from Hints, Enables Easy Toy Building Design”
**Keywords:** 1. Evolved Text-to-Image Generation
**News Content:** Recent research conducted by teams from the University of Southern California (USC) and Harvard University has introduced a groundbreaking method for text-to-image generation based on the concept of hints – known as DreamDistribution. This approach allows users to generate diverse images based on personalized preference distributions of any text-to-image generation model.
DreamDistribution achieves this by inverting the image set into the distribution of its semantic space, generating diverse, personalized images or 3D renderings. By simply providing a set of reference images, such as different images of Gundam toys, DreamDistribution learns a textual hint distribution D* corresponding to this set. During inference, DreamDistribution samples from D* to produce output images with sufficient variation and diversity within the distribution. Moreover, DreamDistribution can adjust the variance of the distribution to control diversity and combine multiple hint distributions to generate mixed concepts images, among other operations.
The experimental results demonstrate that DreamDistribution is capable of generating images with substantial changes in color, perspective, pose, layout, detail design, and other visual attributes that align with the reference images. Compared to the baseline methods, DreamDistribution achieves optimal diversity and quality in image generation.
This study provides a new solution for personalized image generation tasks, making the generated images more diverse and innovative. In the future, researchers aim to further optimize DreamDistribution to enhance its performance on similar 3D generation tasks and apply it to more practical scenarios.
【来源】https://www.ithome.com/0/745/614.htm
Views: 1