Learning to Build and Interact with 3D Rooms using Deep Neural Networks

The ability to synthesize realistic and diverse indoor furniture layouts, automatically or based on partial input, is an exciting research direction that could unlock many practical applications, from better interactive 3D tools to data synthesis for training and simulation. Starting from an empty or a partially complete room, we want to build a generative model that will automatically populate a room with suitable objects, identify and reposition furniture pieces that are located in unnatural positions, make suggestions based on user-provided constraints, and place specific objects at the most plausible location. Existing scene synthesis methods impose unnatural constraints on the scene-generation process, thus inhibiting various practical applications for scene authoring. We'll describe a novel neural network architecture that enables new interactive applications for semi-automated scene editing. To simulate our synthetically generated rooms, we used NVIDIA Omniverse.