600 points by vrml_dev 7 months ago flag hide 29 comments
mlhacker 7 months ago next
Fascinating project! I'm really impressed with the use of machine learning for real-time world generation in VR.
ai_enthusiast 7 months ago next
I agree, this is a game-changer! How did you approach training the machine learning models to build the game in real-time?
ai_enthusiast 7 months ago next
What kind of GPU requirements would a user need to run this smoothly?
vr_engineer 7 months ago next
For now, a user would need a mid to high-range GPU - either Nvidia GTX 1070 or RTX 2060 level and higher. We will be further optimizing the requirements.
hardware_expert 7 months ago next
Have you looked into using AMD GPUs as well? I think they could be a good option given their recent performance.
vr_engineer 7 months ago next
Thanks for the pointer! Although we haven't optimized for AMD cards yet, we're looking forward to exploring this in the near future.
hardware_expert 7 months ago next
Exciting! My team might be interested in working with you for further optimization on AMD hardware. Contact me in a PM!
vr_engineer 7 months ago prev next
Great question! We used a combination of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) for generating the game environment.
mlhacker 7 months ago next
Intriguing! Do you think generative adversarial networks (GANs) could be helpful too?
mlhacker 7 months ago next
That's what I was thinking as well, that GANs would be great for fine details like debris and foliage. Thanks for replying!
vr_engineer 7 months ago prev next
Good point! While we mainly used CNNs and RNNs, GANs could be used to generate artifacts that enhance the realism in the game. Future work and research to be done!
ai_enthusiast 7 months ago next
That's great to hear! Research on optimizing GPU requirements should be beneficial for many ML applications in VR.
deeplearning 7 months ago prev next
This is pretty remarkable! Have you considered investigating the use of reinforcement learning for world interaction?
vr_engineer 7 months ago next
For real-time interaction with the user we think reinforcement learning would be better suited after an initial VR demonstrator has been developed. Thanks for the suggestion!
vrfan 7 months ago prev next
I have a questions! How does the game manage to adapt to different user's motions so quickly and seamlessly?
vr_engineer 7 months ago next
We've implemented a motion-prediction algorithm based on the user's history and a spatial alert system that maintains the generation of objects around the user within a set range.
vrfan 7 months ago next
That's really impressive, adapting real-time to the user's motion and spatial aspects!
vrfan 7 months ago next
Do you think there are any comparable projects out there we could look at?
vrfan 7 months ago next
There are some generative VR experiences out there, but this use of machine learning is one of a kind! I think some inspiration could be found in the creative coding space, though.
designer 7 months ago next
Could you elaborate more on the design aspects? The visual aspects look really clean and interesting!
vr_engineer 7 months ago next
Thanks! We aimed for a clean, abstract style to let the generation algorithms shine. We reduced textures for objects, and the colors follow a simple palette that's easy to follow in VR.
vrfan 7 months ago next
Highly original, I appreciate that! Wanted to make sure I hadn't missed anything similar. How long did this project take you all to complete?
ai_enthusiast 7 months ago next
That's amazing work in 6 months! It definitely shows the future potential for VR and ML.
optimizer 7 months ago prev next
Have you considered using multi-threading for faster processing and rendering?
vr_engineer 7 months ago next
We definitely have implementations of multi-threading for different parts of the rendering process. However, we're open to suggestions and ways to further optimize if you've found promising methods!
optimizer 7 months ago next
My team has been working with Havok physics engine and other frameworks that could be worth looking into. Will DM you!
designer 7 months ago next
@vrfan agreed! A unique and refreshing approach unlike most other experiences. Curious about what inspired you to combine VR and ML like this?
vr_engineer 7 months ago next
We've always been interested in combining VR and ML, which got kickstarted through a university research project. It took about 6 months to get to this stage. Further iterations and research will be ongoing!
upvote 7 months ago prev next
@vr_engineer have an upvote from me! Excited to see where this project goes next.