3Dera

• 

  VR scene generated from prompt "pyramids of giza"

• 

  VR scene generated from prompt "first moon landing"

• 

  VR scene generated from prompt "titanic sinking"

• 

  VR scene generated from prompt "1492 christopher columbus expedition"

• 

  skybox generated from prompt "first moon landing"

• 

  ground texture generated from prompt "first moon landing"

• 

  web page example

Inspiration

All four of us are STEM majors, and all four of us did NOT have very memorable, interesting, or fun experiences learning history in school. But we believe that history as a subject is not boring at all. We wanted to enhance the experience of learning history in a way that is both immersive, interactive, yet educational.

What it does

3Dera takes any well-documented historical event, and it generates a 3D scene that you can enter in VR. These scenes are created when prompted. While in the scene, 3Dera plays an audio recording of a description of the historical event, so that you can learn about the history while being immersed in the scene.

How we built it

There are 3 components behind 3Dera's logic:

1. Object selection from Objaverse (obj library), and placement into a blender scene using x, y, z coordinates and relative size scaling. (Selection is based on the context of the historical event, which is recognized using GPT-4).

2. Skybox (360 view) of the historical event and ground textures are generated by using the Hugging Face API. Generated based on the context of the event as well.

3. We use the Wikipedia API to collect the information about the historical event, then GPT-4 to summarize and process it into an audio file, which will play when the scene is launched.

Challenges we ran into

Objects can come in many sizes, orientations, placements, so standardizing and configuring all of these to match the scene of a historical event was very difficult. Especially because Objaverse library is not standardized by default, and contains objects from many different sources and authors.

Accomplishments that we're proud of

This was a very ambitious project, and we believe that it's a very cool way of learning history: by putting yourself directly into the history.

What we learned

This was most of our team's first time working with Blender and Blender scripting, as well as our first time working with a couple other interesting technologies such as Hugging Face. It was very satisfying seeing the pieces of the system we designed to generate the scenes fall into place into an immersive experience.

What's next for 3Dera

Expand and standardize object library, thus expanding the amount of historical events we can generate scenes for. Better graphics, more accurate/contextual object placements.

Built With

• blender
• fastapi
• huggingface
• objaverse
• openai
• python
• react