This week, I attended the Immersive Technology Conference at Hamilton College to learn and share knowledge about immersive technology which included 3D modelling, Virtual Reality, 360 videos, 3D scanning, and photogrammetry. Read more about it: here.
I learned many projects from different students from both Hamilton College and Colgate University. All of these projects were impressive, and I found out that I have still a lot to learn, e.g. the 3D model can be very detailed and can take a lot of time, VR experience can be even more interactive, which together can make the experience even more immersive 😉
My main goal this week was to reconstruct the Spanish theater, EL CORRAL DEL PRÍNCIPE, which existed between 16th an 18th century. Under commands of Chris Oechler (Profe O.), I and Orrin were trying to build 3D model of this theater in Blender and implement it in Unity in order to make the VR experience of the historical theater.
Theater consisted of that stage, two side walls, and the back. Orrin is more efficient in Blender than I am, so he took the stage and two side walls, and I took the back of the theater. Because the theater consisted of 4 floors, I divided my work into layers (each layer- each floor), so that I could build at least 1 floor each day (4 floors- 4 layers- 4 days).
On the last day, I and Orrin tried to merge two projects together and there was a problem with the roof, because it covered all of the 4 floor, making impossible to see the stage from the 4th floor, so we had rearrange the roof again. At the end, we could finally fix it and we could put the 3D model to Unity and experience it in VR. Right now, the theater is just a grey 3D model, without colors. The next step will be adding the material and texture to the 3D model to make it look more realistic.
Thank you R.C., Profe O., and Eric for the trip and hanging out with us 😀 It was a very unforgettable experience for me. I really enjoyed working and spending time with you guys! Thank you Lord Greg from DHi Hamilton College for hosting us and teaching us important functions in Blender and Unity.
Thank you Profe O. for allowing me to work in your project. Hope that I was not that bad in cooperating.
Thanks Orrin for working with me and helping me. Happy Birthday!!!
Thanks Alyssa for proudly representing our school!
Have a good weekend,
JHA
P.S. I still think our school should have some squash court.
What a story, a 4.5 million population country, Croatia, is one step from being the World Champions. France is a very young, talented, and organized team. Croatia is brave, strong, and stable. Any outcome of this final match will be a good outcome.
Day1
I made more models of the tombstones in Blender in order to match different types of tombstones of Lincoln Cemetery. I found out that there so many types of tombstones and these pieces of rock are not cheap.
Day2
I started adding tombstones to my scene in Unity. I divided the cemetery into rows, so it would be easier for me to place the tombstones. Also, thanks to my drone photogrammetry of the cemetery I did last week, it showed the 3d (more exact) position and shape of each tombstone.
Lincoln Cemetery in Unity: 3D model (left) vs drone photogrammetry (right)
Day3
I still added the tombstones. I have to place them very precisely, so that later there will be space for other tombstones.
P.S. My tennis idol lost on Wednesday, I was so sad that almost lost the motivation to work.
Day4
I finished adding the tombstones with trees and bushes.
Day5
I worked on interactivity in Unity. I added teleportation, portal teleportation, and my photogrammetry model along with 3D model. Although, the computer I am working on is strong, it could not handle with tombstones, which consisted of million of polygons. Therefore, I have to work on improving the tombstone shapes by decreasing the number of polygons, so that the computer can handle with rendering the tombstones.
Also I made some 360 videos of the cemetery, allowing the user to experience more real.
Week3 was not that fortunate as I thought. Poland was going to beat the world at World Cup this year, but unfortunately Poland was beaten by Senegal.
Day 1
Also unfortunately, my initial plans have to be changed a bit, because of a problem that I was supposed to predict. Gettysburg Battlefield belongs to Gettysburg National Military Park, so I am not allowed to fly with a drone above the park and film it. Thus, I got the new topic, which is equally important, or even more, the Lincoln Cemetery in Gettysburg (please, do not confuse with National Cemetery in Gettysburg). Lincoln Cemetery is a cemetery for African American citizens and Civil War soldiers, as even after the Civil War, United States Colored Troops were not allowed to be buried along with white soldiers.
Day2
On a second day I really wanted to build a VR bowling game, because this game would show me how difficult is to build a simple scene and how the objects interact with each other (ball with bowling pins). This was also a good place to play with sound (the ball hitting the pins, the bowling rolling on a lane) and background music. I found a very good tutorial which showed me step by step how to build a bowling game, here (shout out to FuzedVR). After building a bowling game, I realized that 3d modelling will be very important for me, so I started to work on 3D modelling in Blender, an open- source 3D modelling software.
The screenshot of a Bowling “Game” in Unity
Day3
We got new, strong computers, on which me and Orrin will be working. It is vital to have powerful computers which are processing and rendering heavy graphics for Virtual Reality content. Later that day, I was trying to show more of my VR content to Sharon, but unfortunately everything stopped working and crashed working in Unity. I was so sad that I could not show it, I hope that next time I will be able fix it and make it even better, especially for Sharon. In the afternoon, Eric Remy taught us techniques for public speaking and presentation. It was a very valuable lesson, as public speaking is an integral part of our life and it is important to develop it. I will try to build up my confidence and improve my speeches. Thank You, Eric!
Day4
Today, I felt like building more games to get used building environments in Unity. Also, Tyler encouraged me to build a baseball game. The game was quite basic, where the user could grab a bat and hit the incoming balls. Because, the game was so real, it was so difficult to hit the balls. I also tried to build the tennis game, and I used my 3D model of a tennis racket. Tennis game was not that successful as the baseball one. In both of these games, I played with colliders, which are determining the place where the collision happens with different objects that also have colliders.
A screenshot from the Baseball “Game”The 3D model of a racket
Day5 – testing new things, such as portal, new project
After building these small games, I started to test new things that I can use in Unity. I tried to use portals which can teleport the user to different place. This can be very helpful for my new project, especially if I want to implement 3D modelling, photogrammetry, and 360 videos. The portals can be an interesting way of teleporting to a different, parallel “world.”
Finally, Poland is playing against Colombia this Sunday. Hope they will do better than last time.
Finally, the World Cup has started. I am so happy, but also a bit sad. Because of the time difference, it is difficult for me to watch the matches, especially during my working hours, but that’s fine, I can follow the score from time to time and watch highlights.
Apart from World Cup, I have worked on my project and discovered so much about the capabilities of Unity, which allows me to do projects on VR.
Day1
I attended the DSSF workshop about building webpages with HTML & CSS, GitHub, WordPress in the library. It was a very interesting course and I have learned very important basic components in HTML and CSS. Also, I was able to create my first public website under school domain. It is a very simple website though, you can find it here.
Day1- 3
Before Day3 I was studying for the FAA Remote Pilot Test with Alyssa and on Day3 we went together with Dr.Remy to take the test. Fortunately, we have passed the exams and we get the certificate of remote pilot for Small Unmanned Aircraft System, usually called drones. So now I can legally and responsibly fly the drone and implement the drone into my own project, which can be very helpful in creating 3d models using photogrammetry.
Mavic Pro DJI drone
Day4
On Day4, I focused on embedding the 360 videos to VR environment in Unity. Actually, it is a bit funny how the 360 video is projected in Unity. We have to use a sphere to put the 360 video on the surface of the sphere, but then we have to “reverse” the film and put the film not on the outer surface of the sphere but inner surface. Then, we can watch the film from inside the sphere and look around to watch the film like a 360 video. I also used teleporting, so that there is a way for an user to teleport into the film, when she/he wants to, and there is also a way out, after getting bored with the film.
360 video within the sphere in Unity
Math fact: In order to reverse the film to play inside of the sphere rather than outside, we have to reverse the normals of the vectors that are projecting the film on the surface of the sphere.
Day5
Apart from video, what makes VR experience more “realistic” is audio (more human senses are “fooled”). I played with audio by adding some sound effects to objects once triggered or to object collisions, when at least two objects are interacting with each other. Moreover, Unity gives the possibility of using 3D audio by implementing the Doppler effect, which makes the sound have higher frequency when closer to the user. Furthermore, I even tried to add some sound when the user is walking to make it even more real.
Next week I will try to build the scene and start filming some videos to add to my project.
With the beginning stages of my project underway, I have found that Unity3D as well as utilizing HTC Vive with Steam VR is much more difficult than it seems at first glance 😉 Nevertheless, I am very excited to work for the first time in my life with Virtual Reality and be able to build my own projects for it.
Day1
On my first day I met my DTSF team and went to the Library to meet other fellows from DSSF. It was very friendly to get to know other people and their projects. Later, I was planning my timeline for my whole project for 8 weeks ahead. I found out that it’s not easy to plan so far ahead. However, it is important to plan so that you are more confident of what you are doing and reduce the risk of running out of time (or on the other hand if you finish before time, you know what you can do more). I also downloaded the Unity program to get to know the program interface. It is possible to download the program for free for personal use here.
Day2
I started the tutorials to make a simple 3D game on Unity, Roll-a-ball, which showed me so many possibilities of Unity and that there still so much more to discover. The tutorial was quite straightforward and anyone who is interested can do it. I learned to build the simple plane with some cubes and spheres in Unity and could add some physics to the objects. Therefore, the objects will behave more realistic and so the game will be authentic. Even though the game was simple, I was so excited that I could build that game and even tried to test it with the HTC Vive headsets.
The Roll-a-ball game made in Unity
The rest of the week
For the rest of the week, I tried to get to know the Unity more from the VR perspective. I read many articles and youtube tutorials to find more about VR and Unity program. I learned how to integrate the camera in the headset with the camera in the Unity program. In fact, SteamVR plugin, which we can get for free in the Unity asset store, is very helpful for that mission. It allows me to connect with the VIVE camera rig and the controllers.
HTC VIVE headset and controllers
I learned a lot about movements and interactions in VR. I made two models of gun simulations, where you could shoot with a gun, using a trigger button. It also had a bullet that shot out of the gun with the sound effect. The simulation was simple but looked quite real to me. Moreover, I tried some interactions with some objects, e.g. grabbing the cube, throwing it, catching, opening the door, etc. I also focused on movement in VR, e.g. you can walk in VR, by simulating the hand movements, you could use the buttons to move, or you can also teleport, which is very helpful in movements in bigger areas. It is also important to learn the C# language, which is the main language for writing Unity scripts. Fortunately, there are many tutorials showing and teaching C# scripting in Unity. Also, there is a very important toolkit, called VRTK, which has many necessary scripts for SteamVR, so I don’t have to write scripts all over again.
Gun simulation version1Gun simulation version2
As you can see, the Unity above looks different on 2 screens. I used different versions of Unity on different computers and I encountered problems when I tried to import a game from one Unity to another, so now I know that it is better to work on 1 computer only, or use same versions on both computers.
I am looking forward to my project. I will start building more developed environment in the Unity and I hope this will bring me closer to my final goal.
Below is my timeline with end goals for each week, I hope I can at least fulfill them.
Timeline development with milestones:
Week1:
Goals:
I want to start creating environment with assets and material in Unity, eg. Creating a room.
I want to implement it with VR and use with VR headsets.
End Goal for week 1:
I will be able to build simple scenes in Unity.
Week2:
Goals:
I will start building one of the general green area of the Gettysburg and adding hotspots where I can later add 360 video to certain hotspots.
I want to pass drone exam that might be helpful later for me for photogrammetry.
I will find places where I have to take pictures and videos next week.
End Goal for week 2:
I will start building more developed environment and prepare for the next week shooting.
Week3:
Goals:
I will drive to areas to shoot the 360 videos of the Battlefield. I will make photos.
If I will have drone license, I will use droning photos to create the 3D models of statues and monuments in the Battlefield
I will use photogrammetry to create 3D models.
I will try to add videos to my VR environment.
End Goal for week 3:
I will have the initial environment in Unity with videos (and if have drone license, then 3D models)
Week4:
Goals:
I will improve the VR scene and user interaction with 360 videos.
I will add user movement to the scene and boundaries.
Add some mountainous area.
End Goal for week4:
The VR environment will be more interactive for the user.
Week5:
Goals:
I will add more Battlefield areas to the VR scene.
I will make more 360 videos regarding the new area.
End Goal for week5:
I will add more Gettysburg Battlefield places to the environment.
Week6:
Goals:
I will add some physics to the scene by adding some pickable leaves.
I will make the movements more interactive (teleporting), hints (the path) for users where they can go.
I will add specific spots where user can read information to specific things.
End Goal for week6:
I will make the VR environment more interactive for the user to boost the VR experience.
Week7:
Goals:
Hamilton Project
Week8:
Goals:
I will add some final embellishments, such as sound, shades.
I will find any errors and fix them.
Finalizing the project.
I will improve my speaking skills in order to present my project to the public.
