WEEK 6 (July 9-13th)

It’s quite hard to believe that I only have about two weeks left of the DTSF internship. It is a bittersweet thought indeed!

I am happy to say that I made a lot of progress with my project throughout the course of this past week! However, I first needed to gain administrator access on the computer in order to download programs such as Meshmixer that I needed to fix errors in my models. On Monday, I initially wanted to see how my Pix4D project files would look if they were printed with the 3D printer instead of the Carvey machine for comparison reasons. I exported the projects as .stl files and Tyler assisted me with printing my Stine Lake and West Quad model with the 3D printer. However, the triangle meshes did not mesh well together so I used Meshmixer to resolve all 47 defects in the composition of my model and tried to print the model again. I had to change the dimensions and scale of the project in order to decrease the size of the model, otherwise it would have been too large to print. I also exaggerated the z axis so that every subtle change of the elevation of the terrain would be very noticeable.

I then uploaded my fixed 3D model with completely connected triangle meshes to Fusion 360 and from there I uploaded the project to Easel. This allowed me to print the model on the Carvey 3D Carver machine. However, I was encountering many issues generating the toolpath of the model on Fusion 360 once I imported the project. Director Remy had an idea to work around this issue – I could also use a computer plugin program called Paint.net to edit exported photos of heat maps from my Pix4D models demonstrating different terrain levels. This allowed me to create photographic layers of each elevation level of the models that were individually carved by the Carvey machine. I executed this by adjusting the hue and saturation of the heat map demonstrating DTM which I then converted from .png files to .stl. Even though this method worked, Director Remy and I realized that it may not have produced models that were as scaled and accurate as they could have been. Because of this we approached a new tactic of exporting png files of the contour lines of each model into paint.net where I created individual photos of filled in portions, indicating where the Carvey needed to cut each elevation level layer.

I spent a hefty amount of time perfecting and fixing errors in the elevation images used to carve out different levels of the terrain. This task was difficult at times because if the photos were not perfectly sequential in regards to increasing elevation and filled in the exact location of the change in terrain height, the Carvey machine would not run its mission or the model would be carved inaccurately with incorrect dimensions and scales.

This past week I created carved wooden pine models of Quarry Pond, Stine Lake, and West Quad. Director Remy was even kind enough to drive me to Boyer Nurseries and Orchards so that I could use the Mavic Pro drone to take photographs of the open, hilly terrain. The model of the third area of the orchards that we visited was the most difficult to create due to its drastic and abundant amount of elevation levels. However, I was extremely satisfied and proud of the model that was carved of the orchard in the end!

This week Hoang, Orrin, Director Remy, R.C, Professor Oechler and I will be at Hamilton College in New York for an Immersive Technology Conference. I am excited to see what types of projects other students are working on as well as to expand my knowledge of various programs and types of technology!

 

 

Photogrammetry Model of Boyer Nurseries and Orchards Inc using Mavic Pro drone and Pix4D software.

 

Contour image of elevation level set to 10 foot increments, exported from Pix4D.

 

 

Aerial mosaic view of Boyers Orchards, DSM, and DTM (tree elevations not accounted for).

Week 5 Progress

This week I was able to finish my new Penn Hall model. This new model is much more accurate to the real Penn hall. The roof us more accurate that before, the dimensions of the building scale to the actual building. The step count is very close to the actual amount of steps (16). There are no problems with the windows like there were before. The polygon count is significantly lower, and there are no overlapping vertices. This time I was able to make the entire mesh out of a few shapes which made it much easier to apply textures. This time I kept the mesh in mirror mode which meant if I ever needed to make a change or improvement, I would only have to do it to one quarter of the building and it would automatically be applied to the rest of the building.

After the model was finished, I had to add textures to make Penn hall more realistic. I found a good online texture for the bricks that matched up perfectly to their original. For the windows I filled them in with a dark blue material. The hardest part was the roof. There were no textures online that matched the roof, but I had an Idea. Alyssa and I went out to Penn Hall with the drone to take pictures of the roof, I used those pictures to texture wrap the roof,  while there isn’t a normal map for it, the roof still looks much better and realistic.

The next step was to build the environment around Penn Hall. The most difficult thing was the sidewalks. I didn’t think of an efficient way to do it so I made them in Blender by hand. The process was very tedious but I think I made them look decent for now. I will have to go back to them later to make sure they look good enough for the final presentation. I started putting in the trees near Penn Hall as well as the grass. The scene is starting to come together. Going into next week, I will try to get as far as I can in touching up the scene. Afterwards, I will add in the VR Interactions. I do not think this step will be that difficult because I already spent a lot of time with it and there are a lot of base scripts given to you through Unity assets.

Week 5 Update

Day 1 and 2


Day one and two of this week were arguably some of the most productive of my time here so far. At this point I feel like I have a good understanding of how to produce quality production parts from my 3D printer. I was able to print some possible prototypes for the mechanism that pushes the insulin out of the reservoir. They all work on the same principle, so here is an example of one I am testing:

All of the extruders that I am testing are open source and on thingiverse.com .

Day 3


On day 3 I realized that I had completed all of the pieces necessary to have a crude pump except for the code that controlled the extruder. I used the python code mentioned in my last blog post, along with code to control the stepper motor, and merged the two to work together. Once I had successfully merged the two pieces of code into one python program, I used the blood sugar value that is updated every 5 minutes to control a specific amount of steps (at this point it is not calibrated, but it does work!). I then placed the entirety of the code into one big while loop that allows me to pump a desired amount based off of blood sugar values every 5 minutes for a desired amount of time. This was a huge success in my program having the code finished to the point where I can start making physical prototypes, and the project is really coming together now.

Day 4 and 5


Both of these days were fairly slow considering most of the day was spent waiting for pieces to print out. On the bright side I was able to successfully print out a screw and (basically) a threaded nut that works. For anyone who knows 3D printing knows how difficult this task really is on an FDM printer (the type of printer that I am using). This gives me hope for creating an insulin pump that is completely 3D printed aside from the Raspberry Pi and motor. One of the main issues that I have with current diabetes technology are the astronomical prices for the technology, and one of my main points in my project was making an insulin pump that is very cheap to produce, and 3D printing allows for costs to be far lower.

While I was waiting for my prints to emerge out of the build plates I kept busy with Dr. Remy setting up 3D printers that the IT department had ordered. We wanted to see just how fast we could get a printer from the box to a functioning state. We were able to achieve this task in ~51 minutes, which is pretty impressive considering how many moving pieces make up a 3D printer. Also, setting up 3D printers helps gain a further understanding of how the printers work. This is why I suggest new users to order 3D printer kits rather than fully assembled printers. For one, they are much cheaper when they are in the kit form, and you get a better understanding of how they function.

Hopefully by next week I will be able to have my first fully functioning prototype of an insulin pump which is very exciting. More to come in next weeks blog post!

Week5 – Getting Closer

Hello Everyone,

Belgium is very close to the final. After eliminating Japan in 1/8 and Brazil in 1/4, Belgium will be facing France in the semi-finals. France and Belgium are the best 2 teams at this time, and it will be a very interesting and close match.

Day1


Alyssa taught me how to do photogrammetry with a drone using Pix4D. I made the 3D model of a stone using different photogrammetry programs to test which one will be the most useful, such as Pix4D, 3DF Zephyr, and Agisoft Photoscan. All of them work quite similarly, that means they start with building a dense point clouds from 2D photos taken earlier from the drone, then create a 3D mesh from a dense cloud, and finally add the texture to the mesh based on the photos.

Photoscan- dense point cloud
3DF Zephyr – 3D mesh
Pix4D – Textured Mesh

Day2


I started building the fences surrounding the cemetery. They look quite realistic. I had to calculate the measurements of the fences in real life and convert it to the measures in Unity. I also looked at positions on the map of every fence in order to place them correctly in my Unity scene. My next step will be building the tombstones.

Day3


4th of July – Independence Day 🇺🇸

Day4


I started to model tombstones in Blender. Luckily, the shape are quite simple, so it was not difficult for me. The only “problem” was that there were many different shapes tombstones. It took me a bit longer, but it was not even an issue. The more I model in Blender, the more I get used to working in Blender. At the end, I made 9 different shapes of tombstones. The next stage is to add the proper material and more realistic feel.

Day5


To make the tombstones more realistic, I added the real rock texture on the tombstones that resemble the same material from the cemetery. Then I shaped the flat faces of the model to be more “rocky” by adding the roughness on the faces using different brushes. I used some free brushes from BlendSwap, where users can share their own brushes with everyone and append it in Blender. Here, I used the brushes from here (shout out to rubberduck!).

Have a good weekend!

JHA

WEEK 5 (July 2 – 6th)

It seems as though the more challenges are brought forth to me, the more of these challenges I learn to overcome! It is a very rewarding feeling to know that I have taught myself (with the guidance and advice of others) how to operate so many different types of software, programs, and devices!

Officer Taylor, Associate Director of Public Safety at Gettysburg College, saw my updates on my LinkedIn page of my project and reached out to me asking if Gettysburg College could use my photogrammetry models for emergency evacuation plans and other management. I am very excited for the opportunity to not only complete my project on terrain levels and flood patterns but to also help make my college a safer environment. I am currently working with Officer Taylor to work out the details of how the models will be used for this new and exciting extension to my current project!

Earlier this week I worked on finishing up leveling the levels of my isolated terrain level Pix4D model, making a few mistakes along the way. For example, I mistakenly loaded in surfaces that were not a part of the ground level and had to remake my model of East Quad because the changes were already saved and loaded in as part of the processing of the 3D mesh. However, fixing this error did not take too long and I learned to be more efficient and accurate in filling in voids with surfaces and loading them into the project correctly. I also entirely remade a new, improved version of Quarry Pond this week where the entire model is displayed as a single model unlike my last draft. I did this by starting from the very beginning and taking a new set of 3D grid and circular photos of the pond using the Mavic Pro drone and loading the photos as a new project file.

I also created accounts and familiarized myself with the Carvey 3D Carver, X-Carve, along with computer numerical controls (CNCs) in general to gain a better sense of what types of technology I would be utilizing. I even learned a bit about firmwares and gcode, a type of coordination code used for CNCs. Aside from that, I created trial projects using TinkerCAD, Easel, and Fusion 360 which are computer-aided manufacturing (CAM) programs that are usually used interchangeably with the Carvey 3D Carver. These programs are used for designing prototypes and objects before the file is sent to the 3D carver and printed. However, I have yet to discover how to transfer a Pix4D project to the Carvey 3D Carver machine. For this I need to export the  Pix4D project file as either an .obj or .stl file and from there import it into Fusion 360 or TinkerCAD. I would then transfer the project into Easel so that it can be printed by the 3D Carver. So far I am encountering errors that do not allow me to export the projects from Fusion 360 into a .stl file, which is strange. I plan on looking more into this issue tomorrow!

One thing I am very proud of that I did this week was I figured out how to get exact geolocation coordinates from a drone photo (or any photo linked to satellite and GPS) and from a photogrammetry Pix4D project. Discovering how to do this took me an extensive amount of time, much longer than I had anticipated, but the results were nonetheless very rewarding! Essentially, I had to draw and export polylines in Pix4D around the points and load the exported .kml files into Google Earth. Do not be fooled! This process was much more complicated to figure out than one may think!

I am both excited and a bit nervous about my final weeks of the DTSF internship this summer. Because I will be attending the Hamilton College Virtual Reality conference in New York in about one week, I am doing my best to completely finish my project by the end of next week. I need to figure out how to link my Pix4D projects to the Carvey 3D Carver as soon as possible. It is merely one task but it requires an extensive amount of time to figure out how to do.

I am very proud of my work and progress so far and I am looking forward to the opportunities that lie ahead as usual!

 

 

 

 

 

 

 

 

 

 

 

Easel, a CAD program made by Carvey Inventables to be used for the design process of 3D carving projects.

 

 

 

 

 

 

 

 

 

 

 

East Quad in Fusion 360, another CAD program used to design projects.

_____________________________________________________________________________

Pix4D Model of Quarry Pond (updated and fixed)

 

 

Pix4D Models of Isolated Terrain Elevation Levels

Stine Lake and West Quad

 

Quarry Pond

 

East Quad

 

Creek on North Washington St.

Week 4 Progress

It feels like the time spent at gettysburg flew by. I am officially at the half way point in my project. So far everything is going on schedule. I believe I completed all of the seperate aspects of the project and now is the time to bring it all together. These next few weeks are very exiting because I am able to apply what I have been learning throughout the summmer and make it into a finished product.

Most of the work done this week was spent with world building and level design in Unity. I started off by aquiring several assets that can enhance the experience of the VR user. The first asset was called Time of Day. It is a special skydome in unity that allows time to pass, and as that time passes, the light moves with the sun. The other assets were rocks, trees, and grass that helped the user feel like they were in a realistic nature environment.

I started off the first day with a forestry design tutorial. It was roughly 2 hours of content. It took me the first couple days to complete. I got to learn a lot about level design and what goes through a game developers head when making levels. There were a lot of tools in Unity I used that I was not aware of. I intend to use these skills and assets when building the environment for my project.

However, upon completing the forest scene, I ran into a problem that I did not expect. The models I was creating in blender did not have the realistic detail to match the environment. It looked like a cartoon building in a photo-realistic forest. The problem with this is when I attempted to add texture maps and detail to penn hall in the past, it never properly transferred to unity. Currently I am hand-making my own white brick texture for penn hall so I know that it will be able to transfer to unity.

Starting next week I plan to remake penn hall to take out all of the problems that I had with the first one. This new model is going to be much cleaner and it will also have the applied brick texture I was working on.

Week4-Half-full, Half-empty

Hello Everyone,

This week Poland “beats” Japan and goes home in a very bad style. It was a very disappointing World Cup for me and Poland. My another favorite team, Germany, … I have no words.

Day1


I worked on creating portals in Unity. This is a set of 2 mirrors which are visualizing the reflection of what another mirror would show. This portals allow the user to go into the mirror-portal and teleport to the place what the mirror is showing. There is an asset called Vive Stereo Rendering Toolkit which is doing the task.

Day2


I was trying to build the models of the bowling of my own and add some spice to it. First, I created the 3D models of a bowling ball and bowling pins in Blender, with no colors added. Then, I used the ZBrush to add some rock roughness to the bowling ball and pins, ZBrush allows to edit the sculpture of the 3D model. Afterwards, I needed to unwrap the 3D model in order to be able to color the specific places of the ball and pins, UV Layout by headus was a perfect program for that job. Finally, to add some material and coloring to the ball and pins, I used Substance Painter. In Substance Painter, I could connect everything above together and add the final touch. Then, Substance Painter will create the maps (normal, height, albedo, metallic smoothness) necessary for Unity to apply all the modifications to the 3D model of a ball and pins.

Notes: I used the free versions of these programs, so don’t hesitate to try them out.

Day3


Data Visualization workshop

Day three was very interesting, because I attended the Data Visualization workshop taught by Sharon. I learned a lot about how the data should be presented in order to make it easily understandable for the reader. Also, I learned that it is not easy to make a proper chart/graph, and there the aesthetics are very important in order to attract the attention of the reader and make it clear for the reader. Moreover, it is very easy to fool our eyes, we only know what we see, presented to us. But it is important to see the bigger picture, which is often difficult (or even intentionally) to present to the reader.

Day4


I tried to fly the drone with the mobile application DJI 4 Go. This application worked well. Basically the app allows to control the drone using the phone, to see what the drone is seeing and to photo or film the moment. I also tried to use Pix4D mobile app, this app could not be used without having another app Ctr+Dji, which connects to the drone and sends the data to Pix4D. However, because I did not have an appropriate phone, the app was available for the phone in the Play Store, so I downloaded somewhere else.

Later that day, I did a very stupid thing and broke the drone propellers, I really don’t want to talk about it, but want to apologize Eric for it. It was so embarrassing and disappointing. I felt so bad. I am so sorry.

Day5


I know that I am behind my initial timeline, but I still have time and with all the skills that I gained during these 4 weeks, now I am able to build what I need for the rest of my project.

Fixed timeline:

Week5

-Starting building the 3D scene in Blender with texturing and import to Unity, creating the scene.

Week6

-Creating a 3D model using drone photogrammetry and importing it to Unity along side with the 3D model.

-Filming 360 static videos of the cemetery and inserting to the Unity scene as the hotspots.

Week7

-Hamilton conference

Week8

-Improving the project and preparing for the final presentation.

Have a good weekend!

JHA

 

Week 4 Update

Overview


After last weeks successes with getting the readings from my CGM (continuous glucose monitor) into my cloud database, it was now time for me to figure out how I could get those values out of the cloud so that I could use them to run my insulin pump.

Day 1 and 2


At this point I had realized that getting the readings from the database to be usable would be quite the task taking my lack of knowledge in python into consideration. I knew that I would need to use some type of driver to get the data out of the mongo database, however, I had never coded a task like this before so it was quite the learning curve. A driver basically allows the user to get the data from their collections in the database, and to be able to use that data in their programs. Luckily mongoLabs, the company that I am using to host my database has a driver through python called pymongo that allows me to access my data in python. This would be an efficient way for me to use the data that I was collecting from my CGM.

Early on Monday morning I was also surprised with the arrival of my own 3D printer that I had ordered. After seeing how much was possible with the 3D printers, I knew that I needed to have my own, so much of Monday was spent setting up and tinkering with my new printer.

Day 3 and 4


With help from Dr. Remy, the fourth day of this week I was able to have a working program that would fetch the data that I wanted from my collection and output my blood sugar value, the date and time, as well as the direction in which my blood sugar was trending. With this I am able to start programming my final program that will control my insulin pump. 

Here you can see that I ran my python program, and actively collected my blood sugar values every five minutes for over an hour.

Day 5


Now that I had a working program to collect my blood sugar information it was time for me to experiment with python programs that control stepper motors to pump out the insulin. Essentially a stepper motor allows you to program a motor that will turn a specified amount by the program. I do not yet have a working program that controls the stepper motor based off of my blood sugar values, however it should not be too difficult to program, so look out for that in the next blog post.

On day five we also talked about our project timelines and how our projects were coming along. With four weeks down already, the presentation date is fast approaching. I found that I had achieved many of the goals that I had set out to achieve and more. There are endless possibilities in my project and I am working to implement as many features as I can, and I have decided that I will be trying to achieve my stretch goal of creating a truly closed-loop insulin injection system. In the coming weeks I will be perfecting my program that controls the stepper motors, and 3D printing prototypes for the final insulin pump. The project is beginning to take shape and it is a truly rewarding process. At this point it is promising that I will be able to achieve my stretch goal, and maybe even beyond that.

WEEK 4 (June 25-29th)

This week I mainly worked on resolving issues in my photogrammetry models and making sure that they are the most presentable that they can be. One issue with the first draft of my models is that the mosaic editor was only displaying DSM (density surface models) instead of DTM (density terrain models). DTM is more applicable for tracking flood levels because it excludes the height of infrastructure that could give false interpretations of the elevation of terrain. I resolved this issue by resetting the step 3 density point cloud features and reloading the final step of the project processing for all five of my key campus locations (East/West Quad, Stine Lake, creek, Quarry Pond).

Another problem I found was that the mosaic editor of Pix4D did not always give accurate readings of the land – terrain levels are shown with a color scale that ranges from red (highest elevation) to purple (lowest elevation). However, tall buildings were still showing up on my models as indicated by their red color. I noticed that in places such as Stine Lake, man-made features like concrete sidewalks were perceived to be the same height as raised land elevations when they were actually much lower. Also, the creek by North Washington Street behind the Office of Multicultural Engagement building was shown on the DTM color map as being the same elevation as the land around it when it is actually slightly lower. Aside from that, even though I was using the DTM feature, some buildings were still interpreted and read as being part of the terrain. I made an inference that this may have been partially due to the fact that the drone did not get extremely low-angled photographs of different land masses from a close enough point of view. I tried to resolve the issue by taking manual photographs of certain areas with the drone where the DTM was not accurate on the map viewer as well as taking more low height photos in a circular movement around structures such the East Quad. Although the low-angled perimeter photos helped me receive more accurate DTM readings, some of my manual photographs were not processed by Pix4D because the angles and height of the photos varied too much. I was a bit flustered by this but I realize that it is all part of the learning experience. Despite the setbacks I am experiencing, I am still advancing in my project at my own pace. I wonder if there is any way the Pix4D mapper pro app would allow me to fly below 99ft if I was in a completely open area that may be detected by GPS? That may improve the quality of my photos for future models.

During the second half of my week I worked on making my models look more visually appealing. There was a lot of extra terrain that was a part of the rayCloud 3D mesh that I did not want to focus on. I managed to figure out how to use the rayCloud editor to trim, crop, and delete data visuals that I thought diverged from the focus areas of my project. I utilized Pix4D features and tools such as drawing polylines and grouping pixels of the model into different categories and reprocessing the 3D mesh so that the triangle meshes of the model would match up with the point cloud editor version of the projects. I also familiarized myself with SketchFab, a 3D model software that allows me to share my Pix4D models by embedding the html code into various websites. Take a look at some of my models below!

I created copies of each of my five models with isolated terrain.  However, when I edited the buildings and other land features like trees from my models, their absence left gaping voids one each model where the infrastructure use to be. The next steps for my project are to find a way to fill in the voids with preexisting land that is in line with the current terrain level. I also need to learn how to export my terrain models from Pix4D to the Carvey 3D Carver so that I can begin creating my wooden models before the Hamilton conference in two weeks. Aside from that, I must recreate my Quarry Pond model so that is is portrayed as a single model as opposed to two separate, floating land masses as it is currently shown to be in my first draft. My current Quarry model looks like a movie set from Avatar which is not what I want at all!

I had originally planned on starting to use the total station this week but only now do I realize that I will only be able to incorporate the total station at the very end of my project once I finished my wooden, physical models due to time constraints. I also have come to realize that learning how to properly use the drone and its features along with Pix4D had taken me longer than I had anticipated. However, I am still fairly on track with my initial project timeline.

I’ve made wonderful progress in my project but there is still plenty left to accomplish! I feel a bit nervous knowing that I have essentially two weeks left to reach my goals but I am determined to do so nonetheless!

 

 

First drafts of models before trimming and merging:

East Quad

 

Stine Hall and West Quad

 

Quarry Pond

https://sketchfab.com/models/3b628f58f6634fbaa31a9ceeb4d70c71

 

Creek on N. Washington St

Week 3 Progress

Day 1

The first day there were some mistakes with my Penn Hall model that I had to fix. There were a lot of unconnected vertices that I had to manually connect which took most of the day. Unfortunately it took the full 8 hours to fix.

Day 2

There was a lot of progress but I still need time to understand it. I went through a tutorial to learn the basics of VR development. It included features such as walking around, picking up and throwing objects, and opening doors. I made my Penn hall intractable in VR and players can open the doors to go inside. The last half of the day was spend trying to find a way to move between scenes in unity. This is one of the most important skills to learn in unity, especially for my project. This allows me to be able to walk into Penn hall and use a scene change to go to the interior.

While it is possible to create my entire project in an entire scene, it would have been quite difficult and I would have had a lot more limitations. With the scene change, I can now start a fresh new model that contains the interior. Also using scene changes is one of the most common functions of unity. It is commonly used for changing between levels in a game. A developer can use each scene to focus on a specific goal rather than compile everything into one area.

Now that I am able to do this, the next step is to model the interior of penn hall.

Day 3

I spent the morning modeling the second floor of penn hall. I tried to capture as much detail as possible. I also found a better way to change scenes through using a special teleport area. This method is much simpler. All the user has to do is open the doors to the building and move into the interior of penn hall. Then, the user is instantly transported to the next scene.

Days 4-5

The new computers came not much work was done. A lot of the time was spent setting up the new desktops and transferring all of the work from the laptop to the new location. Unfortunately, this was not very successful and I wasn’t able to save all of my work. Other than the models made in blender, everything else could not be transferred properly, such as the Unity projects. I will have to re-implement the interactions but it will be much faster this time around.