What a busy week! It took a couple of days for me to solve the issue with python concurrency. I was finally able to do it by having a code running in the background that continuously takes voice commands while the main program runs the commands given. After solving the issue, I was able to give voice commands even when the drone was still executing prior commands. I thought my work is done after solving this however google speech to text API has not been very nice to me these days. The following pictures are the screenshots of google’s API trying to predict my ‘’take-off’ command. Some of these predictions aren’t even close to sounding similar so you can see how inaccurate they can be sometimes.
I could maybe be able to make it do better by training it if it was a model I created on my own but since it is a pre-built API, I cannot make it perform any better than this. So, I just decided to work with what I had and maybe try to use it in a silent area with a better microphone for better predictions. Furthermore, I added text-to-speech functionality to the drone as well to improve the user experience. The drone would talk back to the user when their command is being processed and when they want to know what about the battery left, flight time etc.
Moving on, I wanted to add gesture control functionality to the drone so I started looking for datasets online to create a model and train it to recognize different gestures. After working on it for a little bit, I realized that the drone’s API already had a gesture recognition feature so I had to change my plan. Then, I moved on to the idea of creating a prototype for a drone assistant that could help physically disabled people to move things around in their apartment. First, I would use QR codes for the drone to scan and go to the destination room. After that, I plan to use object recognition to have the drone recognize objects. I started this by modifying my program so that it can take video streams from the drone. We also had a presentation this week with more people. It helped us a lot to gain more confidence and also to get feedback from the audience.
