Author: Karla Gonzalez

Welcome to the final reflection of my project! This has been one journey and I have learned a lot. I faced and overcame many challenges that I did not think I would encounter. Being a computer science major and only have dealt with software, I remember thinking to myself how well I would handle the hardware side of my project. There are many physical mechanisms that we don’t think how difficult they are until you need to work with the design. One design I remember vividly from my project that made me realize how challenging designing hardware is the hopper component. I had initially started with a water bottle for my hopper but I then designed a cleaner version on TinkerCad. Creating this design was difficult to perfect because I realized that it took a lot of time for engineers to come up with a great design for even a single water bottle. As you can see there were many small bumps in the road that showed me a different perspective of the mechanisms in my project. I remember at first when I said I wanted to create a voice-controlled dispenser, but I had no clue what materials or any details would be entailed. The first week we were told to make a list of materials and budget, this did not come so easy to me as I couldn’t think of what I wanted to use. The opportunities were endless with all the resources in the Innovation and Creativity Lab (ICL). I was glad that we had many chances to practice our elevator pitch because it pushed me to think of possible obstacles and solving my problem from different perspectives. I initially had not even thought about using a voice-controlled method, I thought about a button approach.

This project was very important to me and close to home because I was inspired by my father and uncle who are both visually impaired. Living with my father, I have seen the struggles of just doing basic everyday things like cooking. It can get quite messy at times when measuring everything out. That is why I decided I wanted to work on an impactful project where it can serve as an everyday application. My project is a kitchen tool that can be vocally prompted to measure the desired measurement.

During this project, there were many lows and highs. The first obstacle I encountered was choosing an already existing voice assistant and programming a skill with it. There are many voice assistants out there, I ended up using an Amazon Alexa. I was worried about how much I would be able to implement due to this being my first time. However, it was quite easy to program my skill because there were so many fantastic tutorials out there. I will say there were small technical things to be on the look for when setting the settings for the skill because that is what caused me to not be able to test locally or publish my skill to use on my actual device. But it didn’t take long for me to quickly scratch this implementation. I ran into the issue of connecting the Alexa to an Arduino. I needed an Arduino in order to connect to a driver and stepper motor, which was necessary to actually have the physical parts moving. There was an already existing skill connect these two, but it was taking me too long to comprehend how to integrate the Alexa-to-Arduino skill with my own Voice Measure skill. Therefore, I was fortunate that another peer of mine was also working with a voice assistant. I was able to smoothly transfer to use the Google Speech-to-Text API on a Raspberry Pi. The Raspberry Pi overall was more convenient because I was able to program directly to the pins to control the motor and have faster processing. After this challenge, I really faced problems with sizing my 3D prints to fit nicely. I also had some problems with the motor acting up over time because the motor would be out of sync with the commands coming from the Raspberry Pi. I had to reset the Pi multiple times for this reason. I would have liked to have attached a hat to the Raspberry Pi, but it was coming down to the last few days and I was unsure how many changes and how long it would take to integrate.

Overall, I am very proud of my project and happy that we had the liberty to choose our own projects. I learned not only about technologies but about project management as well. It is a very much needed skill especially because I will be having my senior CS capstone in two years. I will definitely take my managing and budgeting skills for the future. I also built my presentations skills, which is something we all need to practice. During the weekly presentation updates and elevator pitches, I grew much more comfortable in owning what I am working on and learning. I also learned a lot of hardware troubleshooting and use some technologies that I was not initially exposed to. I will be taking all I have learned this summer to stretch my abilities to help in the ICL as the assistant once again.

My future plans for this project are to continue this during the time I work at the ICL this coming year. I hope to implement a Text-to-Speech feature where my dispenser assistant can speak back to the user. I also hope to be able to support liquids. I was successful in calibrating some solids and adding conversion tables to go between units. There is still more I would like to do and I hope I can have the time to continue and talk to more people to see what other useful features I could implement.

Lastly, I just want to say that I feel blessed to have had the opportunity to be a part of DTSF this summer. I want to thank Eric and Josh, our supervisors for challenging and helping us along the way. Alongside, a big thank you to my DTSF peers who made this summer one of the best yet and always were so willing to help at any given time. That being said, thank you for following along with the progress of my summer project.

Welcome to the week 6 update, where I started to connect the software and hardware components. While I much have rather been at the stage of calibrating how much the motor should turn given a specific unit measurement, much progress was still made. This week I started by creating a Python file that would allow me to test that the Raspberry Pi and stepper motor would communicate with each other effectively. I was able to get a file that asks for a number of steps so that the stepper motor would rotate. This was a success and did not take long to function. I also developed my Python file that deals with speech recognition. The code now asks the user for a command. I also had to brainstorm possible situations where the user might be missing information. Handling these situations is important because we want all the necessary information to be able to get a valid measurement. This was all possible because of my knowledge from developing the Alexa Skill earlier on in my project. During the progress of my Alexa skill, I was able to pick up on dealing with missing information and valid commands. One aspect I did implement was making sure that the user’s command has a valid wake phrase, so the speech recognition program knows to take action. Also implementing follow-up questions for the missing information. Even though I did not use the Alexa skill, I was still able to make the most of that learning.

During this software development, I did face some trouble with handling if the user command read the quantity number as a word or an actual number (Ex. one or 1). This was something I, later on, notice while testing. Luckily Python has so many libraries to access that I was able to find one called word2number that takes in a string and picks up on any number word to convert it into a number unit. Sometimes the speech recognition would pick up on the actual number unit, other times it needed the support of the word2number library.

• 

• 

The last thing I worked on for the software was implementing the text-to-speech attribute. Text-to-speech is the component that allows whatever text is written on the computer to be said out loud. This is a very common tool used among the visually impaired community, in order to get an assistant to read out loud what is on the screen. This is important to implement so that the user can get the voice assistant to have an almost natural conversation when there is missing information. I was able to find Google’s text-to-speech, however, I later found out that it is not available for use on Raspberry Pi. I spent many hours trying to find another replacement, but none were successful. I ended up taking a break from it and it will be one of the tasks to come back to over the weekend.

On the hardware side, more 3D printing occurred. After testing the stepper motor with random step values to turn the hardware components, I found that I had to reprint the spiral piece again but with the resin 3D printer. I had to reprint the spiral because it was rubbing a lot with the case inside. Also, the PLA 3D printer made extra material hang from one of the sides which makes substances like the rice more prone to get jammed. Below is an image of the PLA spirals vs the resin spirals. As you can see the resin spirals came out much smoother.

• 

• 

On the note of 3D printing, I also began to print other components to the hardware to clean up. I printed out a substitute to replace the plastic water bottle I was using as the hooper. I will let the following picture show you what I walked into the next day… you can imagine my reaction.

• 

For now, I will continue to use the plastic water bottle as the hooper because I tried several times to print this part and every time, it had a malfunction. Additionally, I printed casing and supports for the electric components this weekend. I took time to measure the electric components and brainstormed ways to neatly compact them in a box where there would be no wires in sight in my final presentation. I also had to find a way for the support that would elevate the spiral case, which could nicely line up with the stepper motor, because the metal rod goes through the spiral and attaches to the stepper motor. For now, the grey elevating support in the picture is perfect, however, the black casing box is currently being reprinted because the stepper motor is too tight and does not fully fit in width. Other than that, the casing and support should be perfect in no time.

• 

• 

• 

Also, I have to wait on a piece for the Raspberry Pi because it needs a dependent to take control of the pulse time. After testing the stepper motor over a long period of time, we noticed that the motor was sounding funky toward the end of the given number of steps. So if we told it to move 200 steps which is a rotation of 180 degrees clockwise, it would make a weird sound at the end of its cycle. This has to do with not having time to keep a constant pulse so that the stepper motor can be under control and in sync with the give commands.

Next week I will be focusing on calibrating as I mentioned at the beginning of my post. I hope to create a Python library, meaning a conversion table to cross between motor rotation and unit measurement. If the user asks for 2 cups of rice, I would be able to access the library and easily get the number of steps the stepper motor should make. This will be more testing to see how much one spiral can hold in each section and its average weight. This will be the last component of my project to finally be able to have a minimum working voice-controlled dispenser. Hope to see you next week!

Hi, welcome to week 5, where the project faced multiple challenges. This week I started off the weekend with more 3D prints, mainly tweaking measurements in the prints. Below are images of the 3D print series I have compiled, keep in mind that is not all of them! This will be an ongoing theme where I will most likely still be doing tweaks to parts. I do still have to print out a part to finally replace the water bottle as a hopper and support for the entire structure itself. This will be done while I complete other parts of my project, as they will take long hours or even days.

• 

• 

• 

• 

Continuing with 3D printing, I was able to test the two model types more at the beginning of the week. I have established that I will be using the spiral mechanism. The reason behind this is because it jammed the least and when I printed a bigger version. This design was much smoother with testing and is now able to support substances from small grains of sugar to pinto bean size. Below I have included images of the spiral in its case with a rod through it and the view from different perspectives. I will have to reprint the black lid because it is currently on the looser side than I want.

• 

• 

• 

• 

• 

• 

• 

Apart from 3D printing, I started to look into the implementation of the Alexa and Arduino. There were a lot of technical issues beginning with finding MAC addresses for them to register them on the college’s network. From there I was having difficulty trying to think of a way to connect my voice measure skill to the skill that many online sources were indicating that I needed to use in order to connect to the Arduino. At this point, it was later in the week and I know I need to move quickly. We decided that it was best to move on using a Raspberry Pi and the speech recognition Google API that a fellow DTSF peer is already using. This way it would be easier to connect audio and the motor components. I did have slow progress with this because I was also out for a day due to sickness. I also had difficulties with the Raspberry Pi, because although it is not the fastest alone, it did have issues with overheating and freezing the screen altogether. The lesson is to safe progress often. I started with the PyCharm IDE, which was recommended by peers, however, I think the Raspberry Pi was not responding kindly because of it. Therefore, I did have to switch to a different IDE, VSCode. I have used this IDE previously, so it was easier to work with. After many mishaps and technical issues, I finally got the Raspberry Pi to pick up on speech recognition. I will have to develop this code further next week to be able to cut the string that the audio picks up to be able to tell how much the user wants to measure and what they want to measure. I also got all the chords to the stepper motor, Raspberry Pi, and power to be all set. I have now to work on making the stepper motor working, which will be the first thing I do next week. Then the task is to implement these two components together and start calibrating to the different substances. Stay tuned for next week’s progress!

Welcome back to the progress of my project! This week I started off with a prototype of my phyisical device. I used a small tube to be a cast for the spiral mechanism. A water bottle as a hooper to hold the substance, in the picture I have places rice. I also had a paint brush to spin the spiral which is in side the tube because the spiral at the time could not fit a metal rod. This is something I had to address later on. I used the materials to quickly test if my deisgn would work before I printed any parts our. Fortunately, I was able to have rice dispensing from the bottom hole at the other end of the tube. In the image you can see that rice has gone through the spiral after a couple turns and fallen onto the plate.

These are some images of the prototype, I am closer to a much smoother working mechanism. I also got the MAC addresses for the arduinos to register them on the network. Next week will be dedicated to connecting the Alexa and Arduino Wifi Board.

Welcome back to my process creating a voice-controlled dispenser. This week I was able to successfully submit my unique Alexa skill pertaining to this project. To reiterate, the skill I developed uses Amazon’s Alexa to command how much quantity a user wants from a substance. This is that the user can ask for “measure 4 cups of rice.” The skill is also able to store the last request that the user made. Previously I was having an issue when my skill was being reviewed by Amazon because I was not providing a valid example of how to interact with the skill. I also wasn’t able to test locally. However, after going through a couple of videos, I figured out that I had not specified the skill name in another required spot in the developer console, as seen in the images below. This was a very simple specification to overlook because the task had already been marked as done before I knew.

After making this small change, I was able to test locally. I also was approved to have my skill published on the Amazon skill store. Here are some samples of interaction with the Voice Measure skill that I created.

• 

• 

Below is also an example to demonstrate that the skill can support when the user only specifies the substance they would like to measure but not the number of cups. Additionally, it is important to note that the skill only supports when users asks for a measurement in cups at this point. The next goal is to update the skill to accept other forms of measurements such as weight, mg, liter, etc.

Apart from the skill, I was also able to draw up a first prototype blueprint. It is important to keep in mind that this design will go through several edits and iterations while testing. To explain, the dispenser has a hopper where the substance that will be measured is placed. Then there is a narrow pipe that has a spiral that spins when attached to a motor to dispense the substance into a container at the bottom end.

I began to create the 3D models on Tinker Cad for this prototype but had to take a step back and consider that these parts would take too long to print out. Therefore, I will only be printing variations of the spiral piece because it is the most important part. Other parts can use different materials lying around the Innovation Lab.

• 

• 

• 

Lastly, the following images are pictures of inspiration that I found for the spiral dispensing mechanism. One is a spiral and the second is a pet feeder design where there are small blades through a cylinder. I will be testing these two designs the following week since printing will take some time.

To conclude, I have shifted to the hardware piece of my project. Next week I hope to test my prototype with the two different spiral designs. I also hope I can begin to implement the Arduino piece into my project. Stay tuned for next time! 🙂

This week I continued the voice design development of my project using Amazon Alexa Developer Console. I have completed the other half of the Alexa tutorial provided by Amazon titled Build an Engaging Alexa Skill. I was not able to publish the tutorial skill because I kept receiving the following error in the image below. When publishing a new skill for Alexa, you must complete a full description of your skills use and background. This includes providing an example of how to invoke the skill. When I ran to certify that the skill was ready to publish, I was returned an error that tied to the sample phrase that I had.

I moved on from publishing the skill provided by the tutorial and began the implementation of my own Alexa skill for my project. This skill is called Voice Measure, which retrieves the number of cups and substance that the user would like to measure. In my last blog post I mentioned slots; To reiterate slots are dependent on the user’s request and customized information. In this skill, there are two slots: the number of cups and substance. The number of cups is the requested number of cups that the user asks for. The substance is what the user will be measuring, so they could ask to measure rice. I was able to maneuver through the tutorial and modify it to my desired skill. However, I did have difficulty with testing my skill locally as I wrote my code. I am unsure why I am unable to test it locally or even publish the skill. I keep receiving the same error as I was at the end of the tutorial. I will be looking into this error at the beginning of next week. I have found some videos on youtube where people program their Alexa skills live. It must be a small bug that is causing this problem. Next week I also will begin and establish a first draft for my physical dispenser.

Hi everyone, my name is Karla Gonzalez. For my project I am creating a voice control dispenser for those who are visually impaired; This way they can control the quantity they would like to measure from the dispenser only using a voice command. This would allow the process of measuring substances to be quicker and easier in the kitchen. As the daughter of someone who is visually impaired, I know that a device such as this one could immensely become handy. I will be designing a dispenser for solids such as rice and flour. My stretch goal is to create a dispenser that can handle liquids such as oil. During this first week, I solidified the plan for the next few weeks and compiled a list of resources to use. These resources include introductory courses and tutorials to familiarize myself with the software end of the project. I went through a couple of sources because some courses required payment to continue after a certain extent of the course. Apart from planning and compiling, I dedicated this week to familiarize myself with the idea of voice design. Voice design is the process of creating and brainstorming all the possible occurrences between a voice assistant and the user to have the most natural conversation. In this instance, I brainstormed different ways the user could ask the voice assistant to dispense x amount of solid and possible measurement units. I also considered different forms that the user can ask for a measurement: weight, unit (mg, g, etc.), cups, etc. More specifically, I will be utilizing an Amazon Alexa. This week I have completed half of the Alexa tutorial provided by Amazon titled Build an Engaging Alexa Skill. Using an Alexa is the most convenient voice assistant to utilize because Alexa can easily adapt unique skills that the user decides to implement. Additionally, Amazon provides many free courses and access to several products for developers. So far the most important basics that I have learned is the breakdown of a user command. Given an example of “Alexa, ask Daily Horoscopes for the Aquarius horoscope,” this can be broken down into mini parts. “Alexa” is the wake word that starts the conversation with Alexa. “Daily Horoscope” is the invocation name which is part where the user sets the innovation name of the skill. “for the Aquarius horoscope” is the intent of what the user wants out of the skill, which here is to find the specified horoscope. Lastly, “Aquarius” is the slot that is dependent on the user’s request and customized information. This means that the user could have asked for the “Pisces horoscope,” which relies on the user specification. I implemented into the skill from the tutorial ways to create unique slots. In the tutorial, they wanted to create slots for the day, month, and year when a user gives their birthday. This section also covered situational problems, where the user can give 2 out of the three slots. Meaning they could give only the day and year. I have yet to implement a situation where the user says “my birthday was yesterday.” Last note on the tutorial so far, I am working on dialog conversation between the Alexa and user, where there is back and forth. Below is an image for a sample of interaction with Alexa so far. Next, I will wrap up this tutorial and begin to implement the skill for this project, and choose a dispenser design.