Week 6: Connecting The Dots

Welcome to the week 6 update, where I started to connect the software and hardware components. While I much have rather been at the stage of calibrating how much the motor should turn given a specific unit measurement, much progress was still made. This week I started by creating a Python file that would allow me to test that the Raspberry Pi and stepper motor would communicate with each other effectively. I was able to get a file that asks for a number of steps so that the stepper motor would rotate. This was a success and did not take long to function. I also developed my Python file that deals with speech recognition. The code now asks the user for a command. I also had to brainstorm possible situations where the user might be missing information. Handling these situations is important because we want all the necessary information to be able to get a valid measurement. This was all possible because of my knowledge from developing the Alexa Skill earlier on in my project. During the progress of my Alexa skill, I was able to pick up on dealing with missing information and valid commands. One aspect I did implement was making sure that the user’s command has a valid wake phrase, so the speech recognition program knows to take action. Also implementing follow-up questions for the missing information. Even though I did not use the Alexa skill, I was still able to make the most of that learning.

During this software development, I did face some trouble with handling if the user command read the quantity number as a word or an actual number (Ex. one or 1). This was something I, later on, notice while testing. Luckily Python has so many libraries to access that I was able to find one called word2number that takes in a string and picks up on any number word to convert it into a number unit. Sometimes the speech recognition would pick up on the actual number unit, other times it needed the support of the word2number library.

The last thing I worked on for the software was implementing the text-to-speech attribute. Text-to-speech is the component that allows whatever text is written on the computer to be said out loud. This is a very common tool used among the visually impaired community, in order to get an assistant to read out loud what is on the screen. This is important to implement so that the user can get the voice assistant to have an almost natural conversation when there is missing information. I was able to find Google’s text-to-speech, however, I later found out that it is not available for use on Raspberry Pi. I spent many hours trying to find another replacement, but none were successful. I ended up taking a break from it and it will be one of the tasks to come back to over the weekend.

On the hardware side, more 3D printing occurred. After testing the stepper motor with random step values to turn the hardware components, I found that I had to reprint the spiral piece again but with the resin 3D printer. I had to reprint the spiral because it was rubbing a lot with the case inside. Also, the PLA 3D printer made extra material hang from one of the sides which makes substances like the rice more prone to get jammed. Below is an image of the PLA spirals vs the resin spirals. As you can see the resin spirals came out much smoother.

On the note of 3D printing, I also began to print other components to the hardware to clean up. I printed out a substitute to replace the plastic water bottle I was using as the hooper. I will let the following picture show you what I walked into the next day… you can imagine my reaction.

For now, I will continue to use the plastic water bottle as the hooper because I tried several times to print this part and every time, it had a malfunction. Additionally, I printed casing and supports for the electric components this weekend. I took time to measure the electric components and brainstormed ways to neatly compact them in a box where there would be no wires in sight in my final presentation. I also had to find a way for the support that would elevate the spiral case, which could nicely line up with the stepper motor, because the metal rod goes through the spiral and attaches to the stepper motor. For now, the grey elevating support in the picture is perfect, however, the black casing box is currently being reprinted because the stepper motor is too tight and does not fully fit in width. Other than that, the casing and support should be perfect in no time.

Also, I have to wait on a piece for the Raspberry Pi because it needs a dependent to take control of the pulse time. After testing the stepper motor over a long period of time, we noticed that the motor was sounding funky toward the end of the given number of steps. So if we told it to move 200 steps which is a rotation of 180 degrees clockwise, it would make a weird sound at the end of its cycle. This has to do with not having time to keep a constant pulse so that the stepper motor can be under control and in sync with the give commands.

Next week I will be focusing on calibrating as I mentioned at the beginning of my post. I hope to create a Python library, meaning a conversion table to cross between motor rotation and unit measurement. If the user asks for 2 cups of rice, I would be able to access the library and easily get the number of steps the stepper motor should make. This will be more testing to see how much one spiral can hold in each section and its average weight. This will be the last component of my project to finally be able to have a minimum working voice-controlled dispenser. Hope to see you next week!

Week 6: A surprise failure

I’m not really sure were to start with this week. The clock is starting to quickly tick down, we’re down to about nine days left as I write this. This week has not been the most productive, but at the same I feel like all I need is one last big push and I’ll be done, hopefully. As the project goes on, the deadline feels more suffocating. Our projects would have gone from awesome to mind-blowing if we had a few more weeks.

Anyway, practice presentations also took up a good part of the week. Between practicing for them and making slides for them, I’d gather we all lost at least half a day if not more. They all went pretty well and we all got some great constructive criticism.

In some fun news, I got to use saws! I say saws because I needed to add some height to my frame, so with some guidance from Josh, I cut two pieces wood, with an electric and hand saw, glued and screwed them together, and zip tied them to the base. Apparently, I got unreasonably excited, judging by how everyone was talking about my excitement well into the afternoon. Anyway, the base adds height and gives my frame some strong support.

The last two days of the week were spent taking apart and putting my project back together, for multiple reasons. First, when I got in on Thursday, the switches I was essentially using as brakes for my rail system broke. I went through several troubleshooting steps. I checked that all the wires were hooked up. I checked that the switches were still working. I saw that they were interfering with each other after Eric had me write a simple program to just print what the signal that the arduino got. I checked that the arduino wasn’t dead or that its ports weren’t dead. After a day of troubleshooting, Eric, one of my supervisors for those who don’t remember did some research and found out that I had messed up the switch wiring. Apparently, you’re supposed to have switch go to the arduino ports and ground, which in electronic is basically a sink back to where the electricity came from to complete a circuit. Honestly, this was the most frustrating thing about this project, the thing that is too simple to fail, failed. It is all working again, at least.

I’ve found that my servo motor was misbehaving as well after some confusion, I realized it was yet again a power supply issue. We tried calibrating the servo several times, as it just kinda moved on its own and had theries that it was some memory issue. I’m not sure what it is right now but I suspect that either servo is shoddily made or the power supply is not up to the task, after some online research. I was actually to work around the issue for a bit and tried giving it it’s own power mostly because I was told it would be best give anything arduino related its own power source.

In better news, I added some code to make the steppers move together simultaneously with a separate program. Previously, I had to move one then the other in separate code, this was not an issue as I did not need them to stop all other codes until recently. The other program lets me stop all other code with no issue and works with the original, allowing me to move control both individually when needed and to have them halt other codes when necessary.

As the final presentation looms over us, I’m trying my best to get everything working in time and I know everyone else is doing the same. See you next week.

Week 5: More complications

My next step in the project was to further advance my program. At that moment, my drone could only follow commands after the execution of the command given beforehand. However, we wanted to be able to continuously give voice commands while the drone is still in execution and be able to execute the first command in the queue. The only way I thought we could be able to do this is by using threading. I went back on trying to use threading to solve this issue. I constantly got errors and did not know what was going on. First of all, I had an issue with storing the result from one thread and passing it to the other. When that didn’t work, I also tried creating a global variable that could be shared by the two threads in my program. However, variable sharing requires synchronization of the threads so that the variable would not be modified by both the threads at the same time. I used locks to do this. I could execute the program at this point but only the voice recognition function was executed and the program did not perform well. I tried modifying and debugging the code multiple times but it seemed like nothing worked.

I had been stuck with the same issue for a week and I was getting a bit frustrated because of that. So, I decided to keep looking at the problem while also doing some research on object recognition because I was already late on where I needed to be at this point in the project. So, I did some digging on the internet and read some articles on how python handles real-time object recognition. I plan on start implementing it next week.

Week 5: More Technical Challenges

Hi, welcome to week 5, where the project faced multiple challenges. This week I started off the weekend with more 3D prints, mainly tweaking measurements in the prints. Below are images of the 3D print series I have compiled, keep in mind that is not all of them! This will be an ongoing theme where I will most likely still be doing tweaks to parts. I do still have to print out a part to finally replace the water bottle as a hopper and support for the entire structure itself. This will be done while I complete other parts of my project, as they will take long hours or even days.

Continuing with 3D printing, I was able to test the two model types more at the beginning of the week. I have established that I will be using the spiral mechanism. The reason behind this is because it jammed the least and when I printed a bigger version. This design was much smoother with testing and is now able to support substances from small grains of sugar to pinto bean size. Below I have included images of the spiral in its case with a rod through it and the view from different perspectives. I will have to reprint the black lid because it is currently on the looser side than I want.

Apart from 3D printing, I started to look into the implementation of the Alexa and Arduino. There were a lot of technical issues beginning with finding MAC addresses for them to register them on the college’s network. From there I was having difficulty trying to think of a way to connect my voice measure skill to the skill that many online sources were indicating that I needed to use in order to connect to the Arduino. At this point, it was later in the week and I know I need to move quickly. We decided that it was best to move on using a Raspberry Pi and the speech recognition Google API that a fellow DTSF peer is already using. This way it would be easier to connect audio and the motor components. I did have slow progress with this because I was also out for a day due to sickness. I also had difficulties with the Raspberry Pi, because although it is not the fastest alone, it did have issues with overheating and freezing the screen altogether. The lesson is to safe progress often. I started with the PyCharm IDE, which was recommended by peers, however, I think the Raspberry Pi was not responding kindly because of it. Therefore, I did have to switch to a different IDE, VSCode. I have used this IDE previously, so it was easier to work with. After many mishaps and technical issues, I finally got the Raspberry Pi to pick up on speech recognition. I will have to develop this code further next week to be able to cut the string that the audio picks up to be able to tell how much the user wants to measure and what they want to measure. I also got all the chords to the stepper motor, Raspberry Pi, and power to be all set. I have now to work on making the stepper motor working, which will be the first thing I do next week. Then the task is to implement these two components together and start calibrating to the different substances. Stay tuned for next week’s progress!

Week 5: Iteration, programming, and slow advancement

The theme of this week, in my case, seems to have been iteration. Over the course of the week, I kept adding and refining details of the assembly. I’ve been slowly improving an testing my code. I’ve added new pieces to the system and clarified some details that I hadn’t been able to properly plan without getting at least this far into the project.

I had some arduino shenanigans again, this time from two facts I was not aware of. The first fact is arduinos cannot receive a program (Arduinos need to get programs from other machines to operate) when something is connected to ports 0 and 1, which I wish I had known before I connected something to them and then spent a day tearing my hair out until I read that little fact that my supervisors, who use arduinos pretty often had not known about in the slightest. The second fact I learned is that if you connect a positive wire back into a positive circuit, it can fry an Arduino. Thankfully, we had a spare it gave me a pretty good scare.

I’m not sure if I found this week or at the end of last week, but we also discovered that there are at least two types of servo’s: standard and continuous rotation. Standard can only move around 180 degrees, while continuous rotation keeps going as long it has power. These two types are basically identical, arduinos even use the same commands for both. I was using a standard in my magnet raising system to grab the cubes, but then saw I needed a continuous rotation to get the most movement out of it.

One is standard and the other is continuous rotation. You can only tell if you get a clear through casing or preserve the stickers on the other side (they’re still there I just wanted to make it clear just how similar the tow types servos are in everything beside there movement).

The bulk of the week was spent programming and debugging, mostly just trying to get my assembly moving in a predictable way without destroying itself. I’m serious about the destroying itself, the motor’s are too simple for built in emergency stops, so they just keep spinning until something breaks, like in my case the hold on the belts they were pulling. I now have functions to move the assembly in all possible directions, trigger the magnet, read from the scale, and to do all of this at once. In other words, I have all the basic functions needed for the start moving and sorting the cubes.

I also spent some time making small additions to the assembly. I added emergency stop switches that have been really helpful, lubricated some sliding parts, and game some parts their own power line. The downside is I now have a rat’s nest of cables, back of the TV bad. Midway through the week, I discussed with my supervisors the best way to make sure my blocks stayed in the same place. We figured the best option was a base with cones sticking out so the blocks could slide I. Going off some by hand testing, the mechanism works pretty well.

The base with some new blocks that accommodate the cones.

I am happy to report that I finally have the assembly moving to pickup a block, putting it in one spot and having it return the block, or at least something very close to that. While I’m a bit disheartened from how slow the progress, I’m quite confident I will have everything ready by the time the presentation comes. Hopefully I will have at least a sort or two working by next week.

Fifth Week – Designing and Data Collection

DESIGNING AND DATA COLLECTION

I came up with a design that suited my environment more. I call it an all-in-one P.O design. It is cylindrical in shape and has an LED and light sensor attached to it. With this design getting a more accurate reading would be little or no problem because it has stability, and the light sensor and LEDs are tightly fitted with zero percent movement. Also, to ensure perfect stability, I used siliconized acrylic latex sealant to make sure there was not any LED movement while taking readings. Below are the diagrams of the pulse oximeter.

A view from Tinkercad of the All-in-one device.
Preparing for slicing

Data collection started immediately after my device was all set up. And currently, a lot of observations have been made and they are as follows:

  • No movement of the device or its components while taking readings for an accurate result.
  • The tip of the LED must be directly facing the light sensor.
  • The position of the finger is important and should not be too deep into the device.
  • The IR values from the data has a saturation of 65535 and for the visible 0.
  • I constructed a graph that shows the plot of the IR values and Visible values, and it was observed that they both had similar visualization but different values i.e. they moved at the same rate.
view of visible(RED) light graph
view of Infra Red (IR) light graph
  • After taking readings on myself, overtime the light ray’s data reduces due to the LED being ON for a long time. But are they ratio significantly close because if they are it does not matter?

So, the ratio between the values in the IR was calculated to see the difference and it turned out they have significant ratios. GOOD NEWS! Then the ratio of the IR and Visible was calculated and they also gave me similar ratios. This is what we wanted so it did not matter if the light ray’s data reduced over time.

  • Two samples were acquired next. Sample A was a male and a person of color with very dark skin tone and Sample B was a male Caucasian. The reading was taken from their middle finger on the right hand, and we made some interesting observations. The average of the ratio of IR and Visible light passing through the finger for the person of color and that of the male Caucasian had a large difference. Could this be because of the thickness of the skin or the clear color difference of the skin?

Next week we would be assessing a larger sample class with different skin colors and using both the index and middle finger on the right hand and on the left hand, we would be using the already manufactured pulse oximeter to see the difference. So basically, taking readings at the same time and seeing the difference between what we built and what’s in the market selling. By doing this we would be able to answer some questions that are arising.

Fourth Week:

I started this week trying to figure out the issue with keeping the drone stable after taking off. We thought of using multi-threading to solve the issue. However, since I had never worked with multi-threading before, especially with python, it took me a while to learn and implement it. I wasn’t able to solve the issue using the technique, so instead, I stepped back and looked for ways I could solve this issue without using any complicated concepts. Then, I found out that there were multiple libraries for the Tello drone apart from the official one that I used earlier. I found easyTello to be the most efficient and flexible library to use. Although it didn’t have a lot of documentation and tutorials on its functionality, it was pretty straightforward. I used it and could finally pass other commands such as go left, land, etc.

My next task was to add more advanced commands. I was able to add some commands but the processing speed of the commands was quite low. I thought the problem might be in the transfer speed of the adapter I was using. To verify this, I tried using an ethernet cable to get internet access and tried connecting my laptop to the drone’s wifi. However, I wasn’t able to do so so we went back on using the adapter for the drone connection. I added some more commands, did some testing and debugging by referring to the API documentation and everything seemed to work fine.

Then, I started working on making my program more flexible in terms of how commands are passed. At that time, I had to wait until a command was executed before I could pass another command. But, I wanted to be able to pass a command sequence and have the drone follow it one by one. I did so by breaking the command sequence into smaller command segments using different keywords such as go or move. Then, I extracted integers from the command sequence and passed it to the drone so that it can execute commands passing any distance or speed according to the choice of the user. My plan for next week is to polish the speech recognition interface and move on to the object recognition part of the project.

Fourth Week- Application of Tinkercad and Cura software for 3D print.

Tinkercad is a user-friendly app for 3D design, electronics, and coding. I also used an Ultimaker Cura to slice my design from Tinkercad. I got acquainted with these softwares and used them for my designs to be able to contain both the LED(RED and IR) and the light sensor.

The above designs were done on Tinkercad
A view of Ultimaker Cura for slicing before its transferred to the 3D printer.

These designs are designed to hold my light sensor and LED lights. I faced a lot of failure from the product of my print. I had to consider how much the curve would be for the finger to rest on. I also had to ensure there was no movement from the sensor, LED’s and inserted finger to ensure an accurate result. Also we don’t want the pulse oximeter to be too bogus we want simplicity but also very effective. I came up with several designs but failed in some way. It was either the holes for the LED’s and sensor were too small or too big. In some cases where the finger was meant to rest wasn’t curved well enough. It was also important that the LED was facing directly on the light sensor(the middle) which was also a discovery I made from the other prints produced. Below are pictures of the 3D prints I made that had one or more of the above explained errors.

Failed 3D prints samples of the LED and light sensor holder.

I have been working on trying to perfect my design before a final print. It gets frustrating but the hunger to get it right is more than the frustration and I know I will get it to work. Professor Eric Remy would say ” We are too focused sometimes on one particular goal we forget that there is more to do and achieve” So I plan on moving forward with my project like working on smoothing my data output and also trying to figure out the right design, maybe it doesn’t have to be what we commonly see in the market and that is two rectangles that hold the LED and light sensor. It would be nice to think outside the box and design something more conducive and suited to my environment. Hopefully by next week I hope to have something better than my above designs.

Week 4: 3D Prints

Welcome back to the progress of my project! This week I started off with a prototype of my phyisical device. I used a small tube to be a cast for the spiral mechanism. A water bottle as a hooper to hold the substance, in the picture I have places rice. I also had a paint brush to spin the spiral which is in side the tube because the spiral at the time could not fit a metal rod. This is something I had to address later on. I used the materials to quickly test if my deisgn would work before I printed any parts our. Fortunately, I was able to have rice dispensing from the bottom hole at the other end of the tube. In the image you can see that rice has gone through the spiral after a couple turns and fallen onto the plate.

First prototype model using tube, water bottle, paintbrush, foam, plate, and rice.
3D printed case for paddle with metal rod through center for large substances.

Since the metal rod did not fit into the spiral and I had to use a paintbrush for now, as seen in the image above, I had to reprint the spiral with a different hole diameter through the center. That being said, this week I also focused on continuing to print parts for my prototype with slight measurement modifications. At first, I began printing the spiral design that I mentioned in last week’s post. I printed these spirals with different hole sizes for the metal rod that will spin the spiral. This cost me a lot of time to be able to test the spirals. However, once I began to progress with my 3D prints, I figured that I could print out smaller ring sample sizes to make sure things were fitting properly. This is better than waiting for an entire 3D object to print in order to test. The image above on the right is a design that supplies a paddle design inside the case that is different than the spiral. This design I imagine will be more useful for large substances such as beans.

This process was not smooth, there were some errors along the way. Below is an image of what I found at the lab the next day. I had put an 8-hour print and came back to this mess. Needless to say at this point, there were many prints made this week.

Technical difficulties along the way…

After printing a series of spirals and cases, I had many tests to conduct with different substances. Below is a test with Rice and beans. At the beginning of my tests, I experienced many jams with the paddle and the rice and beans. This was because of sizing and gaps inside between the spiral and case.

These are some images of the prototype, I am closer to a much smoother working mechanism. I also got the MAC addresses for the arduinos to register them on the network. Next week will be dedicated to connecting the Alexa and Arduino Wifi Board.

Week the 4th: A Balancing Act

Greetings. This week started out promising, but I quickly lost time to another with my frame, this time due to an issue with balancing everything. I ended up losing about two and a half days on this and combined with slow progress on other fronts led to a week that I would not call very productive.

On Monday, I was able to quickly get the mechanism to raise and lower the magnet functioning quite well, though I was slowed down by some issues with the screws (had to fight them to get into place), that I haven’t properly resolved yet. I then setup a prototype assembly, but ran into an issue with balance and the fact that I’d yet to place the contact switches, switches that send a signal when they touch something and a good stop signal for my mechanism. It honestly could have been so much worse if we hadn’t had multiple printers and I hadn’t printed several overnight.

On Tuesday, I tried to get the balance and assembly problems, but was slowed down when we realized that I need some new 3d prints, some modified plates to make sure the bearings moving along the rails stay in place. Also, had the frame kept titling over quite easily. This process took about two days and essentially froze everything else in the project as the bearings are essential for the movement. This process took so long because the 3d printers can only print so fast, it took about 2-3 hours per print in this case, and I need several to test with and four in total for the final job.

Some of my attempts that got so numerous, I had to start marking to stay at least kind of organized

Thursday came along and the prints from last night had come out well enough, but there were still some issues. I spent most of the morning sorting out the good rails and bearings and examining how good the prints came out. As I found out, linear bearings and rails come in varying quality levels and that can affect how easily the system moves. The better the rod and bearing, the smoother the motion. In retrospect, I probably should have talked with my supervisors about the prints a bit earlier or tried to focus in on the task a bit more.

Presenting the graveyard of the misfit 3d prints, most of which are from just this week.

As I write this and I wait for the latest print to come out, I realize that this week seems to have been dominated by waiting for 3d prints, so the frame is more balanced, so I can start actually moving the magnet in a meaningful way. Side Note: writing this a bit early gave some ideas on what to do, the frame and attempting to work on the up and down mechanism.

By Friday, I was ready to try and get everything reassembled. Thankfully this entire detour actually worked out quite well as the mechanism worked beautifully once I got everything fitted back in. The improvement is night and day, the front and back section in particular glides when it use to only slide, as in front skateboard to hoverboard in terms of smoothness. I also managed to connect the up and down mechanism with some workarounds (tape and wires make for shockingly good screw replacements). I also solved an issue with the lower parts wobbling by re-tightening the nuts and some tape. I can finally say that I can start to work on getting the robot to pickup and weigh blocks.

My sorter at the time of writing