Project

# Title Team Members TA Documents Sponsor
27 Real-Time Sign Language Translator
Gene Lee
Kaelan To
Bonhyun Ku design_document1.pdf
design_document2.pdf
final_paper1.pdf
presentation1.pptx
proposal1.pdf
# Real-Time Sign Language Translator

Team Members:
- Gene Lee (genel2)
- Kaelan To (kto3)

# Problem (Describe the problem you want to solve and motivate the need.)
Technology is improving rapidly, and with that, it serves the purpose of making our lives easier. We want to leverage the technology available to us and further integrate those with disabilities more into our society; specifically deaf individuals in an academic setting. When we imagine students with hearing impairments working with others, we think of the other students having to wait for the student to type out their thoughts. However, brainstorming/bouncing off ideas requires rapid discussion in order to spark good ideas. Sometimes typing may not be as fast (which might hinder the group) or even may not be accessible to students (especially in K-12) in classroom settings.

# Solution (Describe your design at a high-level, how it solves the problem, and introduce the subsystems of your project.)
We propose a portable real-time sign language translator to solve this problem. We would utilize computer vision to differentiate the hand signs and feed the visual input to a microcontroller and give audio feedback (sound translation of the hand sign). This portable system would assist them in communicating with teachers, but most definitely help deaf students work in a team with other students efficiently.

# Solution Components
Explain what the subsystem does. Explicitly list what sensors/components you will use in this subsystem. Include part numbers.

## Visual Input
A camera will be used to read input from the user and send that information to the central processor.

## Central Processor
The central processor will decode the input from the camera and send that information to the audio component. We plan on using a pose estimation library on a Raspberry Pi to process the input.

## Audio
Speakers will take the decoded output from the central processor and play it out from the encasing.

## Power
The battery should have a large enough charge to last about 8 hours, which is a typical school day from grades K-12. They also need to be small enough to fit in a relatively small encasing to be more portable.

## Encasing
All the components are expected to fit inside a casing that would be portable enough for a student to carry around from classroom to classroom without much trouble. The goal is to be able to fit all components inside a 15x15x15cm cube or other encasing of equal or smaller volume.

# Criterion For Success (Describe high-level goals that your project needs to achieve to be effective. These goals need to be clearly testable and not subjective.)
Able to identify sign language and translate into English in real-time (threshold set to be within 0.5 seconds)
Able to identify signing at a moderate/conversational level speed (threshold to be set after more discussion/research)
System is lightweight/portable (not hard to carry around)
Battery lifetime of at least one school day(8 hours)

Cypress Robot Kit

Todd Nguyen, Byung Joo Park, Alvin Wu

Cypress Robot Kit

Featured Project

Cypress is looking to develop a robotic kit with the purpose of interesting the maker community in the PSOC and its potential. We will be developing a shield that will attach to a PSoC board that will interface to our motors and sensors. To make the shield, we will design our own PCB that will mount on the PSoC directly. The end product will be a remote controlled rover-like robot (through bluetooth) with sensors to achieve line following and obstacle avoidance.

The modules that we will implement:

- Motor Control: H-bridge and PWM control

- Bluetooth Control: Serial communication with PSoC BLE Module, and phone application

- Line Following System: IR sensors

- Obstacle Avoidance System: Ultrasonic sensor

Cypress wishes to use as many off-the-shelf products as possible in order to achieve a “kit-able” design for hobbyists. Building the robot will be a plug-and-play experience so that users can focus on exploring the capabilities of the PSoC.

Our robot will offer three modes which can be toggled through the app: a line following mode, an obstacle-avoiding mode, and a manual-control mode. In the manual-control mode, one will be able to control the motors with the app. In autonomous modes, the robot will be controlled based off of the input from the sensors.