Project

#	Title	Team Members	TA	Documents	Sponsor
21	Campus Tour Guide by AI-Powered Autonomous System	Bob Jin Hao Ren Weiang Wang Yuntong Gu		design_document1.pdf design_document2.pdf design_document3.pdf proposal1.pdf	Simon Hu
	This [link](https://accurate-ringer-067.notion.site/Campus-Tour-Guide-by-AI-Powered-Autonomous-System-f4d17e16378740e2948f5bef4afd7315?pvs=4) contains the html version of our project description. # Team Members * Hao Ren 3200110807 haor2 * Xuanbo Jin 3200110464 xuanboj2 * Weiang Wang 3200111302 weiangw2 * Yuntong Gu 3200110187 yuntong7 > 💡 Note: this doc provides an overview of the project “Campus Tour Guide by AI-Powered Autonomous System”. We start by re-iterating the problem. We then present our proposal and solution. We also draft an initial plan to help build `v0`solution. # 👀 Problem Anyone entering a place for the first time, like an university, can be quite challenging. Knowing where you are, how to get to your destination, how to optimize your routes, knowing factors that will influence your routes can be complicated. Having a real-time interactive system that guides people through this process is needed. It has been possible yet not able to scale because it’s not open-sourced, and its hardware isn’t standardized, and is expensive. The interaction isn’t versatile enough to adapt well under the ever-changing applications. A cheap and versatile solution is needed. ------ # 💭 Proposal ## Solution Overview Our solution utilizes autonomous UAV to guide our clients, sensing them and the environment, such as obstacles and drone’s location with a sensor module, controlled by a control unit which orchestrate a series of tasks. Our solution is cheap, open-sourced, and versatile to meet the need of a generalized and sustainable long-term solution for our campus and many other applications. ## Solution Components Our solution contains the following parts: a sensor subsystem, a control subsystem, a mobility subsystem, an inter-connect module. ### Sensor Subsystem - Identify obstacles - Identify the person to lead, exclude the other people - GPS location ### Control Subsystem - Deploy routes ### Mobility Subsystem - A drone ### Inter-connect Module - Inter-communication of control unit, peripheral sensors, and the drone - Supply power to the sensor module and control unit. ## Criteria for Success ### Milestone 1 - drone can be controlled and moved independently - GPS can sense the location - Sensors can be powered ### Milestone 2 - Drone can be controlled by control subsystem - control subsystem can receive signal from GPS module and sensors - Routes can be output (not necessarily by moving the drones) ### Milestone 3 - Without obstacle, the system can follow the human - Without obstacle, the system can fly from A to B and slow down / stop when human is too far away - System can identify obstacle and plan a route to avoid them ### Milestone 4 - With obstacle, the system can fly from A to B and slow down / stop when human is too far away - The starting point and ending destination pairs can be selected, e.x. 5 pairs of (A,B) is available. ### Milestone 5 [optional] - An easy web app which sends signal to the system - System can receive our instruction (vocal) and design a destination and lead the clients - Support interactive chatting mode to help understand the surroundings ## Alternatives SKYCALL currently provides a similar version of guiding tour for MIT. But that project isn’t open-sourced and the hardware are not cheap enough, or easy-to-maintain. Our solution is different in that we provide - Cheap solution - Open sourced solution (software + hardware), each component will be documented - Unnecessary functionality will give its way to generality - Versatile enough to support our campus (which is drastically different to MIT) ------ # 🛫 Division of Work - Xuanbo Jin: Xuanbo excels at software works. He should do the algorithm part of the design and also takes part in the firmware integration. - Yuntong Gu: Yutong’s strong background at electrical engineering makes him a great candidate to test the validity of different hardware and connect them to the object. He should also helps the communication between each components. - Weiang Wang: Enabled by weiang’s strong background in electrical engineering, he should actively helps the communication and interfaces between components. - Hao Ren: Hao can do assorted works. Hao should actively do the software and firmware part of the work. Hao should explore the validity of possible direction and iterate the version of the projects properly. Hao should organize the roadmap and update it frequently, examining the priority of each part by experimentation and analysis.

Intelligent Texas Hold 'Em Robot

Xuming Chen, Jingshu Li, Yiwei Wang, Tong Xu

Featured Project

## Problem

Due to the severe pandemic of COVID-19, people around the world have to keep a safe social distance and to avoid big parties. As one of famous Poker games in the western world, the Texas Hold’em is also influenced by the pandemic and tends to turn to online game platform, which, unfortunately, brings much less real excites and fun to its players. We hope to develop a product to assist Poker players to get rid of the limit of time and space, trying to let them enjoy card games just as before the pandemic.

## Solution Overview

Our solution is to develop an Intelligent Texas Hold’em robot, which can make decisions in real Texas poker games. The robot is expected to play as an independent real player and make decisions in game. It means the robot should be capable of getting the information of public cards and hole cards and making the best possible decisions for betting to get as many chips as possible.

## Solution Components

-A Decision Model Based on Multilayer Neural Network

-A Texas Hold'em simulation model which based on traditional probabilistic models used for generating training data which are used for training the decision model

-A module of computer vision enabling game AI to recognize different faces and suits of cards and to identify the game situation on the table.

-A manipulation robot hand which is able to pick, hold and rotate cards.

-Several Cameras helping to movement of robot hand and the location of cards.

## Criterion for Success

- Training a decision model for betting using deep learning techniques (mainly reinforcement learning).

- Using cv technology to transform the information of public cards and hole cards and the chips of other players to valid input to the decision-making model.

- Using speech recognition technology to recognize other players’ actions for betting as valid input to the decision model.

Using the PTZ to realize the movement of the cameras which are used to capture the information of pokers and chips.

- Finish the mechanical design of an interactive robot, which includes actions like draw cards, move cards to camera, move chips and so on. Utilize MCU to control the robot.

Project Videos

Video of A Real Game from Intelligent Texas Hold'em Robot