Auto-Nurse (Speech-Driven Manual Assistant)

Project Develop an intelligent device that assists a human who is performing a complex but well-known
procedure. The prototype device will integrate speech recognition, video analytics, robotic control
and perhaps text-to-speech to manipulate objects at the command of the user. I envision a
scenario in which different shaped objects are placed on a table; for example a square, star, circle
and triangle shape. The user gives a command, such as, “Put the star in my left hand.” The
device moves into a position that allows it to identify the different shaped objects, selects the star
shaped object, turns toward the user, identifies the user’s left hand, and places the object in the
user’s hand. The prototype could be quite simple, in which the geometry of the problem is
constrained for ease of implementation.
Applications
This device can have applications in medical, industrial manufacturing, educational, and
rehabilitative applications. An example medical application is providing surgical instruments to a
doctor in an operating room. An industrial manufacturing example is handing parts or tools to an
assembler upon request. Educational or rehabilitative applications may allow a user to instigate
the action, for example “give me the ball”, or the machine may reverse the roles and instigate the
action saying “roll the ball to me”