Excited to share my summer break project, turning our son Dexie’s teddy bear Ellee into a robot who can move her head and converse like a human, powered by GPT-3, Computer Vision and some robotics running on a Jetson Nano.
She is also able to recognise a person for a more personalised conversation and extract the unrecognised person’s name from the conversation and to register their name & face for future encounter, automatically extending her social network :)
Watch the demo video here to witness her conversation skill!
Ellee is a generative based chatbot who can converse on various topics such as history, physics, maths, even cooking, movies, sports, etc on par with human conversation intelligence. She really understand what you said and is able to look back to past conversation to generate a contextual response.
This is different to a retrieval based chat bot such as Siri, Google Assistant and Alexa which require a huge database of intent, question/answer to match.
She is able to move her head to follow the person she is talking to. The head motion was built using two servo motors with pan and tilt setup.
The technology used:
- GPT-3 (conversation generation, name entity extraction)
- MobileNetSSDV2 object detection (human detection)
- Dlib (face detection & recognition)
- NVIDIA Jetson Nano (the embedded hardware platform)
- Amazon Polly (speech syntesizer)
- Google Speech Recognition (voice recognition)
- Servo motors + PWM drivers (head movement)
Read the full story here and access the full source code here