Furhat SDK
All the power of social robots, right on your desktop
Furhat comes with a very powerful suite of tools allowing you to create, debug, deploy and analyse your applications on your desktop.
Available on Windows, MacOS, and Linux.
Expressive character design
Change the face and voice of the robot to create immersive characters
Wizard-of-Oz interface
Integrated and programmable Wizard dashboard with camera feed
Conversation modeling
Program using Kotlin – the fastest growing programming language in the world
- Row
- 1,3 Quickly define states and triggers
- 3,13 Use built-in Furhat behaviors, or define your own, using Kotlin extension functions
- 16 Use your own intents
- 17 Easily integrate application with back-end
- 22 Use built-in intents
1 var Idle = state { 2 3 onUserEnter {user -> 4 furhat.attend(user) 5 goto(OfferService) 6 } 7 8 } 9 10 var OfferService = state { 11 12 onEntry { 13 furhat.ask("Can I do anything for you?") 14 } 15 16 onResponse<RequestDirections> {response -> 17 furhat.say("Okay, just go ${directions(response.intent.place)}") 18 furhat.say("Anything else I can do for you?") 19 } 20 21 onResponse<No> {response -> 22 furhat.say("Okay, let me know if you need any help") 23 goto(Idle) 24 } 25 26 }
Visual programming
Visual programming environment, based on Google Blockly
SDK Features and Specifications
Supported Operating Systems
Windows, Linux and Mac
Run Time Environment
Java 1.8
Programming environment
Kotlin 1.3
API support for other languages
Python 3, Perl, Bash, C#, C++, Erlang, Go, Groovy, Java, Node.js, Objective-C, PHP, PowerShell, R, Ruby, Rust, Swift, Typescript (Angular1.x, Angular2.x, Fetch, jQuery, Node), and more
Key Simulation Features
- Virtual Animation of Furhat for simulating interactions
- Ability to simulate microphone and camera input using mouse clicks and text input
- Real-time view of interactions using a dialogue dashboard
- Visual animation of the physical space and users around the robot
- A browser-based dashboard to control the robot remotely and access the camera feed of the robot
- And much, much more!
Visual programming
Custom Blockly-like drag-and-drop visual programming tool.
Additional Tools
- Wizard-of-Oz interface for remote control and experimentation
- Log viewer for visualisation of collected data
- Lip-syncing for using pre-recorded audio files
- Audio file hosting for logging interactions
- Web dashboard for remote monitoring of interactions
Logging
Cloud Dialog Logger using AWS servers for logging of all or subset of interaction data to the cloud.
Listening
- Multiple recognition hypotheses
- Priming of specific expected user speech
- Intermediate speech results
- Speaker detection in group interactions
Vision
- Face Detection
- Face Recognition using faceprint
- Smile detection
- Ability to control all of the above using a simple API in the SDK.
User modeling
- 5+ simultaneous users
- User tracking (0,5-3 m)
- Head pose estimation (Pitch/Yaw/Roll)
- User attention detection
- Situation modeling of the physical space in front of the robot
Speaking
- Vocal gestures (in selected voices), SSML tags (Amazon Polly)
- Pitch, Speaking rate, Loudness control
- Ability to switch voices and language programmatically at any point in the interaction
Face & Neck
- 40 blendshapes (poses) for creating custom facial expressions
- Switching facial textures and models programmatically
- Physically swap magnetic masks
- Programmable shaders to adjust colours, lighting and contrast in the face depending on environment light
- Control of 3 degrees of freedom in the neck
- Ability to create facial and neck gestures programmatically and on the fly
More details
Download the Technical Product Overview for extensive details including Furhat’s hardware, software, and SDK
The Furhat SDK – now freely available to all
COVID-19 has made working from home the new normal, which makes experimenting with social robots in physical environments very challenging. That’s why we have decided to open our Furhat SDK to all, available without charge. Tap into the world of social robots from the comfort of your living room and build the future of our changing world.