Furhat SDK

All the power of social robots, right on your desktop

Furhat comes with a very powerful suite of tools allowing you to create, debug, deploy and analyse your applications on your desktop. Available on Windows, MacOS, and Linux.

Expressive character design

Change the face and voice of the robot to create immersive characters

One of Furhat‘s unique features is its incredibly alive face. With the Furhat SDK you can:

  • Create immersive characters with personalities suitable for your application
  • Change everything from the age, gender and ethnicity to species, voice and personality
  • Choose from a wide selection of pre-built expressions or design your own
  • Control Furhat’s lips, eyes, gestures, neck movement
  • And much more!

Wizard-of-Oz interface

Integrated and programmable Wizard dashboard with camera feed

Furhat‘s SDK allows you to design your own remote control panel where you can:

  • Easily define your own buttons and behaviors using Kotlin or Blockly
  • Collect data from the Wizard controller and from the robot’s sensors simultaneously
  • Design buttons that can change depending on the dialogue state
  • Be able to trigger complex behaviours with single buttons
  • Run the Wizard in the browser and have a direct visual feed from the robot’s camera

Conversation modeling

Program using Kotlin – the fastest growing programming language in the world

  • Row
  • 1,3 Quickly define states and triggers
  • 3,13 Use built-in Furhat behaviors, or define your own, using Kotlin extension functions
  • 16 Use your own intents
  • 17 Easily integrate application with back-end
  • 22 Use built-in intents 
1  var Idle = state {
2
3      onUserEnter {user ->
4          furhat.attend(user)
5          goto(OfferService)
6      }
7
8  }
9
10 var OfferService = state {
11
12     onEntry {
13         furhat.ask("Can I do anything for you?")
14     }
15
16     onResponse<RequestDirections> {response ->
17         furhat.say("Okay, just go ${directions(response.intent.place)}")
18         furhat.say("Anything else I can do for you?")
19     }
20
21     onResponse<No> {response ->
22         furhat.say("Okay, let me know if you need any help")
23         goto(Idle)
24     }
25
26 }

Visual programming

Visual programming environment, based on Google Blockly

One of the newest additions to the Furhat SDK is support for easy visual programming using Blockly. With Blockly you will be able to:

  • Learn the basics of how the robot works in minutes
  • Build concept interactions in minutes
  • Quickly run skills directly on the robot and see the execution in real-time
  • Fun and creative tool for learning how to program interactions and to explore Furhat’s capabilities with no technical requirements
  • Powerful for designing skill concepts, defining Wizard-of-Oz interactions, etc. directly in the browser
  • Plug & Play with zero setup time

SDK Features and Specifications

Supported Operating Systems

Windows, Linux and Mac

Run Time Environment

Java 1.8

Programming environment

Kotlin 1.3

API support for other languages

Python 3, Perl, Bash, C#, C++, Erlang, Go, Groovy, Java, Node.js, Objective-C, PHP, PowerShell, R, Ruby, Rust, Swift, Typescript (Angular1.x, Angular2.x, Fetch, jQuery, Node), and more

Key Simulation Features

  • Virtual Animation of Furhat for simulating interactions
  • Ability to simulate microphone and camera input using mouse clicks and text input
  • Real-time view of interactions using a dialogue dashboard
  • Visual animation of the physical space and users around the robot
  • A browser-based dashboard to control the robot remotely and access the camera feed of the robot
  • And much, much more!

Visual programming

Custom Blockly-like drag-and-drop visual programming tool.

Additional Tools

  • Wizard-of-Oz interface for remote control and experimentation
  • Log viewer for visualisation of collected data
  • Lip-syncing for using pre-recorded audio files
  • Audio file hosting for logging interactions
  • Web dashboard for remote monitoring of interactions

Logging

Cloud Dialog Logger using AWS servers for logging of all or subset of interaction data to the cloud.

Listening

  • Multiple recognition hypotheses
  • Priming of specific expected user speech
  • Intermediate speech results
  • Speaker detection in group interactions

Vision

  • Face Detection
  • Face Recognition using faceprint
  • Smile detection
  • Ability to control all of the above using a simple API in the SDK.

User modeling

  • 5+ simultaneous users
  • User tracking (0,5-3 m)
  • Head pose estimation (Pitch/Yaw/Roll)
  • User attention detection
  • Situation modeling of the physical space in front of the robot

Speaking

  • Vocal gestures (in selected voices), SSML tags (Amazon Polly)
  • Pitch, Speaking rate, Loudness control
  • Ability to switch voices and language programmatically at any point in the interaction

Face & Neck

  • 40 blendshapes (poses) for creating custom facial expressions
  • Switching facial textures and models programmatically
  • Physically swap magnetic masks
  • Programmable shaders to adjust colours, lighting and contrast in the face depending on environment light
  • Control of 3 degrees of freedom in the neck
  • Ability to create facial and neck gestures programmatically and on the fly

More details

Download the Technical Product Overview for extensive details including Furhat’s hardware, software, and SDK

The Furhat SDK – now freely available to all COVID-19…

The Furhat SDK – now freely available to all

COVID-19 has made working from home the new normal, which makes experimenting with social robots in physical environments very challenging. That’s why we have decided to open our Furhat SDK to all, available without charge. Tap into the world of social robots from the comfort of your living room and build the future of our changing world.

Request SDK