1

Leap Motion – Gesture Control in Fibaro Smart Home

by crissxcross4. January 2016

May the force be with you! We had the honor to be able to look behind the scenes of the new Star Wars trailer and show you, how from now on, you can operate your Fibaro Smart Home, as controlled from higher powers, via Gesture Control.

No, of course we did not participate in the real making of the Star Wars trailer, the siio Blog is too small to do so at this point ;-) . We were talking about the new video of “Mein-cleveres-Haus.de. On their YouTube Channel dressed up in a Star Wars costume, the guys from mein-cleveres-haus showed, how blinds can be opened magically via hand gesture and how the Sonos speaker became louder via “swipe”. The announced Fibaro Swipe is not yet available anywhere (as we reported), so how did the boys do that?

We wondered about that ourselves and were allowed to exclusively take a look behind the scenes.

Leap Motion for Gesture Controlling.

1) The Leap Motion, which you can buy on Amazon for about 80 Euros, served as Hardware. Those that look it up on Amazon will find two versions of the device. The more expansive version (around 230 Euros) comes along with the SDK Kit, the “cheap” version is enough to rebuild this tutorial, though.

2) Furthermore, you need the App “HandWAVE” which you can get for free from the Leap App Store. You can find it on here.

3) Last but not least, you need the program “wget“ for Windows. (It would also work without the little program and you could perform it via Browser – though, in comparison, those need more space and are slower.) You can, for instance, get wget Beispiel hier.

You can see, we have a Hardware, as well as Software, which works with a Windows PC. That means, our gesture control only works, whenever the PC, Barbone, Notebook, or whatever is turned on. Though, it should be noted that the Hardware Leap Motion also works with the Raspberry Pi. It might be worth a try in the future, to also rebuild it on a small single board PC.

HandWave

As the very first, I download and install the Leap Motion Software. Doing this, a registration in the Leap Motion App Store will be needed as well. Afterwards, you can start the “Leap Motion App Home”. You can now go to the App Store and download the HandWAVE App mentioned above.

Leap Motion Home App

Leap Motion Home App

Who needs even more possibilities, can also buy the APP “GameWAVE

for around 4$. It can even differentiate whether you perform the gesture with 1, 2, 3, or more fingers, and therefore offers many more possibilities.

After the first start, you need to perform a Calibration in HandWave. Therefore, simply follow the instructions on the screen. Afterwards, the APP HandWave has to be adjusted. In order to do so, simply create a new Configuration in the dialogue “File-> New”.

Configuring the Leap Motion Gesture Control

Configuring the Leap Motion Gesture Control

At this point, you can now assign a function to any gesture that will later on be performed, if Leap Motion recognizes this gesture. In order to do so, simply click on the field with the 3 points (…)

Deposit commands to the Leap Motion Gesture Control

Deposit commands to the Leap Motion Gesture Control

Under “Launch File” you can now insert files that contain the appropriate command. At the end of this tutorial, we put together a *.zip file for you, which, amongst other things, also contains the so called Batch files (*.cmd).

After inserting those Batch files, the whole thing will, for instance, look like that:

Depositing Commands to the Leap Motion Gesture Control

Depositing Commands to the Leap Motion Gesture Control

You should preferably save the configuration now. In order to do so, simply click “File -> Save as” in the upper area of the program and choose the target on your hard drive. It would be ideal, to copy the file together with the tool “wget”, which was mentioned above, into a register. In our example, we chose the common path: C:\@leap .

Adjusting the Batch Files

In order for all of this to work for you, you certainly need to adjust these Batch Files (*.cmd), because amongst other things they also contain the IP address to the Fibaro Home Center 2, as well as the username and the password.

  • The path to the program “wget” is in the very front of the Batch File. In our case it is c:\@leap\wget -> if you installed the program under another path, adjust it on here
  • The username and the password for your Fibaro Home Center 2 follow afterwards
  • The local IP address of your Fibaro Home Center 2 is written behind http:// . You certainly need to adjust it.

Afterwards, you can already see that an API Call is following. That may seem familiar to you, since Daniel used the same for his tutorial with the Voice Control via Google nowearlier. That means, we use the Fibaro API for this purpose. This way, you can directly contact devices, press “Buttons” in virtual devices and also start whole scenes. Our example certainly requires a few adjustments:

Adjusting the Batch File

Adjusting the Batch File

First of all, the values of “deviceID” – have to be replaced by the IDs of your devices. Here is an example on how to find out the ID:

Fibaro - Geräte ID herausfinden

Fibaro – Geräte ID herausfinden

  • replace: aaa by the ID of your RollerShutter
  • replace: bbb by the ID of the Dimmer
  • replace: ccc by the ID of the Leap Motion VD (more information further below)

Example: Move your hand up (blinds go up)

c:@leapwget –user=admin –password=admin “http://192.168.100.24/api/callAction?deviceID=aaa&name=close” -O dummy

Example: Hand-swipe to the right (light off)

c:@leapwget –user=admin –password=admin “http://192.168.100.24/api/callAction?deviceID=bbb&name=turnOff” -O dummy

Example: Pivot your hands contra clockwise (turning light (dimmer) darker)

c:@leapwget –user=admin –password=admin “http://192.168.100.24/api/callAction?deviceID=ccc&name=pressButton&arg1=2″ -O dummy

On demand, which controller was used for the Sonos in the video, Thomas Radtke from mein-cleveres-haus told us:

It was the “INTUITECH Sonos Virtual Device”, we are willingly to deliver it to our customers on demand. We only used a few of the functions available, in our video: “Switch Chanel”, “Sonos on/off” and the volume control with simple “Wrist showing down once/showing up once”.

In order to be able to use the example, we used a simplified version and the Fibaro Sonos Plugin for the volume control. This way, it is possible to control the volume with multiple hand gestures (showing down and if needed showing up). Replace the “xxx” in our example by the ID of the Sonos Plugin.

By the way, a VD with a list of the channels and e.g. Sonos TTS/MP3 like the one by Jean-Christophe Vermandé, is good enough, to switch the Sonos Channel with Fibaro.

Example: Making a fist (Sonos on/off)

c:@leapwget –user=admin –password=admin “http://192.168.100.24/api/callAction?deviceID=xxx&name=pressButton&arg1=3″ -O dummy

By doing that, you actually already mastered the most complex part. Let us now switch to the web surface of your Fibaro Home Center 2.

Creating Fibaro Scenes and VDs

On here, we need two global variables. So just go to “Control -> Controlling Variables” in the menu and apply the following two variables under “Predefined Values”:

  1. lmDimmChange with the values: 0, 1 and 2
  2. scVolChange with the values: 0, 1 and 2
Applying global variables to Fibaro

Applying global variables to Fibaro

The variable “lmDimmChange“ will later on be interesting for the dimming of the lights and the variable “scVolChange” is needed for the controlling of Sonos.

Applying global variables to Fibaro

Applying global variables to Fibaro

Leap Motion Virtual Device

This is where the Leap Motion Virtual device gets in, which we already mentioned in the part “Adjusting the Batch Files”. Pretty simple: Go to “Modules” in the menu of the Fibaro Home Center 2, then on the left to “Delete or add a device” and afterwards in the paragraph “Virtual Module” select -> file. On there, you can insert the Leap Motion VD, which you can download on here.

There are no adjustments necessary on the virtual device, since it only triggers scenes.

Scenes for the Gesture Control

Now, we directly get to the earlier mentioned scenes. In order for your Smart Home to work as “spaced out” as in the video of “mein-cleveres-haus”, you need two scenes. We are just going to name the following scene: Im-dimm-action and it is responsible for the controlling of the dimmers. You certainly need to insert the ID of your dimmer (for example of the Fibaro Dimmer 2 Module) in line 9.

If from now on, you also want to control your Sonos via Gesture Control, you also need a second scene. This one is responsible for the Volume Control of the Sonos. You need to insert the ID of your Sonos Plugin in line 8.

In case you already have the Software Version v4.058 or higher on your Fibaro Home Center, set the “Max. running instances:” to at least the value 3.

That is it already:

We assembled the needed Virtual Device, the scenes and the Batch files in a *.zip package for you, you can download it on here.

Now, may the Force be with you!

About The Author
crissxcross
Hi, I'm crissxcross, qualified IT system electronics. Professionally, I work as a product marketing manager in the Smart Home. And private I am self a Smart Homie :-) If I do not sit in front of the monitor like I photograph and am a big fan of Apple. I am the founder of the SiiO.de blog.
  • 7. July 2016 at 9:46

    Wow! Nice step by step tutorial in integrating Leap Motion – Gesture Control to Fibaro Smart Home. I hope that you have a video tutorial for this.

Leave a Response