Gesture recognition with Kinect and CRFSharp

Dec 25, 2012 at 5:22 PM

Hi all, I'm a student. I've never used before CRFSharp and I need an help.
I'm working on a project for gesture recognition, this is my application's video:

http://www.youtube.com/watch?v=PDkfgxEV8TI

I'm using DTW (dynamic time warping) for recognition, but this is very slow.
I have ten sequence of movements (example, Hand Left on, Hand right on, ecc.) 
I post here an example of my data:

<?xml version="1.0" encoding="utf-8"?>
<!--Questo file contiene le sequenze regitrate dal Kinect.-->
<Sequence ID_sequence="10">
<Observation ID_observation="0">
<Joint joint_element="Head">
<X>-0,0681642160598825</X>
<Y>-0,829331295395237</Y>
</Joint>
<Joint joint_element="HandLeft">
<X>-0,704363565952119</X>
<Y>1,60185907740724</Y>
</Joint>
<Joint joint_element="HandRight">
<X>0,681642160598825</X>
<Y>1,67002329346712</Y>
</Joint>
<Joint joint_element="Spine">
<X>0</X>
<Y>0,727084971305413</Y>
</Joint>
<Joint joint_element="FootLeft">
<X>-0,261296161562883</X>
<Y>3,73767118061689</Y>
</Joint>
<Joint joint_element="FootRight">
<X>0,261296161562883</X>
<Y>3,70358907258695</Y>
</Joint>
</Observation>
<Observation ID_observation="1">
.......

I can also obtain data of distance or angles between the body joints.

I work with positive and negative numbers (double) .
Can I create a machine learning algorithm with CRF for these value? 

Thanks you very much

Coordinator
Jan 4, 2013 at 12:48 AM

What is your specific problem ? To classify different gesture by model ? or any other specific problem ?

May 10, 2013 at 11:31 AM
Edited May 10, 2013 at 11:32 AM
According to code snippet of Decoder.Segment, the feature set is defined by string.
Thus, the feature to feed in training or test should be discrete. It does not allow real-value feature.
right?
Coordinator
May 10, 2013 at 4:57 PM
Edited May 10, 2013 at 4:58 PM
Yes. In CRFSharp, all features are binary value, and of course, their weight are float type.

If your features are discrete type, as we know, it's easy to map them into binary type, for example: set FEATURENAME_DISCRETE_VALUE as feature name in binary type.

If your features are real-value, currently, CRFSharp does not support it directly. However, it's really a good idea that I should think about. :)

If you want to use CRFSharp to handle real-value features now, maybe we can try to find the relationship in these real-value features, such as real-values' order (<, =, >), and use these relationship as feature set in CRFSharp.