It’s been 3 months since the latest release of AIRkinect, which brought you OpenNI support on windows, but today we’re very happy to bring you 2.2! Some of the new stuff we have is:
- Support for seated skeleton tracking (MS SDK)
- Near mode support with skeleton tracking (MS SDK)
- Option to choose which users to track (MS SDK)
- Skeleton bone tracking (both)
We’ve also fixed some bugs concerning initialization failures on the MS SDK version & coordinates not being mirrored correctly on OpenNI.
One of the big new structures you’re going to see is skeleton bones. Skeleton bones is actually the joint rotation info which is moved from the skeleton joints structures into a structure of it’s own. This is mainly because of the differences between OpenNI & the MS SDK. The MS SDK tracks more joints & exposes more joint orientations. OpenNI stores joint rotation info in the start joint of a bone (as an absolute orientation offset from the T-pose), the MS SDK in the end joint. We wanted to create a structure which would be the same in both openNI & windows & exposes.
There is a set of 10 bones which are available on both OpenNI & the MS SDK:
SkeletonBone.NECK SkeletonBone.SPINE SkeletonBone.LEFT_UPPER_ARM SkeletonBone.LEFT_LOWER_ARM SkeletonBone.RIGHT_UPPER_ARM SkeletonBone.RIGHT_LOWER_ARM SkeletonBone.LEFT_UPPER_LEG SkeletonBone.LEFT_LOWER_LEG SkeletonBone.RIGHT_UPPER_LEG SkeletonBone.RIGHT_LOWER_LEG
You can get absolute rotation information as a Matrix3D from a given bone with the following code:
user.getBoneByName(SkeletonBone.LEFT_UPPER_ARM).orientation.absoluteOrientationMatrix
Other information you can get is the name of the start joint, name of the end joint and name of the parent bone. If you are using the MS SDK, you get information on 9 more bones (in regular tracking, not seated mode):
MSSkeletonBone.LEFT_COLLAR MSSkeletonBone.LEFT_HAND MSSkeletonBone.RIGHT_COLLAR MSSkeletonBone.RIGHT_HAND MSSkeletonBone.LOWER_SPINE MSSkeletonBone.LEFT_HIP MSSkeletonBone.LEFT_FOOT MSSkeletonBone.RIGHT_HIP MSSkeletonBone.RIGHT_FOOT
Again, you can get those bones by the getBoneByName method. There’s also a collection of bones you can loop through (similar like you can loop through all the joints of a user):
user.skeletonBones
contains a vector of SkeletonBone instances.
So, skeleton bones contain absolute orientation values for bones. Now, if you want to use the native, platform specific rotation values, we have added that information in the joints where that platform stores those values. So if you have an OpenNI Skeleton joint, you will have access to:
openNiJoint.nativeOrientationConfidence:Number openNIJoint.nativeOrientation:Matrix3D
And in the MS SDK:
msJoint.nativeHierarchicalRotationQuaternion:Quaternion msJoint.nativeHierarchicalRotationMatrix:Matrix3D msJoint.nativeAbsoluteRotationQuaternion:Quaternion msJoint.nativeAbsoluteRotationMatrix:Matrix3D
Theres one big change concerning joint information, and that’s where you can find the position values. In earliers versions of airkinect, you had direct properties on joints / users (like joint.depthPosition, joint.depthRelativePosition, joint.position, etc…). We have moved those values (word, rgb & depth position data) to a separate position structure.
So instead of:
joint.position.x joint.rgbPosition.x
etc…
You will use:
joint.position.world.x joint.position.rgb.x
etc…
This will require some basic refactoring of your existing applications, but shouldn’t be that difficult to implement.
There is one more thing to announce: from now on our native C++ sources are open source as well! So if you run into bugs, or want to add new features yourself, please fork & play with the code! Hopefully this will speed up bugfixes / improvements to the native code too 
As always, you can find the latest release & docs on github (http://as3nui.github.com/airkinect-2-core/), all our repositories are on https://github.com/AS3NUI/
We hope you like this update, looking forward to seeing the projects you’re building with AIRKinect!
47 Responses to “AIRKinect 2.2 – now with open-source native code”
as3NUI provide Natural User Interface integration to Adobe AIR based products. NUI is the future of interaction and we provide a way to integrate this with Adobe technologies. This is a community based development, that has been created by a community for a community. If interested in development, please contact developer@as3NUI.com with ideas and projects related to NUI.
Muchas Gracias por el esfuerzo y el tiempo dedicado,su librerìa se ha vuelto muy importante para los que desarrollamos kinect.
Muchas gracias !
Sounds nice
Thank you guys once again! Does this version support tracking 2 skeletons? And if not, are there plans to do so in the near future?
I started using AIRKinect since version 1.6, and as I remember it was already able to track 2 skeletons. cmiiw
Thanks for that. I do see that 2 skeletons are working in the example. Don’t know why I thought otherwise. Maybe because I code alone mostly.
Hi, I have problems starting the application with the new airkinect sdk 2.2, I get as an error “initial content not found”, I share with you the project to see if they can help me.
https://dl.dropbox.com/u/862935/prueba.rar
can someone help me with this?
Glad it finally support a higher resolution in mac version. But I have trouble using it..
When I change it to CameraResolution.RESOLUTION_1280_960, it dosen’t show anything. But other resolution sizes are working fine.
I have installed the most updated version of driver – OpenNI-Bin-Dev-MacOSX-v1.5.4.0.
Any help?
I seem to be having the same issue on my PC windows 7. I thought maybe it was because my XBox360 Kinect didn’t support that size.
We don’t support larger resolutions on OpenNI yet. I have added it to the issue list to look into that.
Hello,
I’m trying to put the Kinect working with the code you have on github, but when i compile allways get the error “[Fault] exception, information=TypeError: Error #1009: Cannot access a property or method of a null object reference.”
and the line is this one “context = ExtensionContext.createExtensionContext(“com.as3nui.nativeExtensions.air.kinect”, (shared) ? “shared” : null);”
the var context allways return me NULL
That sounds like a driver issue. Make sure you install the 32-bit version of OpenNI (on windows), or the latest MS SDK drivers.
I have follow this installation guide
http://code.google.com/p/simple-openni/wiki/Installation#Windows
and check if all drivers are installed under PrimeSensor : Kinect Camera, Kinect Motor, Kinect Audio
do i need some “bridge” to kinect work with flash?
hey,
Can someone help me ?
Make sure you use the Installation Package Win32, and not the Win64!
Hello
as was having problems with kinect, i erased all the drivers and software related to kinect, and installed it again following the tutorial from this link (http://code.google.com/p/simple-openni/wiki/Installation # Windows) and installed all the 32bit drivers.
But I still have the same error.
“Context = ExtensionContext.createExtensionContext (” com.as3nui.nativeExtensions.air.kinect “(shared)?” Shared “: null);”
the var context I allways return NULL
tnks
I’m having the exact same problem where “Context = ExtensionContext.createExtensionContext (” com.as3nui.nativeExtensions.air.kinect “(shared)?” Shared “: null);” always returns null and causing my app to crash.
Have you found a solution to this?
More details of the issue here:
http://forum.as3nui.com/viewtopic.php?f=13&t=2800&p=3450#p3450
Hello.
I have problems starting the application with the new version (2.2).
I have experienced an shutdown error in “setting.userMask = true”
You’ll need to provide a little more details: what’s the system/drivers you’re using, what’s the full list of settings you are using. Did you experience problems with previous versions of airkinect?
really , c++ source code ? cool !
i dont have many money but i could buy you guys a beer
How to get user’s SkeletonJoint in version2.2 (MSSDK)??
This code in version2.1(MSSDK) is work fine, but in version2.2(MSSDK), device.users.length is always 0, why?
for each(var user:User in device.users){
...............
}
How to get axis-z of joint?
I can’t find it from position.rgb and position.depth.
rgb & depth are 2D values if you want to overlay on the rgb or depth camera image. The z values are in the world positions.
Thanks, I figure out these problem from example, I didn’t know example is updated.
But how to get the old rotationX, rotationY, rotationZ of every Bone? I have no idea how to use the absoluteOrientationMatrix for this. Can you help me? I have the mssdk.
You can use the builtin decompose method for that:
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/geom/Matrix3D.html#decompose()
3D rotations are best described in matrices / quaternions to prevent “gimbal locking”.
Thanks for the help
I just tried out 2.2 but on all the demo’s using the skelletons I’m getting:
INFO: Starting Device
INFO: Starting Skeleton Tracking
ERROR: Failed to Initalize Skeleton Tracking
INFO: Stopping Kinect
The 2.1 examples are still behaving as expected though, so I’m ruling out any driver issues, what else can it be?
It might just be driver issues, as we use new functionality from the 1.5 SDK in 2.2 (which we didn’t use in 2.1). So make sure you have the latest MS SDK drivers installed (Louis had the same error: http://forum.as3nui.com/viewtopic.php?f=13&t=1527)
Hey guys ! Tanx for the great work/help !
I’m getting this error while trying to debug in FlashDevelop: “The content cannot be loaded because there was a problem loading an extension: Error: Requested extension com.as3nui.nativeExtensions.air.kinect could not be found.”
What is correct ID of the extension to put in the XML, please ! =]
[]‘s
Hi I have another kind of problem in FlashDevelop, if you manage to solve yours could share the project?, That will allow me to work with FlashDevelop, my problem was as follows:
“Hi, I have problems starting the application With The airkinect new sdk 2.2, I get an error as” initial content not found “, I share with you the project to see if THEY CAN help me.”
https://dl.dropbox.com/u/862935/prueba.rar
Hey Luis !
Your first problem was that you didn’t added the flag “-extdir extension/debug/” in the “Packager.bat” file. You must add this that AND in the Run.bat.
After that, the window opened, but had another problem inside I coudn’t manage.
Let’s see if someone else can help us.
[]‘s
Perfecto, era esta lìnea de còdigo que estaba mal escrita :
adl “%APP_XML%” “%APP_DIR%”-extdir extension/debug/
lo reemplace por
adl “%APP_XML%” “%APP_DIR%” -extdir extension/debug/
Muchas gracias
Hi. There is a method AIRKinectTransformSmoothParameters() in the previous version of AIRKinect, but I can not find it in AIRKinect2, did this method get removed in AIRKinect2? is there something similar in AIRKinect2?
I really want this method since I need the cursor image that linked to user’s hand joint to be smoother.
any help would be much appreciated
CAI
One more question:
Which drivers should I use with this API ? I found more than one on the internet.. =[
Tanx !
[]‘s
Why the function “chooseSkeletons” is only available for MSSDK?
thank you!
I need to use this feature with OpenNI. How can active a skeleton for a specific user with OpenNi? Is it possible ?
thank you
Hi,
I’m trying to make a small game with collision detection from the 3D character demo.
My problem is that I can get the position of the hands from the RGB feed but the 3D character position is different from the RGB skeleton. I’ve tried to modify the camera3D parameters but even if it’s better, the RGB and 3D character joints are still not exactly the same. Do you know how I can make them identical or how I can get the position of the 3D character hands?
Thanks.
Nice job!
Great job!
I have one question though:
If i disable the skeleton detection i should still get events when a user is added, right?
I see the users (with depthShowUserColors) but the event never gets dispatched.
On the other hand if i set skeletonEnabled true, then both USERS_ADDED and USERS_WITH_SKELETON_ADDED get dispatched.
Is it a bug or am i doing something wrong?
Thanks!
Hi,
Is higher resolution of video supported yet. Somehow I cannot go over 640 x 380 resolution. Have been stuck at this for over 2 months now…..
C’est article est interesssant, je le relais à mes contacts !! :p
Feel free to surf to my blog post :: Micah
Right here is the perfect blog for anybody who would like to find out
about this topic. You know so much its almost hard to argue with you (not that I really will need to…HaHa).
You definitely put a new spin on a topic that has been discussed for many
years. Great stuff, just wonderful!
It is really a great and helpful piece of information. I’m glad that you simply shared this useful info with us. Please keep us up to date like this. Thanks for sharing.
How would I identify either an user is facing towards the Kinect device or the user is in backward posture?
It always took 3~5 seconds to know it, Can it get faster to identify when the user just entering detect area?
Just want to say your article is as astonishing. The clearness on your
put up is simply great and that i could think you are an expert in this subject.
Well together with your permission allow me to clutch your RSS feed to keep updated
with forthcoming post. Thanks one million and please carry on the enjoyable work.