DepthMap class


This class offers a convenient way to convert the packed depth frames received by a Kinect sensor into a DepthMap. The DepthMap is internally represented as a 3D surface parameterized as a quadratic mesh (for more technical details you can read IEEE Transactions on Cybernetics, 2013, or watch the VIDEO DEMO). The DepthMap can be saved as a 3D object in VRML format. It can also be rendered in openGL as a textured object, non-textured object, or 3D mesh using JogAmp's JOGL Java library. The use of JOGL library is optional and is not required if you don't use the drawing methods provided in the DepthMap class.

This class is part of the J4K library, which is included in the University of Florida Digital Worlds (ufdw.jar) Java library. This library was developed by Prof. Angelos Barmpoutis, and further extended by the students and faculty who work in the SAGE program (Serious and Applied Gaming Environments) at the Digital Worlds Institute. You can find more information about how to join our graduate and undergraduate programs at this link (www.digitalworlds.ufl.edu).

If you use this Java library in your research, please cite the following article, which introduced this library:
A. Barmpoutis. 'Tensor Body: Real-time Reconstruction of the Human Body and Avatar Synthesis from RGB-D', IEEE Transactions on Cybernetics, Special issue on Computer Vision for RGB-D Sensors: Kinect and Its Applications, October 2013, Vol. 43(5), Pages: 1347-1356. Download PDF - Kinect Avatars paper [VIDEO DEMO]


package edu.ufl.digitalworlds.j4k;

public class DepthMap {

The following methods construct a new object of the DepthMap class in different ways.

    /*This method constructs a default DepthMap of a given size*/
    public DepthMap(int w, int h);

    /*This method constructs a DepthMap out of a depth frame received 
         from a Kinect sensor and is given as an XYZ array.*/

    public DepthMap(int w, int h, float XYZ[]);

You can get the values of various parameters of the DepthMap class using the following methods.

    /*This method returns the width of the DepthMap (in pixels).*/
    public int getWidth();

    /*This method returns the height of the DepthMap (in pixels).*/
    public int getHeight();
    
    /*This method returns true if there is a non-zero depth value at
      the pixel (x,y).*/

    public boolean validDepthAt(int x, int y);

    /*This method returns true if there is a non-zero depth value at
         the pixel location (x,y) given as index=y*width+x.*/

    public boolean validDepthAt(int index);

The DepthMap class offers several different ways of visualizing the depth frames in 3D using openGL. The following methods require the JOGL Java library.

    /*This method displays the DepthMap as a 3D surface.  The U,V 
      texture coordinates are used, if available, otherwise the 
      normal vectors of the 3D surface are computed for shading 
      purposes. This method requires the JOGL Java library.*/

    public void draw(GL2 gl);
    
    /*This method displays the DepthMap as a 3D surface by skipping
      every 'skip' pixels in both dimensions. The U,V texture 
      coordinates are used, if available, otherwise, the normal 
      vectors of the 3D surface are computed for shading purposes.
      This method requires the JOGL Java library.*/

    public void draw(GL2 gl,int skip);

    /*This method displays the DepthMap as a 3D surface. The normal 
         vectors of the 3D surface are computed for shading purposes. 
      This method requires the JOGL Java library.*/

    public void drawNormals(GL2 gl);
    
    /*This method displays the DepthMap as a 3D surface by skipping
      every 'skip' pixels in both dimensions. The normal vectors of 
      the 3D surface are computed for shading purposes. This method 
      requires the JOGL Java library.*/

    public void drawNormals(GL2 gl,int skip);
    
    /*This method displays the DepthMap as a 3D surface using the 
      U,V texture coordinates, if available. This method requires 
      the JOGL Java library.*/

    public void drawTexture(GL2 gl);
    
    /*This method displays the DepthMap as a 3D surface by skipping
      every 'skip' pixels in both dimensions. The U,V texture 
      coordinates are used, if available. This method requires the 
      JOGL Java library.*/

    public void drawTexture(GL2 gl,int skip);
    
    /*This method displays the DepthMap as a 3D surface. The normal 
         vectors of the 3D surface are computed for shading purposes. 
      The U,V texture coordinates are also used, if available. This
      method requires the JOGL Java library.*/

    public void drawTextureNormals(GL2 gl);    
    
    /*This method displays the DepthMap as a 3D surface by skipping
      every 'skip' pixels in both dimensions. The normal vectors of 
      the 3D surface are computed for shading purposes. The U,V 
      texture coordinates are also used, if available. This method 
      requires the JOGL Java library.*/

    public void drawTextureNormals(GL2 gl,int skip);

    /*This method displays the DepthMap as a 3D rectangular grid.
      The normal vectors are computed for shading purposes. This 
      method requires the JOGL Java library.*/

    public void drawMesh(GL2 gl);

    /*This method displays the DepthMap as a 3D rectangular grid by
         skipping every 'skip' pixels in both dimensions. The normal 
         vectors are computed for shading purposes. This method 
      requires the JOGL Java library.*/

    public void drawMesh(GL2 gl, int skip);

The following methods perform calculations related to the 3D visualization of a depth frame. Furthermore, there are methods for masking the depth data in different ways.

    /*With this method you can provide a U,V texture mapping in the
      form of float array that hold pixel locations in an image.*/
 
    public void setUV(float[] UV);
    
    /*This method creates a uniform U,V texture mapping for this 
      DepthMap.*/

    public void setUVuniform();

    /*This method sets the player index for this DepthMap.*/
    public void setPlayerIndex(short[] depth_frame, byte[] player_index);

    /*With this method you can specify the maximum allowed difference 
      in depth values within a quad. If a quad does not satisfy this 
      criterion it is not rendered.*/

    public void setMaximumAllowedDeltaZ(double dz);

    /*This method returns the current maximum allowed difference in 
      depth values within a quad.*/
     
    public float getMaximumAllowedDeltaZ();

    /*This method masks out all pixels with depth value larger than
      a given threshold.*/

    public void maskZ(float threshold);

    /*This method masks out all pixels outside the rectange defined
      by a given corner point (x,y) and of size width x height.*/

    public void maskRect(int x, int y, int width, int height);

    /*This method performs smoothing of the depth frame.*/
    public void smooth();

    /*This method repeats the smoothing process for several times.*/
    public void smooth(int times);

Below, there are several methods for saving the depth frames in VRML format.

    /*This method exports the DepthMap in VRML (wrl) format.*/
    public void saveWRL(String filename);
    
    /*This method exports the DepthMap in VRML (wrl) format by 
      skipping every 'skip' pixels in both dimensions.*/

    public void saveWRL(String filename, int skip)        

Finally, you can have direct access to the data of a depth frame by using the following public arrays.

    /*The following public arrays contain the parameters of the 
      DepthMap. You can directly read or write to these arrays to
      alter the DepthMap in your custom algorithms.*/
 
    public float realX[]=null;
    public float realY[]=null;
    public float realZ[]=null;

    public float U[]=null;
    public float V[]=null;
    
    /*The player array provides a player ID at each pixel of the
      DepthMap. The playerID information is included in the packed
      depth frames received from the Kinect.*/

    public byte player[]=null;
    
    public boolean mask[]=null;

    /*You can provide a 4x4 transformation matrix to be applied to the
      DepthMap. The transformation should be given in a 1D array form
      arranged by columns.*/

    public double transformation[];    
}

What to do next:
  • You can download several source code examples from the Source Code Examples website.
  • You can also read a brief tutorial on "How to write your own Kinect-Java programs".


  • Disclaimer: The names JAVA and KINECT and their associated logos are trademarks of their respective copyright owners Oracle and Microsoft. None of these companies endorse, fund, or are in any way associated with the J4K library.

    Disclaimer: This software is provided for free without any warranty expressed or implied for academic, research, and strictly non commercial purposes only. By downloading this library you accept the Terms and Conditions.

    University of Florida, Digital Worlds Institute, P.O.Box 115810, 101 Norman Gym, Gainesville, FL 32611-5810, USA