[SOLVED] Read raw/csv Point Cloud data into GVDB

hi

i have raw file with 3 floats per line denoting points in my point cloud. I see that the programming guide mentions that the mechanism to input data into gvdb from RAW format is discussed in chapter 10 but i cannot find something about this in chapter 10.

Can you please share the information on how to read in a point cloud into gvdb? Is it by using insertpoint in a for loop?
kindly advise.
rohit

I have just updated the Github repository (6/20/2017).

Look at the gFluidSurface example. There you will find a new #define USE_CPU_COPY which demonstrates how to load point data onto the GPU from CPU source buffers and convert to voxels. You could also read this CPU data from disk, such as a CSV file.

Rama

Hi Rama

Thanks a lot for being so kind to provide these examples. I think I can make more progress.

If I do understand the code changes correctly, I must change my RAW file with only values to a file with VTK header and then it can be imported using the kind of code you have in source/gvdb_library/src/gvdb_volume_gvdb.cpp starting line 251

Am I right?

I ask this since I am wondering if I can also test this somehow? I am going to try to build gFluidSurface and see if I can spot the exact routines or if I can test it somehow.

Kindly advise.
rohit

Please note that RAW is a generic file format often used for many things.

For example, you talk about RAW for point cloud data, where you say “3 floats per line denoting points.” However, RAW might also be used to store small 3D volumes of voxels with a fixed dimension. Also note that the VTK format can store either point clouds or volumes.

If you want to import Point Clouds, then I suggest that you look at the gFluidSurface example and go from there. You should be able to remove the fluid sim, and just load the points from disk and store them in CPU memory. Then put them on GPU as the sample shows how.

If you want to import Volume data, then you can look at ImportVTK, and/or the Resample function. Note that gvdb.ImportVTK currently only supports small dense volumes. It cannot yet load point clouds stored in VTK.

Hi Rama

I was able to import my point cloud into gvdb with the following piece of code.

    DataPtr m_pntpos, m_pntclr;
int m_numpnts=0;
int devid=-1;
std::vector<Vector3DF> positions;
FILE *fp;
float a1,b1,c1;
fp=fopen(argv[1], "rt");
while (fscanf(fp, "%f %f %f", &a1, &b1, &c1) != EOF) {
		positions.push_back(Vector3DF(a1, b1, c1));
		m_numpnts++;
}
fclose(fp);
std::cout<<"Number of points read in are "<<m_numpnts<<"."<<std::endl;
std::cout<<"Size of Vector 3DF is "<<sizeof(Vector3DF)<<"."<<std::endl;
gvdb.SetVerbose(true);
gvdb.SetProfile(false);
gvdb.SetCudaDevice(devid);
gvdb.Initialize();
gvdb.AllocData(m_pntpos, m_numpnts, sizeof(Vector3DF), true );
gvdb.AllocData(m_pntclr, m_numpnts, sizeof(uint), true );

memcpy(m_pntpos.cpu, positions.data(), sizeof(Vector3DF)*m_numpnts);
gvdb.CommitData(m_pntpos);
gvdb.CommitData(m_pntclr);
gvdb.SetPoints(m_pntpos, m_pntclr);

Now I have some more questions.

I read (in the programming guide) that in order to voxelize one must call the functions in the following order.

gvdb.Configure ( 3, 3, 3, 3, 5 );
gvdb.AddChannel ( 0, T_UCHAR, 1 );
gvdb.SetVoxelSize ( 0.4, 0.4, 0.4 );

Model* m = gvdb.getScene()->getModel(0);
gvdb.SolidVoxelize ( 0, m, &xform );

Here how do i define the xform ? I figured it is a matrix4F type but right now that i am importing my point cloud into gvdb do i need to define something?

Also how do i use the insert points function

Vector3DF translate;
gvdb.InsertPoints(m_numpnts, translate, true);
?

Furthermore i want to use gvdb to replicate what i did with openVDB. To find nearest neighbors of voxels and also the number of points per voxel.

I read that in case I want more than 2 neighbors of a voxel I must use the apron function to increase the apron size. Am i right?

If yes with a larger apron size i suppose i can write my own kernel as described in the programming guide to make a kernel to access voxel values [sec 5.4.2 page 39] and launch them [5.4.4 page 41-42]

However I cannot find a routine for finding the number of points in some particular voxel.

Can you please suggest some pointers on how to implement these functionalities?

with kind regards
Rohit

Hello, Rama! Thank you for all the great work on GVDB! And your attention to this forum. Really
appreciate the care and consideration.

I have a situation similar to above, in which I’m trying to code up a custom point
cloud reader for getting point data (coming from disk/CPU) into GVDB for voxelization.
I’ve read the programming guide and you give a really good sketch of how to do this.
Going a step further, you added the code to main_fuild_surface.cpp as a realization
of the sketch ine the guide. Thank you for that.

I have written something almost exactly what is above using color as a T_UCHAR4 for
channel 0, but I now need something that is a step or two beyond that.

I need to be able to use a different data type for channel 0. I need to use T_FLOAT
and use a transfer function to display it which would make this idea exactly
like your “density for channel 0” idea in the guide. The problem is I think I need
something beyond using getDataPtr(), CommitData(), or SetPoints() to do this. And
I’m saying this because I’ve tried using these along with InsertPoints(),
ScatterPointDensity(), and Compute() and gotten something unexpected. While I obtain
a voxelization, essentially that voxelization is empty. It doesn’t contain the channel
0 data. Instead, channel 0 is initialized to zero for every voxel and ignores anything
I do with AllocData(), CommitData(), and other function mentioned above. I believe I
need to launch a kernel and use surf3Dwrite() to write my density data to channel 0.
And I’m up for doing that, if necessary, but I just thought there would be an easier
way to accomplish filling channel 0 with real data.

Hi Notzeus,

Currently, the SolidVoxelize function writes into a T_UCHAR or a T_FLOAT channel, and it does not overwrite any existing data found there. Whereas the ScatterPointDensity does overwrite every voxel in the target channel. Therefore, you have a few options.

  1. Modify ScatterPointDensity to avoid writing data if something is already in the channel. Then you can use SolidVoxelize to write mesh voxelization data first.
  2. Keep ScatterPointDensity as it is, run SolidVoxelize into a second channel, and then write a custom kernel to transfer the voxel data into your scatter data.

There are lots of variations on this. I’m only guessing what you’re trying to do. For anything more, I’d need a better idea of the goal. In the future, we hope these functions will be more flexible with respect to channel targets. However, we can’t anticipate every need, which is exactly why custom kernels exist.

Regards,
Rama

Hello Rama,

I seem to have made some progress using ScatterPointDensity() since my last post; however, thank you for your response. It is very insightful.

My understanding of SolidVoxelize() is that it is to be used with a polygonal mesh; whereas I am working with point cloud data (with no defined polygons) and figured it was incompatible with this purpose. Am I wrong in this understanding?

What I am basically trying to accomplish is to write a VBX file. To aid the discussion, we could talk using your explosion.vbx as a working example. How did you create this file?

I see two pathways for creating this file. 1) Use OpenVDB to input an explosion point cloud and output a VDB file, then load the VDB file into GVDB and save it as a VBX. Or, 2) Write a parser to input the point cloud data directly into GVDB and save the VBX.

I prefer 2) because it is more direct and requires no dependency on OpenVDB. The step in the process I am missing, or need help with, is getting the data into channel 0.

Your explosion example is perfect because it uses a transfer function for displaying the volume which is also what I need to do. I believe the fact that it uses this function means that the data in channel 0 is of type T_FLOAT, but I am not sure because that data is hidden in the VBX file (for which I have not yet written a parser using the provided file spec).

If you could provide a CSV file with the explosion point cloud data, perhaps, I could run it through the current incarnation of my parser and see if it produces the same resultant PNG as the one using LoadVBX(). If it doesn’t, I could iterate on it until it did.

It would be nice if there were a sample provided with the GVDB source that shows how to get a RAW point cloud into GVDB using T_FLOAT for channel 0. I’d be happy to provide this, if I could get mine to a satisfactory state. I just need point cloud data that I can share with the sample. The data I’m currently using cannot be shared.

Hope this post helps explain what I am trying to accomplish.

Just confirmed indirectly that my float data is not making it into channel 0 using ScatterPointDensity. Previously thought I made progress but I changed the rendering mode to SHADE_TRILINEAR but was fooled.

Need to figure out how to fill channel 0 with valid data.

Hi Notzeus,

The best reference for loading point cloud data is currently the gFluidSurface sample.
You can essentially ignore the ‘fluid’ part, and think of that as a set of points which are changing on every frame. I understand this may not be your case, but the basic input is still a set of points, and the rest of the sample shows how to convert from points-to-voxels.

The steps are then:

  1. Transfer the points from CPU to GPU. This is done with PrepareAux, SetDataCPU and CommitData.
    (you will not see this in gFluidSurface because the fluid points are already on GPU)

  2. Inform GVDB about the point data by calling SetPoints

  3. Activate topology. This is one reason you might see nothing. If you have not activated any bricks, then there are no voxels to write or draw. You activate regions in space by calling gvdb.ActivateSpace. When done activating, call FinishToplogy and UpdateAtlas.

  4. Perform the point-to-voxels by calling InsertPoints and then ScatterPointDensity. The insertion call is necessary before scatter.

The order of calls and the arguments are detailed in the gFluidSurface example.

Regards,
Rama