从D435获得完整的点云但数据集不完整

我找到了SDK 2.0的点云脚本（不幸的是在Python中），它指定了如何处理无效像素，显然给它们的深度值为零而不是删除它们。
//设置深度值为零的无效像素，用于表示无数据 
pcl :: PointXYZRGB点; 
if（depth_value == 0）{ 
if（mUseUncolorizedPoints）{ 
point.x = NAN; 
point.y = NAN; 
point.z = NAN; 
} 
奇怪的故障SR300深度与颜色对齐·问题＃1027·IntelRealSense / librealsense·GitHub



以下为原文

I found a point cloud script for SDK 2.0 (not in Python, unfortunately) that specifies how to deal with invalid pixels, apparently giving them a depth value of zero instead of removing them.
 
// Set invalid pixels with a depth value of zero, which is used to indicate no data
            pcl::PointXYZRGB point;
            if (depth_value == 0) {
                if (mUseUncolorizedPoints) {
                    point.x = NAN;
                    point.y = NAN;
                    point.z = NAN;
                }
              
Weird glitch SR300 depth aligned to color · Issue #1027 · IntelRealSense/librealsense · GitHub

谢谢！
我代表另一个团队成员发帖，我现在不看他们的代码，但我知道我们一直在使用一个特定的函数来获得一个函数的整个点云，而你发布的链接正在做我所做的事情
希望能够逐像素地找到并生成点云（并且在没有好数据的情况下输入NaN值）。
这应该是我需要的！



以下为原文

Thanks! I'm posting on behalf of another team member and am not looking at their code right now, but I know we've been using a particular function to get the entire point cloud with one function, whereas the link you posted is doing what I was hoping to find and generating the point cloud pixel-by-pixel (and entering NaN values where there is no good data). This should be exactly what I need!

大！
请稍后告诉我们您的工作原理。
祝你好运！



以下为原文

Great!   Let us know later how it work outs for you, please.  Good luck! 

嘿马蒂，
最后尝试用C ++实现这个解决方案并遇到了一些最终解决的问题 - 所以最后我有一个完整的点云，这就是我的目标。
但是，当我尝试按照您提供的链接中的代码时，我必须处理两个问题（这非常有用！）：
1.据我所知，当前版本的SDK似乎没有提供“deproject”功能。
我不得不使用“rs2_deproject_pixel_to_point”，但这很好用。
2.使用if-else结构来捕获坏点并将NaN值点插入点云是不成功的，并且导致将RGB数据映射到点云的奇怪问题。
这是两个图像：在左侧，我们看到在将深度为零的每个点的x-y-z数据设置为NAN之后的点云;
在右侧，我们看到完整的点云，而没有设置任何点到NAN。
由于某种原因，在点云中使用NAN数据会导致RGB数据被解释的问题（尽管x-y-z数据看起来很好）。
很高兴能够将点作为点云的一部分存在但不会显示使用NaN值，但对于我的应用程序，我可以与它们一起坐在原点。
再次感谢您的帮助，我在下面附上我的代码以防其他人想要访问它（抱歉，复制过程中显示缩进信息丢失了......）。
干杯，
布赖恩
#include“stdafx.h”
＃包括
＃包括
#include“opencv2 / imgproc / imgproc.hpp”
＃包括
＃包括
＃包括
＃包括
＃包括
使用命名空间cv;
使用命名空间std;
int main（int argc，char * argv []）
{
// RealSense管道
rs2 ::管道管道;
rs2 :: config cfg;
//深度流配置
cfg.enable_stream（RS2_STREAM_DEPTH，1280,720，RS2_FORMAT_Z16,30）;
//颜色流配置
cfg.enable_stream（RS2_STREAM_COLOR，1280,720，RS2_FORMAT_RGB8,30）;
//启动管道并获取传感器配置文件数据
auto pipeProfile = pipe.start（cfg）;
auto sensor = pipeProfile.get_device（）。first（）;
auto depthScale = sensor.get_depth_scale（）;
//深度和颜色流
auto align = new rs2 :: align（RS2_STREAM_COLOR）;
auto depth_stream = pipeProfile.get_stream（RS2_STREAM_DEPTH）.as（）;
auto color_stream = pipeProfile.get_stream（RS2_STREAM_COLOR）.as（）;
auto depthToColor = depth_stream.get_extrinsics_to（color_stream）;
auto colorToDepth = color_stream.get_extrinsics_to（depth_stream）;
auto depthIntrinsic = depth_stream.get_intrinsics（）;
auto colorIntrinsic = color_stream.get_intrinsics（）;
// OpenCV和PCL窗口
cv :: namedWindow（“Image”，cv :: WINDOW_AUTOSIZE）;
cv :: namedWindow（“Depth”，cv :: WINDOW_AUTOSIZE）;
pcl :: visualization :: CloudViewer查看器（“查看器”）;
//主循环（当PCL查看器打开时）
while（！viewer.wasStopped（））{
//获取新框架
rs2 :: frameset frames = pipe.wait_for_frames（）;
auto processedFrames = align-> process（frames）;
rs2 :: frame frameDepth = processedFrames.first（RS2_STREAM_DEPTH）;
rs2 :: frame frameRGB = processedFrames.first（RS2_STREAM_COLOR）;
//框架大小
const int w = frameRGB.as（）。get_width（）;
const int h = frameRGB.as（）。get_height（）;
// OpenCV格式的RGB和深度图像
cv :: Mat image（cv :: Size（w，h），CV_8UC3，（void *）frameRGB.get_data（），cv :: Mat :: AUTO_STEP）;
cv :: Mat image_depth（cv :: Size（w，h），CV_16U，（uchar *）frameDepth.get_data（），cv :: Mat :: AUTO_STEP）;
//生成RGB和深度的输出图像（标准化深度）
cv :: Mat image_bgr，image_map，image_nrm;
cv :: cvtColor（image，image_bgr，cv :: COLOR_RGB2BGR）;
image_depth.convertTo（image_nrm，CV_32F）;
双dMin，dMax;
minMaxIdx（image_depth，＆amp; dMin，＆amp; dMax）;
image_nrm / = 65535.0;
normalize（image_nrm，image_nrm，0,1，NORM_MINMAX）;
image_nrm * = 255;
image_nrm.convertTo（image_nrm，CV_8U）;
applyColorMap（image_nrm，image_map，2）;
//更新图像窗口
imshow（“Image”，image_bgr）;
imshow（“深度”，image_map）;
waitKey（10）;
//新点云
pcl :: PointCloud :: Ptr cloud（new pcl :: PointCloud）;
int N_bad = 0;
//坏点计数器（即深度= 0）
//对于每个像素......
for（int u = 0; u points.push_back（point）;
}
}
//将云输出到查看器
viewer.showCloud（云）;
}
//终止
返回0;
}



以下为原文

Hey Marty,
 
Finally tried to implement this solution in C++ and ran into a few problems that were eventually solved -- so in the end I have a full point cloud which is what I was aiming for. However, here are two issues that I had to deal with as I tried following the code in the link you provided (which was extremely helpful!):
 
1. The function "deproject" does not seem to be available in the current version of the SDK as far as I can tell. I had to use "rs2_deproject_pixel_to_point" instead but this works fine.
 
2. The use of an if-else structure to catch bad points and insert NaN value points into the point cloud was unsuccessful and led to a strange problem with mapping RGB data to the point cloud. Here are two images: on the left we see the point cloud after setting x-y-z data to NAN for each point where the depth is zero; on the right we see the full point cloud without setting any points to NAN. For some reason the use of NAN data in the point cloud leads to problems with how the RGB data is interpreted (although the x-y-z data appears to be fine). It is nice to be able to have points exist as part of the point cloud but not show up using NaN values but for my application I can live with them sitting at the origin.
 

Thanks again for your help, I'm attaching my code below in case someone else wants to access it (sorry, it appears the indentation was lost when copying over...).
 
Cheers,
 
Brian
 
#include "stdafx.h"#include #include #include "opencv2/imgproc/imgproc.hpp"#include #include #include #include #include using namespace cv;using namespace std;int main(int argc, char * argv[]){// RealSense pipeliners2::pipeline pipe;rs2::config cfg;// depth stream configcfg.enable_stream(RS2_STREAM_DEPTH, 1280, 720, RS2_FORMAT_Z16, 30);// colour stream configcfg.enable_stream(RS2_STREAM_COLOR, 1280, 720, RS2_FORMAT_RGB8, 30);// start pipeline and get sensor profile dataauto pipeProfile = pipe.start(cfg);auto sensor = pipeProfile.get_device().first();auto depthScale = sensor.get_depth_scale();// depth and color streamsauto align = new rs2::align(RS2_STREAM_COLOR);auto depth_stream = pipeProfile.get_stream(RS2_STREAM_DEPTH).as();auto color_stream = pipeProfile.get_stream(RS2_STREAM_COLOR).as();auto depthToColor = depth_stream.get_extrinsics_to(color_stream);auto colorToDepth = color_stream.get_extrinsics_to(depth_stream);auto depthIntrinsic = depth_stream.get_intrinsics();auto colorIntrinsic = color_stream.get_intrinsics();// OpenCV and PCL windowscv::namedWindow("Image", cv::WINDOW_AUTOSIZE);cv::namedWindow("Depth", cv::WINDOW_AUTOSIZE);pcl::visualization::CloudViewer viewer("Viewer");// main loop (while PCL viewer is open)while (!viewer.wasStopped()) {// get new framesrs2::frameset frames = pipe.wait_for_frames();auto processedFrames = align->process(frames);rs2::frame frameDepth = processedFrames.first(RS2_STREAM_DEPTH);rs2::frame frameRGB = processedFrames.first(RS2_STREAM_COLOR);// frame sizeconst int w = frameRGB.as().get_width();const int h = frameRGB.as().get_height();// RGB and depth images in OpenCV formatcv::Mat image(cv::Size(w, h), CV_8UC3, (void*)frameRGB.get_data(), cv::Mat::AUTO_STEP);cv::Mat image_depth(cv::Size(w, h), CV_16U, (uchar*)frameDepth.get_data(), cv::Mat::AUTO_STEP);// generate output images for RGB and depth (normalize depth)cv::Mat image_bgr, image_map, image_nrm;cv::cvtColor(image, image_bgr, cv::COLOR_RGB2BGR);image_depth.convertTo(image_nrm, CV_32F);double dMin, dMax;minMaxIdx(image_depth, &dMin, &dMax);image_nrm /= 65535.0;normalize(image_nrm, image_nrm, 0, 1, NORM_MINMAX);image_nrm *= 255;image_nrm.convertTo(image_nrm, CV_8U);applyColorMap(image_nrm, image_map, 2);// update image windowsimshow("Image", image_bgr);imshow("Depth", image_map);waitKey(10);// new point cloudpcl::PointCloud::Ptr cloud(new pcl::PointCloud);int N_bad = 0; // bad point counter (i.e., depth = 0)// for each pixel...for (int u = 0; u < h; u++) {for (int v = 0; v < w; v++) {// get depth from image and scaleuint16_t depth_value = image_depth.at(u, v);float depth_in_meters = depth_value * depthScale;// new pointpcl::PointXYZRGB point;// set bad points to NANif (depth_value < 0) {N_bad++;point.x = NAN;point.y = NAN;point.z = NAN;point.r = 255;point.g = 255;point.b = 255;}else {// use intrinsics to map image coordinates to real world coordinatesfloat depth_pixel[2];depth_pixel[0] = v;depth_pixel[1] = u;float depth_point[3];rs2_deproject_pixel_to_point(depth_point, &depthIntrinsic, depth_pixel, depth_in_meters);point.x = depth_point[0];point.y = depth_point[1];point.z = depth_point[2];// get corresponding RGB data and map onto pointauto pix = image.at(u, v);point.r = pix[0];point.g = pix[1];point.b = pix[2];}// add point to cloudcloud->points.push_back(point);}}// output cloud to viewerviewer.showCloud(cloud);}// terminatereturn 0;}

好报道！
我很高兴为你效劳。
非常感谢您与社区分享您的经验和代码。



以下为原文

Great report!  I'm really glad it worked out for you.  Thanks so much for sharing your experience and code with the community.