Unstable performance when using filters=pointcloud in d435

[jediofgever]

Hi,
When I start camera node with pointclouds enabled, I get very random performance. sometimes it can deliever a frame in 10 seconds and sometimes it performs as expected. I have built realsense with CUDA support and the computer specs are absolutely above required ones. I am using an additional usb cable to increase range to put camera on my robotic arm. I have checked the performance with and without this additional cable and I still have the same issue with original cable. Is there someone who met this problem ?

[doronhi]

I haven't met the problem you describe.
Could you begin by specifying the type of computer, OS version (including kernel version - uname -r) and the versions of librealsense2 and realsense2_camera (Printed by the realsense2_camera node)?

I assume you launch the node using roslaunch realsense2_camera rs_camera.launch filters:=pointcloud and measure the frame rate of the point cloud using rostopic hz /camera/depth/color/points. Do you see the frame rate drop from 30Hz (or whatever you specified) to around 1 or less or does the frame rate go up and down?

Can you run roslaunch realsense2_camera rs_camera.launch align_depth:=true and measure the frame rate of the aligned images: rostopic hz /camera/aligned_depth_to_color/image_raw?

Are there any warning messages printed by realsense2_camera node?

[jediofgever]

Hi @doronhi,
To start with info quickly ;

OS: UBUNTU 18.04 LTS
KERNEL:  5.0.0-37-generic
RealSense: ROS v2.2.11
LibRealSense: v2.31.0
Intel RealSense D435I

Here is output of when I listen to the average rate with point cloud;

no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 15.161
	min: 0.062s max: 0.067s std dev: 0.00175s window: 9
average rate: 15.048
	min: 0.028s max: 0.105s std dev: 0.01145s window: 24
average rate: 15.024
	min: 0.028s max: 0.105s std dev: 0.00892s window: 39
average rate: 15.012
	min: 0.027s max: 0.105s std dev: 0.01257s window: 54
average rate: 15.009
	min: 0.027s max: 0.105s std dev: 0.01111s window: 69
average rate: 15.005
	min: 0.027s max: 0.105s std dev: 0.01006s window: 84
average rate: 15.001
	min: 0.027s max: 0.105s std dev: 0.00925s window: 99
average rate: 14.885
	min: 0.027s max: 0.122s std dev: 0.01040s window: 107
no new messages
average rate: 11.798
	min: 0.027s max: 1.948s std dev: 0.18045s window: 109
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 7.027
	min: 0.027s max: 6.891s std dev: 0.65298s window: 117
average rate: 7.483
	min: 0.027s max: 6.891s std dev: 0.61493s window: 132
average rate: 7.889
	min: 0.027s max: 6.891s std dev: 0.58286s window: 147
average rate: 8.252
	min: 0.027s max: 6.891s std dev: 0.55532s window: 162
average rate: 8.582
	min: 0.027s max: 6.891s std dev: 0.53135s window: 177
average rate: 8.880
	min: 0.027s max: 6.891s std dev: 0.51023s window: 192
average rate: 9.151
	min: 0.027s max: 6.891s std dev: 0.49145s window: 207
average rate: 9.400
	min: 0.027s max: 6.891s std dev: 0.47461s window: 222
^Caverage rate: 9.539
	min: 0.027s max: 6.891s std dev: 0.46530s window: 231
ubscribed to [/camera/depth/color/points]

and the output when alignment is enabled;

subscribed to [/camera/aligned_depth_to_color/image_raw]
no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 9.206
	min: 0.109s max: 0.109s std dev: 0.00000s window: 2
average rate: 1.995
	min: 0.109s max: 0.894s std dev: 0.39272s window: 3
average rate: 1.476
	min: 0.109s max: 1.030s std dev: 0.40610s window: 4
average rate: 1.316
	min: 0.109s max: 1.030s std dev: 0.37945s window: 5
average rate: 1.243
	min: 0.109s max: 1.030s std dev: 0.35088s window: 6
average rate: 1.191
	min: 0.109s max: 1.030s std dev: 0.32982s window: 7
average rate: 1.160
	min: 0.109s max: 1.030s std dev: 0.31017s window: 8
average rate: 1.137
	min: 0.109s max: 1.030s std dev: 0.29400s window: 9
average rate: 1.124
	min: 0.109s max: 1.030s std dev: 0.27862s window: 10
average rate: 1.111
	min: 0.109s max: 1.030s std dev: 0.26623s window: 11
average rate: 1.097
	min: 0.109s max: 1.030s std dev: 0.25628s window: 12
average rate: 1.088
	min: 0.109s max: 1.030s std dev: 0.24683s window: 13
average rate: 1.079
	min: 0.109s max: 1.030s std dev: 0.23839s window: 14
average rate: 1.073
	min: 0.109s max: 1.030s std dev: 0.23051s window: 15
average rate: 1.069
	min: 0.109s max: 1.030s std dev: 0.22315s window: 16
no new messages
no new messages
average rate: 0.939
	min: 0.109s max: 3.007s std dev: 0.54605s window: 17
no new messages
average rate: 0.891
	min: 0.109s max: 3.007s std dev: 0.57618s window: 18
no new messages
no new messages
average rate: 0.813
	min: 0.109s max: 3.062s std dev: 0.71483s window: 19
average rate: 0.822
	min: 0.109s max: 3.062s std dev: 0.69791s window: 20
^Caverage rate: 0.829
	min: 0.109s max: 3.062s std dev: 0.68160s window: 21

Are there any warning messages printed by realsense2_camera node?

yes it prints out that;

No stream match for pointcloud chosen texture Process - Color

and time to time it gives this ERROR;

ERROR [140645725357824] (synthetic-stream.cpp:47) Exception was thrown during user processing callback!

As seen on logs, its performing very randomly

[jediofgever]

what is your kernel version ?, I tried on another computer where environmental setup is the same except for the kernel version being 4.15.0-72-generic, then the cam seems to perform as expected.
Kernel version might be triggering this, is it possible that you try on a system with kernel version 5.0.0-37-generic ?

and overall is there any recommended kernel version especially for Ubuntu 18.04 ?
Thank you for your time

[doronhi]

There is a problem with kernel 5.0.0-37-generic. The timestamps stop arriving after a random length of time. That causes issues with synchronizing depth and color images and hence the problems with aligning and pointcloud. I also encountered it a couple of days ago in 4.18.0-15-generic.
The issue is currently being investigated and I assume a fix will be published very soon.

[lyubu]

Hello! My version is Ubuntu18.04 with kernel 5.0.0-37.
When I run
roslaunch realsense2_camera rs_camera.launch filters:=pointcloud enable_infra1:=false enable_infra2:=false
and
rosrun rviz rviz
And I cant see my pointcloud2 topic. Is this what you mean (the problem is kernel 5.0.0-37)?
@doronhi @jediofgever

[jediofgever]

Hi @lyubu ,
As @doronhi confirmed that currently there is an issue with kernel 5.0.0-37,I suggest you to use another kernel version until a fix is published. You can switch kernel versions in the boot up menu of ubuntu. When you restart your machine, choose advanced options for Ubuntu, from the list choose a previous kernel version, for example 4.15.0-74-generic

[ri-ceres]

We are also experiencing the same issue here and have tried a number of different combinations of Ubuntu kernel, realsense firmware, and realsense sdk versions. One source of error seems to be related to firmware versions. 5.11.15.0 seems to work as expected with kernel 5.0.0-37-generic until a device with firmware 5.12.1.0 is plugged in. After this happens, both versions of firmware fail to produce a pointcloud at 30fps, and we see the same dropping of frames as reported by @jediofgever. This has been replicated across 3 different machines over the last week, and even after downgrading our cameras to 5.11.15.0 and recompiling the SDK we have been unable to recover the pointcloud functionality.

Any additional help with this issue would be appreciated.

OS: UBUNTU 18.04 LTS
KERNEL:  5.0.0-37-generic
RealSense: ROS v2.2.12
LibRealSense: v2.31.0, v2.32.0
Intel RealSense d435, d435i
Firmware: 5.11.15.0, 5.12.1.0

Unfortunately, our X399 motherboards have xHCI USB issues with Ubuntu kernels < 5.0, so we have not been able to test with kernel versions 4.15 or 4.18.

[ri-ceres]

Following up on the previous comment, it appears that issue 1024 is related in our case.

Downgrading to librealsense v2.30.1 has restored normal operation of our cameras. Further testing is needed, but after multiple restarts and extended uptime (~45min) there have been zero failure cases. This has been replicated on two separate machines.

Our stable configuration looks like this:

OS: UBUNTU 18.04 LTS
KERNEL:  5.0.0-37-generic
RealSense-ROS: v2.2.12
LibRealSense: v2.30.1
Firmware: 5.11.15.0

[ri-ceres]

After testing various firmware versions including 5.12.0 and 5.12.1, this appears to be entirely related to librealsense version 2.31.0 on our systems, and the solution is to rollback to 2.30.1. The potential firmware implication mentioned above was coincidence due to the systems running the older version of librealsense while testing firmware 5.11.15.0.

[lyubu]

I have uninstalled my librealsense-2.31.0 and installed librealsense-2.30.0 and point cloud has been displayed. Beside this, I did nothing extra. So this means, the display problem is only librealsense-2.31.0. Thanks for your advice so much!!
@ri-ceres
By the way, librealsense-2.30.1 doesn't suit my kernel. Because after I download the package and tried to install, it said that 2.30.1 suits only with kernel 4.4, so my kernel 5.0.0-37 isn't her Mr.Right.
But 2.30.0 also works.

I have not tried to change my kernel from 5.0.0-37 to 4.15, because the operating is a little complicate and I worried it will influence my other packages which I have installed. But thanks for your advice too!!
@jediofgever

Ubuntu 18.04 LTS
KERNEL: 5.0.0-37-generic
librealsense: 2.30.0

[PeterJochem]

How to I download the prior version of librealsense? Do I have to do this from source? Can I use apt somehow?
sudo apt-get install librealsense2=2.30.0 tells me that the version does not exist