Save/Load Atlas Functionality #310

neo-knight-td · 2021-05-04T15:59:17Z

For Saving an Loading an Atlas session. This solves the serialization bug of version V.0.3 (from which most of the code is borrowed). Map merging is also solved. An example script is made available.

…NBestCandidates

fliegend · 2021-05-06T12:13:42Z

include/Map.h


 #include <boost/serialization/base_object.hpp>
+#include <boost/serialization/map.hpp>
+#include <boost/serialization/set.hpp>


These are important

fliegend · 2021-05-06T12:22:58Z

include/Atlas.h

+
+        //Save/Load all objects
+        ar & mspMaps;
+        ar & mvpBackupMaps;


is this necessary?

Now that you mention it, serializing mspMaps should not be required given that these are backed up in PreSave and rebuilt in PostLoad. I have not yet tested without that line if it works. If it's the case I'll make the change ;) Thanks !

you are right, either mspMaps or mvpBackupMaps is ok. https://github.com/UZ-SLAMLab/ORB_SLAM3/pull/310/files#r627846281

fliegend · 2021-05-06T12:25:25Z

CMakeLists.txt

 include/MLPnPsolver.h
 include/TwoViewReconstruction.h
 include/Config.h
+include/serialize_tuple.h


why not directly use boost library? It works with me.

When I serialized large maps, I had a bug when recovering the MapPoints's observations from the backup. Serializing mObservations directly solved the problem. However those observations contain a tuples. If I'm not mistaken, by default, there is no tuple serialization in Boost. That's why this is required.

I didn't meet this kind of problem. Will your program break down with red error? Recover from backup is ok from my side.

not sure if this is helpful. https://github.com/UZ-SLAMLab/ORB_SLAM3/pull/310/files#r627844802
But I think add serialize_tuple.h is also a good option:)

fliegend · 2021-05-07T00:21:22Z

src/MapPoint.cc

+        else
+        {
+            EraseObservation(pKFi);
+        }


I comment out line 605-608 when recover mObservations from backup, probably this the bug

I comment out line 605-608 when recover mObservations from backup, probably this the bug

Hello,
Could we add wechat (william_hjx)? I have few questions about the commented lines. Thx!

fliegend · 2021-05-07T00:26:26Z

src/Atlas.cc

+    mspMaps.clear();
+    unsigned long int numKF = 0, numMP = 0;
+    map<long unsigned int, KeyFrame*> mpAllKeyFrameId;
+    for(Map* pMi : mvpBackupMaps)


I use for(Map* pMi : mspMaps) here directly, since mspMaps is already stored and mvpBackupMaps may be not necessary

PattyLiang · 2021-06-08T08:48:00Z

@neo-knight-td why are you comment all of inertial variables? Is that due to the serialization problem of std::vector<integrable> mvMeasurements? Or inertial variables are useless?

phdsky · 2021-07-28T03:51:40Z

Good job! I have tested it, only sometimes when loading a map it may occur core dump, but works well after loading the map again in stereo mode. Stereo_Inertial not working when loading map, just core dump.

Jiaxin8122 · 2021-08-21T15:13:53Z

I was using only monocular to run this case but failed with my rosbag...Map can be saved and reloaded but when I changed into relocalization mode, frames will disappear in screen. So everything got blank ...Are there some variables I didn't save or load?
Thx!

Jiaxin8122 · 2021-08-23T08:58:23Z

src/KeyFrame.cc

+    SetPose(Tcw);
+
+    /*
+    // Reference reconstruction


Why comment on these lines?

Jiaxin8122 · 2021-08-23T09:02:07Z

src/MapPoint.cc

+        else
+        {
+            EraseObservation(pKFi);
+        }


I comment out line 605-608 when recover mObservations from backup, probably this the bug

Hello,
Could we add wechat (william_hjx)? I have few questions about the commented lines. Thx!

neo-knight-td · 2021-08-23T10:39:37Z

@neo-knight-td why are you comment all of inertial variables? Is that due to the serialization problem of std::vector<integrable> mvMeasurements? Or inertial variables are useless?

@PattyLiang As far as I was concerned, I did not need the inertial variables to be serialized. Therefore I commented those lines.

neo-knight-td · 2021-08-23T10:43:20Z

Good job! I have tested it, only sometimes when loading a map it may occur core dump, but works well after loading the map again in stereo mode. Stereo_Inertial not working when loading map, just core dump.

@phdsky Thank you ! With which maps did you encounter problems at loading ? Maps from which sequences ? Stereo or Mono ?

neo-knight-td · 2021-08-23T10:45:32Z

I was using only monocular to run this case but failed with my rosbag...Map can be saved and reloaded but when I changed into relocalization mode, frames will disappear in screen. So everything got blank ...Are there some variables I didn't save or load?
Thx!

@Jiaxin8122 What do you mean by "change in relocalization mode" ? You trigger the change manually ? On which sequences do you experience such errors ?

Jiaxin8122 · 2021-08-23T11:20:05Z

I was using only monocular to run this case but failed with my rosbag...Map can be saved and reloaded but when I changed into relocalization mode, frames will disappear in screen. So everything got blank ...Are there some variables I didn't save or load?
Thx!

@Jiaxin8122 What do you mean by "change in relocalization mode" ? You trigger the change manually ? On which sequences do you experience such errors ?

Yes, I use the bottom from GUI to begin the LocalizationMode. The save and load function in my case looks nice. But when I wanna load the saved map to relocalize, this error happened. Also, just curious why you commented the save map point id and observation.

neo-knight-td · 2021-08-23T11:49:01Z

I was using only monocular to run this case but failed with my rosbag...Map can be saved and reloaded but when I changed into relocalization mode, frames will disappear in screen. So everything got blank ...Are there some variables I didn't save or load?
Thx!

@Jiaxin8122 What do you mean by "change in relocalization mode" ? You trigger the change manually ? On which sequences do you experience such errors ?

Yes, I use the bottom from GUI to begin the LocalizationMode. The save and load function in my case looks nice. But when I wanna load the saved map to relocalize, this error happened. Also, just curious why you commented the save map point id and observation.

I never tried the LocalizationMode only (even from non loaded sequences). I don't exactly know what it does in the code when this button is pressed. I will need to dig a bit in order to answer your question. Also, regarding the commented lines, it's been quite a while since I did this so I don't remember the exact reason but certainly this was to avoid other core dumps.

I come back to you when I have more answers to your questions ;)

Jiaxin8122 · 2021-08-23T11:58:03Z

I was using only monocular to run this case but failed with my rosbag...Map can be saved and reloaded but when I changed into relocalization mode, frames will disappear in screen. So everything got blank ...Are there some variables I didn't save or load?
Thx!

@Jiaxin8122 What do you mean by "change in relocalization mode" ? You trigger the change manually ? On which sequences do you experience such errors ?

Yes, I use the bottom from GUI to begin the LocalizationMode. The save and load function in my case looks nice. But when I wanna load the saved map to relocalize, this error happened. Also, just curious why you commented the save map point id and observation.

I never tried the LocalizationMode only (even from non loaded sequences). I don't exactly know what it does in the code when this button is pressed. I will need to dig a bit in order to answer your question. Also, regarding the commented lines, it's been quite a while since I did this so I don't remember the exact reason but certainly this was to avoid other core dumps.

I come back to you when I have more answers to your questions ;)

That's really helpful. Thx a lot!

neo-knight-td · 2021-08-24T08:14:04Z

@Jiaxin8122 What did you use as sequences in Monocular ? I tested on MH01 from EuRoC, played with the Localization Mode button with a loaded sequence but I have no "frame disappearing" or whatsoever. Could you provide some screenshots ?

neo-knight-td · 2021-09-18T10:30:29Z

Code Update :

The code for Monocular is integrated. The example file is adapted. Also, unserialized MspMaps (was not needed as there is a backup of it).

Code for inertial will come later.

neo-knight-td · 2021-09-18T10:32:22Z

@Jiaxin8122,

Sorry for the delay, quite busy at the moment :/

Unfortunately I could not replicate your bug. I added the code I use for Monocular. Try to use use it and tell me what you get with it. I tried switching on and off the Localization Mode button but did not ended with such blancks or whatsoever :/

Could you provide your code/bagfile ?

ps : Regarding the Localization Mode, the only thing it does is turning off the Local Mapping thread ;)

Jiaxin8122 · 2021-09-18T12:21:45Z

@Jiaxin8122,

Sorry for the delay, quite busy at the moment :/

Unfortunately I could not replicate your bug. I added the code I use for Monocular. Try to use use it and tell me what you get with it. I tried switching on and off the Localization Mode button but did not ended with such blancks or whatsoever :/

Could you provide your code/bagfile ?

ps : Regarding the Localization Mode, the only thing it does is turning off the Local Mapping thread ;)

Hi, thx for your answer. I have solved my problem because there is a bug in the tracking thread.

Thanks for your helping again!

makolele12 · 2021-10-20T13:37:40Z

Hi, I am testing your code and found a problem which can be replicated following the next steps:

Run ORB-SLAM3 and make sure that there is at least a map merging
Save the map
Load the same map
Try to save again the map (over the previously saved map)
--> The map will not be successfully saved (segmentation fault)

After some investigation, I´ve found out that the problem occurs here. Apparently, one (or some) of the pMPi pointers are invalid. Therefore, GetObservations().size() fails.

Do you know where the problem could be?

makolele12 · 2021-10-22T12:03:20Z

Hi, I am testing your code and found a problem which can be replicated following the next steps:

Run ORB-SLAM3 and make sure that there is at least a map merging

Save the map

Load the same map

Try to save again the map (over the previously saved map)
--> The map will not be successfully saved (segmentation fault)

After some investigation, I´ve found out that the problem occurs here. Apparently, one (or some) of the pMPi pointers are invalid. Therefore, GetObservations().size() fails.

Do you know where the problem could be?

After some more debugging I have found out the problem is here.

Apparently at one given iteration, executing EraseObservation() corrupts somehow mspMapPoints. Hence, at the next step pMPi points to an invalid location.

JACKLiuDay · 2022-04-14T05:57:30Z

Hi, I am testing your code and found a problem which can be replicated following the next steps:

Run ORB-SLAM3 and make sure that there is at least a map merging

Save the map

Load the same map

Try to save again the map (over the previously saved map)
--> The map will not be successfully saved (segmentation fault)

After some investigation, I´ve found out that the problem occurs here. Apparently, one (or some) of the pMPi pointers are invalid. Therefore, GetObservations().size() fails.
Do you know where the problem could be?

After some more debugging I have found out the problem is here.

Apparently at one given iteration, executing EraseObservation() corrupts somehow mspMapPoints. Hence, at the next step pMPi points to an invalid location.

Hi guy, thx for your contribution about the project. I have been testing ORB_SLAM3 mono_imu as saving map for weeks. But it often seg fault.
#535
So in your reply, the bug is caused by the
pMPi->EraseObservation(it->first); //We need to find where the KF is set as Bad but the observation is not removed
So we need to find some KF that not removed by observation? What would happen, if I deleted this line just ignore it.

Thomas Dutrannois added 13 commits May 4, 2021 17:32

Boost serialization code + PreSave/PostLoad functions

980ac9b

Serialization code + added constructor for KFDB loading

0ce0d6d

Serialization code + added include for tuple serialization

3e58a1f

Load/Save Atlas functions declaration

ae0ed23

PreSave/PostLoad code added

b3a7adc

Functions for Save/Load + constructor + avoid infinite loop in Detect…

c4ce511

…NBestCandidates

avoid failure in 1st LBA post merge

9e603a7

added condition in color index

3ae3981

PreSave/PostLoad functions + avoid getting empty camera center

2dd5fef

Added include and examples

a9948ee

Save/Load Functions

ea2da47

Set StepbyStep

a77e821

Added include and example code

dcbb819

This was referenced May 4, 2021

how to use the SaveAtlas() and LoadAtlas() for relocalization? #128

Closed

in ros save map? #17

Closed

Save/Load Atlas V0.4 #301

Closed

fliegend reviewed May 6, 2021

View reviewed changes

fliegend reviewed May 7, 2021

View reviewed changes

wangtsiao mentioned this pull request Aug 19, 2021

Saving all the Trajectory if not Merging #361

Open

Jiaxin8122 reviewed Aug 23, 2021

View reviewed changes

Thomas Dutrannois added 4 commits September 18, 2021 11:52

Add Mono scripts in CMakeLists.txt

114227e

Unserialize MspMaps

c9316ed

Mono Scripts

ce8a338

Add Mono code lines

d2f6ddd

knazneen123 mentioned this pull request Nov 30, 2021

Floating point exception #430

Closed

jdtardos closed this Dec 23, 2021

Ryan-Red mentioned this pull request Jun 24, 2022

Save and Load maps Soldann/MORB_SLAM#5

Closed

Save/Load Atlas Functionality #310

Save/Load Atlas Functionality #310

Uh oh!

Conversation

neo-knight-td commented May 4, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PattyLiang commented Jun 8, 2021

Uh oh!

phdsky commented Jul 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jiaxin8122 commented Aug 21, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

neo-knight-td commented Aug 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neo-knight-td commented Aug 23, 2021

Uh oh!

neo-knight-td commented Aug 23, 2021

Uh oh!

Jiaxin8122 commented Aug 23, 2021

Uh oh!

neo-knight-td commented Aug 23, 2021

Uh oh!

Jiaxin8122 commented Aug 23, 2021

Uh oh!

neo-knight-td commented Aug 24, 2021

Uh oh!

neo-knight-td commented Sep 18, 2021

Uh oh!

neo-knight-td commented Sep 18, 2021

Uh oh!

Jiaxin8122 commented Sep 18, 2021

Uh oh!

makolele12 commented Oct 20, 2021

Uh oh!

makolele12 commented Oct 22, 2021

Uh oh!

JACKLiuDay commented Apr 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

phdsky commented Jul 28, 2021 •

edited

Loading

neo-knight-td commented Aug 23, 2021 •

edited

Loading

JACKLiuDay commented Apr 14, 2022 •

edited

Loading