Abstract

This paper presents the design of a two key-frame visual-inertial navigation system (2KF-VINS) using a combined Lie group SE2(3) extended Kalman filter (EKF) design framework. The conventional 2KF-VINS filter is unobservable for translations along all three axes and rotation about the gravity direction. As a result, the filter suffers from estimation inconsistencies related to unobservable transformations of the estimation problem. The proposed combined Lie group SE2(3) framework remedies this issue by implicitly preserving the observability consistency property of the filter. Monte Carlo numerical simulations are used to validate the theoretical performance of the rightSE2(3) 2KF-VINS, along with experimental validation using the EuRoC micro aerial vehicle (MAV) dataset to evaluate the performance in real-world scenarios. Additionally, the proposed algorithm is compared with state-of-the-art MSCKF, RI-MSCKF, leftSO(3), and rightSO(3) 2KF-VINS versions with identical and realistic tuning parameters to validate the performance related to the accuracy, consistency, and computational speed of the method.

References

1.
Rogne
,
R. H.
,
Bryne
,
T. H.
,
Fossen
,
T. I.
, and
Johansen
,
T. A.
,
2018
, “
Redundant MEMS-Based Inertial Navigation Using Nonlinear Observers
,”
ASME J. Dyn. Syst., Meas., Control
,
140
(
7
), p.
071001
.10.1115/1.4038647
2.
Abeywardena
,
D.
,
Huang
,
S.
,
Barnes
,
B.
,
Dissanayake
,
G.
, and
Kodagoda
,
S.
,
2016
, “
Fast, on-Board, Model-Aided Visual-Inertial Odometry System for Quadrotor Micro Aerial Vehicles
,” IEEE International Conference on Robotics and Automation (
ICRA
), Stockholm, Sweden, May 16–21,
IEEE
, pp.
1530
1537
.10.1109/ICRA.2016.7487290
3.
Wu
,
K.
,
Zhang
,
T.
,
Su
,
D.
,
Huang
,
S.
, and
Dissanayake
,
G.
,
2017
, “
An Invariant-EKF VINS Algorithm for Improving Consistency
,” IEEE/RSJ International Conference on Intelligent Robots and Systems (
IROS
), Vancouver, Canada, Sept. 24–28,
IEEE
, pp.
1578
1585
.10.1109/IROS.2017.8205965
4.
Sun
,
K.
,
Mohta
,
K.
,
Pfrommer
,
B.
,
Watterson
,
M.
,
Liu
,
S.
,
Mulgaonkar
,
Y.
,
Taylor
,
C. J.
, and
Kumar
,
V.
,
2018
, “
Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight
,”
IEEE Rob. Autom. Lett.
,
3
(
2
), pp.
965
972
.10.1109/LRA.2018.2793349
5.
Mourikis
,
A. I.
, and
Roumeliotis
,
S. I.
,
2007
, “
A Multi-State Constraint Kalman Filter for Vision-Aided Inertial Navigation
,”
IEEE International Conference on Robotics and Automation
, Rome, Italy, Apr. 10–14,
IEEE
, pp.
3565
3572
.10.1109/ROBOT.2007.364024
6.
Leutenegger
,
S.
,
Lynen
,
S.
,
Bosse
,
M.
,
Siegwart
,
R.
, and
Furgale
,
P.
,
2015
, “
Key Frame-Based Visual-Inertial Odometry Using Nonlinear Optimization
,”
Int. J. Rob. Res.
,
34
(
3
), pp.
314
334
.10.1177/0278364914554813
7.
Bloesch
,
M.
,
Omari
,
S.
,
Hutter
,
M.
, and
Siegwart
,
R.
,
2015
, “
Robust Visual Inertial Odometry Using a Direct EKF-Based Approach
,” IEEE/RSJ International Conference on Intelligent Robots and Systems (
IROS
), Hamburg, Germany, Sept. 28–Oct. 2,
IEEE
, pp.
298
304
.10.1109/IROS.2015.7353389
8.
Qin
,
T.
,
Li
,
P.
, and
Shen
,
S.
,
2018
, “
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
,”
IEEE Trans. Rob.
,
34
(
4
), pp.
1004
1020
.10.1109/TRO.2018.2853729
9.
Delmerico
,
J. A.
, and
Scaramuzza
,
D.
,
2018
, “
A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
,” IEEE International Conference on Robotics and Automation (
ICRA
), Brisbane, Australia, May 21–25, pp.
2502
2509
.10.1109/ICRA.2018.8460664
10.
Huang
,
G. P.
,
Mourikis
,
A. I.
, and
Roumeliotis
,
S. I.
,
2010
, “
Observability-Based Rules for Designing Consistent EKF SLAM Estimators
,”
Int. J. Rob. Res.
,
29
(
5
), pp.
502
528
.10.1177/0278364909353640
11.
Hesch
,
J. A.
,
Kottas
,
D. G.
,
Bowman
,
S. L.
, and
Roumeliotis
,
S. I.
,
2014
, “
Consistency Analysis and Improvement of Vision-Aided Inertial Navigation
,”
IEEE Trans. Rob.
,
30
(
1
), pp.
158
176
.10.1109/TRO.2013.2277549
12.
Ryan
,
J. G.
, and
Bevly
,
D. M.
,
2014
, “
On the Observability of Loosely Coupled Global Positioning System/Inertial Navigation System Integrations With Five Degree of Freedom and Four Degree of Freedom Inertial Measurement Units
,”
ASME J. Dyn. Syst., Meas., Control
,
136
(
2
), p.
021023
.10.1115/1.4025985
13.
Johansen
,
T. A.
,
Hansen
,
J. M.
, and
Fossen
,
T. I.
,
2017
, “
Nonlinear Observer for Tightly Integrated Inertial Navigation Aided by Pseudo-Range Measurements
,”
ASME J. Dyn. Syst., Meas., Control
,
139
(
1
), p.
011007
.10.1115/1.4034496
14.
Li
,
X.
,
Wang
,
Y.
, and
Khoshelham
,
K.
,
2019
, “
Comparative Analysis of Robust Extended Kalman Filter and Incremental Smoothing for UWB/PDR Fusion Positioning in NLOS Environments
,”
Acta Geod. Geophys.
,
54
(
2
), pp.
157
179
.10.1007/s40328-019-00254-8
15.
Huang
,
G.
,
2019
, “
Visual-Inertial Navigation: A Concise Review
,” International Conference on Robotics and Automation (
ICRA
), Montreal, Canada, May 20–24,
IEEE
, pp.
9572
9582
.10.1109/ICRA.2019.8793604
16.
Paul
,
M. K.
,
Wu
,
K.
,
Hesch
,
J. A.
,
Nerurkar
,
E. D.
, and
Roumeliotis
,
S. I.
,
2017
, “
A Comparative Analysis of Tightly-Coupled Monocular, Binocular, and Stereo VINS
,” IEEE International Conference on Robotics and Automation (
ICRA
), Singapore, May 29–June 3,
IEEE
, pp.
165
172
.10.1109/ICRA.2017.7989022
17.
Taffanel
,
A.
, “
Crazyflie Nano Quadcopter
,”
Bitcraze AB
, Sweden.https://www.bitcraze.io/products/crazyflie-2-1/
18.
Preiss
,
J. A.
,
Honig
,
W.
,
Sukhatme
,
G. S.
, and
Ayanian
,
N.
,
2017
, “
Crazyswarm: A Large Nano-Quadcopter Swarm
,” IEEE International Conference on Robotics and Automation (
ICRA
), Singapore, May 29–June 3,
IEEE
, pp.
3299
3304
.10.1109/ICRA.2017.7989376
19.
Dusha
,
D.
, and
Mejias
,
L.
,
2012
, “
Error Analysis and Attitude Observability of a Monocular GPS/Visual Odometry Integrated Navigation Filter
,”
Int. J. Rob. Res.
,
31
(
6
), pp.
714
737
.10.1177/0278364911433777
20.
Kelly
,
J.
, and
Sukhatme
,
G. S.
,
2011
, “
Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-Calibration
,”
Int. J. Rob. Res.
,
30
(
1
), pp.
56
79
.10.1177/0278364910382802
21.
Nguyen
,
T.
,
Mann
,
G. K. I.
,
Vardy
,
A.
, and
Gosine
,
R. G.
,
2019
, “
Developing Computationally Efficient Nonlinear Cubature Kalman Filtering for Visual Inertial Odometry
,”
ASME J. Dyn. Syst., Meas. Control
,
141
(
8
), p.
081012
.10.1115/1.4042951
22.
Wong
,
X. I.
, and
Majji
,
M.
,
2018
, “
Extended Kalman Filter for Stereo Vision-Based Localization and Mapping Applications
,”
ASME J. Dyn. Syst., Meas., Control
,
140
(
3
), p.
030908
.10.1115/1.4037784
23.
Gomaa
,
M. A.
,
De Silva
,
O.
,
Mann
,
G. K.
, and
Gosine
,
R. G.
,
2021
, “
Observability-Constrained VINS for MAVs Using Interacting Multiple Model Algorithm
,”
IEEE Trans. Aerosp. Electron. Syst.
,
57
(
3
), pp.
1423
1442
.10.1109/TAES.2020.3043534
24.
Thalagala
,
R. G.
,
2019
, “
Comparison of State Marginalization Techniques in Visual Inertial Navigation Filters
,”
M.Sc. thesis
, Memorial University of Newfoundland, St. John's, Newfoundland and Labrador, Canada.http://research.library.mun.ca/id/eprint/14086
25.
Brossard
,
M.
,
Bonnabel
,
S.
, and
Barrau
,
A.
,
2018
, “
Invariant Kalman Filtering for Visual Inertial SLAM
,” 21st International Conference on Information Fusion (
FUSION
), Cambridge, UK, July 10–13,
Institute of Electrical and Electronics Engineers
, pp.
2021
2028
.10.23919/ICIF.2018.8455807
26.
Barrau
,
A.
, and
Bonnabel
,
S.
,
2015
, “
An EKF-SLAM Algorithm With Consistency Properties
.”10.48550/arXiv.1510.06263
27.
Huang
,
G. P.
,
Mourikis
,
A. I.
, and
Roumeliotis
,
S. I.
,
2008
, “
Analysis and Improvement of the Consistency of Extended Kalman Filter Based SLAM
,”
IEEE International Conference on Robotics and Automation
, Pasadena, CA, May 19–23,
IEEE
, pp.
473
479
.10.1109/ROBOT.2008.4543252
28.
Fernando
,
E.
,
De Silva
,
O.
,
Mann
,
G. K.
, and
Gosine
,
R.
,
2022
, “
Towards Developing an Indoor Localization System for MAVs Using Two or Three RF Range Anchors: An Observability Based Approach
,”
IEEE Sens. J.
,
22
(
6
), pp.
5173
5187
.10.1109/JSEN.2021.3116930
29.
Brossard
,
M.
,
Barrau
,
A.
, and
Bonnabel
,
S.
,
2019
, “
Exploiting Symmetries to Design EKFs With Consistency Properties for Navigation and SLAM
,”
IEEE Sens. J.
,
19
(
4
), pp.
1572
1579
.10.1109/JSEN.2018.2882714
30.
Heo
,
S.
, and
Park
,
C. G.
,
2018
, “
Consistent EKF-Based Visual-Inertial Odometry on Matrix Lie Group
,”
IEEE Sens. J.
,
18
(
9
), pp.
3780
3788
.10.1109/JSEN.2018.2808330
31.
Barrau
,
A.
, and
Bonnabel
,
S.
,
2017
, “
The Invariant Extended Kalman Filter as a Stable Observer
,”
IEEE Trans. Autom. Control
,
62
(
4
), pp.
1797
1812
.10.1109/TAC.2016.2594085
32.
Bonnabel
,
S.
,
2012
, “
Symmetries in Observer Design: Review of Some Recent Results and Applications to EKF-Based SLAM
,”
Lecture Notes in Control and Information Sciences
, Springer, London, Vol.
422
, pp.
3
15
.10.1007/978-1-4471-2343-9
33.
Zhang
,
T.
,
Wu
,
K.
,
Song
,
J.
,
Huang
,
S.
, and
Dissanayake
,
G.
,
2017
, “
Convergence and Consistency Analysis for a 3-D Invariant-EKF SLAM
,”
IEEE Rob. Autom. Lett.
,
2
(
2
), pp.
733
740
.10.1109/LRA.2017.2651376
34.
Barczyk
,
M.
, and
Lynch
,
A. F.
,
2013
, “
Invariant Observer Design for a Helicopter UAV Aided Inertial Navigation System
,”
IEEE Trans. Control Syst. Technol.
,
21
(
3
), pp.
791
806
.10.1109/TCST.2012.2195495
35.
De Silva
,
O.
,
Mann
,
G. K. I.
, and
Gosine
,
R. G.
,
2018
, “
The Right Invariant Nonlinear Complementary Filter for Low Cost Attitude and Heading Estimation of Platforms
,”
ASME J. Dyn. Syst., Meas., Control
,
140
(
1
), p.
011011
.10.1115/1.4037331
36.
De Silva
,
O.
,
Mann
,
G. K.
, and
Gosine
,
R. G.
,
2012
, “
Development of a Relative Localization Scheme for Ground-Aerial Multi-Robot Systems
,”
IEEE International Conference on Intelligent Robots and Systems
, Vilamoura-Algarve, Portugal, Oct. 7–12, pp.
870
875
.10.1109/IROS.2012.6386015
37.
Farrell
,
J. A.
,
2008
,
Aided Navigation Systems: GPS and High Rate Sensors
, 1st ed.,
McGraw-Hill
,
New York
.
38.
Huang
,
G.
,
Kaess
,
M.
, and
Leonard
,
J. J.
,
2014
, “
Towards Consistent Visual-Inertial Navigation
,” IEEE International Conference on Robotics and Automation (
ICRA
), Hong Kong, China, May 31–June 7,
IEEE
, pp.
4926
4933
.10.1109/ICRA.2014.6907581
39.
De Silva
,
O.
,
Mann
,
G. K.
, and
Gosine
,
R. G.
,
2017
, “
Observability Analysis of Relative Localization Filters Subjected to Platform Velocity Constraints
,”
ASME J. Dyn. Syst., Meas. Control
,
139
(
5
), p.
051009
.10.1115/1.4035295
40.
Bonnabel
,
S.
,
Martin
,
P.
, and
Salaün
,
E.
,
2009
, “
Invariant Extended Kalman Filter: Theory and Application to a Velocity-Aided Attitude Estimation Problem
,”
Proceedings of the IEEE Conference on Decision and Control
, Shanghai, China, Dec. 15–18, pp.
1298
1304
.10.1109/CDC.2009.5400372
41.
Barczyk
,
M.
, and
Lynch
,
A. F.
,
2012
, “
Integration of a Triaxial Magnetometer Into a Helicopter UAV GPS-Aided INS
,”
IEEE Trans. Aerosp. Electron. Syst.
,
48
(
4
), pp.
2947
2960
.10.1109/TAES.2012.6324671
42.
Barrau
,
A.
, and
Bonnabel
,
S.
,
2018
, “
Invariant Kalman Filtering
,”
Ann. Rev. Control, Rob., Auton. Syst.
,
1
(
1
), pp.
237
257
.10.1146/annurev-control-060117-105010
43.
Brossard
,
M.
,
Bonnabel
,
S.
, and
Barrau
,
A.
,
2018
, “
Unscented Kalman Filter on Lie Groups for Visual Inertial Odometry
,”
IEEE International Conference on Intelligent Robots and Systems
, Madrid, Spain, Oct. 1–5, pp.
649
655
.10.1109/IROS.2018.8593627
44.
Forster
,
C.
,
Carlone
,
L.
,
Dellaert
,
F.
, and
Scaramuzza
,
D.
,
2015
, “
IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation
,”
Robotics: Science and Systems
.http://www.roboticsproceedings.org/rss11/p06.pdf
45.
Burri
,
M.
,
Nikolic
,
J.
,
Gohl
,
P.
,
Schneider
,
T.
,
Rehder
,
J.
,
Omari
,
S.
,
Achtelik
,
M. W.
, and
Siegwart
,
R.
,
2016
, “
The EuRoC Micro Aerial Vehicle Datasets
,”
Int. J. Rob. Res.
,
35
(
10
), pp.
1157
1163
.10.1177/0278364915620033
You do not currently have access to this content.