KeyWords: 环境感知,多模态融合
核心参考:
0.为何要融合
1. 具体应用
深度补全
语义分割
目标检测及跟踪
特种场景
???真的能做得更好
找找应用场景!
未来趋势
2D——3D
单任务——多任务
信号级——多级融合
早期融合
晚期融合
时间上下文
性能相关
融合数据的特征/信号表示形式(Feature/Signal Representation)
几何约束
时间上下文Encoding Temporal Context
NAS
工程上
外参自校准
相机内参+相机&LiDAR外参
运动引导96+无目标97
时间同步
晶振时间戳(基本完备)
深度补全
激光点云具有稀疏性,进行上采样
同甘融合来引导上采样
encoder-decoder结构
3D目标检测
2D、3D语义分割、实例分割
Tracking
融合策略
决策层融合
决策+特征层融合
特征层融合
参考文献
[1] Cui et.al., Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review, 2020.
[2] Qi et.al., Frustum Pointnets for 3d Object Detection from RGB-D Data, 2018.
[3] Yang et.al., IPOD: Intensive Point-based Object Detector for Point Cloud, 2018.
[4] Zhao et.al., 3D Object Detection Using Scale Invariant and Feature Re-weighting Networks, 2019.
[5] Chen et.al., Multi-View 3D Object Detection Network for Autonomous Driving, 2016.
[6] Ku et.al., Joint 3D Proposal Generation and Object Detection from View Aggregation, 2017.
[7] Liang et.al., Deep Continuous Fusion for Multi-Sensor 3D Object Detection, 2018.
[8] Vora et.al., PointPainting: Sequential Fusion for 3D Object Detection, 2019.
[9] Sindagi et.al., MVX-Net: Multimodal VoxelNet for 3D Object Detection, 2019.