⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠
Excalidraw Data
Text Elements
Perceptions
World Modelling
MPC
Object Tracks /eve/perception/tracks
What we are concerned with
LiDar (space) + Camera (semantics) + Radar (Velocity) = Tracked Objects for WM
Watod
Container2
Container3
Container1
ros msg w/ DDS/cyclone
node
node
Camera 0-5
LiDar 0-2
Radar 0-3
/eve/camera_0-5
CAR
/eve/lidar_0-2
/eve/radar_0-3
*note: right now we will only be using the camera, 1 LiDar, and 1 radar
HD Map
Predictor
- OSM
Costmap Generator
-
Figure out where things go
-
Street map
- Majority of processing should happen in Eve’s reference Frame
- Provides hard lines for boundaries
- A good amount of this comes from LiDar
PERCEPTIONS STACK
1
2
Object detection (LiDar, Camera, Radar but for now focus on camera)
Traffic light detection
Traffic sign detection
- No sensor fusion
/eve/perception/agent_camera_detections
/eve/perception/traffic_sign_detections
/eve/perception/traffic_light_detections
/vision_msgs/Detection2DArray.msg
/vision_msgs/Detection2DArray.msg
/vision_msgs/Detection2DArray.msg
0: car 1: pedestrian 2: cyclist
0: red 1: yellow 2: green
/eve/perception/agent_classes
/eve/perception/traffic_light_classes
/vision_msgs/VisionInfo.msg
camera detections
Lidar Object Detection
/eve/perception/agent_lidar_detections
Radar Object Detection
/eve/perception/agent_radar_detections
/sensor_msgs/PointCloud2
/custom_radar_message
/vision_msgs/Detection3DArray.msg
Tracking
2D to 3D converter
Camera Depth Estimation
/eve-percetion_camera_depth_estimates
/PerceptionMsgs/AgentCamera.msg
which Depth maps to use (coming from the 6 cameras depending on object positioning), 2D detection to 3D box (traditional CV?)
Frustum projection → euclidean/generic clustering → 3D box?
3D Tracking
Track with camera 3D detections, use lidar for correction, use radar for velocity/accuracy
kalman filter? Estimates future trajectory
*Kalman Filters Predict + Update
State: anything you want to estimate (x,y,z,dx,dy,dz) States Covariance: Uncertainty of state estimate
Used to filter out stuff you don’t want
/eve/perception/agent_tracks
/PerceptionMsg/AgentTracks.msg
/vision_msgs/Detections3DArray + trackID
extends to this
—PERCEPTIONS ENDS HERE— —WORLD MODELLING BEGINS—
Databases XML files that load on launch for msg
Embedded Files
63acef331d61d0ad2e03689a9d99a75c49fb0bf4: cost map.png
