Behavior Modeling of AV Users

Behavior Modeling of AV Users

Last Updated: 01/27/2025 | All information is accurate and up-to-date

Main Goals

  • Modeling the behavioral patterns and characteristics of AV users.
    • Drivers, occupants, pedestrians and other road users potentially interacting with AVs.
    • Guide the design, development and evaluation of human-centric AI functions in intelligent transportation.

Research Examples

Naturalistic driving studies
Naturalistic Driving Studies
Behavior Modeling of Vulnerable Road Users
Behavior Modeling of Vulnerable Road Users
Driver Distraction and Relative Risks
Driver Distraction and Relative Risks

Naturalistic Driving Study #1

110-car Naturalistic Driving Study

  • Goal: behaviors of pedestrians and bicyclists in a naturalistic road environment when encountering cars.
  • Method: Synchronized data from three sources were recorded continuously when the subject is driving through the onboard data collection device hidden behind the rearview mirror.
  • Scope:
    • Number of primary subjects: 116
    • Duration: 12 months
    • Collected raw data size:
      • Driving distance: ~1.44 million miles
      • Driving time: > 40,000 hours
      • Video data collected: 90 TB
    • Dataset:
      • 62,000 pedestrians and 13,000 bicyclists with behavior, environment, and risk labels.
Data Processing Pipeline
Data Processing Pipeline

Naturalistic Driving Study #2

  • Naturalistic panoramic road-scene data collection
    • 200 hours of naturalistic data collection in 4 US Regions
    • All VRUs: Pedestrians, E-scooter Riders, Bicyclists
    • Global scene reconstructions (LiDAR + 360 Camera + RTK GPS)
    • Comprehensive annotations (Visual, Map, Action, and Intention labels)

Data Collection Sites

A map of the United States showing all the different cities that the experimental vehicle was used (Los Angeles, San Diego, Austin, Lafayette, Indianapolis, Washington DC, New York, Providence, Salem and Boston.)

Experiment Vehicle

A diagram showing how the camera system is set up within the vehicle
A diagram of a car looking straight down from above, showing the position of the 360-degree camera on the roof.
Two photos of the experimental car. One showing the equipment mounted in the truck and the other showing the camera system mounted to the roof.

Automatic and Manual Annotations of VRU-Encountering Scenes

A diagram showing the sensor-fusion-based scene reconstruction pipeline.
Sensor-fusion-based Scene Reconstruction Pipeline

Visual, movement and action annotations for more than 600k frames to measure the behaviors of VRUs and the contextual scenes.

Two images of computer screens showing the annotations for more than 600k frames.

Scene Reconstruction and Behavior Variable Calculation for >2000 VRUs

A view on the driver's side from the roof-mounted camera.
A forward view of a car from the roof-mounted camera.
A view on the passenger's side from the roof-mounted camera.
An aerial view of a city intersection where three roads cross each other.
Scenario VariableseScooter #1
Front eScooter Speed (mph)16.04
Ego Speed (mph)12.47
Closest Passing Distance (m)n/a
Crossing Distance (m)32.12
Crossing Angle (deg)61.45
Smallest TTC (sec)n/a

Descriptive Vulnerable Road User Behavior Models

Behavior modeling based on naturalistic observations can contribute important details for encountering scenarios.

  • Pedestrians and bicyclists moving speeds at different scenarios
  • Pedestrian and bicyclist limb motion
  • Pedestrian and bicyclist appearance locations
  • Calculation of time-to-potential-conflict (TTC)
  • Vehicle traveling speed at different scenarios
    • 200 hours of naturalistic data collection in 4 US Regions
    • All VRUs: Pedestrians, E-scooter Riders, Bicyclists
    • Global scene reconstructions (LiDAR + 360 Camera + RTK GPS)
    • Comprehensive annotations (Visual, Map, Action, and Intention labels)

E-scooter and cyclist behavior analysis during car encounters based on naturalistic observations.

A pie chart showing the breakdown of the eScooter's performance.
A pie chart showing the breakdown of the bicyclist's performance.