Automated & Cooperative Driving
Perception, prediction, and control for automated vehicles, built on AI and tested on a full-size vehicle.
Why it matters
Automated vehicles can only deliver on safety and efficiency if they perceive the road reliably, predict what surrounding drivers will do, and make decisions that hold up in mixed traffic. Each of those steps breaks in conditions that matter to agencies: weak GPS, occluded lane markings, dense merges, and roads that were never designed for automated driving.
What we do
We develop the perception, prediction, and control algorithms behind automated and cooperative driving, and we use AI wherever it adds capability that classical models lack: learned localization, topology reasoning, reinforcement learning, and, more recently, foundation models and agentic AI for driving decisions. We keep analytical models where they give guarantees that learning alone cannot.
What sets the work apart is the test loop. Algorithms move from our CARLA–SUMO co-simulation platform to a full-size autonomous vehicle and then to the Larson Transportation Institute test track, one of 10 USDOT-designated Automated Vehicle Proving Grounds.
We also study the other side of deployment: how roads, pavement, and policy have to change for automated vehicles to operate at scale.
What we work on
Perception and localization
Algorithms that let a vehicle know where it is and what the road looks like when GPS is weak and lanes are occluded: LiDAR-based cooperative localization between a leader and a follower vehicle, LiDAR SLAM for truck automation, monocular visual odometry, infrastructure-assisted cooperative perception, all-weather LiDAR segmentation, and lane graph prediction that stays reliable under occlusion.
- Lightweight LiDAR-Based Cooperative Localization Model for Asymmetric Leader-Follower Cooperative Driving Automation System (IEEE T-ITS, 2026)
- Confidence-Gated Topology Reasoning with Fiducial Marker Priors for Occlusion-Robust Lane Graph Prediction (ICRA 2026, 2026)
- Infrastructure Sensor-Based Cooperative Perception for Early Stage Connected and Automated Vehicle Deployment (JITS, 2024)
Prediction, behavior, and scenario generation
Models of how human-driven and automated vehicles actually move, so that planners, digital twins, and test scenarios behave like real traffic: online lane-change inference, stochastic lateral wandering in mixed traffic, behavior-consistent digital twin trajectories, and collision-constrained generation of safety-critical scenarios.
- CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation (ECCV 2026, 2026)
- Online Inference of Lane Changing Events for Connected and Automated Vehicle Applications with Analytical Logistic Diffusion Stochastic Differential Equation (TR Part C, 2022)
- Behavioral Investigation of Stochastic Lateral Wandering Patterns in Mixed Traffic Flow (TR Part C, 2023)
Planning and control
Control methods that range from analytical to learning-based: freeway merging control with state-dependent discharge rates, deep reinforcement learning for longitudinal trajectories, Monte Carlo tree search at arterial intersections, platooning, and simulation-based verification of lane keeping assistance. A current project applies agentic AI to goal-directed, cooperative decision-making.
- Analytical Formulation of Autonomous Vehicle Freeway Merging Control with State-Dependent Discharge Rates (IEEE T-ITS, 2025)
- Online Longitudinal Trajectory Planning for Connected and Autonomous Vehicles in Mixed Traffic Flow with Deep Reinforcement Learning Approach (JITS, 2023)
- A Monte Carlo Tree Search-Based Mixed Traffic Flow Control Algorithm for Arterial Intersection (TRR, 2020)
Infrastructure and policy for automation
What roads and rules need to change before automated vehicles operate at scale: how automated driving changes pavement roughness and long-term condition, AI-based pavement condition prediction, the PennDOT Act 130 assessment of CAV impacts on Pennsylvania, and an analysis of U.S. automated vehicle policy.
- Highly Automated Vehicle Policy in the United States: Six Questions Exposing the Structural Misalignment Between Perception, Regulation, and Enforcement (TR Today, 2026)
- A Machine Learning-Based Approach to Assess Impacts of Autonomous Vehicles on Pavement Roughness (Phil. Trans. R. Soc. A, 2023)
- Bridging Multi-Level Vehicle Behavioural Patterns with Long-Term Pavement Condition: A Bidirectional Modelling Approach for Progressively Mixed Autonomous Traffic Flow (Transportmetrica B, 2024)
Representative work
Confidence-Gated Topology Reasoning with Fiducial Marker Priors for Occlusion-Robust Lane Graph Prediction
Predicts the lane graph an automated vehicle needs for planning, and keeps the prediction reliable when lanes are occluded by gating topology reasoning on confidence and using fiducial markers as priors.
CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation
Generates safety-critical driving scenarios with flow matching under collision constraints, for testing automated driving systems.
Lightweight LiDAR-Based Cooperative Localization Model for Asymmetric Leader-Follower Cooperative Driving Automation System
A lightweight LiDAR-based model that lets the follower vehicle localize itself relative to the leader in an asymmetric leader-follower system, reducing reliance on GPS.
Analytical Formulation of Autonomous Vehicle Freeway Merging Control with State-Dependent Discharge Rates
An analytical formulation of autonomous vehicle merging control at freeway on-ramps, with discharge rates that change with the traffic state.
Highly Automated Vehicle Policy in the United States: Six Questions Exposing the Structural Misalignment Between Perception, Regulation, and Enforcement
Examines U.S. policy for highly automated vehicles through six questions and shows where public perception, regulation, and enforcement are structurally misaligned.
Videos
Penn State Student Researchers Demo Self-Driving Vehicle with President Neeli Bendapudi
Autonomous vehicle demo with real-time LiDAR and camera data
CARMA cooperative driving automation stack tested on the Penn State campus
Waypoint following on the LTI test track
Sponsored projects
All projectsAgentic AI for Autonomous Vehicles: Enhancing Goal-Directed and Cooperative Decision-Making at Penn State
Institute for Computational and Data Sciences, Penn State
2025–2026
PIHarry West Teaching Award: Creating a Shared Automated Vehicle Dataset for Transportation Engineering Teaching
Penn State
2024–2025
PIPennDOT Act 130 Reporting: Evaluating the Impact of Connected and Automated Vehicles to the Commonwealth of Pennsylvania
Pennsylvania Department of Transportation
2024–2025
PIIntegrating Arts with Autonomous Vehicles for Transportation Engineering Education
Leonhard Center, Penn State
2023–2024
PISocially Responsible AI for Autonomous Vehicles
Center for Socially Responsible Artificial Intelligence, Penn State
2022–2023
PIAI-Based Prediction Models for Transportation Infrastructure Asset Management Data Hub – Phase I
U.S. Department of Transportation (NCTriDurLE UTC)
2022–2023
PSU PIPreparation of Pavement Infrastructure for Connected and Autonomous Vehicle Deployment – Phase I
U.S. Department of Transportation (NCTriDurLE UTC)
2022–2023
PSU PINational Center for Transportation Infrastructure Durability and Life Extension (TriDurLE), a National University Transportation Center
U.S. Department of Transportation
2020–2022
Co-PI
Recent publications
All 21 in this thrustFrom Clear to Adverse: A Survey of All-Weather LiDAR Semantic Segmentation for Autonomous Driving
FGD-VO: Enhancing Monocular Visual Odometry for Autonomous Vehicles with Reliable Inter-Frame Correspondence Modeling
Lightweight LiDAR-Based Cooperative Localization Model for Asymmetric Leader-Follower Cooperative Driving Automation System
Highly Automated Vehicle Policy in the United States: Six Questions Exposing the Structural Misalignment Between Perception, Regulation, and Enforcement
CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation
Multi-Label Scene Classification for Autonomous Vehicles: Acquiring and Accumulating Knowledge from Diverse Datasets
Confidence-Gated Topology Reasoning with Fiducial Marker Priors for Occlusion-Robust Lane Graph Prediction
Evaluating the Impact of Connected and Automated Vehicles to the Commonwealth of Pennsylvania
Analytical Formulation of Autonomous Vehicle Freeway Merging Control with State-Dependent Discharge Rates
SimLKAS: A Simulation-Based Framework for the Verification and Validation of Lane Keeping Assistance Systems
People
Work with us
We work with agencies and companies that want to test automated driving algorithms on a real vehicle and a closed test track, or need an independent evaluation of how automation affects their roads.
Email Dr. Hu