Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach

Perimeter metering control based on macroscopic fundamental diagrams has attracted increasing research interests over the past decade. This strategy provides a convenient way to mitigate urban congestion by manipulating vehicular movements across homogeneous regions without modeling the detailed behaviors and interactions involved with individual vehicle presence. In particular, multi-region perimeter metering control holds promise for efficient traffic management in large-scale urban networks. However, most existing methods for multi-region control require knowledge of either the environment traffic dynamics or network properties (i.e., the critical accumulations), whereas such information is generally difficult to obtain and subject to significant estimation errors. The recently developed model-free techniques, on the other hand, have not yet been shown scalable or applicable to large urban networks. To fill this gap, this paper proposes a scalable model-free scheme based on multi-agent deep reinforcement learning. The proposed scheme features value function decomposition in the paradigm of centralized training with decentralized execution, coupled with critical advances of single-agent deep reinforcement learning and problem reformulation guided by domain expertise. Comprehensive experiment results on a seven-region urban network suggest the scheme is: (a) effective, with consistent convergence to final control outcomes that are comparable to the model predictive control method; (b) resilient, with superior learning and control efficacy in the presence of inaccurate input information from the environment; and (c) transferable, with sufficient implementation prospect as well as real time applicability to unencountered environments featuring increased uncertainty.

© This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/

Files

Metadata

Work Title Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach
Access
Open Access
Creators
  1. Dongqin Zhou
  2. Vikash V. Gayah
License CC BY-NC-ND 4.0 (Attribution-NonCommercial-NoDerivatives)
Work Type Article
Publisher
  1. Transportation Research Part C: Emerging Technologies
Publication Date March 1, 2023
Publisher Identifier (DOI)
  1. https://doi.org/10.1016/j.trc.2023.104033
Deposited March 14, 2023

Versions

Analytics

Collections

This resource is currently not in any collection.

Work History

Version 1
published

  • Created

    March 14, 2023 13:36 by Researcher Metadata Database

  • Added MR-RL_R2_2023.01.18.pdf

    March 14, 2023 13:36 by Researcher Metadata Database

  • Added Creator Dongqin Zhou

    March 14, 2023 13:36 by Researcher Metadata Database

  • Added Creator Vikash V. Gayah

    March 14, 2023 13:36 by Researcher Metadata Database

  • Published

    March 14, 2023 13:36 by Researcher Metadata Database

  • Updated Work Title Show Changes

    May 02, 2023 16:19 by pmk5516

    Work Title
    • Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach
    • ! Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach
  • Updated Description Show Changes

    May 05, 2023 12:35 by pmk5516

    Description
    • <p>Perimeter metering control based on macroscopic fundamental diagrams has attracted increasing research interests over the past decade. This strategy provides a convenient way to mitigate urban congestion by manipulating vehicular movements across homogeneous regions without modeling the detailed behaviors and interactions involved with individual vehicle presence. In particular, multi-region perimeter metering control holds promise for efficient traffic management in large-scale urban networks. However, most existing methods for multi-region control require knowledge of either the environment traffic dynamics or network properties (i.e., the critical accumulations), whereas such information is generally difficult to obtain and subject to significant estimation errors. The recently developed model-free techniques, on the other hand, have not yet been shown scalable or applicable to large urban networks. To fill this gap, this paper proposes a scalable model-free scheme based on multi-agent deep reinforcement learning. The proposed scheme features value function decomposition in the paradigm of centralized training with decentralized execution, coupled with critical advances of single-agent deep reinforcement learning and problem reformulation guided by domain expertise. Comprehensive experiment results on a seven-region urban network suggest the scheme is: (a) effective, with consistent convergence to final control outcomes that are comparable to the model predictive control method; (b) resilient, with superior learning and control efficacy in the presence of inaccurate input information from the environment; and (c) transferable, with sufficient implementation prospect as well as real time applicability to unencountered environments featuring increased uncertainty.</p>
    • Perimeter metering control based on macroscopic fundamental diagrams has attracted increasing research interests over the past decade. This strategy provides a convenient way to mitigate urban congestion by manipulating vehicular movements across homogeneous regions without modeling the detailed behaviors and interactions involved with individual vehicle presence. In particular, multi-region perimeter metering control holds promise for efficient traffic management in large-scale urban networks. However, most existing methods for multi-region control require knowledge of either the environment traffic dynamics or network properties (i.e., the critical accumulations), whereas such information is generally difficult to obtain and subject to significant estimation errors. The recently developed model-free techniques, on the other hand, have not yet been shown scalable or applicable to large urban networks. To fill this gap, this paper proposes a scalable model-free scheme based on multi-agent deep reinforcement learning. The proposed scheme features value function decomposition in the paradigm of centralized training with decentralized execution, coupled with critical advances of single-agent deep reinforcement learning and problem reformulation guided by domain expertise. Comprehensive experiment results on a seven-region urban network suggest the scheme is: (a) effective, with consistent convergence to final control outcomes that are comparable to the model predictive control method; (b) resilient, with superior learning and control efficacy in the presence of inaccurate input information from the environment; and (c) transferable, with sufficient implementation prospect as well as real time applicability to unencountered environments featuring increased uncertainty.
  • Deleted MR-RL_R2_2023.01.18.pdf

    May 05, 2023 12:36 by pmk5516

  • Added Paper 2 - Scalable multi-region perimeter metering control for urban networks A multi-agent deep reinforcement learning approach.pdf

    May 05, 2023 12:37 by pmk5516

  • Updated Work Title Show Changes

    May 24, 2023 10:34 by pmk5516

    Work Title
    • ! Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach
    • Scalable multi-region perimeter metering control for urban networks: A multi-agent deep reinforcement learning approach
  • Updated

    April 04, 2024 10:22 by [unknown user]