Deep reinforcement learning is expected to advance 80 years of congestion modeling-based research. Scientists demonstrated in what is known as The MegaVander Test that connected autonomous vehicles (AVs) to function as mediators for mixed autonomic transportation with the aim to ease traffic and conserve energy.

Those findings were shared in the latest InnovaTalk webinar, hosted by the VinFuture Foundation, on the topic Smart Transportation and Mobility Solutions for Urban Areas on September 19th. In the webinar, top professionals from the fields of smart transportation and technology, including both Vietnamese and global experts, highlighted the latest technologies aimed at easing traffic and enhancing transport system management. Moreover, the webinar facilitated interactions among experts and participants who were representatives from academic institutions and Vietnamese businesses, providing crucial industry knowledge and avenues for collaboration.

Dr. Padmanabhan Anandan, a world-renowned expert in computer vision and artificial intelligence, and also a Member of the VinFuture Prize Council, was the chair of the webinar. With over three decades of experience spanning both academia and industry, he holds an admirable record of 18,000 citations on Google Scholar. He is now the founder of AI Matters Advisors LLC and chairman of the Governing Board at Telangana AI Mission (T-AIM).

The distinguished speaker of the webinar was Prof. Alexandre Bayen from the University of California, Berkeley. He currently serves as the Associate Provost for Moffett Field Program Development and the Liao-Cho Professor of Engineering. Representing the Vietnamese transportation technology community was Dr. Le Nhan Tam, Chief Technology Officer of Microsoft Vietnam. Dr. Tam has many years of experience in the IT field and has held various positions from a researcher to an expert in the industry. Before Microsoft, Dr. Tam was the Lead Architect and Country Technical Manager of IBM in Vietnam.

The prospects of mixed autonomy and a mega controller

“Smart transportation and smart mobility have been a dream for decades, but right now they have reached a point where things are beginning to shape,” Dr. Anandan set the tone for the discussion. He believed this was possible thanks to three factors. The first was the ubiquity of sensors that gave us a global picture of what was happening on the road. The second was the intense communication level between the cars and the system through the internet network, and the third was the fact that many aspects of car driving nowadays could be controlled by software.

Scientists proposed that these factors make the case for leveraging technology to enhance urban planning, reduce congestion, and increase safety. “The potential is endless,” Dr. Anandan emphasized as he introduced the distinguished speaker of the webinar, Prof. Alexandre Bayen, to present the latest advances in transportation management technology.

The MegaVander Test is set in the context of mixed autonomy in traffic – a condition that implies the coexistence of controlled automated vehicles (CAVs) and human-driven vehicles on the same road. As Prof. Bayen and his colleagues predicted, this is the reality for the next 50 years.

According to Prof. Bayen, a principal contributor to persistent traffic congestion is the stop-and-go wave phenomenon perpetuated by human-driven vehicles. This not only hampers the optimization of the scarce highway capacities but also squanders energy. This is where CAVs enter the fray as a viable solution. Equipped with GSM or 5G chips, these vehicles excel in high-level information transmission amongst themselves, laying a strong case for their integration into broader transportation planning systems. Through such technology-aided coordination, the potential for smooth traffic flow appears significantly brighter.

The crux of this system is integrating driverless cars into traffic flow to act as speed moderators among human-driven vehicles. By strategically positioning them, traffic fluctuations are minimized, leading to energy savings. Preliminary findings indicate a potential 24% reduction in total energy consumption.

The control system’s algorithm is designed to ensure a safe distance between vehicles and manage speed on highways. Researchers must anticipate and set an optimal speed to ensure the system’s functionality, even in the absence of car-to-car communication. To achieve this, they rely on transportation data, often sourced from third-party companies. However, these data can be sometimes inadequate with some crucial parameters missing altogether.

To solve the problem of lack of data and loss of communication, the team has implemented two layers of controllers. The first is the expert controller, which still relies heavily on human calculations, and the second is the imitation learning controller, which employs a feedback control strategy across all simulation scenarios.

Game-changing effect of machine learning

Machine learning has brought about transformative shifts in transportation research as Prof. Bayen highlighted two prominent approaches: imitation learning and deep reinforcement learning. Imitation learning, which involves learning actions based on demonstrated behaviors, empowers the mega-controller system to operate even in the absence of connectivity.

“What you do is you train the imitation learning to operate using a series of scenarios that have been seen before so that in the event no information is passed to it, it still can recognize what is happening and be able to operate fairly well,” Prof. Bayen explained.

On the other hand, CAVs utilize reinforcement learning when they receive information. This enables them to evaluate situations and make decisions based on a predefined algorithm. At its core, reinforcement learning operates within an optimization framework, where the algorithm learns through a system of scores and rewards. The research team has furthered this approach by developing a hybrid model that educates the algorithm about the uncertainties present in their environment.

That model was a success. According to Prof. Bayen, when FLOW project the launched in 2017, it marked the first instance in the history of transportation engineering where microsimulation was integrated with reinforcement learning on the cloud.

The MegaVander Test has been conducted on real highways, and the team is now in the final stages of analyzing data to assess its impact. Preliminary findings have already shown its effectiveness in reducing congestion and saving energy. Prof. Bayen found it particularly intriguing that even human-operated vehicles sandwiched between the controlled cars end up conserving energy, as they are essentially compelled to adopt a similar driving style on the highway.

The future impacts of mixed or full autonomy are still highly uncertain and depend on our actions. Currently, transportation accounts for 28 percent of the United States’ total energy consumption. According to research, vehicle automation has the potential to reduce that impact by 60% if done correctly but could possibly worsen it by 200% if done incorrectly.

Implications for Vietnam

Dr. Le Nhan Tam, the Chief Technology Officer of Microsoft Vietnam, contributed to the discussion by presenting what he envisions as pivotal solutions for advancing smart mobility in developing nations like Vietnam. He outlined five solutions, namely, fare and toll management, traffic and transit management, fleet and asset management, parking management, and the establishment of a data marketplace.

Dr. Tam underscored the crucial role of a data marketplace, aligning with a point earlier made by Prof. Bayen. He stressed the importance of creating a reliable collaborative framework on data among various stakeholders to securely input, store, analyze, and leverage data for communal benefit.

He explained his vision for a future data marketplace: “We can build an ecosystem of transportation providers to collect and share data from the vehicles, analyze data to build the data marketplace and even better, a national open civic database. With those data available, third-party services and companies can use [them] to develop their own solution or business models, and in turn, provide back the data for smart cities infrastructure to enhance the overall system.”

Questions from the audience also centered on how these new technologies could be implemented in Vietnam, especially when mixed autonomy seems to be a distant possibility. Prof. Bayen responded and recommended that the initial steps should be taken on the freeway rather than in the city, as it is a simpler task, especially in areas near major cities with stop-and-go traffic. The subsequent stage could involve the utilization of a Vehicle-to-Infrastructure (V2I) communication model, enabling vehicles to communicate with infrastructure such as traffic lights.

“Currently, it is only implemented on transit vehicles, but maybe VinFast vehicles have features that are going to enable that,” Prof. Bayen said. He also stated that in the situation of Hanoi with mixed vehicles, buses or trucks can play the role of good flow smoothers because they are heavy, do not usually accelerate too much, and are also easier to manage by specific regulations.

Although new cities in developing countries have rather low-quality technological infrastructure, Prof. Bayen believed that there are opportunities for those city planners to do it right from the beginning. “The key idea to realize is that congestion results from an imbalance in supply and demand, not poor driving. When planning a new city, it’s crucial to match the transportation network to the population size. Doing this simplifies everything that follows,” Prof. Bayen said.


Potential and challenges in the pursuit of the hydrogen economy