ICANEWS

Engineering & Technology

Offline Reinforcement Learning for Rotation Profile Control in Tokamaks

This research investigates the application of offline reinforcement learning (RL) and offline model-based RL algorithms to control the plasma rotation profile w

Overview

This research investigates the application of offline reinforcement learning (RL) and offline model-based RL algorithms to control the plasma rotation profile within tokamaks. The methodology specifically trained these algorithms using historical operational data from the DIII-D tokamak. The developed policy, which models plasma dynamics probabilistically, was subsequently deployed on the DIII-D Tokamak, yielding promising real-world observations.

Research Context

Tokamaks are considered primary candidates for achieving practical fusion energy. However, various critical control challenges within these devices remain difficult or unresolved. One such challenge involves the control of the plasma rotation profile. The rotation profile significantly affects plasma stability, confinement, and transport properties. While the average plasma rotation can be managed, controlling the comprehensive rotation profile presents difficulties due to its high dimensionality, reliance on multiple actuators, and dependence on dynamic plasma conditions. Learning-based control methods, including RL, offer a potential solution to this complex control problem, capable of modeling intricate interactions to enable effective multi-input multi-output control. A significant hurdle in developing such policies is the absence of accurate simulators capable of precisely modeling rotation profile dynamics.

Approach

The study focused on developing and evaluating offline RL and offline model-based RL algorithms for rotation profile control. The training of these algorithms exclusively utilized historical data acquired from the DIII-D tokamak. The final proposed method incorporated probabilistic models of plasma dynamics. These probabilistic models were used to generate rollouts, which are sequences of states and actions, for the purpose of RL training. Following training, the resultant policy was deployed on the DIII-D Tokamak.

Findings

  • Offline RL and offline model-based RL algorithms were investigated for controlling the plasma rotation profile.
  • Training of these algorithms was conducted exclusively using historical data from the DIII-D tokamak.
  • The final method employed probabilistic models of plasma dynamics to generate rollouts for RL training.
  • Deployment of the trained policy on the DIII-D Tokamak resulted in promising real-world observations.

Why This Matters

The ability to control the plasma rotation profile is crucial for advancing fusion energy due to its strong influence on stability, confinement, and transport within tokamaks. Addressing this challenge using learning-based control methods trained on existing data could provide a pathway to manage complex, multi-input multi-output control problems in these devices, contributing to the development of practical fusion energy.

Key Limitations Mentioned by Researchers

The research highlighted key challenges and insights derived from the process of training and deploying an RL policy on a complex physical device, particularly when relying on limited historical data.

Key Findings