Overview
This research introduces a wind-relative reinforcement-learning framework designed for olfactory search in turbulent environments. The framework enables an agent to navigate turbulent plumes using a single internal variable and actions relative to a locally estimated wind direction. The wind direction is processed through an exponential memory kernel. This approach addresses the challenges of intermittent odor detections and fluctuating wind directions encountered in turbulent conditions, which often render standard search strategies unreliable.
Research Context
Olfactory search in turbulent environments poses a sensorimotor problem that many animals efficiently solve, yet its replication in artificial systems remains challenging. The difficulty arises from the intermittent nature of odor detections and the strong fluctuations in wind direction characteristic of turbulent conditions. Such environmental dynamics can undermine the effectiveness of conventional search strategies.
Approach
The developed framework incorporates a reinforcement-learning agent. This agent utilizes one internal variable: the elapsed time since the last odor detection. Actions are selected by the agent relative to a locally estimated wind direction. This wind direction estimate is refined through an exponential memory kernel. The policies derived from this framework were trained and subsequently evaluated within direct numerical simulations of turbulence. These simulations were configured to capture the multi-scale characteristics of both velocity and odor fields, representative of natural environments. Evaluations were conducted under two conditions: the presence of a mild mean wind and in isotropic turbulence (absence of a mean wind).
Findings
- In the presence of a mild mean wind, the learned policy exhibited superior performance compared to established biomimetic policies, including cast-and-surge. This performance advantage was observed irrespective of the wind memory time utilized.
- The learned policy demonstrated an ability to adapt its movement patterns based on the quality of its wind-estimation.
- In isotropic turbulence, the search performance peaked at an intermediate wind memory time. This finding suggests that temporal wind integration functions as a regime-dependent resource in such conditions.
Why This Matters
The findings emphasize the necessity of developing and validating olfactory-navigation strategies under realistic turbulent conditions. The research offers a compact design principle for minimal robotic olfactory navigation. Furthermore, it provides testable predictions concerning biological search behavior.