Modeling Decision Making in Complex Naturalistic Environments

Mugan, Ugurcan

doi:https://doi.org/10.21985/n2-mgq6-wb98

Work

Modeling Decision Making in Complex Naturalistic Environments

Public

Download PDF

Decisions in naturalistic environments usually feature delayed and uncertain outcomes that can only be reached after a sequence of actions are performed. For example, canonical stalking and pursuit strategies used by terrestrial predators often involve stages of concealment, pauses where the predator remains motionless, and high speed chase sequences. The decision-making processes that underlie action choices across these stages are influenced by sensory perception, time, and the distributed clutter in the environment. While the computations that evaluate these factors are known to occur within the nervous system, the ecological factors that advantage different types of behavioral control paradigms are not well understood. Studies of animal decision making have revealed two distinct, competing control paradigms: habitual and goal-directed (or plan-based). Habitual control appears to be universal amongst vertebrates, both terrestrial and aquatic; in contrast, behavioral and neural evidence for goal-directed control seems to only exist for mammals and birds and is either absent or ambiguous for reptiles, amphibians, and fish. The universality of habitual control may reflect the conserved organizational structure of the basal ganglia from lamprey, jawless fish that preceded mammals by 560 million years, to mammals. The apparent absence of universality of goal-directed control may therefore reflect the novelty of the prefrontal cortex in mammals (and possible functional homologs in birds) which interacts with the hippocampus during planning. This thesis provides theoretical insight into the possible reasons for this uneven distribution of goal-directed control across species by modeling the change in sensory ecology and environmental complexity that occurred during the water-to-land transition. The first part of this thesis models the visual ecology of vertebrates that bracket the water-to-land transition with respect to viewing medium and morphological parameters. Analysis of the selective regime shifts within the time-calibrated phylogenetic tree distributions suggests that larger eyes were most likely favored in animals that were primarily aquatic. Notably, the increase in eye size also coincided with a distinct change in the placement of eyes. Although eyes were placed laterally in the early phase of the tetrapod evolution, similar to other fish, the eyes moved into a position on the top of the head in the transitional group where the first shift in eye size is observed. To understand the basis of the changes in location and size of eye sockets, we estimated the visual capabilities of early tetrapods using first-order optical physics. These results show that the inferred increase in eye size achieves very little additional performance for eyes that are underwater. Therefore, this increase in eye size was likely driven by aquatic tetrapods surfacing their eyes above the water line and hunting like crocodiles. The increased visual awareness brought on by aerial vision, could possibly favor animals evolving morphological adaptations---such as weight bearing limbs---and neurobiological adaptations---such as goal-directed control. The second part of this thesis investigates the effect of topological complexity and range of visual perception on the selective benefit of plan-based control, by modeling predator-prey interactions as a reinforcement learning problem. When a situation affords a long latency between stimulus and response, deliberative behavioral control may be used to generate behavior that is strategic, variable, and hard to predict by an opportunity (e.g. prey), or adversary (e.g. predator). As this latency decreases, the habit-based system takes over to generate responses that are fast, less variable, and easier to predict. As outlined in the first part of this thesis, a sensory consequence of the vertebrate invasion of land was a massive increase in visual range, which allowed animals to see distant potential drivers of behavior, and afforded long delays between stimulus and response. Therefore, this increase in time-to-act permits---but does not necessitate---the contemplation of multiple futures. Importantly, this work shows that the concomitant increase in spatial complexity, due to richer structures (due to vegetation and terrestrial topology) found in terrestrial environments, may have been essential in increasing the number of potential future scenarios. Therefore, aquatic domains---due to both low sensory range causing time-limited decision making and to low spatial complexity---do not advantage extensive planning. The number of possible futures that arise as a consequence of value diversity is dependent on the task environment, and therefore influences the choice of behavioral control paradigm during dynamic tasks (e.g. predator-prey interactions). The distribution of environmental connectedness, quantified based on an abstract representation of the environment, influences the arbitration between behavioral control paradigms, and provides a unifying account of experimental data.

Creator