what is the relation between sense of agency, exploration of the world, reward system and intrinsic motivation

The relationship among sense of agency, exploration of the world, reward system, and intrinsic motivation is foundational for understanding how humans (and many animals) initiate, sustain, and find satisfaction in self-driven learning and creative behaviors. These constructs form an interconnected loop, each reinforcing the others and collectively enabling adaptive behavior and psychological well-being. Below, each is described and the evidence for their interrelations is synthesized with reference to empirical findings.

---

Sense of Agency and Its Neurocognitive Basis

The sense of agency (SoA) is the subjective experience that one's intentions and actions are the cause of effects in the external (or internal) world. SoA arises from both retrospective processes—comparing intended and actual outcomes—as well as prospective components generated during action selection[1][2][3]. Prospective senses of agency rely on internal monitoring in prefrontal and parietal networks, particularly involving the angular gyrus and dorsolateral prefrontal cortex, which track whether an action is self-initiated and likely to bring about the desired effect[1]. When action selection is smooth and well-matched to subsequent outcomes, the SoA is heightened, reinforcing the perception of effective self-causation[1][2].

---

Exploration of the World: Manifestation of Agency and Motivation

Exploration encompasses self-initiated behaviors aimed at interacting with, learning from, or manipulating one's environment to reduce uncertainty or seek novelty[4][5]. Exploration is both a manifestation and a driver of SoA: possessing a robust sense of agency makes individuals more likely to engage with and experiment on their environment, as their actions are expected to produce informative changes[1][2][5]. Conversely, exploration provides feedback loops that refine and reinforce agency, as successful interaction clarifies action-outcome contingencies and builds competence[4][5].

---

Reward System: Intrinsic Reinforcement and Learning Progress

The reward system—centered on the brain's dopaminergic pathways, including the ventral tegmental area and nucleus accumbens—responds to both external rewards and self-generated signals such as curiosity, mastery, and the reduction of prediction error[4][5]. Exploration and the exercise of agency can activate reward circuits even in the absence of overt external incentives, with dopamine signaling the value and salience of new or unexpected outcomes[4][5]. Importantly, research and computational modeling suggest that “learning progress”—the expected reduction in prediction error as a result of one’s own actions—serves as a fundamental intrinsic reward[5]. This encourages individuals (and developing children in particular) to seek not pure novelty or complete predictability, but tasks of intermediate difficulty that maximize learning gains[5].

---

Intrinsic Motivation: Foundation and Outcomes

Intrinsic motivation is the drive to engage in activities for their inherent satisfaction, curiosity, or interest, rather than for separable external rewards[4][6][7]. According to Self-Determination Theory (SDT), intrinsic motivation is optimized when fundamental needs for autonomy, competence (that is, the ability to effect change, closely related to SoA), and relatedness are met[4][6]. Neuroimaging evidence reveals that intrinsically motivated behaviors engage reward-related neural circuits similar to those activated by extrinsic rewards, indicating that internal drivers such as curiosity and competence yield genuine hedonic and motivational value[4]. Intrinsic motivation is closely tied to exploratory behavior and SoA; when individuals feel capable of influencing their environment (high SoA), their intrinsic motivation for exploration and learning is strengthened[4][7][5][8].

---

Interconnections and Feedback Loops

1. Agency as Enabler of Intrinsic Motivation

A strong sense of agency satisfies the autonomy and competence components of SDT, directly supporting intrinsic motivation[4][6][2]. The subjective feeling of control over outcomes (agency) is pleasurable and reinforcing, thereby activating the reward system even in the absence of external incentives[4][5].

2. Agency and Exploration

SoA encourages exploration: when individuals believe their actions are effective, they are more willing to try new things, take on challenges, and test hypotheses about the world[1][5]. Exploration, in turn, fine-tunes agency by providing evidence of cause-effect relationships and supporting skill acquisition[5].

3. Exploration and Reward System

Exploration is intrinsically rewarding when it yields learning progress or satisfaction of curiosity. The dopaminergic system signals not just external rewards, but also the internal “aha” moments associated with new understanding or mastery[4][5]. Computational models highlight that exploratory actions are maintained not by random foraging but by the maximization of expected learning progress—i.e., how much the agent expects to reduce ignorance or error by exploring[5].

4. Reward System and Intrinsic Motivation

Intrinsic motivation is fundamentally supported by internal rewards; neurobiological studies confirm activation of reward-related brain circuits during engagement in intrinsically motivated tasks[4][6]. This convergence underlies why individuals will often invest considerable energy in learning or creative endeavors without any external reinforcer[4][5][8].

5. Exploration, Agency, and Self-Development

Exploration guided by intrinsic motivation and agency leads to mastery, self-efficacy, and ultimately development of complex skills and adaptive behaviors[5][4][8]. This is especially clear in childhood, where open-ended exploration, guided by curiosity and supported by growing agency, structures learning and developmental trajectories[5].

---

Integrated Model

These components can be represented as a cyclical model:

1. Agency enables exploration
2. Exploration yields learning progress
3. Learning progress stimulates dopaminergic reward (intrinsic reward)
4. Reward enhances intrinsic motivation
5. Intrinsic motivation sustains and amplifies agency and further exploration

This positive feedback cycle is summarized as follows:

$$\begin{aligned} &\text{Agency} \\ &\quad \rightarrow \text{Exploration} \\ &\quad \rightarrow \text{Learning Progress (Reward)} \\ &\quad \rightarrow \text{Intrinsic Motivation} \\ &\quad \rightarrow \text{Agency} \end{aligned}$$

---

Empirical and Theoretical Highlights

• Intrinsic motivation and exploratory/mastery behaviors are ancient and phylogenetically conserved, supported by dopaminergic (reward) systems[4][5].
• Computational models and robotic experiments confirm that learning-driven exploration based on maximizing learning progress organizes behavior into ever-more-complex developmental trajectories[5].
• The introduction of external rewards can sometimes undermine intrinsic motivation, especially if they diminish perceived agency or autonomy[7], though this effect is modulated by the type and context of reward[9][10].
• Emotional factors (such as clarity and regulation) also correlate with intrinsic motivation and, through the experience of agency, influence willingness to explore and learn[8].

---

Conclusion

The sense of agency, exploration of the world, the reward system, and intrinsic motivation form an interdependent motivational architecture. Agency underwrites the willingness to explore, exploration activates the reward system via learning progress or curiosity satisfaction, and intrinsic motivation (grounded in this neural and experiential loop) sustains engagement, mastery, and self-driven growth[4][6][1][7][2][5][8]. This integrated cycle is central not only to individual learning, creativity, and development, but also to addressing key aspects in education and mental health.

The relationship between mechanistic models of the sense of agency (SoA)—epitomized by the comparator or forward model—and a more "rich" sense of agency integrated with exploration, curiosity, and intrinsic motivation involves crucial differences in explanatory scope, level of analysis, and underlying neural systems. Yet, these perspectives are complementary, and their integration is increasingly recognized as essential for a unified account of agency in both cognitive neuroscience and motivational psychology.

---

Mechanistic Models: Comparator and Forward Model

The mechanistic or computational view—such as the comparator model—posits that SoA arises primarily from the match between predicted sensory consequences of one's actions (generated by internal forward models) and the actual sensory feedback received. When prediction and outcome align, a pre-reflective (often unconscious) sense of self-causation emerges [1][2]. This model is grounded in motor control, neuroanatomically anchored in cerebellar-cortical-spinal circuits, and is critical for rapid and accurate discrimination of self-generated versus externally-caused events [1][2][3].

Such a mechanistic approach excels at explaining the immediate sense of agency for simple, well-delineated motor actions ("I moved my hand and it felt like I did it") and offers insight into disorders in which this low-level SoA is disrupted, such as schizophrenia [4][1].

Nonetheless, this framework centers on the how of agency—detailing the neural computations and sensory-motor contingencies that signal authorship—while remaining largely agnostic to why actions are generated or what makes agents persist in or select particular actions in the first place.

---

Rich Agency: Curiosity, Exploration, and Intrinsic Motivation

In contrast, a "rich" sense of agency incorporates the motivational, emotional, and volitional dimensions connected to self-endorsed exploration, learning, and seeking novelty—domains foundational to intrinsic motivation [5][6][7]. Here, agency is not simply a feeling associated with action-effect congruence, but also a reflective, affectively charged experience grounded in autonomy, choice, and meaning-making.

Self-Determination Theory (SDT), for example, conceptualizes intrinsic motivation as an agentic tendency to explore, seek challenges, and master new skills, even absent external rewards. Intrinsically motivated behavior is closely tied to a subjective sense of agency and autonomy, as individuals perceive their actions as arising from their own interests and volition rather than external controls [5][8]. Importantly, SDT and related work emphasize that this motivational agency is essential for sustained engagement, creativity, optimal development, and psychological wellness [5][6][8].

At the neurobiological level, intrinsic motivation—along with the sense of agency and exploration—engages large-scale neural systems beyond the classic sensorimotor comparator network. Notably, dopaminergic reward circuits signal the value of novel, self-initiated, and learning-propelling activities, while prefrontal and default mode networks mediate volitional control, self-referential processing, and the setting and maintenance of intrinsic goals [5][7][3].

---

Distinctions and Relations

Level of Explanation

• Mechanistic models: Lower-level, sensorimotor, pre-reflective; how the brain attributes causality to simple actions via prediction-error minimization [1][2][3].
• Rich agency: Higher-level, encompassing cognitive, affective, and motivational processes; why agents originate, choose, and sustain actions that embody exploration and self-development [5][6][4][7][8].

Temporal Scale

• Mechanistic SoA operates over milliseconds to seconds, essential for immediate action monitoring and control [1][2].
• Rich agency unfolds over minutes to years—critical for learning trajectories, skill development, and lifelong adaptation [5][7][8].

Neural Substrates

• Mechanistic SoA: Cerebellum, sensorimotor cortex [1].
• Rich agency: Dopaminergic circuit (ventral striatum, midbrain), prefrontal regions, default mode network [5][7][3].

Phenomenology and Function

• Mechanistic: "Did I cause this action/effect?"—often implicit [2][3].
• Rich: "Am I the originator of my choices? Am I acting out of curiosity or learning needs?"—explicit, reflective, value-laden [6][8].

---

Complementarity and Integration

Rather than being competing accounts, these approaches reflect distinct but interactive layers in the architecture of agency.

Hierarchical Integration

A burgeoning view conceptualizes agency as hierarchical or layered [4][3]:

• Low-level: Comparator/forward models monitor and affirm basic authorship of actions.
• High-level: Volitional, motivational systems (exploration, curiosity) set and endorse action goals, leveraging comparator processes to bring about desired effects.
• Malfunction at either level impairs the total sense of agency, explaining why disruptions in SoA are evident not just in movement disorders but also in motivational deficits (e.g., depression, anhedonia) [6][4][8].

Curiosity-Driven Learning and Forward Models

Computational and robotic research suggests that intrinsic motivation and curiosity can be formally modeled as agents seeking to maximize learning progress, i.e., the expected reduction in prediction error—not just at the sensory-motor level, but in the acquisition of new skills, knowledge, or environmental regularities [7]. Here, forward models are critical not only for action monitoring but also for predicting the outcomes of novel or exploratory behaviors, linking both mechanistic and rich agency within an overarching framework [7][3].

Self-Determination and Autonomy as Necessary for Rich Agency

Empirical research demonstrates that externally controlled actions (e.g., those undertaken solely for external reward) often undermine intrinsic motivation and diminish the subjective feeling of agency, whereas behaviors freely chosen out of interest or challenge enhance both motivation and the rich sense of being an agent [9][8]. This underscores the motivational necessity of a robust sense of authorship not just for immediate action control, but for the emergence and maintenance of curiosity-based exploratory behavior [5][9][8].

---

Synthesis

• The mechanistic view explains how basic sensorimotor contingencies support the immediate, often unconscious feeling that one’s actions matter—essential for on-line control and self-other distinction [1][2][3].
• The rich view of agency explains how curiosity, exploration, and intrinsic motivation enable agentic learning, goal-setting, and identity formation—processes fundamental for development and adaptive functioning [5][6][7][8].
• Neuroscience and computational modeling increasingly bridge these accounts: forward prediction mechanisms scaffold both low-level SoA and, when harnessed by volitional-motivational systems, support the self-determined pursuit of learning and curiosity [5][4][7][3].

In sum, a layered account is warranted: basic comparator models are necessary substrates, but not sufficient for the rich, sustained agency found in intrinsically motivated exploration. The intersection of these models illuminates how humans become not just agents of movement, but agents of inquiry, growth, and meaning-making.