선택에 대한 보상의 신뢰할당에 관한 기저핵 정보처리 과정 연구
Neural Processes Underling Temporal Credit Assignment
초록/요약
I investigated how different parts of rodent basal ganglia is engaged in the value-based reinforcement learning by recording neural activity in striatum and globus pallidus. At the first experiment, I specifically focused on the role of striatum in the temporal credit assignment. Temporal credit assignment is necessary to properly connect temporally discontiguous choices and their outcomes, because choice outcomes are often revealed after a delay in the real world. To elucidate how the striatum handles the temporal credit assignment problem, I examined neuronal activity in the dorsomedial and dorsolateral striatum (DMS and DLS, respectively) of rats performing a novel behavioral task in which a choice has to be remembered until its outcome is revealed for correct credit assignment. Choice signals appeared sequentially, initially in the DMS and then in the DLS, during the period in which the animal had to remember its choice for correct credit assignment. Action value signals were found in both structures, and were elevated further in the DMS, but not in the DLS, when the choice outcome was revealed. In a behavioral context requiring temporal credit assignment, the DMS and DLS might be sequentially engaged in transmitting choice signals, and the DLS might be in charge of updating the value of the chosen action by integrating choice, action value, and reward signals together. In the second experiment, I focused on the role of globus pallidus. I recorded globus pallidus neurons in a dynamic foraging task on a modified figure 8-shaped maze. As the parts of the downstream structure of striatum, the globus pallidus (GP) showed similar activity with striatum in terms of choice, its outcome, and chosen value. I also founded persistent, but weak, previous choice signals. My results argue against serial processing of information from striatum to globus pallidus. Collectively, these results show that rodent striatum and globus pallidus are engaged in value-based reinforcement learning, and suggest sequential engagement of the DMS and DLS, is temporal credit assignment.
more목차
I. INTRODUCTION 1
A. Temporal credit assignment in reinforcement learning 1
B. Neural substrates of reinforcement learning 4
C. Roles of striatum in reinforcement learning 8
D. Roles of globus pallidus in reinforcement learning 11
E. Aims of the thesis 13
II. MATERIALS AND METHODS 14
A. Experiment in striatum 14
1. Subject 14
2. Apparatus 14
3. Behavior task 17
4. Behavior stages 20
5. Models of behavior 23
6. Unit recording 27
7. Isolation and classification of units 29
8. Multiple regression analysis 32
9. Statistical analysis 33
B. Experiment in globus pallidus 34
1. Subjects 34
2. Behavior task 34
3. Behavior stages 34
4. Unit recording 37
5. Logistic regression analysis of behavior data 39
6. Reinforcement learning model of behavior 40
III. RESULTS 43
A. Experiment in striatum 43
1. Choice behavior during TCA task 43
2. Temporal profiles of neuronal activity 47
3. Neural signals for choice and reward 50
4. Neural signals related to value 56
B. Experiment in globus pallidus 60
1. Choice and behavior during DAWH task 60
2. Temporal profiles of neuronal activity 62
3. Neural signals for choice and reward 65
4. Neural signals related to value 69
IV. DISCUSSION 72
A. Complementary role of DLS and DMS in TCA task 72
B. Different value signals in TCA task 74
C. Neural processes underlining TCA 75
D. Neural processes underlining sequential engaging in choice 77
E. Neural processes in globus pallidus 79
V. CONCLUSION 81
REFERENCES 83
국문요약 102
