Clustered C-fos Activation in Rat Hippocampus at the Acquisition Stage of Appetitive Instrumental Learning

To address the issue of how hippocampal neurons are involved into learning progress, we studied c-Fos expression in rat hippocampal subfields at different stages of appetitive instrumental learning. To model the first stage of learning, we pre-trained animals to approach the lever to obtain the food, and then made this behavior ineffective by not reinforcing it during the last session (" mis-match " group). Another group just acquired lever-pressing behavior at that day (" acquisition " group). Animals of the third group performed this well-trained behavior (" performance " group). The number of Fos-positive neurons in all hippocampal regions of the " mismatch " group animals was higher than in the ones of the home cage control group animals. The number of Fos-positive neurons was increased in CA1 and CA3 areas, but not in the dentate gyrus of both the " acquisition " and " performance " group animals as compared with the control group. We also found segmented c-Fos expression, which was more evident in " acquisition " group animals. Thus, our results suggest that during the first (mismatch) stage of learning hippocampal neurons are activated in an equally distributed manner. The following clustered pattern of activated CA1 neurons during the acquisition stage may reflect specialization of these neurons in respect to the specific lever-pressing behavior.


Introduction
Most behaviors are not learned at once.Instrumental learning progress is often characterized by learning curves based on the success rate [1].Such progress is mediated by discrete, functional reorganizations of neuronal groups subserving the learned behavior, which suggest existence of more or less distinct stages [2].It is quite common to characterize these stages by a different number of mistakes [3], but more detailed descriptions of each stage based on the underlying neuronal processes are still missing.Specifically, behavioral characteristics of each stage of instrumental learning should be related to patterns of neuronal activity that underlie this stage.
One process that takes place during learning is the development of neuronal activity selectively related to the learned behavior.Such experience-dependent "behavioral specialization" of neurons [4] is a stable characteristic [5] [6], which have many examples [7]- [11].Appearance of such specific activations might be already evident during the earliest newly learned behavioral acts [12] [13].Such changes in neuronal activity were found just before, at, or immediately after the time when the correct behavior was learned [14].Experience-dependent changes in activity of single neurons might result in the described phenomenon of correlated or synchronized activities between neurons [15] [16] and establishment of specific neuronal ensembles, groups or systems of coactive neurons where activities are specifically related to the learned task.
At the level of subcellular biochemical processes, newly synchronized neuronal activations may be accompanied by changes in neuronal gene transcription.Activation of gene transcription, specifically as well as immediate early gene (IEG) induction, is related to acquisition of new experience [17]- [21].This allows using IEG expression imaging as a tool to map patterns of experience-dependent changes in neuronal activity across various brain regions [22].
Neuronal activity changes related to acquisition of instrumental appetitive task have been found in several brain regions including hippocampus [23] [24].Hippocampal neuronal activity plays a crucial role in establishment of action-outcome relationship during instrumental learning [25] [26].Learning dynamics of an operant conditioning task has been shown to be correlated with changes of the intrinsic frequency and amplitude of hippocampal ripple oscillations associated with network synchronization [16], which suggests the recruitment of hippocampal neurons into synchronized ensembles responsible for this learning.Differential involvement of hippocampal subfields into operant odor-discrimination learning has been shown to depend on the stage of the task [27]- [29].However, hippocampal subfields consist of many different neurons that show differential involvement, for example, into the delayed-nonmatching-to-sample task [30].To examine patterns of involvement of hippocampal neurons into the sequential stages of appetitive instrumental learning, we first pre-trained rats to approach the lever in order to receive food, and then during the final session modeled three consecutive stages of instrumental lever-pressing learning in three groups of animals.Animals of the first group ("mismatch" group) demonstrated mostly the previously acquired lever-approaching behavior, which was now ineffective because it was not reinforced.The "acquisition" group animals demonstrated initially scattered lever-pressing behavior."Performance" group animals demonstrated over-trained lever-pressing behavior.Thus, the sequential stages of instrumental learning were the followings: mismatch of the previous experience, initial acquisition of new behavior and correct performance.We found that the overall pattern of hippocampal subfield activations was similar in all groups, but the pattern of neuronal activations inside the CA1 area depended on the learning stage.

Subjects
Twenty four male Long-Evans hooded rats (5 -9 months old) were housed in individual cages.They were food deprived to 85% of their free-feeding body weight and maintained at this level throughout the experiment.Water was available ad libitum.All animal procedures in these studies were in accordance with the National Institutes of Health "Guidelines for the Care and Use of Animals for Experimental Procedures", which were approved by the Russian Academy of Sciences.

Apparatus
All behavioral training took place in an operant chamber of 40 × 40 × 50 cm.The chamber was fitted with an automated plastic food-cup in the corner and a wall-mounted lever located in the other corner along the same wall.The food-cup and the lever were equipped with photodiodes.A button controlled by an experimenter was located outside of the cage and allowed filling the food-cup at any required time.Lever-presses and food-cup checks by an animal were registered by Ikegami data-recorder DTR 1204× (Nihon Kohden, Kogyo Co., Ltd., Tokyo, Japan).

Behavioral Training
Training was conducted daily in 30-min sessions.Animals were progressively shaped across days to acquire the definitive behavior [31].The experimenter delivered a food reward to the subject for approaching the food-cup (two days), then for turning away from the food-cup toward the lever (two days), then for moving half a way toward the lever (two days), then for approaching the lever on the distance of less than 1 cm (two days).Thus, rats were initially pre-trained to approach a lever in order to receive a food from a feeder (Figure 1).On the last experimental day the first stage of learning was modeled by making this behavior ineffective ("mismatch" group, n = 5).Rats of the second group learned to press a lever ("acquisition" group, n = 7).Animals of the third group were trained for lever-pressing task over a period of five days and performed this well-trained behavior during the final experimental session ("performance" group, n = 6).Thus "acquisition" group animals were sacrificed for immunohistochemistry after their first lever-pressing session, and "performance" group animals were sacrificed after their fifth level-pressing session.In order to equalize the total number of sessions in the experimental chamber between all groups the first stage of training (approaching the food-cup) was prolonged up to five days for "mismatch" and "acquisition" group.Animals of the control group (n = 6) were kept in their home cages with free access to food and water and killed at the same time as trained animals.Behavioral measures included the number of presses completed and the number of food-cups checked, along with the timestamp of each event.
To assess instrumental performance, percentage of correct trials (lever-press followed by food-cup check) was calculated as: % of correct trials = Number of lever-presses/Total number of food-cup checks × 100.Mann-Whitney rank sum test was used for analysis of variables between groups, and Wilcoxon test was used for analysis of variables inside groups.All statistical tests were performed in Statistica 5.0.

Immunohistochemistry
Seventy-five minutes after the final experimental session, animals were overdosed with halothane.Their brains were removed and frozen for analysis.Coronal 20 µm cryostat brain sections were taken through the hippocampus (−4.0 to −5.0 mm to bregma) [32].The sections prepared for immunohistochemistry were dried overnight and fixed in 4% paraformaldehyde in 0.1 M phosphate-buffered saline (PBS), pH 7.4, for 15 min.Fixed sections were washed (3x5 min) in 0.1 M PBS and placed into a blocking solution (2.5% normal serum/0.1 M PBS) for 30 min.The sections were then incubated in Fos rabbit polyclonal antibody ("Calbiochem", Ab-5, Cat.#PC38, USA), diluted 1:2000 with 0.1 M PBS, for 18 h.The sections were washed (6 × 5 min) with 0.3% Triton X-100 in 0.1 M PBS, and incubated with biotinylated goat anti-rabbit secondary antibody ("Vector Laboratories", USA) diluted 1:300 in PBS for 2 h.They were then washed (5 × 5 min) and processed with the 1% streptavidin-biotin complex (PK-6101, "Vector Laboratories", USA) for 1 h.After 4 × 5 min washes the sections were placed in a solution of 0.06% diaminobenzidine (DAB, Sigma, USA) and 0.003% H 2 O 2 for 6 min.The sections were then washed in tap water, counterstained, dehydrated and coverslipped with the mounting medium.For Fos staining we used a conventional procedure as it is often used [33] [34] without fluorescent double-labe- ling for Fos and mature neuron marker NeuN because Fos is known as a marker of neuronal activation [35] [36], and it was shown in studies that all cells labeled for Fos also were labeled for NeuN, which supports that only neurons expressed Fos in brains during learning [37].

Data Analysis
Images of the hippocampal slices were digitized at 20× magnification under Olympus BX-50 microscope (Japan) by WV-CP230 camera (Panasonic, Japan) and analyzed using AnalySis 3.0 image analysis software (SiS, Germany).The number of Fos-positive cells was counted in the hippocampal subfields CA1, CA3 and the dentate gyrus.Counts were taken from 10 consecutive sections in each rat.In this study we did not intend to provide principal neuronal numbers in the areas, so we used non-stereological approach, which is considered to be biased because of the appearance probability of objects in an image due to their size, shape and orientations [38] [39].Such considerations are irrelevant for our study which allows comparing relative numbers of stained cells in the same structures of different group animals, whose brains underwent the identical procedure.Such conventional approaches are still widely used [33]- [40].Counting was performed by an investigator blind to the experimental group assignment of animals.Kruskal-Wallis ANOVA median test and Mann-Whitney rank sum test for pairwise comparisons were used to compare the numbers of Fos-positive neurons between the groups.A probability level of <0.05 was accepted as statistically significant.All statistical tests were performed in Statistica 5.0.

Results
To reveal consecutive learning stages of appetitive instrumental skill acquisition we recorded rats' behavior in the experimental chamber equipped with a lever situated on a distance from the feeder.The animals were progressively shaped across several sessions to acquire the behavior of lever-approaching (Figure 1).Behavior during the final training session was classified according to the following categories: previously learned, but recently ineffective behavior; effective lever-pressing behavior; explorative behavior.Combinations of these categories were used to distinguish "mismatch", "acquisition" and "performance" stages of the instrumental behavior (Figure 2).
Rats of "mismatch" group demonstrated ineffective (i.e.unreinforced) lever-approaching behavior and explorative behavior.Rats of "acquisition" group demonstrated ineffective lever-approaching behavior, explorative behavior and effective (i.e.reinforced) behavior of lever-pressing.Rats of "performance" group demonstrated effective lever-pressing behavior.We analyzed the number of food-cup checks and the number of leverpresses in all the groups.During the final session, animals of "mismatch" group made 136 ± 9 food-cup checks (Figure 3(a)).These animals made significantly fewer food-cup checks than "acquisition" group animals did (Mann-Whitney, z = 3.36, P < 0.01) (Figure 4(a)).However, the number of food-cup checks during the first half of the session did not differ between "mismatch" group rats (88 ± 4) and "acquisition" group rats (110 ± 11) (Mann-Whitney, z = 1.99,P = 0.05).There were no significant differences in the number of food-cup checks between "performance" group (261 ± 33) and "acquisition" group (271 ± 21) animals.Rats of "acquisition" group learned the lever-pressing behavior (Figure 3(b)).This behavior developed after a period of unreinforced lever-approaching behavior.Rats of this group had a significant increase of lever-presses during the second half of this session (59.2% ± 8.1% correct trials) as compared to the first half (23.8±3.8%)(Wilcoxon, z = 2.52, P < 0.05).The mean percentage of correct trials for the rats of this group was 45.4% ± 6.2% (Figure 4(b)).Rats of "performance" group pressed the lever extensively (76.1% ± 3.7% presses) (Figure 3(c)) and made significantly more correct trials than "acquisition" group animals did (Mann-Whitney, z = 2.90, P < 0.01).
To further investigate differences between the consecutive stages of this learning we assessed homogeneity of variances between "mismatch" group and "acquisition" group by using Levene test for equality of variances.Variances were not equal across the groups (F = 4670; df1 = 1; df2 = 20; P = 0.043).
Analysis of consecutive brain sections showed that CA1 region contained segments (200 -500 µm) that had no c-Fos-positive neurons albeit Fos-positive cells were evident in the cortex above the hippocampus area in all the brain sections (Figure 6).Such segmented c-Fos expression was found in 6 out of 7 "acquisition" group animals and only in 1 rat out of 5 in "mismatch" group.Half of the animals of "performance" group had such segmented activity in CA1 region of the hippocampus.Neither of the groups had such segmented c-Fos activity in CA3 region or the dentate gyrus.

Discussion
The results described above demonstrated that distribution of Fos-activated neurons in the hippocampus depended on the learning stage of the appetitive instrumental behavior.We modeled the earliest stage of learning in our experiments ("mismatch" group) by making previously reinforced and effective behavior non-reinforced and ineffective.Fos induction in neurons was already evident at this earliest stage of learning.The mismatch stage is usually characterized by memory retrieval, which may induce reconsolidation and extinction [41] [42].Fos expression was shown both during extinction [43] and reconsolidation [44].Retrieval of operant skill memory in our case was manifested in those types of behavior (approaching the lever and the food-cup checking) that were acquired during the intermediate stages of previous shaping.The set of neurons activated during learning and the one reactivated during memory retrieval are shown to be largely overlapping [45].Contextual modulation of neuronal activity was demonstrated after extinction of fear memory [46].All these findings suggest that during performance of previously acquired behavior task-related neurons are getting reactivated and might be recruited into newly formed groups subserving newly learned behavior.Our results described above demonstrate that "mismatch" and "acquisition" stages of learning are characterized by neuronal Fos-expression in all three hippocampal subfields in a similar manner.Thus not the new skill acquisition results in neuronal Fos induction but rather extinction, re-learning or reorganization of the previous experience.Nowadays the fact that learning is not happening on tabula rasa and based on previous memory gets more and more attention [47] [48].Reorganization of previous experience is manifested as explorative behavior which includes also ineffective behavior elements.Ineffective behavior during the first trials of learning might lead to neuronal changes, e.g.changes in "neuronal phenotype" [49] and create "the potential" for formation of the following memory [50].Neuronal gene expression changes during the first stage of learning might only prime the subsequent long-term changes and form a background for the following selection of specific neuronal groups as proposed by the selection theories of learning [4] [51].
We have demonstrated that the number of Fos-positive neurons after "mismatch" stage is similar across different individuals, unlike the situation after "acquisition" stage.This may imply that memory retrieval is a similar process among individuals given that they leaned similar behavior, but acquisition of a new memory differs due to different exploration trials or trial-and-error behavior that animals perform during acquisition stage.Skill memories of different individuals might become similar due to the process of consolidation over time, which is thought to begin shortly after learning [52] [53].Consolidation or reorganization of acquired memory was also shown to happen during sleep [54] [55].It has been demonstrated that proportion of neurons responsive to the imprinting stimulus reaches a maximum the day after training, and that sleep is necessary for this maximum to be achieved [56].Thus the process of consolidation leads to the reorganization of neuronal ensembles.Such reorganization could develop in a similar way across the animals of "mismatch" group due to the identity of their training history.Patterns of task-related neuronal activations in the cortex were shown to depend on the previous training history [12] [57].All the mentioned data and our results are in a line with suggestion that during the first stage of learning-"mismatch" stage-previously learned behaviors and previously acquired neuronal ensembles are temporary reorganized; this process was called "accommodative reconsolidation" [58].
Not all the stages of learning were characterized by clustered c-Fos induction in hippocampal neurons.Our data demonstrated that clustering was the most evident during "acquisition" stage of learning.At this stage the first correct trials could be achieved by recruitment of suitable neurons in a new neuronal group.Clustered organization is one of the general principles of brain functioning.The brain is not homogeneous in its nature.It is organized in such a way that similar cells tend to segregate together [59].This general principle allows considering the brain as a set of more or less discrete, so called, structures based on morphological characteristics of cells: cortices, nuclei, layers etc.However, none of brain structures really works as a unit and usually exhibits regionally differentiated activation or even clustered activation.Functional clusters might be found inside the structures: body representations in the primary motor cortex and the primary somatosensory cortex [60], ocular dominance columns and orientation columns in the primary visual cortex [61] and others.Clustering of functional cell types is not limited to the primary sensory areas.It has been demonstrated that hippocampal neurons are distributed in functional segments along the length of the hippocampus; moreover such segmentation appears to depend on task-related specificity of neurons [30].How, when and why such functional clusters are formed remains poorly understood.A more detailed understanding of functional clusters formation would provide important insight into general principles of brain functions.
The appearance of clustered activation of CA1 neurons mostly during "acquisition" stage might reflect the behavioral specialization of these neurons in respect to the specific lever-pressing behavior.Such data suggest that different CA1 neurons may play different roles in pattern completion (retrieval) and pattern separation (new encoding) processes similar to the dentate gyrus neurons [62].It has been shown that CA1 area of the hippocampus contains many neurons whose activity is related to performance of an instrumental appetitive skill and alcohol-acquisition skill [23].Because the number of Fos-positive neurons in CA1 area did not differ between "mismatch" and "acquisition" groups, it suggests that process of Fos-expression in some of CA1 neurons was deactivated, which resulted in the appearance of areas that did not contained Fos-positive neurons after the "acquisition" stage.It was shown that the number of neurons containing Fos protein was reduced within the first 30 minutes after the end of light exposure [63].Little is known about the processes of c-fos mRNA and Fos protein decay in neurons.However, some of the mechanisms of Fos degradation in cells in vitro have been suggested [64].It might be proposed that coordinated activation of a suitable neuronal group should lead to the active process of Fos deactivation in the neurons.

Conclusion
Thus, in this research we investigated how hippocampal neurons were getting sequentially involved into instrumental learning.We found that the overall pattern of hippocampal subfield activations was similar in all groups-"models" of sequential stages of instrumental learning, but the pattern of neuronal activations inside the CA1 area depended on the learning stage.This pattern was mostly characterized by clustering of Fos-positive neurons.

Figure 2 .
Figure 2. Behavioral patterns of animals of the three experimental groups: "mismatch" group (MG); "acquisition" group (AG); "performance" group (PG) during the final training session.

Figure 3 .
Figure 3. Frequency of different behavioral acts (food-cup checking, lever-pressing) during the final training session of three representative animals out of the three experimental groups: "mismatch" group (a); "acquisition" group (b); "performance" group (c).

Figure 4 .
Figure 4. Food-cup checking activity (a) and lever-pressing activity (b) during the final training session of animals of the three experimental groups: "mismatch" group (MG); "acquisition" group (AG); "performance" group (PG).