Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-02-06T11:26:29.715Z Has data issue: false hasContentIssue false
  • English
  • Français

Neural hybrid model of semantic object memory: Implications from event-related timing using fMRI

Published online by Cambridge University Press:  12 February 2004

MICHAEL A. KRAUT ,
VINCE CALHOUN ,
JEFFERY A. PITCOCK ,
CATHERINE CUSICK  and
JOHN HART
MICHAEL A. KRAUT
Affiliation:
Dept. of Radiology, Division of Neuroradiology, The Johns Hopkins University School of Medicine, Baltimore, MD
VINCE CALHOUN
Affiliation:
Dept. of Psychiatry, Yale University, New Haven, CT
JEFFERY A. PITCOCK
Affiliation:
Depts. of Geriatrics, Neurology, and Radiology, Donald. W. Reynolds Center on Aging, University of Arkansas for Medical Science, Little Rock, AR
CATHERINE CUSICK
Affiliation:
Dept. of Structural and Cellular Biology, Neuroscience Program, Tulane University, New Orleans, LA
JOHN HART
Affiliation:
Depts. of Geriatrics, Neurology, and Radiology, Donald. W. Reynolds Center on Aging, University of Arkansas for Medical Science, Little Rock, AR
Rights & Permissions [Opens in a new window]

Abstract

Previous studies by our group have demonstrated fMRI signal changes and synchronized gamma rhythm EEG oscillations between thalamus and cortical regions as subjects recall objects from visually presented features. Here, we extend this work by estimating the time course of fMRI signal changes in the cortical and subcortical regions found to exhibit evidence for task-related activation. Our results indicate that there are separate loci of signal changes in the thalamus (dorsomedial and pulvinar) that exhibit notable differences in times of onset, peak and return to baseline of signal changes. The signal changes in the pulvinar demonstrate the slowest transients of all the cortical and subcortical regions we examined. Evaluation of cortical regions demonstrated salient differences as well, with the signal changes in Brodmann area 6 (BA6) rising, peaking, and returning to baseline earlier than those detected in other regions. We conclude that BA6 mediates early designation or refinement of search criteria, and that the pulvinar may be involved in the binding of feature stimuli for an integrated object memory. (JINS, 2003, 9, 1031–1040.)

Keywords

Semantic memoryThalamusfMRI
Type
THEMATIC ARTICLES
Information
Journal of the International Neuropsychological Society , Volume 9 , Issue 7 , November 2003 , pp. 1031 - 1040
Copyright
© 2003 The International Neuropsychological Society

INTRODUCTION

Delineating the neural bases of semantic memory, even for single entities (objects, features of objects, categories, actions, etc.) has proved difficult, due in part to variable definitions of semantic terms, and to several different models of the functional organization of semantic memory. The emergence of new investigative techniques has aided in detecting regions associated with representations in semantic memory (Martin et al., 1996), but few studies have focused on the mechanisms by which these representations are manipulated.

Previous investigations of organizations of semantic memory for objects have provided evidence that categories (Caramazza & Shelton, 1998) and features (Chao et al., 1999; Miceli et al., 2001) are dissociable entities that are neurally encoded at some level. Considerable research has focused on the relationships between these entities of categories, objects, and features with one focus being semantic associations. Kounios et al. (2001) have outlined a theoretical framework for semantic associations that posited compositional associations (two items are associated but retain their individuality in the relationship) and noncompositional associations (the two items become fused semantically—computer and virus become computer virus) (Fodor & Pylyshyn, 1988; see Kounios et al., 2001 for detailed explanation). The noncompositional type of association can be extended to other semantic relationships. One theoretically significant association is that features that are common to an object could “fuse” or “bind” to activate a previously stored semantic object representation (Eichenbaum & Bunsey, 1995; Gray, 1999; Kraut et al., 2002a; 2002b; Singer & Gray, 1995). Kraut et al. (2002a; 2002b) developed an fMRI-based paradigm to address this relationship. They presented word pairs, and the subjects had to determine whether the combination of words elicited the recall of an object; for example, the word-pair desert and humps would typically elicit the recall of the word camel.

The cognitive operations putatively engaged by this object recall task differ from those for standard semantic association at the later stages of the semantic search (Kounios et al., 2001; Smith et al., 1974). Termination of a semantic search in typical semantic association likely occurs when the subject determines that relevant conjunction criteria for association between the two items' semantic fields have been met. In contrast, the object recall task of Kraut et al. (2002a, 2002b) differs in that the semantic search can proceed via two routes including (1) the semantic fields of potential target objects are searched until both stimulus features are detected in a single object's field(s), and/or (2) the two features, by their saliency to a specific object, the frequency of their co-occurrence, or idiosyncratic nature of the feature combination, focuses the search to a specific target object's semantic field(s). In either case, the end result for the feature pairs that fuse/bind is object recall. The fMRI signal changes detected during both trials that resulted in object recall and some that did not were located in bilateral Brodmann area 6 and the ventral temporo-occipital cortices. In previous investigations, the ventral temporo-occipital regions have been associated with visual object memory stores, while BA6 has been imputed in word generation (Crosson et al., 1999), in addition to its long-studied role in motor preparation. In only those trials that resulted in object recall, there were significant signal changes in the thalamus (Kraut et al., 2002a). Similar results were found when a picture and a word were used as the feature stimuli (Kraut et al., 2002b). For these multimodal stimuli, the thalamic activation was bilateral, which has been reported when picture and word stimuli are processed (Underwood & Whitfield, 1985). Of note, the thalamic signal changes were not determined to be related to lexical access of the target or feature names since the signal changes were not present in paralleled control tasks that required similar name generation (Kraut et al., 2002a, 2002b, 2002c). Nor was it related to general decision making, motor preparedness, or motor response as again control tasks did not consistently result in thalamic activation with these operations (Kraut et al., 2002a, 2002b, 2002c).

The thalamus is uniquely positioned in this semantic network to mediate interactions between widely separated cortical regions encoding for feature representations, via its extensive thalamo-cortical connections, role as a relay station, and serving as an alternate route for cortico–cortical interactions (e.g., via cortico–thalamo–cortical connections; Guillery, 1995; Sherman, 2001). Moreover, the thalamus has been shown in animals and humans to modulate fast (gamma) cortical EEG rhythms which have been postulated to synchronize neuronal firing that has been implicated in mediating cognitive operations (Llinas et al., 1999; Ribary et al., 1991). Synchronizing fast (gamma) rhythms had not been previously postulated as a plausible neural mechanism to mediate object recall in semantic memory until Slotnick et al. (2002) studied a patient with bilateral surgically implanted thalamic electrodes (part of a research protocol to electrically disrupt seizure propagation). Event-related thalamic local field potentials (LFPs) and scalp EEG were recorded while the patient performed the object recall and control tasks. During all tasks, a spatially widespread decrease in low-frequency rhythm power was detected. For the object recall trials only, this decrease was followed by an increase in spatially specific fast rhythm power at thalamus and occipital scalp electrodes. A power decrease in surface-EEG detectable low frequency signals has previously been shown to occur in humans during various cognitive tasks. More importantly, Slotnick et al. proposed that the spatially circumscribed fast rhythm burst, reflecting thalamically mediated (likely glutaminergic) excitatory activity may mediate feature binding during recall.

The thalamus appears to act as a subcortical focus in an integrated network for semantic memory, modulating gamma rhythms that may bind different memory representations of an object from the various sensorimotor and higher order cognitive systems into an integrated object concept. The previous studies have provided information on the locations of some of the nodes of this network subserving object recall, but there is little information on the time course of the interactions between these regions. Estimating the time course of activation for each region would provide useful insight into the inter-regional processing relationships upon which object recall is based in this formulation. To assess these factors in a noninvasive fashion in normal control subjects, we performed the semantic object recall task using event related fMRI, and evaluated the time courses of the regional signal changes.

METHODS

Research Participants

Nine normal right-handed individuals, 5 women and 4 men, between the ages of 23 and 34 (M age of 27.8 years, SD = 3.62) participated in the semantic object from features tasks. Details regarding the stimuli, the means of their delivery, fMRI signal acquisition and initial processing have been published previously (Kraut et al., 2002a, 2002b).

Stimuli and Procedures

The stimuli consisted of visually presented word pairs printed in lower case, with one word above the other. There were two types of word pairs: (1) 16 O+ pairs, where the two words describing features of an object combine to elicit an object that was not presented (e.g., the words desert and humps, which produce the object camel); and (2) 16 word pairs that do not combine to activate an object not presented, O− pairs, and are semantically unrelated (e.g., humps and alarm). To minimize stimulus-specific effects based on the words themselves (vs. the relationship between the words), the same feature words used in the O+ pairs were the stimuli in the O− pairs, being paired in these cases with a semantically unrelated word (e.g., humps and alarm).

The participants in the imaging study were pre-instructed as to the meaning of “object recall from features,” as this term pertained to our experiment, immediately prior to the scanning session. The O+ and O− feature word pairs were all admixed, pseudorandomized, and presented to the participants one at a time with the instructions: “Push the button if the words combine together to form an object not presented in the stimulus words themselves.”

These stimuli subtended approximately the central 3° of visual field. Word pairs were presented for 2.7 s, with 15 s between stimuli. Total run time for the experiment was 8 min, 15 s.

Scanning Procedures

The fMRI data were acquired in a 1.5 T MRI scanner, using a standard head coil. Functional MRI data covering the entire supratentorial brain were acquired using echo planar imaging with 15 seven-millimeter thick sections using a TR/TE=1000/50 ms, and flip angle of 70°. The field of view was 24 cm, with a 64 × 64 acquisition matrix, yielding pixels that were 3.75 × 3.75 mm in plane.

Image Analysis

As a first-step analysis, we segregated the recorded responses to each of the stimulus subtypes, in order to average and evaluate the responses separately. Thus, we analyzed the responses of the participants to the O+ stimulus pairs separately from the responses to the O− pairs in the object activation experiment.

We performed an event-related analysis for each individual using SPM99 (K. Friston, Wellcome Department of Cognitive Neurology, London, UK). Because the acquisition was 2D, nominally contemporaneous slices were acquired as much as ∼TR apart. Therefore, the slices were time-adjusted using sinc interpolation (Calhoun et al., 2000). The data were motion-corrected, spatially smoothed (6 × 6 × 8 mm Gaussian kernel), and normalized into a standardized Talairach template.

Signal changes were modeled as delta-functions located at stimulus presentation onset convolved with a canonical hemodynamic response function, and then entered into a linear model. The data were effectively band-pass filtered by temporally smoothing using a 4-s Gaussian kernel and removing harmonic frequencies up to one-half the fundamental frequency of the word-pair occurrences (15 s).

The model was fitted to each participant's data (at p < .001) and a group t-statistic image was created and thresholded at t = 1.66 (p < .05) which were then overlaid on T1 images for display. The images from the fMRI data sets were displayed on standardized templates derived from Montreal Neurologic Institute (MNI) and converted to Talairach and Tournoux (1988), after warping to the atlas was performed (Figure 1). Using the MRIcro (ver. 1.33; http:// www.psychology.nottingham.ac.uk/staff/cr1/mricro.html) program and a canonical MNI normalized single subject template, the masks for the regions of interests were drawn. The coordinates for each area were defined using the Talairach and Tournoux atlas.

Regional fMRI signal changes (group data) for feature pairs that elicited object recall. Note the signal changes in the thalamus, both dorsally and posteriorly, as well as those in the dorsomedial frontal lobes (BA6), the primary visual and the ventral temporo–occipital regions. Data thresholded at p < .05.

We constructed region of interest (ROI) masks to include all of the voxels that exceeded the p < .05 threshold from the group and then a latency calculation was performed for each ROI. The voxels within the ROI, that persisted within the group statistics map were used to calculate latency values for each region of interest using a weighted least squares (WLS) approach (Calhoun et al., 2000). Weighting was applied to the upslope (to estimate onset time), the downslope (to estimate return-to-baseline time), and both (to estimate the peak time). Results for the mean latencies ± standard error of mean (SEM) in seconds (s) were calculated for each of the ROIs of interest. Then paired t tests between the mean latencies were performed. Mean event-average time courses from each of the ROIs were calculated (see Figure 4).

RESULTS

There are significant signal changes with object recall noted in the relevant regions of BA6, dorsomedial nucleus, pulvinar nucleus, primary visual cortex, and the ventral temporo-occipital regions in the group data (p < .05) using SPM (see Figure 1). There were no significant signal changes in the thalamus for feature pairs that did not elicit an object (see Figure 2). What is notable are the distinctly different, and significant, signal changes in the pulvinar and dorsomedial nucleus in the thalamus when object recall occurs (see Figure 3).

Signal changes detected for feature pairs that did not result in object recall. Signal changes in the thalamus are not detectable at this statistical threshold.

Surface-rendered, cut-away image of the brain showing, from a more advantageous perspective, the signal changes in two different regions of the thalamus: anteriorly in the dorsomedial thalamic nucleus (designated by left arrow), and posteriorly in the pulvinar (designated by right arrow). (Note signal in BA6 and V1/early extrastriate cortex.)

Time courses of the signal changes (group data) in each of the designated regions of interest. Signal intensity in arbitrary units.

Using the previously defined ROIs, we calculated event-averaged time courses of blood flow changes for the following significant regions: BA6 (pre-SMA), dorsomedial nucleus, pulvinar nucleus, ventral temporo–occipital region (the “what” system for object recognition), and primary visual cortex (V1). These are plotted as percent MR signal change for the group versus time in seconds (see Figure 4). It appears at first glance that most regions initiate onset at the same relative time, possibly indicating parallel processing between the regions, at this level of sensitivity and using blood flow as a measure. However, it is clear that the pulvinar has a later time to onset, time to peak, and peak-onset times than the other regions and that BA6 is earlier than all other regions in these same measures (see Figure 5 for specific comparison of these regions).

Waveforms (group data) from the BA6 and pulvinar regions shown separately, demonstrating the differences in time course between these two loci of activation.

To confirm this, we next calculated the onset time, return-to-baseline time, and peak time for each region across all 9 subjects (see Table 1). Each subject's mean time for a given region was compared to another region for onset time, return-to-baseline time, and peak time. The results of that analysis showed that the pulvinar was significantly different (two tailed paired t test, ps < .03 for all three of these times) from BA6, dorsomedial nucleus (except for time to onset which is p = .058), and ventral temporo–occipital region (except for time to onset which is p = .083). There were no significant differences for any of the times between the pulvinar and primary visual cortex (ps > .18). BA6 differed significantly (two tailed paired t test, ps < .04) from all regions for all three times, except for the time to onset for primary visual cortex (p = .058) and the ventral temporo–occipital region (p = .064).

Means and standard deviations (s) of the upslope, peak, and downslope of the hemodynamic response curves for the anatomic regions of interest

DISCUSSION

The data presented here complement those of previous functional imaging as well as electrophysiologic studies, in that few if any fMRI studies in the past have focused on the time courses of the signal changes, and most electrophysiologic studies do not afford the whole-brain coverage included in our data. These findings extend the previous results (Kraut et al., 2002a, 2002b; Slotnick et al., 2002) in two important ways. First, they demonstrate two distinct loci of thalamic signal change, one in the dorsomedial region, and the other more posteriorly, in the pulvinar. Second, analysis of the regionally specific signal change waveform morphology demonstrates distinct patterns that are likely, at least in part, reflective of different time courses of neural activity in different brain regions.

The data show that the signal changes centered in BA6 have a steep climb toward an early peak, and an early drop-off; this latter contention is strengthened by the relative concavity of the falling edge of the signal changes here. The ventral temporo-occipital regions exhibit a later increase in signal, and the waveform is prolonged compared to that of BA6 (Figure 4). The signal changes detected in primary visual cortex and the dorsomedial thalamus demonstrate almost identical rising phases, with the thalamic signal changes falling off slightly more slowly. Interestingly, the signal changes in the primary visual region both rise and fall more slowly than those in the non-primary, ventral temporo–occipital region. Finally, the signal changes in the pulvinar exhibit distinctly slower signal increase and decrease phases than the other regions in which we evaluated the signal change dynamics.

Our data help inform further development of a neural-based model of semantic object recall referred to as the Neural Hybrid Model (Ver. 1.0; Hart et al., 2002). In the framework of this model, we have proposed that there are cortical and subcortical regions that likely encode for representations associated with objects in sensorimotor and higher order cognitive systems (e.g., lexical–semantic). Both feature- and category-based representations have been proposed for several sensorimotor/cognitive domains. While there are variable proposals as to the functional-anatomic organization of these representations, there is little disagreement that these types of representations exist in some cognitive domains (but see Devlin et al., 2002; Moore & Price, 1999). For example, objects that intuitively possess a manipulable feature have been associated with signal changes in premotor areas on functional imaging studies, independent of their category designation (Kraut et al., 2002c). Alternatively, there have been several examples in lesion studies of at least a categorical organization to the lexical semantic system (Caramazza & Shelton, 1998) with a suggestion of a featural organization as well (Hart & Gordon, 1992; Haxby et al., 2001; Miceli et al., 2001). Irrespective of the specific functional anatomic assignments, there have been suggestions of some aspects of these representations for objects in the tactile, somatomotor, lexical–semantic, and visual domains, with the visual representations appearing to be the most salient given that the target stimuli are objects.

Another notion that has emerged from both the lesion and functional imaging findings has been the suggestion that there are focal, circumscribed brain regions that regularly are activated in functional imaging studies or disrupted in lesion studies for specific semantic processes or operations (e.g., category membership, feature identification) across object types. For example, providing a verb/action for a noun/object has been associated in multiple functional imaging studies (including PET and fMRI) with a focal circumscribed region in the left dorsolateral prefrontal cortex (Binder et al., 1996; Fiez, 1997), with further investigations suggesting that the specific role the region plays is in semantic selection (Thompson-Schill et al., 1997). Data gathered in patients with focal lesions (Kemmerer & Tranel, 2000; Thompson-Schill et al., 1998) corroborated the critical role that the left inferior frontal gyrus plays in verb generation, as have the data from cortical stimulation studies (Ojemann et al., 2002). In contrast, the temporoparietal junction region has been shown to mediate object-based category, property, definition and synonym functions during stimulus presentation from multiple modalities (auditory-word, visual-word, and pictures; Hart & Gordon, 1990) and with knowledge of associations between concepts and attribute judgment for these concepts during functional imaging studies (Vandenberghe et al., 1996). Thus far, although the regions detected have been associated with multimodal processing, it appears theoretically plausible that unimodal regions may be isolated with more sensitive investigative techniques.

The mechanisms by which these brain regions function in concert to integrate mental representations are not understood. The present study and previous related experiments using fMRI (Kraut et al., 2002a, 2002b) and electrophysiologic measures (Slotnick et al., 2002) have directly explored semantic object recall from features to assess plausible neural mechanisms that mediate such recall across multiple memory representations. The study by Slotnick et al. is one of the few that have explored rigorously the relationships between cortical and subcortical fMRI signal changes and their electrophysiological concomitants, and emphasizes the specific changes in regional neural ensemble activity that occur with successful object recall. Slotnick et al. showed that during all trials of the semantic object recall task there is a global decrease in alpha band EEG power which was followed by an increase in spatially specific gamma band EEG power in the thalamus and occipital scalp electrodes for only those trials where feature binding resulted in semantic object recall.

In cats, the analogs of the low frequency electrical rhythms detected by Slotnick et al. appear to be controlled by means of spatially widespread, inhibitory (i.e., GABA-ergic) projections from the thalamic reticular nucleus (RE) to thalamocortical (TC) cells in other thalamic nuclei and mediate cyclic activity by rebound excitation. Spatially widespread connections from cortex to RE have also been shown to modulate low-frequency synchronization. In contrast, high-frequency thalamocortical rhythms in the cat have been proposed to reflect spatially specific, monosynaptic excitatory (e.g., glutamatergic) cortico–thalamo–cortical pathways that do not necessarily engage RE.

Based upon these findings and the framework of this semantic object recall task, the low-frequency synchronization may functionally represent a tonic state of cortical inhibition, and thus the widespread decrease in low-frequency EEG power could reflect cortical disinhibition. The spatially specific fast rhythm burst (gamma), may mediate or at least be related to feature binding during recall via the synchronization of neural regions associated with feature representations of the object to be recalled (Joliot et al., 1994; Klimesch, 1996; Singer, 1993; Singer & Gray, 1995; Steriade, 2000).

Within this framework, not every sensorimotor/cognitive domain will necessarily have featural information about an object, nor will all features have the same strength of association to a given object. The varying strength of association for each cognitive domain's featural component of semantic memory for a given object is likely based on the individual's encoding process and experience with that object over time, thus allowing for the individual variability noted. Obversely, common distinguishing features across groups of objects likely allow for the observed regularities in object recall across individuals (e.g., all animals have a strong visual–perceptual loading for features). The network of feature representations for each object thus will vary by object and reflect varying weights and saliency of each feature to that object's overall representation. In addition, how that network is accessed can vary, and this access may be reflected in the strength of activation of all or part of the network. It may be that some features or set/domain of features may have stronger links to others and function as “pointers” in the network. For example, referent and associated connections for a hammer (hammer, tool, nail, repair, etc.) may serve as pointers to some of the sensorimotor features and trigger their activation, lower their activation threshold, or activate the entire feature network for hammer more effectively than the tactile feature of a hammer does. The extent or presence of such triggering may differ, depending upon the specific item or task.

Our data show that the dynamics of both the cortical and subcortical task related signal changes are complex, likely reflecting recurring inter-regional interactions. Taken in conjunction with the electrophysiology results, these findings suggest that the pulvinar region engages later in the process and is thus likely the mediator/modulator of the selective gamma burst rhythm that we propose binds or unites the features in this instance of object recall. The prolonged fMRI response suggests that the pulvinar is more likely engaged after the semantic fields of each feature have been searched and a common object is detected. From the previous studies, the gamma synchronizing rhythm is most readily detected when an actual object is recalled and thus is likely not the principal mechanism for the search for the object. However, whether the gamma rhythm heralds the detection of the common object or unites the features to represent the object is unclear and cannot be determined from the present studies. Further elucidation will require investigation with electrophysiological markers with more precise temporal resolution and optimally with electrode tip placement closer to or in the pulvinar nucleus itself.

The multiple reciprocal connections between the pulvinar and numerous cortical association cortices provide many potential pathways by which this nucleus could mediate integration of features to form object representations. For example, a role for pulvinar in binding of shape and color has been proposed (Ward et al., 2002). The region of the pulvinar most strongly activated in these experiments appears to be medial pulvinar. In macaques, the medial pulvinar connects particularly widely—including prefrontal cortex, auditory-related and multimodal parts of the superior temporal gyrus, and vision-related posterior parietal cortex. Medial portions of the pulvinar connect with inferotemporal visual cortex as well as somatosensory cortex of the insula and with the amygdala (for review, see Gutierrez et al., 2000). Thus, the V1/occipitotemporal–pulvinar connections as well as medial pulvinar connections with wide array of cortical areas (superior temporal gyrus, inferior parietal lobule) may well be involved in a semantic object recall network.

Given its numerous anatomic connections, the dorsomedial nucleus (DM) activity more likely is involved as part of a search or object generation pathway in conjunction with BA6, to which it has direct connections (Ilinsky et al., 1985; Inase et al., 1996), setting or monitoring criteria that an object would meet to fulfill the search. The dorsomedial nucleus may interact with or trigger the pulvinar via corticothalamic feedback connections, since sectors of prefrontal cortex interconnect with both DM and the pulvinar (Gutierrez et al., 2000; Preuss & Goldman-Rakic, 1987; Yeterian & Pandya, 1988). Finally, DM may directly activate other specific prefrontal portions of the network. This is relevant given that prefrontal cortex has been shown to be involved in spatial and object-centered working memory (Wilson et al., 1993), as well as language functions.

There are several possibilities as regards the role of the pre-SMA region (BA6). Crosson et al. (1999) showed that this region is active during word generation. Kraut et al. (2002a, 2002b) suggested that this activation is likely associated with a search strategy, which may be semantic or language related, while others have suggested that its function is in preparedness to make a motor response (Thompson-Schill et al., 1997). However, the absence of its activation in very similar tasks requiring a motor response suggests either (1) semantic or generic search strategy; or (2) generation of an object concept framework from featural input. Nadeau and Crosson (1997) have suggested that the frontal lobe, via the inferior thalamic peduncle, reticular nucleus, and centromedian complex selectively engages portions of a network of regions necessary to represent an object or concept. We would suggest that BA 6, along with its dorsomedial nucleus connections, could be playing a role in this process.

The somewhat prolonged time course of activation for primary visual cortex is intriguing in that this likely would have been one of the initial regions activated in a visually driven task. We would thus have expected to see the signal in primary visual cortex increase early, and perhaps decrease early, with completion of visual evaluation of the stimuli. The delay in primary visual cortex response with respect to that of higher-order regions, suggests that this region is mediating some mental visualization of the target object stimulus or features that are contributing to the integrated object concept in memory during task performance (subjectively, subjects do report seeing internal images of the objects recalled). This is further supported indirectly by there being no significant differences in time course between primary visual cortex and pulvinar, given pulvinar's putative role in feature binding during integrated semantic object recall.

It is clear that when evaluating the time courses of fMRI signal changes, one must take into account the considerable likelihood that regionally disparate hemodynamics will contribute to differences in the morphologic features of the waveforms. Nonetheless, the data presented here provide motivation for further studies using electro- or magneto-encephalographic techniques, as well as further functional imaging studies in which stimulus and task parameters are varied in an effort to accentuate differences in the timing of regional activation. We also acknowledge that some aspects of semantic object recall presented here have been accounted for in other cognitive frameworks, particularly feature and category representation (Devlin et al., 2002; Gainotti, 2000; Moore & Price, 1999; Tyler & Moss, 2001; Warrington & McCarthy, 1994). Overall, however, we believe that coordinated processing of category-level information is fundamental to object recall and likely integrated within the proposed mechanism. Irrespective of the hypotheses for functional organization of some components of the object memories (e.g., categories, features), the present model is one possible set of mechanisms by which semantic object recall could be instantiated, and could be applied to several of the models of knowledge organization that have not specified comparable mechanisms. We do not posit that it operates exclusively to accomplish such object recall; however, we also believe schema analogous to that presented here could well exist in other cognitive domains. We are currently carrying out experiments to test these hypotheses.

ACKNOWLEDGMENTS

We would like to thank Sarah Kremen, Jessica B. Segal, Lauren Moo, and Scott Slotnick for their invaluable contributions to this project. We would like to also acknowledge Dr. D. T'ib for his critical comments on previous versions of this manuscript and express sincere wishes for a complete recovery.

References

REFERENCES

Binder, J.R., Swanson, S.J., Hammeke, T., Morris, G., Mueller, W., Fischer, M., Benbadis, S., Frost, J., Rao, S., & Haughton, V. (1996). Determination of language dominance using functional MRI: A comparison with the Wada test. Neurology, 46, 978–984.CrossRefGoogle Scholar
Calhoun, V., Adali, T., Kraut, M., & Pearlson, G. (2000). A weighted least-squares algorithm for estimation and visualization of relative latencies in event-related functional MRI. Magnetic Resonance in Medicine, 44, 947–954.3.0.CO;2-5>CrossRefGoogle Scholar
Caramazza, A. & Shelton, J.R. (1998). Domain-specific knowledge systems in the brain: The animate-inanimate distinction. Journal of Cognitive Neuroscience, 10, 1–34.Google Scholar
Chao, L.L., Haxby, J., & Martin, A. (1999). Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nature Neuroscience, 2, 913–919.CrossRefGoogle Scholar
Crosson, B., Sadek, J.R., Bobholz, J.A., Gokcay, D., Mohr, C., Leonard, C., Maron, L., Auerbach, E., Browd, S., Freeman, A., & Briggs, R. (1999). Activity in the paracingulate and cingulate sulci during word generation: An fMRI study of functional anatomy. Cerebral Cortex, 9, 307–316.CrossRefGoogle Scholar
Devlin, J.T., Russell, R.P., Davis, M.H., Price, C., Moss, H., Fadili, M., & Tyler, L. (2002). Is there an anatomical basis for category-specificity? Semantic memory studies in PET and fMRI. Neuropsychologia, 40, 54–75.CrossRefGoogle Scholar
Eichenbaum, H. & Bunsey, M. (1995). On the binding of associations in memory: Clues from studies on the role of hippocampal region in paired-associate learning. Current Directions in Psychological Science, 4, 19–23.Google Scholar
Fiez, J.A. (1997). Phonology, semantics, and the role of the left inferior prefrontal cortex. Human Brain Mapping, 5, 79–83.Google Scholar
Fodor, J.A. & Pylyshyn, Z.W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3–71.Google Scholar
Gainotti, G. (2000). What the locus of brain lesion tells us about the nature of the cognitive defect underlying category-specific disorders: a review. Cortex, 36, 539–559.Google Scholar
Gray, C.M. (1999). The temporal correlation hypothesis of visual feature integration: Still alive and well. Neuron, 24, 31–47.CrossRefGoogle Scholar
Guillery, R.W. (1995). Anatomical evidence concerning the role of the thalamus in corticocortical communication: a brief review. Journal of Anatomy, 187, 583–592.Google Scholar
Gutierrez, C., Cola, M.G., Seltzer, B., & Cusick, C. (2000). Neurochemical and connectional organization of the dorsal pulvinar complex in monkeys. Journal of Comparative Neurology, 419, 61–86.3.0.CO;2-I>CrossRefGoogle Scholar
Hart, J. & Gordon, B. (1990). Delineation of single-word semantic comprehension deficits in aphasia, with anatomical correlation. Annals of Neurology, 27, 226–231.Google Scholar
Hart, J. & Gordon, B. (1992). Neural subsystems for object knowledge. Nature, 359, 60–64.Google Scholar
Hart, J., Moo, L., Segal, J.B., Adkins, E., & Kraut, M. (2002). Neural substrates of semantics. In A. Hillis (Ed.), Handbook of language disorders, (pp. 207–227). Philadelphia: Psychology Press.
Haxby, J., Gobbini, M.I., Furey, M.L., Ishai, A., & Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293, 2425–2430.Google Scholar
Ilinsky, I.A., Jouandet, M.L., & Goldman-Rakic, P.S. (1985). Organization of the nigrothalamocortical system in the rhesus monkey. Journal of Comparative Neurology, 236, 315–330.Google Scholar
Inase, M., Tokuno, H., Nambu, A., Akazawa, T., & Takada, M. (1996). Origin of thalamocortical projections to the presupplementary motor area (pre-SMA) in the macaque. Neuroscience Research, 25, 217–227.CrossRefGoogle Scholar
Joliot, M., Ribary, U., & Llinas, R. (1994). Human oscillatory brain activity near 40 Hz coexists with cognitive temporal binding. Proceedings of the National Academy of Sciences U.S.A., 91, 11748–11751.Google Scholar
Kemmerer, D. & Tranel, D. (2000). Verb retrieval in brain-damaged subjects: 1. Analysis of stimulus, lexical, and conceptual factors. Brain and Language, 73, 347–392.Google Scholar
Klimesch, W. (1996). Memory processes, brain oscillations and EEG synchronization. International Journal of Psychophysiology, 24, 61–100.Google Scholar
Kounios, J., Smith, R.W., Yang, W., Bachman, P., & D'Esposito, N. (2001). Cognitive association formation in human memory revealed by spatiotemporal brain imaging. Neuron, 29, 297–306.Google Scholar
Kraut, M.A., Kremen, S., Segal, J.B., Calhoun, V., Moo, L., & Hart, J. (2002a). Object activation from features in the semantic system. Journal of Cognitive Neuroscience, 14, 24–36.Google Scholar
Kraut, M.A., Kremen, S., Moo, L.R., Segal, J., Calhoun, V., & Hart, J. (2002b). Object activation in semantic memory from visual multimodal feature input. Journal of Cognitive Neuroscience, 14, 37–47.Google Scholar
Kraut, M., Moo, L., Segal, J., & Hart, J. (2002c). Neural activation during an explicit categorization task: Category- or feature-specific effects? Brain Research Cognitive Brain Research, 13, 213–220.Google Scholar
Llinas, R.R., Ribary, U., Jeanmonod, D., Kronberg, E., & Mitra, P. (1999). Thalamocortical dysrhythmia: A neurological and neuropsychiatric syndrome characterized by magnetoencephalography. Proceedings of the National Academy of Sciences USA, 96, 15222–15227.CrossRefGoogle Scholar
Martin, A., Wiggs, C.L., Ungerleider, L.G., & Haxby, J.V. (1996). Neural correlates of category-specific knowledge. Nature, 379, 649–652.Google Scholar
Miceli, G., Fouch, E., Capasso, R., Shelton, J., Tomaiuolo, F., & Caramazza, A. (2001). The dissociation of color from form and function knowledge. Nature Neuroscience, 4, 662–667.Google Scholar
Moore, C.J. & Price, C.J. (1999). A functional neuroimaging study of the variables that generate category-specific object processing differences. Brain, 122, 943–962.CrossRefGoogle Scholar
Nadeau, S.E. & Crosson, B. (1997). Subcortical aphasia. Brain and Language, 58, 355–402.Google Scholar
Ojemann, J., Ojemann, G., & Lettich, E. (2002). Cortical stimulation mapping of language cortex by using a verb generation task: Effects of learning and comparison to mapping based on object naming. Journal of Neurosurgery, 97, 33–38.CrossRefGoogle Scholar
Preuss, T.M. & Goldman-Rakic, P.S. (1987). Crossed corticothalamic and thalamocortical connections of macaque prefrontal cortex. Journal of Comparative Neurology, 257, 269–281.Google Scholar
Ribary, U., Ioannides, A.A., Singh, K.D., Asno, R., Bolton, J., Lado, F., Mogilner, A., Llinas, R. (1991). Magnetic field tomography of coherent thalamocortical 40-Hz oscillations in humans. Proceedings of the National Academy of Sciences USA, 88, 11037–11041.Google Scholar
Sherman, S.M. (2001). Thalamic relay functions. Progress in Brain Research, 134, 51–69.Google Scholar
Singer, W. (1993). Synchronization of cortical activity and its putative role in information processing and learning. Annual Review of Physiology, 55, 349–374.Google Scholar
Singer, W. & Gray, C.M. (1995). Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience, 18, 555–586.Google Scholar
Slotnick, S., Moo, L., Kraut, M., Lesser, R., & Hart, J. (2002). Thalamic modulation of cortical rhythms during semantic memory recall in humans. Proceedings of the National Academy of Sciences USA, 99, 6440–6443.CrossRefGoogle Scholar
Smith, E.E., Shoben, E.J., & Rips, L.J. (1974). Structure and process in semantic memory: A featural model for semantic decisions. Psychological Review, 81, 214–241.Google Scholar
Steriade, M. (2000). Corticothalamic resonance, states of vigilance and mentation. Neuroscience, 101, 243–276.Google Scholar
Talairach, J. & Tournoux, P. (1988). Co-planar sterotaxic atlas of the human brain: 3-dimensional proportional system: An approach to medical cerebral imaging. New York: Thieme.
Thompson-Schill, S., D'Esposito, M., Aguirre, G., & Farah, M. (1997). Role of left inferior prefrontal cortex in retrieval of semantic knowledge: A reevaluation. Proceedings of the National Academy of Sciences USA, 94, 14792–14797.CrossRefGoogle Scholar
Thompson-Schill, S., Swick, D., Farah, M., D'Esposito, M., Kan, I., & Knight, R. (1998). Verb generation in patients with focal frontal lesions: A neuropsychological test of neuroimaging findings. Proceedings of the National Academy of Sciences USA, 95, 15855–15860.Google Scholar
Tyler, L. & Moss, H. (2001). Towards a distributed account of conceptual knowledge. Trends in Cognitive Science, 5, 244–252.CrossRefGoogle Scholar
Underwood, G. & Whitfield, A. (1985). Right hemisphere interactions in picture–word rocessing. Brain and Cognition, 4, 273–286.Google Scholar
Vandenberghe, R., Price, C., Wise, R., Josephs, O., & Frackowiak, R. (1996). Functional anatomy of a common semantic system for words and pictures. Nature, 383, 254–256.Google Scholar
Ward, R., Danziger, S., Owen, V., & Rafal, R. (2002). Deficits in spatial coding and feature binding following damage to spatiotopic maps in the human pulvinar. Nature Neuroscience, 5, 99–100.CrossRefGoogle Scholar
Warrington, E. & McCarthy, R.A. (1994). Multiple meaning systems in the brain: A case for visual semantics. Neuropsychología, 32, 1465–1473.Google Scholar
Wilson, F.A., Scalaidhe, S.P., & Goldman-Rakic, P.S. (1993). Dissociation of object and spatial processing domains in primate prefrontal cortex. Science, 260, 1955–1958.CrossRefGoogle Scholar
Yeterian, E.H. & Pandya, D.N. (1988). Corticothalamic connections of paralimbic regions in the rhesus monkey. Journal of Comparative Neurology, 269, 130–146.Google Scholar
Figure 0

Regional fMRI signal changes (group data) for feature pairs that elicited object recall. Note the signal changes in the thalamus, both dorsally and posteriorly, as well as those in the dorsomedial frontal lobes (BA6), the primary visual and the ventral temporo–occipital regions. Data thresholded at p < .05.

Figure 1

Signal changes detected for feature pairs that did not result in object recall. Signal changes in the thalamus are not detectable at this statistical threshold.

Figure 2

Surface-rendered, cut-away image of the brain showing, from a more advantageous perspective, the signal changes in two different regions of the thalamus: anteriorly in the dorsomedial thalamic nucleus (designated by left arrow), and posteriorly in the pulvinar (designated by right arrow). (Note signal in BA6 and V1/early extrastriate cortex.)

Figure 3

Time courses of the signal changes (group data) in each of the designated regions of interest. Signal intensity in arbitrary units.

Figure 4

Waveforms (group data) from the BA6 and pulvinar regions shown separately, demonstrating the differences in time course between these two loci of activation.

Figure 5

Means and standard deviations (s) of the upslope, peak, and downslope of the hemodynamic response curves for the anatomic regions of interest