Perception – Gain Control



Long Term Memory – Reinforcement Learning

|Task Name |Description |Cognitive Construct Validity |Neural Construct Validity |Reliability |Psychometric Characteristics |Animal Model |Stage of Research |

|Probabilistic |Subjects select one stimulus from an array of |Acquisition of a multiple choice visual |Interactions between declarative |As this is a "learning" |Practice effects are unknown, but are |Analogous tests for mice, |There is evidence that this |

|Reversal Learning |concurrently presented stimuli (preferably 3); |discrimination is believed to reflect |systems (hippocampal and |task, test-retest |the subject of active investigation. |rats and monkeys exist. |specific task elicits deficits|

| |typically, more than one set of discriminanda are |implicit learning processes. Optimal |prefrontal) and implicit systems |reliability is unavailable.|Depending upon how the test is | |in schizophrenia. |

| |presented, in a randomized order across trials. Each |performance of a visual discrimination is |(striatum) are predicted across | |conducted, there can be ceiling |(Lee et al., 2007) | |

| |choice is followed by positive or negative feedback |thought to reflect rule maintenance. |initial learning of a | |effects for discrimination learning. | |We need to assess psychometric|

| |(correct choice or incorrect choice). The relationship |Reversal of a learned visual discrimination|discrimination, such that | |The use of multiple discriminada sets,|(Boulougouris, Dalley, & |characteristics such as |

| |between the stimulus presented and the feedback given |is thought to measure cognitive control |increasing activation in the | |at least 3 stimuli in each set and |Robbins, 2007) |test-retest reliability, |

| |can be deterministic or probabilistic. Subjects must,|over pre-potent responding (response |striatum, as a function of trials| |probabilistic feedback can avoid this.|(Chudasama & Robbins, 2006)|practice effects, and |

| |based upon feedback alone, learn which cue in each |inhibition). |performed, occurs along with | | | |ceiling/floor effects for this|

| |discriminanda set are associated with positive feedback | |learning. The ventrolateral | | | |task. |

| |and then maintain optimized performance of the rule | |and ventromedial frontal cortex | | | | |

| |(rule maintenance). Once the subject meets pre-set | |are particularly involved in the | | | |We need to study whether or |

| |performance criteria, feedback is adjusted without | |effective inhibition of responses| | | |not performance on this task |

| |warning. In other words, the rules are "reversed" in one| |at reversal. | | | |changes in response to |

| |or more of the discriminanda sets. Subjects must | | | | | |psychological or |

| |update their behavior based upon the change in rules. | |(Dias, Robbins, & Roberts, 1996, | | | |pharmacological intervention |

| |Optimal reversal performance is often considered to | |1997) | | | | |

| |involve cognitive control over pre-potent responding. | |(Fellows & Farah, 2005) | | | |BUT: (Cools, Altamirano, & |

| | | |(Fellows & Farah, 2003) | | | |D'Esposito, 2006; Cools, |

| |(Waltz & Gold, 2007) | | | | | |Lewis, Clark, Barker, & |

| |(Lee, Groman, London, & Jentsch, 2007) | | | | | |Robbins, 2007) |

| |(Frank & Claus, 2006) | | | | | | |

| | | | | | | | |

| |MANUSCRIPTS ON THE WEBSITE: | | | | | | |

| | | | | | | | |

| |Cools, R., Altamirano, L., & D'Esposito, M. (2006). | | | | | | |

| |Reversal learning in parkinson's disease depends on | | | | | | |

| |medication status and outcome valence. Neuropsychologia,| | | | | | |

| |44(10), 1663-1673. | | | | | | |

| | | | | | | | |

| |Waltz, J. A., & Gold, J. M. (2007). Probabilistic | | | | | | |

| |reversal learning impairments in schizophrenia: further | | | | | | |

| |evidence of orbitofrontal dysfunction. Schizophrenia | | | | | | |

| |Research, 93(1-3), 296-303. | | | | | | |

| | | | | | | | |

|Pizzagalli Reward |Recently, we described a probabilistic reward task based|In prior studies in non-clinical samples, |In healthy controls, we have also|Information about |Practice effects might emerged with |A federal grant is |This specific task needs to be|

|Task |on a differential reinforcement schedule that allowed us|subjects reporting elevated depressive |shown that response bias was |test-retest reliability can|repeated administration. This can be |currently under review to |studied in individuals with |

| |to objectively assess participants’ propensity to |symptoms showed reduced responsiveness to |reduced under an acute stress |be found in Pizzagalli et |circumvented by asking participants to|develop a rodent model of |schizophrenia. |

| |modulate behavior as a function of reward history. In |the more frequently rewarded stimulus |condition (Bogdan & Pizzagalli, |al. (2005). After an |respond to different stimuli in |our probabilistic reward | |

| |this 15-20-min computerized task, participants are |(Pizzagalli et al., 2005). Moreover, reward|2006) and by a pharmacological |average of 38 days, the |different sessions. For an example, |task. |Data already exists on |

| |confronted with a choice between two responses that are |responsiveness negatively correlated with |manipulation affecting the |test-retest correlation |see: Bogdan & Pizzagalli, 2006 | |psychometric characteristics |

| |linked to different probabilities of reward. Due to this|self-reported anhedonic symptoms (Bogdan |dopaminergic system (Pizzagalli |between bias scores was | | |of this task, such as |

| |probabilistic nature, participants cannot infer which |and Pizzagalli, 2006; Pizzagalli et al., |et al., 2008) Conversely, |0.57. | | |test-retest reliability, |

| |stimulus is more advantageous based on the outcome of a |2005), and predicted these symptoms one |response bias was increased when | | | |practice effects, |

| |single trial and so need to integrate reinforcement |month later (Pizzagalli et al., 2005). |a nicotine patch was applied to | | | |ceiling/floor effects. |

| |history over time in order to optimize their choices. |These findings have recently been extended |healthy non-smokers (Barr et al.,| | | | |

| |Behavioral performance is analyzed using |to two clinical groups: euthymic patients |2007). Finally, ongoing ERP and | | | |There is evidence that |

| |signal-detection theory by calculating both response |with bipolar disorder (Pizzagalli et al., |fMRI studies have provided | | | |performance on this task can |

| |bias toward the more frequently rewarded stimulus and |2008) and unmedicated subjects with |preliminary evidence that | | | |improve in response to |

| |overall discriminability (in addition to reaction time |unipolar depression (Pizzagalli et al., |response bias is associated with | | | |psychological or |

| |and hit rates). Unix scripts are available for automatic|under review) were characterized by reduced|feedback-related negativity (FRN)| | | |pharmacological interventions.|

| |data quality check, outlier detection, and computation |reward learning and reward responsiveness, |and activation in the dorsal | | | | |

| |of behavioral variables. |respectively. |anterior cingulate and basal | | | | |

| | | |ganglia. | | | | |

| |(Pizzagalli, Jahn, & O'Shea, 2005) |(Pizzagalli, Jahn, & O'Shea, 2005) | | | | | |

| | |(Pizzagalli, Bogdan, Ratner, & Jahn, 2007) |(Pizzagalli, Bogdan, Ratner, & | | | | |

| |MANUSCRIPTS ON THE WEBSITE: |(Pizzagalli, Goetz, Ostacher, Iosifescu, & |Jahn, 2007) | | | | |

| | |Perlis, 2008) |(Frank, Seeberger, & O'Reilly R, | | | | |

| |Pizzagalli, D. A., Goetz, E., Ostacher, M., Iosifescu, |(Bogdan & Pizzagalli, 2006) |2004) | | | | |

| |D. V., & Perlis, R. H. (2008). Euthymic Patients with | |(Bogdan & Pizzagalli, 2006) | | | | |

| |Bipolar Disorder Show Decreased Reward Learning in a | |(Barr, Pizzagalli, Culhane, Goff,| | | | |

| |Probabilistic Reward Task. Biol Psychiatry. | |& Evins, 2007) | | | | |

| | | | | | | | |

| |Pizzagalli, D. A., Jahn, A. L., & O'Shea, J. P. (2005). | | | | | | |

| |Toward an objective characterization of an anhedonic | | | | | | |

| |phenotype: a signal-detection approach. Biological | | | | | | |

| |Psychiatry, 57(4), 319-327. | | | | | | |

|Probabilisitic |The probabilistic selection (PS) task measures |Performance in this task is defined by the |Probabilistic positive and |In healthy populations, |Practice effects have been assessed in|Currently being used for |There is evidence that this |

|Selection Task |participants' ability to learn from positive and |ability to choose the probabilistically |negative feedback learning are |there is not a very large |Frank & O'Reilly (2006). On average |rat testing in |specific task elicits deficits|

| |negative feedback, both on a trial-to-trial basis, and |most optimal stimulus. Relative positive |sensitive to dopaminergic |test-retest reliability |participants are faster to learn the |collaboration with Claudio |in schizophrenia. |

| |in integrating reinforcement probabilities over many |and negative feedback learning is similarly|manipulation. Increases in |(some participants can show|task after multiple sessions, but this|DaCunha's lab in Brazil. | |

| |trials. It also probes performance in a novel 'test' |modulated by dopamine manipulation in this |dopaminergic stimulation, likely |a positive learning bias in|practice does affect relative positive| |Data already exists on |

| |phase which permits evaluation of whether one implicitly|task and others that are meant to measure |in the striatum, lead to better |one session and a negative |versus negative feedback learning. | |psychometric characteristics |

| |learned more from the positive or negative outcomes of |similar constructs using different stimuli,|positive learning but cause |bias in another). On the | | |of this task, such as |

| |their decisions. This "Go" or "NoGo" learning bias is |motor responses, and task rules. |impairments in negative feedback |other hand, there are |(Frank & O'Reilly R, 2006) | |test-retest reliability, |

| |very sensitive to dopaminergic manipulation and | |learning. Dopamine depletions, as|robust genetic effects in | | |practice effects, |

| |dopamine-related genetics. | |in Parkinson's disease and older |this task, with large | | |ceiling/floor effects. |

| |(Frank, Seeberger, & O'Reilly R, 2004) | |seniors (greater than 70 years of|effect sizes, so on average| | | |

| | | |age) are associated with |the task must measure a | | |There is evidence that |

| | | |relatively better negative |reliable trait | | |performance on this task can |

| | | |feedback learning. Genes that |characteristic. Thus | | |improve in response to |

| |(Waltz et al., 2007) | |control dopamine function in the |individual variability is | | |psychological or |

| |(Frank, Moustafa, Haughey, Curran, & Hutchison, 2007) | |striatum are predictive of |likely due to either (i) | | |pharmacological interventions.|

| | | |probabilistic positive and |state-dependent changes; or| | | |

| |MANUSCRIPTS ON THE WEBSITE: | |negative learning, whereas genes |(ii) intrinsic noise in the| | | |

| | | |that control dopamine function in|measure (for example a | | | |

| |Frank, M. J., Seeberger, L. C., & O'Reilly R, C. (2004).| |prefrontal cortex are predictive |participant might be | | | |

| |By carrot or by stick: cognitive reinforcement learning | |of rapid trial-to-trial learning |labeled a 'positive' | | | |

| |in parkinsonism. Science, 306(5703), 1940-1943. | |from negative feedback. |learner after choosing | | | |

| | | |Negative feedback learning is |stimulus A, but any one | | | |

| |Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. | |also associated with enhanced |subject might happen to | | | |

| |M. (2007). Selective reinforcement learning deficits in | |error-related negativity (brain |prefer stimulus A from the | | | |

| |schizophrenia support predictions from computational | |potentials originating from |outset (before receiving | | | |

| |models of striatal-cortical dysfunction. Biological | |anterior-cingulate cortex) and |any reinforcement) simply | | | |

| |Psychiatry. | |activation of this same region in|due to its surface | | | |

| | | |fmri. |features). These problems | | | |

| | | | |can be mitigated by testing| | | |

| | | |(Frank et al., 2007) |participants in multiple | | | |

| | | |(Frank, Woroch, & Curran, 2005) |blocks with different | | | |

| | | |(Klein et al., 2007) |stimuli and averaging. | | | |

| | | |(Frank & O'Reilly R, 2006) | | | | |

|Weather Prediction |This is a 200 trial learning task. The stimuli for this|Research on how subjects solve this task |Evidence that this task measures |Unknown |Unknown |Unknown |This task does not seem to |

|Task |task consist of four “tarot” cards that each have a |suggests three different possible learning |striatally based learning comes | | | |elicit deficit sin |

| |unique appearance that consists of an arrangement of |strategies (Gluck, Shohamy, & Myers, 2002):|from studies showing impaired WPT| | | |schizophrenia. |

| |circles, squares, triangles or diamonds. On any given |1) multi-cue – respond to each pattern on |performance in patients with | | | | |

| |trial, between 0 and 3 of these cards are presented, and|the basis of associated of all four cues |Parkinson’s disease (Knowlton, | | | |We need to assess psychometric|

| |there are 14 possible card combinations. Each tarot card|with each outcome; 2) one-cue – respond on |Mangels et al., 1996), | | | |characteristics such as |

| |is associated with one of two outcomes (rain or sun) |the basis of the presence or absence of a |Huntington’s Chorea (Knowlton, | | | |test-retest reliability, |

| |with a fixed probability. The probability of each |single cue, ignoring the other cues; or 3) |Squire et al., 1996), and | | | |practice effects, and |

| |outcome (rain or sun) on any given trial is computed as |singleton – learn only about patterns (4 |Tourette’s Syndrome (Keri, | | | |ceiling/floor effects for this|

| |the conditional probability of each outcome and card |total) that have only one cue present an |Szlobodnyik, Benedek, Janka, & | | | |task. |

| |occurring together. After the set of cards is presented |all other absent. Participants who adopt a|Gadoros, 2002). Neuroimaging | | | | |

| |for that trial, participants have up to 5 seconds to |multi-cue strategy do better than |studies in healthy individuals | | | |There is evidence that |

| |respond. Subjects are then presented with their |participants who adult either of the other|reveal activation of the striatum| | | |performance on this task can |

| |prediction accuracy and the actual weather for that |two strategy, for which performance does |(Gluck, Poldrack, & Keri, 2008; | | | |improve in response to |

| |trial (rain or sun). A running score on the side of the|not differ (Gluck et al., 2002). |Poldrack & Foerde, 2008; Poldrack| | | |psychological or |

| |screen increases or decreases as a function of the | |& Gabrieli, 2001; Poldrack, | | | |pharmacological interventions.|

| |accuracy of the subject’s predictions. |It is also thought that early learning on |Prabhakaran, Seger, & Gabrieli, | | | | |

| |(Knowlton, Mangels, & Squire, 1996; Knowlton, Squire, & |the WPT is dependent on implicit, |1999). | | | | |

| |Gluck, 1994) |striatally governed processes, while later | | | | | |

| | |learning can engage explicit classification|There is evidence that | | | | |

| |MANUSCRIPTS ON THE WEBSITE: |learning processes (Price, 2005). |individuals with schizophrenia | | | | |

| | | |are not impaired in performance | | | | |

| |Gluck, M. A., Shohamy, D., & Myers, C. (2002). How do |Parkinson’s patients do poorly on versions |on this task (Keri et al., 2005; | | | | |

| |people solve the "weather prediction" task?: individual |of the task involving feedback based |Keri et al., 2000; Weickert et | | | | |

| |variability in strategies for probabilistic category |learning, but are not impaired on versions |al., 2002), unless they are | | | | |

| |learning. Learn Mem, 9(6), 408-418. |requiring observational learning (Shohamy |taking antipsychotics that block | | | | |

| | |et al., 2004). Parkinson’s patients are |dopamine receptors in the | | | | |

| |Keri, S., Juhasz, A., Rimanoczy, A., Szekeres, G., |less likely than control to use a multi-cue|striatum (Beninger et al., 2003).| | | | |

| |Kelemen, O., Cimmer, C., et al. (2005). Habit learning |strategy and more likely to use a singleton| | | | | |

| |and the genetics of the dopamine D3 receptor: evidence |strategy (Shohamy et al., 2004). | | | | | |

| |from patients with schizophrenia and healthy controls. | | | | | | |

| |Behav Neurosci, 119(3), 687-693. | | | | | | |

| | | | | | | | |

REFERENCES:

Barr, R. S., Pizzagalli, D. A., Culhane, M. A., Goff, D. C., & Evins, A. E. (2007). A Single Dose of Nicotine Enhances Reward Responsiveness in Nonsmokers: Implications for Development of Dependence. Biol Psychiatry.

Beninger, R. J., Wasserman, J., Zanibbi, K., Charbonneau, D., Mangels, J., & Beninger, B. V. (2003). Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: a double dissociation. Schizophr Res, 61(2-3), 281-292.

Bogdan, R., & Pizzagalli, D. A. (2006). Acute stress reduces reward responsiveness: implications for depression. Biol Psychiatry, 60(10), 1147-1154.

Boulougouris, V., Dalley, J. W., & Robbins, T. W. (2007). Effects of orbitofrontal, infralimbic and prelimbic cortical lesions on serial spatial reversal learning in the rat. Behav Brain Res, 179(2), 219-228.

Cools, R., Altamirano, L., & D'Esposito, M. (2006). Reversal learning in parkinson's disease depends on medication status and outcome valence. Neuropsychologia, 44(10), 1663-1673.

Cools, R., Lewis, S. J., Clark, L., Barker, R. A., & Robbins, T. W. (2007). L-dopa disrupts activity in the nucleus accumbens during reversal learning in parkinson's disease. Neuropsychopharmacology, 32(1), 180-189.

Chudasama, Y., & Robbins, T. W. (2006). Functions of frontostriatal systems in cognition: comparative neuropsychopharmacological studies in rats, monkeys and humans. Biological Psychiatry, 73(1), 19-38.

Dias, R., Robbins, T. W., & Roberts, A. C. (1996). Dissociation in prefrontal cortex of affective and attentional shifts. Nature, 380, 69-72.

Dias, R., Robbins, T. W., & Roberts, A. C. (1997). Dissociable forms of inhibitory control within prefrontal cortex with an analog of the Wisconsin Card Sort Test: restrictions to novel situations and independence from "on-line" processing. The Journal of Neuroscience, 17(23), 9285-9297.

Fellows, L. K., & Farah, M. J. (2003). Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain, 126(Pt 8), 1830-1837.

Fellows, L. K., & Farah, M. J. (2005). Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cereb Cortex, 15(1), 58-63.

Frank, M. J., & Claus, E. D. (2006). Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev, 113(2), 300-326.

Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A, 104(41), 16311-16316.

Frank, M. J., & O'Reilly R, C. (2006). A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci, 120(3), 497-517.

Frank, M. J., Seeberger, L. C., & O'Reilly R, C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940-1943.

Frank, M. J., Woroch, B. S., & Curran, T. (2005). Error-related negativity predicts reinforcement learning and conflict biases. Neuron, 47(4), 495-501.

Gluck, M. A., Poldrack, R. A., & Keri, S. (2008). The cognitive neuroscience of category learning. Neurosci Biobehav Rev, 32(2), 193-196.

Gluck, M. A., Shohamy, D., & Myers, C. (2002). How do people solve the "weather prediction" task?: individual variability in strategies for probabilistic category learning. Learn Mem, 9(6), 408-418.

Keri, S., Juhasz, A., Rimanoczy, A., Szekeres, G., Kelemen, O., Cimmer, C., et al. (2005). Habit learning and the genetics of the dopamine D3 receptor: evidence from patients with schizophrenia and healthy controls. Behav Neurosci, 119(3), 687-693.

Keri, S., Kelemen, O., Szekeres, G., Bagoczky, N., Erdelyi, R., Antal, A., et al. (2000). Schizophrenics know more than they can tell: probabilistic classification learning in schizophrenia. Psychol Med, 30(1), 149-155.

Keri, S., Szlobodnyik, C., Benedek, G., Janka, Z., & Gadoros, J. (2002). Probabilistic classification learning in Tourette syndrome. Neuropsychologia, 40(8), 1356-1362.

Klein, T. A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D. Y., & Ullsperger, M. (2007). Genetically determined differences in learning from errors. Science, 318(5856), 1642-1645.

Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996). A neostriatal habit learning system in humans. Science, 273(5280), 1399-1402.

Knowlton, B. J., Squire, L. R., & Gluck, M. A. (1994). Probabilistic classification learning in amnesia. Learn Mem, 1(2), 106-120.

Knowlton, B. J., Squire, L. R., Paulsen, J. S., Swerdlow, N. R., Swenson, M., & Butters, N. (1996). Dissociations within nondeclarative memory in Huntington's Disease. Neuropsychology, 10, 538-548.

Lee, B., Groman, S., London, E. D., & Jentsch, J. D. (2007). Dopamine D2/D3 receptors play a specific role in the reversal of a learned visual discrimination in monkeys. Neuropsychopharmacology, 32(10), 2125-2134.

Pizzagalli, D. A., Bogdan, R., Ratner, K. G., & Jahn, A. L. (2007). Increased perceived stress is associated with blunted hedonic capacity: potential implications for depression research. Behav Res Ther, 45(11), 2742-2753.

Pizzagalli, D. A., Goetz, E., Ostacher, M., Iosifescu, D. V., & Perlis, R. H. (2008). Euthymic Patients with Bipolar Disorder Show Decreased Reward Learning in a Probabilistic Reward Task. Biol Psychiatry.

Pizzagalli, D. A., Jahn, A. L., & O'Shea, J. P. (2005). Toward an objective characterization of an anhedonic phenotype: a signal-detection approach. Biological Psychiatry, 57(4), 319-327.

Poldrack, R. A., & Foerde, K. (2008). Category learning and the memory systems debate. Neurosci Biobehav Rev, 32(2), 197-205.

Poldrack, R. A., & Gabrieli, J. D. (2001). Characterizing the neural mechanisms of skill learning and repetition priming: evidence from mirror reading. Brain, 124(Pt 1), 67-82.

Poldrack, R. A., Prabhakaran, V., Seger, C. A., & Gabrieli, J. D. (1999). Striatal activation during acquisition of a cognitive skill. Neuropsychology, 13(4), 564-574.

Price, A. L. (2005). Cortico-striatal contributions to category learning: dissociating the verbal and implicit systems. Behav Neurosci, 119(6), 1438-1447.

Shohamy, D., Myers, C. E., Grossman, S., Sage, J., Gluck, M. A., & Poldrack, R. A. (2004). Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain, 127(Pt 4), 851-859.

Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007). Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biological Psychiatry.

Waltz, J. A., & Gold, J. M. (2007). Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophrenia Research, 93(1-3), 296-303.

Weickert, T. W., Terrazas, A., Bigelow, L. B., Malley, J. D., Hyde, T., Egan, M. F., et al. (2002). Habit and skill learning in schizophrenia: evidence of normal striatal processing with abnormal cortical input. Learn Mem, 9(6), 430-442.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download