Y Niv and PR Montague, Theoretical and empirical studies of learning, Studies In History And Philosophy Of Science Part B, 2009 pp. 331-351.
Publisher Summary This chapter introduces the reinforcement learning framework and gives a brief background to the origins and history of reinforcement learning models of decision- making. Reinforcement learning provides a normative framework, within which conditioning can be analyzed. That is, this suggests a means by which optimal prediction and action selection can be achieved, and exposes explicitly the computations that must be realized in the service of these. In contrast to descriptive models that describe behavior as it is …
(web, pdf)
Ned Block, On a confusion about a function of consciousness, Behavioral And Brain Sciences, 18 (1995) 227-247.
Consciousness is a mongrel concept: there are a number of very different “consciousnesses.” Phenomenal consciousness is experience; the phenomenally conscious aspect of a state is what it is like to be in that state. The mark of access-consciousness, by contrast, is availability for use in reasoning and rationally guiding speech and action. These concepts are often partly or totally conflated, with bad results. This target article uses as an example a form of reasoning about a function of “consciousness” based on the phenomenon of blindsight. Some information about stimuli in the blind field is represented in the brains of blindsight patients, as shown by their correct “guesses.” They cannot harness this information in the service of action, however, and this is said to show that a function of phenomenal consciousness is somehow to enable information represented in the brain to guide action. But stimuli in the blind field are both access-unconscious and phenomenally unconscious. The fallacy is: an obvious function of the machinery of accessconsciousness is illicitly transferred to phenomenal consciousness.
(web, pdf)
Stanislas Dehaene and L Naccache, Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework., Cognition, 79 (2001) 1-37.
This introductory chapter attempts to clarify the philosophical, empirical, and theoretical bases on which a cognitive neuroscience approach to consciousness can be founded. We isolate three major empirical observations that any theory of consciousness should incorporate, namely (1) a considerable amount of processing is possible without consciousness, (2) attention is a prerequisite of consciousness, and (3) consciousness is required for some specific cognitive tasks, including those that require durable information maintenance, novel combinations of operations, or the spontaneous generation of intentional behavior. We then propose a theoretical framework that synthesizes those facts: the hypothesis of a global neuronal workspace. This framework postulates that, at any given time, many modular cerebral networks are active in parallel and process information in an unconscious manner. An information becomes conscious, however, if the neural population that represents it is mobilized by top-down attentional amplification into a brain-scale state of coherent activity that involves many neurons distributed throughout the brain. The long-distance connectivity of these 'workspace neurons' can, when they are active for a minimal duration, make the information available to a variety of processes including perceptual categorization, long-term memorization, evaluation, and intentional action. We postulate that this global availability of information through the workspace is what we subjectively experience as a conscious state. A complete theory of consciousness should explain why some cognitive and cerebral representations can be permanently or temporarily inaccessible to consciousness, what is the range of possible conscious contents, how they map onto specific cerebral circuits, and whether a generic neuronal mechanism underlies all of them. We confront the workspace model with those issues and identify novel experimental predictions. Neurophysiological, anatomical, and brain-imaging data strongly argue for a major role of prefrontal cortex, anterior cingulate, and the areas that connect to them, in creating the postulated brain-scale workspace.
(web, pdf)
Angel J Gallego and Roman Orus, Language Design as Information Renormalization, , 2017.
Here we consider some well-known facts in syntax from a physics perspective, allowing us to establish equivalences between both fields with many consequences. Mainly, we observe that the operation MERGE, put forward by N. Chomsky in 1995, can be interpreted as a physical information coarse-graining. Thus, MERGE in linguistics entails information renormalization in physics, according to different time scales. We make this point mathematically formal in terms of language models. In this setting, MERGE amounts to a probability tensor implementing a coarse-graining, akin to a probabilistic context-free grammar. The probability vectors of meaningful sentences are given by stochastic tensor networks (TN) built from diagonal tensors and which are mostly loop-free, such as Tree Tensor Networks and Matrix Product States, thus being computationally very efficient to manipulate. We show that this implies the polynomially-decaying (long-range) correlations experimentally observed in language, and also provides arguments in favour of certain types of neural networks for language processing. Moreover, we show how to obtain such language models from quantum states that can be efficiently prepared on a quantum computer, and use this to find bounds on the perplexity of the probability distribution of words in a sentence. Implications of our results are discussed across several ambits.
(web, pdf)
P W Glimcher, Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis, Proceedings Of The National Academy Of Sciences Of The United States Of America, 108 (2011) 15647-15654.
A number of recent advances have been achieved in the study of midbrain dopaminergic neurons. Understanding these advances and how they relate to one another requires a deep understand- ing of the computational models that serve as an explanatory framework and guide ongoing experimental inquiry. This intertwin- ing of theory and experiment now suggests very clearly that the phasic activity of the midbrain dopamine neurons provides a global mechanism for synaptic modification. These synaptic modifications, in turn, provide the mechanistic underpinning for a specific class of reinforcement learning mechanisms that now seem to underlie much of human and animal behavior. This review describes both the critical empirical findings that are at the root of this conclusion and the fantastic theoretical advances from which this conclusion is drawn.
(web, pdf)
Thomas Metzinger, The Problem of Mental Action - Predictive Control without Sensory Sheets, Philosophy And Predictive Processing, Chapter 19, 1-26. Frankfurt am Main.
In mental action there is no motor output to be controlled and no sensory in- put vector that could be manipulated by bodily movement. It is therefore un- clear whether this specific target phenomenon can be accommodated under the predictive processing framework at all, or if the concept of “active inference” can be adapted to this highly relevant explanatory domain. This contribution puts the phenomenon of mental action into explicit focus by introducing a set of novel conceptual instruments and developing a first positive model, concen- trating on epistemic mental actions and epistemic self-control. Action initiation is a functionally adequate form of self-deception; mental actions are a specific form of predictive control of effective connectivity, accompanied and possibly even functionally mediated by a conscious “epistemic agent model”. The overall process is aimed at increasing the epistemic value of pre-existing states in the conscious self-model, without causally looping through sensory sheets or using the non-neural body as an instrument for active inference.
(pdf)
Aaron Schurger and Sebo Uithol, Nowhere and Everywhere: The Causal Origin of Voluntary Action, Review Of Philosophy And Psychology, 6 (2015) 761-778.
The idea that intentions make the difference between voluntary and non-voluntary behaviors is simple and intuitive. At the same time, we lack an understanding of how voluntary actions actually come about, and the unquestioned appeal to intentions as discrete causes of actions offers little if anything in the way of an answer. We cite evidence suggesting that the origin of actions varies depending on context and effector, and argue that actions emerge from a causal web in the brain, rather than a central origin of intentional action. We argue that this causal web need not be confined to the central nervous system, and that proprioceptive feedback might play a counter-intuitive role in the decision process. Finally we argue that the complex and dynamic origins of voluntary action and their interplay with the brain’s propensity to predict the immediate future are better studied using a dynamical systems approach.
(web, pdf)
Rüdiger J Seitz and Hans-Ferdinand Angel, Belief formation – A driving force for brain evolution, Brain And Cognition, 2020 vol. 140 p. 105548.
The topic of belief has been neglected in the natural sciences for a long period of time. Recent neuroscience research in non-human primates and humans, however, has shown that beliefs are the neuropsychic product of fundamental brain processes that attribute affective meaning to concrete objects and events, enabling individual goal setting, decision making and maneuvering in the environment. With regard to the involved neural processes they can be categorized as empirical, relational, and conceptual beliefs. Empirical beliefs are about objects and relational beliefs are about events as in tool use and in interactions between subjects that develop below the level of awareness and are up-dated dynamically. Conceptual beliefs are more complex being based on narratives and participation in ritual acts. As neural processes are known to require computational space in the brain, the formation of inceasingly complex beliefs demands extra neural resources. Here, we argue that the evolution of human beliefs is related to the phylogenetic enlargement of the brain including the parietal and medial frontal cortex in humans.
(web, pdf)
Mark J Wagner and Liqun Luo, Neocortex–Cerebellum Circuits for Cognitive Processing, Trends In Neurosciences, 43 (2020) 42-54.
Although classically thought of as a motor circuit, the cerebellum is now understood to contribute to a wide variety of cognitive functions through its dense interconnections with the neocortex, the center of brain cognition. Recent investigations have shed light on the nature of cerebellar cognitive processing and information exchange with the neocortex. We review findings that demonstrate widespread reward-related cognitive input to the cerebellum, as well as new studies that have characterized the codependence of processing in the neocortex and cerebellum. Together, these data support a view of the neocortex–cerebellum circuit as a joint dynamic system both in classical sensorimotor contexts and reward-related, cognitive pro- cessing. These studies have also expanded classical theory on the computations performed by the cerebellar circuit.
(web, pdf)