The semiotic approach to the analysis of cognition Daniil Berezhnoy Kira Nikolskaya Dep. of Higher Nervous Activity, Moscow State University, Russia

C. Darwin ( ) I. Sechenov ( ) W. Kohler ( ) E. Tolman ( ) "the difference in mind between man and the higher animals, great as it is, certainly is one of degree and not of kind" (Darwin, 1871, p. 128) Animal cognition – the key to the human mind In the phylogeny of thought and speech, we can certainty distinguish the pre-speech phase of the intelligence development and pre-intellectual phase in the development of speech. (Vygotsky, 1929)

Modern cognitive science Concentrated on the study of human cognitive processes. Evolutionary approach disappears. Animals are used only as a convenient material for the neurophysiological experiments. Theoretical basis - human psychology. Behavioristic approach to the animal studies: no mind, only behavioral realisation.

Behaviorism Establishment of the strict correspondence between stimulus and reaction. What is learning? Cognitivism Establishing cause-effect relationships and their use for the organization of behavior R1R1 S1S1 R2R2 S2S2 RnRn SnSn behavior S1S1S1S1 S3S3S3S3 SnSnSnSn S4S4S4S4 S2S2S2S2 C1C1C1C1 C2C2C2C2 C4C4C4C4 C3C3C3C3 CnCnCnCn C5C5C5C5 R1R1R1R1 R2R2R2R2 R3R3R3R3 RnRnRnRn cognition environment

Back to animal cognition

Animal behavioral models Open-field test T-maze Scinner box

Sign system Communicative system syntactic semantic pragmatic What is this? What to do ? How to do? Information about environment topology Semiotic view on the problem Information about task structure Information about behavioral plan bearer of signs bearer of language + environment living system Time, sec EOhMJB+aJMIKA+s 0-40 eKwisIhMwJB 0-60 sJsisQkkGSisUsisis 1-20 USkGQMoisIKAsKLHPoh 1-40 HLIoMsQisQMoMJBs 1-60 MIoIOEOMsisMOEsOT ss4isis3sis34Ss456is68s Eh4Ss4321sis1Ssis4hEOh 2-60 IKA+asskeKIMJB assJMIKAs 3-40 RXFPhHLKA+KIoGSUsi 3-60 isZJMLHmP1234Eo

L e a r n i n g associate (summation) forecast of result Perception of the problem situation What is this? comparison (selection) compilation (integration) Estimation of the method of action What to do? Generalization How to do ? forecast of action forecast of the action outcome Process = Structure = Function = Behavioral realization= Associative - provoked response Choice of action behavior (program of actions) How do we understand learning (operational aspect)

Levels of forecasting Forecast of result (meaning of the symbol) Forecast of action (probabilistic situation) Forecast of the action outcome (meaning of the action) If A Then B If A Do R2(+) Do R1(-) ? C1C1C1C1 C2C2C2C2 C4C4C4C4 C3C3C3C3 CnCnCnCn C5C5C5C5 R1R1R1R1 R2R2R2R2 R3R3R3R3 RnRnRnRn Result (+)

Experimental environment Start food area Non-food area BBAA C C D D P P T T EnEn EnEn semantic elementcrossing in the maze HF I X R K L AA PP DD EnEn TT G S M U BB Z J Q CC EnEnEnEn AABB D C TP Route diversity number of 13-link routes (length of locomotor task solution) is equal Semantic graph of the operant task 7 4 number of 4-link semantic sequences is equal 7 4

What to do? (semantics) Alphabet - 7 units number of 4-link semantic sequences is equal 7 4 if you leave the maze after getting food and come back there again, then food would always be available in food feeders. Exit from the food- area Cause: no more food Cause: possibility of multiple reinforcement Entrance to the food- area (maze) Feeders true false full empty Non-food area Cause: fear (high uncertainty)

Pragmatic aspects of decision (plan of behavior) causal conditioned reflex The aim of learning – getting maximum result by minimum expense chain conditioned reflex En F 1 F 2 Ex 1. A Free A Food n 2. En F 1 F 2 Ex SnSn S2S2 S1S1

Psycholinguistic analysis of cognitive activity Text as a sequence of semantic symbols Trial Semantic sequence 1 EBABAPBET 2 EABAPAP 3 EBT 4 EABAP 5 EBTBABABPEBT 6 EBAP 7 EBABAP Text as a sequence of syntactic symbols (routs) Text as a sequence of pragmatic symbols Trial Pragmatic sequence 1 EnF 1 F 2 F 1 ExF 2 EnEx EnF 1 F 2 ExF 1 Ex EnF 1 Ex EnF 1 F 2 F 1 Ex EnF 1 ExF 1 F 2 F 1 F 2 F 1 ExEnF 1 Ex EnF 1 F 2 Ex Trial Syntactic sequence EOMJBJMIKAKI MJBJQGSUUSGQMIKAK LHPHLIMQQMMJBJMIIO EOMJBJMOEOMQGT E EOIKAKIMJBJMIKAR XFPHLKAKIMJJQGSUZJ MILHP E EOMJ BJQGT…...

Fish (carps) Reptiles (turtles) Mammals: insectivores (hedgehogs) rodents (mice and rats) carnivores (ferrets) monkeys (macaque rhesus) cetaceans (dolphins) Invertebrates Crustaceans Crabs Insects (ants, beetles) Vertebrates Participants of our experiments

Learning as creation of language: letters to sentences. During learning, the information is being re-coded, in such a way that amount of information contained by elementary information unit increases while the total number of these units (7±2) does not change. Levels of re-coding 2-n; segments as letters syntactic level 4-n; links as single words semantic level n; solution routes as sentences pragmatic level

The re-coding process A B a semantic level syntactic level ABAB pragmatic level

C T En ABD T P DA P C B Symmetry operator: modeling of the environment spatial structure endstart aim Analysis – symmetry operation Operational information unit – sensorimotor ring O Synthesis –route formation via association process O Chained ringed structure

Inversion Operator: Probing semantic connections for the readout direction invariance. A P D O T R A

aa bb cc aa bb cc aa bbcc aa bbcc aa bb cc Behavioral plan as a result of semantic connections combinatory interchange inside whole structure Semantic node on the step NSemantic node on the step N+1 Combinatory interchange operator: pragmatic information recognition

Syntactic variation Semantic variation Pragmatic variation Solution Symmetry operator Inversion operator Combinatory interchange operator Evolutionary algorithm of learning

Invitation to cooperate Possible ways of model development: 1) Comparison of the revealed cognitive patterns with the data in the field of human problem solving and linguistic activity. 2) The software simulation of the derived patterns and their appliance in artificial intelligence research. Comparison with existing solution finding algorithms (genetic, evolutionary algorithms in programming). 3) The use of symbolisation and semiotic approach as a method to formally describe and identify significant patterns in the dynamics of other systems (e.g. analysis of neurophysiological data). 4) Involvement of methodological approaches from the field of structural linguistics, cryptography, theory of artificial languages to extend the capabilities of symbolic data analysis (revealing the internal rules).