Lecture Topics

•    sensory outfit and ecological niches (sensors co-evolve with behaviour, with occupation of specific environments)
•    super-human senses in the animal kingdom: UV, IR, polarised light, electricity, magnetism, …
•    sensory processing strategies and sensory inegration: cortical networks
•    combining sensory infromation in speech: recognising emotions, lip reading, cortical processing of sign language, ventrilocism
•    classical theories of perception: Gestalt psychology, constructivist approach, direct perception
•    perception as information processing: neuroscience
•    Q&A about revision, examination – coursework feedback

comparing the senses

5 senses (proverbial, Aristotelian tradition)

•  perhaps a better physical classification: electromagnetic, mechanical, chemical stimuli
•  are these all channels of sensory input?
•  what about temperature, pain? other senses?

Super(?)-human senses

why don’t we have X-ray vision ? who has ?

the limitations of human sensory systems can best be demonstrated by looking at the ‘sensory ecology’ - in their particular habitats, animals have developed during evolution senses which are unknown to humans

The Ecology of Ultraviolet Vision in Birds & Insects

UV light (wavelength < 400nm) exposure can have immediate and/or long term effects on human eyes.
Photokeratitis : something like a painful sunburn on the front surface of the eyes (snow blindness)
many animals, however, use UV vision (and use protective screening mechanisms in the retina to protect against damage)

>> birds see the world in a very different way to humans !!! 

	peacock feather in colour & UV (from Bristol Ecology of Vision group)
the ability to detect UV light affects their behaviour recognition of plumage patterns that is invisible to humans
	this ability is used in communication, in particular mating systems  (Bennett et al. 1996)

in this context, various theories based on co-evolution are developed concerning animal and plant coloration (mimicry, fruit colours etc.)

similar abilities in many insects, which is important for them in the context of food selection and orientation:
for example, this honey bee is homing in on a colt's foot (Tussilago farfara L.) flower (which appears just as brigth yellow to the human eye);
the bee is guided through the prominent UV colour patterns to the centre of the flower head
the flower is usingthis pattern as advertising sign, because it needs the bee to transport its pollen
 

photographed in UV flash (c) Bjørn Rørslett

we find the same story underwater, where UV can help to counteract poor lighting conditions:

	In normal daylight, these two species of Damsel fish, Pomacentrus amboinensis and Pomacentrus moluccensis, appear plain yellow, and are difficult to distinguish
	In bright UV light, using a UV sensitive camera, and to any animal with UV vision, these two species show very distinct facial patterns (by courtesy of Uli Siebeck, Brisbane)

some interesting adaptations to underwater vision can be found in humans, as well, but no UV sensitivty!

The Visual Ecology of Polarized Light

many animals, particularly invertebrates, are sensitive to the plane of polarized light (preferred orientation of electromagnetic waves)
– humans are completely blind to this property !
this provides these animals with an extra dimension of visual information: functional significance poorly understood

example : dungbeetles
living on firm coastal sands in burrows, the dung beetle returns to the burrow along the shortest possible route from extended foraging trips; in this environment visual landmarks are scarce >> they rely on cues in the sky to find way back, using polarisation-sensitive photoreceptors in the dorsal parts of their eyes (see also Dacke et al. 1999)

what do humans do?  

humans use tools, such as sports sunglasses : advanced multi-layer lens technology in polarising sun glasses, eliminating surface glare and reducing unwanted reflections (fishing!)

Infrared Vision

and humans?

humas again need tools, for example: Thermal Imaging cameras that are often used to detect insulation, electrical and mechanical problems, as indicated by heat loss ... (here: a thermal image of a dog)

  this technology is used in particular for night vision equipment !!!

Sensing magnetic fields

humans are also using tools to exploit the magnet field of the earth for navigation purposes :
compass (magnetic needle passively orienting parallel to magnetic field)

navigating birds have been demonstrated to use the earth magnetic field for orientation on their extended journeys (thousands of miles)	little is known about the location and function of magnetic sensors & the strategies to exploit such signals

other suspects for the use of magnetic information:

fish (salmon)   turtles   insects (Monarch)

Electric field

some fish can generate electric fields (electrogenic) and/or detect electric fields (electroreceptive)
which are used for active or passive orientation mechanisms: electrolocation

(see Nelson & MacIver 1999)

	Apteronotus albifrons (black ghost knifefish)
electrolocation is also known for the Australian Platypus

Echolocation in Humans

bats are known to use echo signals to fly in the dark >> humans?
a California teenager who is blind and overcomes his disability by experiencing life with all his other senses...

Sensory processing strategies

• senses and behaviour match ecological niches
• active exploration gives access to richer information
• combination of senses: improves sensitivity and selectivity >>> sensory integration

>>> sensory integration

how can cross-modal combination of information be investigated experimentally ? (an example for vision-touch integration: recognise objects by vision and/or haptics)

(large interest from engineering: sensory fusion in robots, VR, video games) >> how do nervous systems combine sensory information ?

Multisensory regions in the human brain

how and where are the different senses integrated in the human brain ?

fMRI is the tool to study some cases of multisensory integration: most prominent :
combination of visual and auditory information (Calvert et al 1998)

• red: viewing mouth movements without sound
• blue: listening to speech
• yellow: activated by both signals

these could be the regions for crossmodal identification !

Audio-visual speech integration

one step further – conjunction: is coherence between visual AND auditory information crucial?

what happens in the multisensory brain regions when listeners receive conflicting signals (McGurk Effect) ? (seeing someone saying 'ga' while listening to sound 'ba' leads to perception of 'da') (King and Calvert 2001)

Encoding of sign language

•   how the deaf process signed languages in the brain ?
•   specialised structures for decoding linguistic patterns in general?

                example for sign language: butterfly                  

what is the cortical location for language error handling ? signed sentences mostly encoded in left hemisphere (like parsing spoken language hearing people) identical brain structures for similar tasks in the left inferior frontal cortex in deaf and hearing people meaningless grammatical hand movements >> greater blood flow in the planum temporale (like spoken equivalent in hearing people)

reading emotions

observers are asked to rate the sadness of a series of (real) faces with different expressions between happy and sad

sadness ratings follow a ‘psychometric curve’: from low to high for happy to sad faces combining the face display with a sad or happy voice shifts the psychometric curve to higher or lower sadness ratings

emotional voices influence the categorization of facial expression !!
(De Gelder & Bertelson 2003)

for some more intersting observations about about facial expressions click here ...

transfer across sensory modalities

ventriloquism: speaking or uttering sounds so that they seem to come from the speaker’s dummy or some other source than the speaker for some nice moves, click here for 'read my lips'

synaesthesia: a mixing of senses causing a person to experience such things as colored hearing, gustatory sights, and auditory smells… (one in every 25,000 people !) (Ramachandran & Hubbard 2003)

sensory substitution: replacing a (lost/missing) sense with some other sense! the vOICe system translates arbitrary video images into sounds - this means that you can see with your ears, whenever you want to

an interesting case study (on YouTube)... (see also Stoffregen TA, Pittenger JB 1995)
Human echolocation : see a California teenager who is blind overcome his disability by experiencing life with all his other senses...

Theories of Perception

Gestalt psychology

Gestalt theory has its focus on the principles of perceptual organisation: ‘the whole is more than the sum of the parts’
‘laws’ describing perceptual phenomena <<you may want to ask yourself: are these explanations?>>
(Wertheimer, Koffka, Kohler)

praegnanz: good shape (vision, acoustical) similarity proximity good continuation familiarity (illusion from Kanizsa 1976)

constructivist approach

emphasizing top-down processes in perception: the mind tries to make the best sense of ambiguous data
(Neisser, Gregory)

active & constructive process iterative comparison of sensory input with internal (stored) knowledge hypotheses & expectations generate specific errors (illusions) physiological basis ? revival: Bayesian estimation ('Dalmatian' from Marr 1982)

direct perception

emphasizing bottom-up processing, exploiting richness of information content in sensory data
direct use of sensory input for behavioural control without need of high-level representation
(Gibson)

comprehensive capture of information in optic array unambiguous information about spatial layout (flowfield) affordances: meaning in behavioural context resonance: process to extract information (filter tuning?) flowfield sketch: pilot approaching runway, from Gibson 1979

the information processing approach

in these lectures the focus was on a neuroscientific & computational approach to perception, which describes the first information processing steps as basis for cognitive psychology

common themes for all 5 major sensory channels are the following key concepts:

• receptor: transformation from external to neural signals
• filter: encoding of information, tuning to specific properties
• receptive field: physiological basis for localisation and tuning
• representation: cortical processing and mapping
• illusions: processing can misrepresent the physical world
• active sensing: sensory system link with exploratory behaviour

how does this approach relate to other concepts of perception?

summary: integration & conceptual framework

•   humans rely on a variety of senses to organise their behaviour
•   many animals have sensory systems that are not available to humans - adaptation to specific environments and behavioural strategies
•   different types of sensory information is integrated in the human brain (audio-visual binding is important for understanding language)
•   ‘classical’ approaches to understand human perception include Gestalt, constructivist and direct perception
•   the currently dominating view focuses on information processing in biological systems (~cognitive psychology)

some things to remember about revision and exams

•   how do you revise for Sensation and Perception? (work in small groups)
•   reading materials : handouts, textbook, lecture webpages, references >>> start at http://www.pc.rhul.ac.uk/staff/J.Zanker/PS1061/PS1061.htm
•   exams format: 2 hours, answer 2 out of 4 questions
•   discuss with peers!
•   practise sample exam script (webpage)…
•   course intranet forum for FAQ

Reading:

•    Zanker, J.M. (2010) Sensation, Perception, Action - an evolutionary perspective. Palgrave (152.1 ZAN) : chapters 12 and 13
•    Goldstein, EB (2002)  Sensation and Perception (7th ed.), Wadsworth (152.1 GOL)
•    Eysenck (2001) Principles of Cognitive Psychology. Psychology Press (153.4 EYS) chapter 2 for conceptual frameworks

Specific References:

• Bennett ATD, Cuthill IC, Partridge JC & Maier EJ. (1996) "Ultraviolet vision and mate choice in zebra finches." Nature 380, 433-435, click here (download from Journal)
• Calvert, G A, Brammer, M J, Iversen, S D 1998 "Crossmodal identification" Trends in Cognitive Sciences 2, 247-253, click here (download from Journal)
• Dacke M, Nilsson D-E, Warrant EJ, Blest AD, Land MF& O´Carroll DC (1999) "Built-in polarizers form part of a compass organ in spiders" Nature 401, 470-473
• de Gelder B, Bertelson P (2003) "Multisensory integration, perception and ecological validity." TICS 7(10), 460-476. click here (download from Journal)
• Kanizsa G (1976) "Subjective Contours" Scient.Am. 234, 48-52. click here (download from virtual resources)
• Gibson JJ (1979) The ecological approach to visual perception. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
• King, AJ, Calvert, GA 2001 "Multisensory integration: Perceptual grouping by eye and ear" Current Biology 11, R322-R325. click here (download from Journal)
• Land, MF, Nilsson D-E (2002)  Animal Eyes.  Oxford: Oxford University Press
• Marr D (1982) Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco: Freeman & Co.
• Moller, P. (1995). Electric Fishes: History and Behavior. London: Chapman & Hall
• Nelson, M.E. and MacIver, M.A. (1999) Prey capture in the weakly electric fish Apteronotus albifrons: sensory acquisition strategies and electrosensory consequences. J. Exp. Biol. 202, 1195-1203
• Newman E, Hartline P (1982) The infrared "vision" of Snakes. Scientific American 213:116-127. click here (download from virtual resources)
• Nishimura H, Hashikawa K, Doi K, Iwaki T, Watanabe Y, Kusuoka H, Nishimura T, Kubo T (1999) "Sign language 'heard' in the auditory cortex" Nature 397, 116
• Petitto LA, Zatorre RJ, Gauna K, Nikelski EJ, Dostie D, Evans AC (2000) "Speech-like cerebral activity in profoundly deaf people processing signed languages: Implications for the neural basis of human language" PNAS 97, 13961–13966
• Ramachandran VS, Hubbard EM (2003) "Hearing colours, tasting shapes" Scientific American, 288(5): 52-9. click here (download from virtual resources)
• Siebeck U E, 2004 "Communication in coral reef fish: the role of ultraviolet colour patterns in damselfish territorial behaviour" Animal Behaviour 68 273-282. click here (download from virtual resources)
• Stoffregen TA, Pittenger JB (1995) "Human Echolocation as a Basic Form of Perception and Action" Ecological Psychology, Vol. 7(3), 181-216. click here (download from virtual resources)

to download a pdf copy of lecture summary, click here

to download a pdf copy of lecture slides, click here

back to course outline
last update 30-01-2012
Johannes M. Zanker