Lecture Topics

•    the basic brain function: neural encoding to create an internal representation of the outside world
•    studying single neurons: electrophysiological analysis of receptive field structure
•    extracting image features: parallel and serial processing in the sensory system
•    receptive field organisation (e.g. opponency) as basis of perception (example: simultaneous contrast)
•    receptive fields act like filters for spatial detail: spatial frequency channels
•    principles of information encoding: contrast enhancement and redundancy reduction

encoding of information

	what happens when you use your mobile phone ? sender: sound is first converted into electrical signals and then into radio signals receiver: radio signals are converted into electrical signals and then into sound signals (carrying messages, 'information') are converted (is 'encoded') from one physical medium into another and transitted from sender to receiver
in the same sense, the brain encodes information, thus converting physical events (e.g. a flower) into mental events (e.g. the conscious experience, abstract concept or name of a flower)

•   What are the constraints & strategies of encoding information in nervous systems?
•   What is the neural code?

electrical signals: spikes (carry information along a neuron, rate code)  chemical signals: transmitters (transmit information between neurons – across synapses, concentration code)

	electrical signals: spikes (carry information along a neuron: rate code) - how many spikes/second?
	chemical signals: neurotransmitters (carry information between neurons – across synapses: concentration code) - how much neurotransmitter?

Responses in ‘neural code’ – what is meant by ‘neural activity’?

• spikes (rate): as above
• graded electric potentials: gradual changes in voltage differences across the membrane of a neuron (between inside and outside the neuron)

studying single neurons

the function of nervous systems is analysed by means of ‘electrophysiological’ recordings: electrodes are placed on the surface of neuronal tissues, in nerves, or in the proximity or inside of individual neurons, in order to observe nerve cell activity. Pioneering work in this area was carried out by Kuffler (1952) and Hubel & Wiesel (1962, Nobel Prize in Physiology or Medicine 1981).

•   stimulus: needs to have adequate modality (stimulus attributes have to match the sensor, e.g. light for visual system, sound for auditory system) and has to arise from a restricted location (e.g. a particular region in the visual field)
•   responses are recorded - activity of neurons in ‘neural code’: graded electric potentials (voltage changes) or action potential = spikes (rate code: activity level is reflected by number of spikes - spike frequency)

retinotopic map: parallel processing

neurones are organised in large arrays sampling the complete visual field, point by point: creating a 'retinotopic' map ! neighbouring points in the visual field are represented in neighbouring neurons

	the eye operates in a fashion that can be compared to a digital camera :   image is encoded in pixels (two-dimensional)   visual field (field of view)   spatial resolution limits   parallel processing   (many pixels transmitted at the same time)
but much better performance !!!    higher sensitivity    wider operating range    intelligent ! (ca. 1 MegaPix, smart encoding)

Retinotopic maps

Relative spatial relationships are preserved !!! But: upside-down image on retina (tissue of photo-sensors at the back of the eye) ...

vision: from light to spikes

the information encoding in the visual system: light is converted into electrical signals

	light : electromagnetic waves (energy) - photons
	photoreceptors in the eye (retina): membrane potential (light is converted by means of photopigments in the outer segments) rods (luminance) & cones (colour)
	interneurones : generating spikes to be further processed in the retina and transmitted to the brain the information is now ready for computation !!!

These first steps of visual information processing are carried out in the retina (the sensory epithelium of the eye - interface between the physical stimuli and the nervous system). Physiological aspects of neural processing will be covered by Brain & Behaviour PS1060 in year 1 and PS2061 in second year. Here (PS1061) we focus on principles of information processing and their relation to perception.

neural computation: convergence, excitation and inhibition

Neurons sample sensory space (arrays of photoreceptors, for instance, capture light from the visual field in a ordered manner), and neurons interact with each other, thus processing the collected information. This processing constitutes a receptive field and its structure which is explained schematically in the following diagrams.

	no interaction direct representation of each sampling point (each receptor signal is encoded in a single interneurone) size is not encoded (interneurone response is the same for small and extended patches of light) low sensitivity (each interneurone collects light from a minimum region in visual space without summation) high resolution (many points are sampled separately)
	information from neighbouring sensors is combined (integration): simplest case is spatial pooling: convergence graded response to stimulus size (interneurone response larger when more receptors are stimulated) sensitivity increased (summation of light in interneurone) reduced resolution (fewer sampling points)
	responses from neighbouring regions are subtracted: lateral inhibition integration with different signs in second stage interneurone: a simple neural computation ! graded response with optimum size: when a stimulus is bigger than the optimum size, it will start exciting adjacent first stage interneurons which inhibit the second stage interneuron and thus reduce its response

Receptive Field

Definition: The location in space where the presence of a visual stimulus can produce a change in the response of a neuron.
Simple mapping of receptive fields – shining small spots of light and checking where a change in response occurs.
What is our receptive field in the previous examples?

Plotting receptive fields with spotlight example: retinal ganglion cells

receptive field function

spatial integration in two-dimensional case

the receptive field is structured plotting receptive fields of neurones with spotlight Retina ganglion cells: (after Kuffler 1952) regions on the screen which activate (green circles) and deactivate (red circles) the particular neurone

lateral inhibition is the key to understand the function of receptive field structures

	schematic sketch of a 'concentric' receptive field: regions of the visual field that excite (green) and inhibit (red) a sensory neuron, respectively
opponency: signals from neighbouring regions (driven by incoming light: yellow blobs) are balanced against each other

electrophysiological recordings from neurons with concentric (On-Off) receptive fields, using different stimulus configurations, demonstrate that the optimum stimulus for a receptive field of a given size is a light beam that just covers the on-centre of this receptive field; larger light beams reduce the response to spontaneous activity, and exclusive illumination of the off-surround of the recptive field inhibits all neural activity

this neural response can be understood as the integration (balancing) of excitation from the centre and inhibition from the surround of the receptive field

feature extraction: orientation

by combining simple steps of information processing, the brain encodes a range of features - for example, some neurones respond preferably to specific orientation - this is achieved by structured receptive fields (more complex than the simple concentric receptive field structures shown above)

	receptive fields plotted with a light spot for 2 exemplary cortical neurons after Hubel & Wiesel 1962 regions on the screen which activate (green circles) and deactivate (red circles) areextended in a particular orientation and aligned with each other
hierachical integration of neurons generates novel receptive field properties (Hubel & Wiesel 1962): orientation tuning of a cortical simple neurone is derived by integrating concentric receptive fields arranged in a linear array: the optimum stimulus now is a line!
	orientation tuning curves can measured in neuronal responses and in perception !!!

neurons encode features

each image pixel is analysed (in parallel) to represent (encode) a variety of local features such as brightness, colour, size, orientation, texture, motion
such 'intelligent' encoding is the basis for perception. Similar processing steps are used fro early analysis in machine vision systems.

feature specificity is generated through hierarchical computations : appropriate changes in receptive field structure (serial processing)

parallel & serial processing

the brain can be decomposed a hierarchical network of brain regions (cloured boxes in the figure below) that have a dense network of connections (black and red lines in the figure) >>>>>> for the full story, see van Essen et al 1992

• advanced feature extraction thus is achieved through (sequential) serial processing - this leads to characteristic channels carrying specific stimulus information: perceptual filters
• specialization for image properties in parallel processing streams (functional components), in addition to parallel processing in arrays of neurons that carry out the same processing on many image points at the same time (retinotopic maps)
• perceptually, we experience the combination of stimulus features: at some higher stage of the visual system the different streams are merged to a coherent scene analysis, sometimes referred to as 'cue combination' or 'sensory fusion'

a schematic picture of parallel and serial processing in the brain

visual informtaion is processed in parallel in several streams that may be dedicated to different image properties and may feed into different aspects of thinking and behaviour within each stream, we find sequential processing steps that extract the image properties reuquired for scene analysis within each processing step, a large number of image points are processed in parallel by arrays of neurons organised in maps

simultaneous contrast illusions

what is contrast ??

low contrast	high contrast

Definition: Brightness contrast (perceived intensity difference) is the perceived relative difference between two image regions of different luminance (physical intensity)

a demonstration of brightness contrast :

compare the brightness of the vertical lines - are they identical? YES but they don't look like this - brightness and colour are perceived relative to their surroundings !

can we explain such brightness illusions in terms of neural encoding ?

centre-surround receptive fields

excitation and inhibition create opponency in concentric receptive fields, weighing up the amount of light hitting the centre and the surround, respectively

On-centre-Off-surround : excitation from centre - inhibition from surround > responding to bright spots	Off-centre-On-surround : inhibition from centre - excitation from surround > responding to dark spots

‘looking through’ centre-surround receptive fields explains simultaneous contrast illusions: the amount of light falling on the centre/surround determines the amount of neural excitation/inhibition, and thus the final output of the sensory neuron

This called filtering:

• The information is being transformed
• Certain regions are emphasised
• The receptive field acts as a filter – allowing signal from around boundaries through, but not transmitting signal from uniforms areas

strong inhibition from bright surround > central area is perceived as darker	weak inhibition from dark surround> central area is perceived as brighter

this phenomenon demonstrates how illusions can be used to understand how receptive field properties are related to perception

receptive fields serve multiple fuctions: contrast enhancement (here), spatial filtering (next), redundancy reduction (below)

spatial filtering

another property of centre-surround receptive field is to make it preferentially respond to stimuli of a particular size: this property is called size tuning (like tuning your stereo to your favourite radio station, the is tuned to an optimum size stimulus) – the receptive field acts like a spatial filter (cf. lecture 1)

wide stimulus stripes   > large excitation   > large inhibition   small response	medium stimulus stripes   > large excitation   > small inhibition   large response	narrow stimulus stripes   > small excitation   > small inhibition   small response

the size of a receptive field determines spatial detail visible
separate images are visible at different spatial scales (levels of detail)
for the full story, see Wilson 1991

selecting spatial detail in images

we can actively select levels of spatial detail in images which we are attending to, while ignoring others
two examples to observe different image priopeties at different spatial scales (click on links to get full-sized images from original sources of thumbnails)

Self-Portrait, Etching Rembrandt van Rijn, 1654 Boerner Gallery New York	Lincoln in Dalivision, Lithograph Salvador Dali, 1977 Fundación Gala-Salvador Dalí, Figueras

spatial frequency channels - what does that mean?

think of the brain as a gravel pit :	parallel sets of receptive fields (filter banks) are operating in the visual stream
feed in a mix of rocks, pebbles, sand, ... … and filter through a set of sieves and you collect different grains in different bags !!!	the retinal image is represented in different spatial frequency channels (see Wilson 1991)

boundary contrast enhancement

what happens to receptive field outputs at a luminance border ??

	in regions of equal luminance (in the middle of dark or bright regions) excitation and inhibition cancel each other
	at the boundary excitation and inhibition are not balanced and thus increase the relative difference, 'sharpen' the transition between dark and bright

opponency (generated by lateral inhibition) enhances perceived contrast of a luminance border
(contrast : relative difference in luminance)

simultaneous contrast revisited

perceptual observation: luminance of background affects brightness of central grey
lateral inhibition: perceived intensity is reduced by activation in nearby regions = contrast enhancement by opponency

such an explanation based on receptive field properties works at low spatial scale

however:

•   no enhancement of local contrast is perceived at luminance borders
•   the squares perceptually appear to be filled in with a single level of grey...

remaining issues:

•   simple receptive fields can’t be the full explanation
•   how can we measure perceived features objectively ?
•   what could be the function ('purpose') of the opponency encoding strategy ?

redundancy reduction

redundancy: parts of a message can be removed without loosing essential information (Shannon 1949)
… when you are silent on your mobile phone, the transmission line can be used for someone/ something else …

contrast enhancement goes along with reduction of response in regions without change in stimulus intensity (redundant signal components)

so the effect of concentric receptive fields is to transmit no signals where/when there is no change in the input
this will help to:

•   remove redundancy
•   achieve economical encoding
•   contribute to noise reduction

JPEG compression

common technique in contemporary IT : don’t transmit redundant signals (use expensive equipment for something else in the meantime) by applying optimising encoding strategies - there are standard methods of image compression that you are know from the internet ...

original image 110 x 100 pixels TIFF file: 37 KB	compressed image JPEG file:1 KB

images contain large amounts of information (because they are two-dimensional)
telephone / internet lines are expensive and large traffic slows everything down
therefore data compression is vital (saves telecommuunication companies millions of pounds) !!!
in the most common compression methods (e.g. JPEG) redundant signals are removed by means of various types of opponency operators !

what do you see in pictures ?

low-level analysis: encoding features - brightness, colour, contrast, spatial detail, orientation, texture high-level analysis: understanding meaning – the famous smile   Leonardo da Vinci La Gioconda, 1503-1506 Louvre, Paris

Summary: receptive & perceptive fields

•    the sensory system makes the physical world accessible to the brain/mind
•    basic unit of brain function is the neurone, sensory neurons are characterised by receptive field
•    signals arising from sensor arrays are analysed in a complex network of parallel and serial processing in the brain
•    receptive fields > encode specific image features > foundation of perception
•    smart techniques of image encoding, as used in the brain, are cheap, fast, robust, reliable = highly evolved

General Reading:

•    Zanker, J.M. (2010) Sensation, Perception, Action - an evolutionary perspective. Palgrave (xxxxx) : chapter 2
•    chapter 3 (ignore the first physiological sections) of Goldstein, E.B. (2007) Sensation and Perception (7th ed.) Wadsworth-Thompson (152.1 GOL)
•    chapters 4 & 5 in R.L.Gregory 1994 “Eye and Brain” London: Oxford University Press. (152.14 GRE)
•    chapters 5 in V.Bruce, P.R.Green & M.Georgeson 1996 Visual Perception: Physiology, Psychology and Ecology (3rd ed.) Hove: Psychology Press (152.14BRU)

Specific References:

•   DeAngelis, G.C., I. Ohzawa, and R.D. Freeman, Receptive-field dynamics in the central visual pathways. Trends in Neuroscience, 1995. 18: p. 451-458, click here (download from Journal)
•   Field, D.J., What is the Goal of Sensory Coding? Neur.Comp., 1994. 6: p. 559-601
•   Fiorentini, A., et al., The Perception of Brightness and Darkness: Relations to Neuronal Receptive Fields, in The Neurophysiological Foundations of Visual Perception, L. Spillmann and J.S. Werner, Editors. 1989, p129-161, click here (download from virtual resources)
•   Gregory R.L. and E.H.Gombrich 1973 “Illusion in Nature and Art” Duckworth Ltd., London (152.148GRE)
•   Hartline H K, Ratliff F, 1957 "Inhibitory Interaction of receptor units in the eye of limulus" J.Gen.Physiol. 40 357-376
•   Hubel, D.H., The Visual Cortex of the Brain. Scient.Am., 1963. 209: p. 54-63, click here (download from virtual resources)
•   Hubel, D.H., Exploration of the primary visual cortex, 1955-78. Nature, 1982. 299: p. 515-524
•   Hubel, D.H., Eye, Brain and Vision. 1988, New York: Freeman & Co.
•   Hubel D H, Wiesel T N, 1962 "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex" J.Physiol. 160 106-154
•   Kuffler, S.W. Neurons in the retina: Organization, inhibition and excitatory problems. Cold Spring Harbour Symposia in Quantitative Biology 17: 281-292, 1952
•   Ramachandran, V. S., Rogers-Ramachandran, D. (2006) Cracking the Da Vinci Code. Scientific American Mind, June 14-16, click here (download from virtual resources)
•   Ratliff, F., Contour and Contrast. Scient.Am., 1972. 226: p. 90-101, click here (download from virtual resources)
•   Shannon C E, Weaver W, 1949 The Mathematical Theory of Communication (Urbana & Chicago: University of Illinois Press)
•   Van Essen D C, Anderson C H, Felleman D J, 1992 "Information processing in the primate visual system: An integrated systems perspective" Science 255 419-423
•   Wilson, H.R., Pattern discrimination, Visual Filters, and Spatial Sampling Irregularity, in Computational Models of Visual Processing, M.S. Landy and J.A. Movshon, Editors. 1991, MIT Press: Cambridge MA. p. 153-168
•   Zeki, S., The visual image in mind and brain. Sci.Am., 1992. 267: p. 68-76.

to download a pdf copy of lecture summary, click here

to download a pdf copy of lecture slides, click here

back to course outline
last update 16-10-2011
Johannes M. Zanker