PS1061: Sensation and Perception 2014-15
Term 2,    Thursday 11 am-1 pm     (Windsor Auditorium)

Lecture 2: Learning to Read the Neural Code

Course co-ordinator: Johannes M. Zanker, j.zanker@rhul.ac.uk, (Room W 246)


Lecture Topics


encoding of information




what happens when you use your mobile phone ?

sender: sound is first converted into electrical signals and then into radio signals

receiver: radio signals are converted into electrical signals and then into sound

signals (carrying messages, 'information') are converted (is 'encoded')
from one physical medium into another and transitted from sender to receiver

in the same sense, the brain encodes information, thus converting physical events (e.g. a flower) into mental events (e.g. the conscious experience, abstract concept or name of a flower)






 

electrical signals: spikes (carry information along a neuron: rate code)
- how many spikes/second?

Responses in ‘neural code’ – what is meant by ‘neural activity’?


studying single neurons

the function of nervous systems is analysed by means of ‘electrophysiological’ recordings: electrodes are placed on the surface of neuronal tissues, in nerves, or in the proximity or inside of individual neurons, in order to observe nerve cell activity. Pioneering work in this area was carried out by Kuffler (1952) and Hubel & Wiesel (1962, Nobel Prize in Physiology or Medicine 1981).


retinotopic map: parallel processing

neurons are organised in large arrays sampling the complete visual field, point by point: creating a 'retinotopic' map ! neighbouring points in the visual field are represented in neighbouring neurons


the eye operates in a fashion that can be compared to a digital camera :

  •   image is encoded in pixels (two-dimensional)
  •   visual field (field of view)
  •   spatial resolution limits
  •   parallel processing
  •   (many pixels transmitted at the same time)

but much better performance !!!
  •    higher sensitivity
  •    wider operating range
  •    intelligent ! (ca. 1 MegaPix, smart encoding)

Retinotopic maps

Relative spatial relationships are preserved !!! But: upside-down image on retina (tissue of photo-sensors at the back of the eye) ...


vision: from light to spikes

the information encoding in the visual system: light is converted into electrical signals


light : electromagnetic waves (energy) - photons


photoreceptors in the eye (retina): membrane potential (light is converted by means of photopigments in the outer segments)

rods (luminance) & cones (colour)


interneurons
: generating spikes to be further processed in the retina and transmitted to the brain

the information is now ready for computation !!!

These first steps of visual information processing are carried out in the retina (the sensory epithelium of the eye - interface between the physical stimuli and the nervous system). Physiological aspects of neural processing will be covered by Brain & Behaviour PS1060 in year 1 and PS2061 in second year. Here (PS1061) we focus on principles of information processing and their relation to perception.


neural computation: convergence, excitation and inhibition

Neurons sample sensory space (arrays of photoreceptors, for instance, capture light from the visual field in a ordered manner), and neurons interact with each other, thus processing the collected information. This processing constitutes a receptive field and its structure which is explained schematically in the following diagrams.

no interaction
direct representation of each sampling point (each receptor signal is encoded in a single interneuron)

  • size is not encoded (interneuron response is the same for small and extended patches of light)
  • low sensitivity (each interneuron collects light from a minimum region in visual space without summation)
  • high resolution (many points are sampled separately)

information from neighbouring sensors is combined (integration):
simplest case is spatial pooling: convergence

  • graded response to stimulus size (interneuron response larger when more receptors are stimulated)
  • sensitivity increased (summation of light in interneuron)
  • reduced resolution (fewer sampling points)

responses from neighbouring regions are subtracted: lateral inhibition
integration with different signs in second stage interneuron: a simple neural computation !

  • graded response with optimum size: when a stimulus is bigger than the optimum size, it will start exciting adjacent first stage interneurons which inhibit the second stage interneuron and thus reduce its response

Lateral inhibition (see Hartline & Ratliff 1957) leads to opponent behaviour ('opponency'): signals from neighbouring regions are weighed up against each other and the difference between these regions determines the final output. This is a very simple form of neural computation that leads to surprising results.


Receptive Field

Definition: The location in space where the presence of a visual stimulus can produce a change in the response of a neuron.
Simple mapping of receptive fields – shining small spots of light and checking where a change in response occurs.
What is our receptive field in the previous examples?

Plotting receptive fields with spotlight example: retinal ganglion cells


receptive field function

spatial integration in two-dimensional case


the receptive field is structured

plotting receptive fields of neurons with spotlight
Retina ganglion cells: (after Kuffler 1952)

regions on the screen which activate (green circles) and deactivate (red circles) the particular neuron

lateral inhibition is the key to understand the function of receptive field structures

schematic sketch of a 'concentric' receptive field: regions of the visual field that excite (green) and inhibit (red) a sensory neuron, respectively

opponency: signals from neighbouring regions (driven by incoming light: yellow blobs) are balanced against each other


electrophysiological recordings from neurons with concentric (On-Off) receptive fields, using different stimulus configurations, demonstrate that the optimum stimulus for a receptive field of a given size is a light beam that just covers the on-centre of this receptive field; larger light beams reduce the response to spontaneous activity, and exclusive illumination of the off-surround of the recptive field inhibits all neural activity



this neural response can be understood as the integration (balancing) of excitation from the centre and inhibition from the surround of the receptive field


feature extraction: orientation

by combining simple steps of information processing, the brain encodes a range of features - for example, some neurons respond preferably to specific orientation - this is achieved by structured receptive fields (more complex than the simple concentric receptive field structures shown above)

receptive fields plotted with a light spot for 2 exemplary cortical neurons after Hubel & Wiesel 1962

regions on the screen which activate (green circles) and deactivate (red circles) areextended in a particular orientation and aligned with each other

hierachical integration of neurons generates novel receptive field properties (Hubel & Wiesel 1962): orientation tuning of a cortical simple neuron is derived by integrating concentric receptive fields arranged in a linear array: the optimum stimulus now is a line!

orientation tuning curves can measured in neuronal responses and in perception !!!



neurons encode features

each image pixel is analysed (in parallel) to represent (encode) a variety of local features such as brightness, colour, size, orientation, texture, motion
such 'intelligent' encoding is the basis for perception. Similar processing steps are used fro early analysis in machine vision systems.


feature specificity is generated through hierarchical computations : appropriate changes in receptive field structure (serial processing)


parallel & serial processing

the brain can be decomposed a hierarchical network of brain regions (cloured boxes in the figure below) that have a dense network of connections (black and red lines in the figure) >>>>>> for the full story, see van Essen et al 1992, they can be thought of as carrying out image processing.

a schematic picture of parallel and serial processing of the image in the brain

 

visual informtaion is processed in parallel in several streams that may be dedicated to different image properties and may feed into different aspects of thinking and behaviour

within each stream, we find sequential processing steps that extract the image properties reuquired for scene analysis

within each processing step, a large number of image points are processed in parallel by arrays of neurons organised in maps

 



simultaneous contrast illusions

what is contrast ??

low contrast
high contrast

Definition: Brightness contrast (perceived intensity difference) is the perceived relative difference between two image regions of different luminance (physical intensity)

a demonstration of brightness contrast :

compare the brightness of the vertical lines - are they identical?

YES

but they don't look like this - brightness and colour are perceived relative to their surroundings !

can we explain such brightness illusions in terms of neural encoding ?



centre-surround receptive fields

excitation and inhibition create opponency in concentric receptive fields, weighing up the amount of light hitting the centre and the surround, respectively

On-centre-Off-surround : excitation from centre - inhibition from surround > responding to bright spots
Off-centre-On-surround : inhibition from centre - excitation from surround > responding to dark spots

‘looking through’ centre-surround receptive fields explains simultaneous contrast illusions: the amount of light falling on the centre/surround determines the amount of neural excitation/inhibition, and thus the final output of the sensory neuron

This called filtering:

strong inhibition from bright surround > central area is perceived as darker
weak inhibition from dark surround> central area is perceived as brighter

this phenomenon demonstrates how illusions can be used to understand how receptive field properties are related to perception

receptive fields serve multiple fuctions: contrast enhancement (here), spatial filtering (next), redundancy reduction (below)


spatial filtering

another property of centre-surround receptive field is to make it preferentially respond to stimuli of a particular size: this property is called size tuning (like tuning your stereo to your favourite radio station, the is tuned to an optimum size stimulus) – the receptive field acts like a spatial filter (cf. lecture 1)

    •   wide stimulus stripes
    •   > large excitation
    •   > large inhibition
    •   small response
    •   medium stimulus stripes
    •   > large excitation
    •   > small inhibition
    •   large response
    •   narrow stimulus stripes
    •   > small excitation
    •   > small inhibition
    •   small response

the size of a receptive field determines spatial detail visible
separate images are visible at different spatial scales (levels of detail)
for the full story, see Wilson 1991

selecting spatial detail in images

we can actively select levels of spatial detail in images which we are attending to, while ignoring others
two examples to observe different image properties at different spatial scales (click on links to get full-sized images from original sources of thumbnails)

Self-Portrait, Etching
Rembrandt van Rijn, 1654
Boerner Gallery New York
Lincoln in Dalivision, Lithograph
Salvador Dali, 1977
Fundación Gala-Salvador Dalí, Figueras


spatial frequency channels - what does that mean?

think of the brain as a gravel pit :

parallel sets of receptive fields (filter banks) are operating in the visual stream

feed in a mix of rocks, pebbles, sand, ...
… and filter through a set of sieves
and you collect different grains in different bags !!!

the retinal image is represented in different spatial frequency channels (see Wilson 1991)



boundary contrast enhancement

what happens to receptive field outputs at a luminance border ??

in regions of equal luminance (in the middle of dark or bright regions) excitation and inhibition cancel each other

at the boundary excitation and inhibition are not balanced and thus increase the relative difference, 'sharpen' the transition between dark and bright

opponency (generated by lateral inhibition) enhances perceived contrast of a luminance border
(contrast : relative difference in luminance)


simultaneous contrast revisited

perceptual observation: luminance of background affects brightness of central grey
lateral inhibition: perceived intensity is reduced by activation in nearby regions = contrast enhancement by opponency

such an explanation based on receptive field properties works at low spatial scale

however:

remaining issues:


redundancy reduction

redundancy: parts of a message can be removed without loosing essential information (Shannon 1949)
… when you are silent on your mobile phone, the transmission line can be used for someone/ something else …

contrast enhancement goes along with reduction of response in regions without change in stimulus intensity (redundant signal components)

so the effect of concentric receptive fields is to transmit no signals where/when there is no change in the input
this will help to:

JPEG compression

common technique in contemporary IT : don’t transmit redundant signals (use expensive equipment for something else in the meantime) by applying optimising encoding strategies - there are standard methods of image compression that you know ...

original image 110 x 100 pixels TIFF file: 37 KB
compressed image JPEG file:1 KB

images contain large amounts of information (because they are two-dimensional)
telephone / internet lines are expensive and large traffic slows everything down
therefore data compression is vital (saves telecommuunication companies millions of pounds) !!!
in the most common compression methods (e.g. JPEG) redundant signals are removed by means of various types of opponency operators !


what do you see in pictures ?

low-level analysis:
encoding features - brightness, colour, contrast, spatial detail, orientation, texture


high-level analysis:
understanding meaning – the famous smile

 

Leonardo da Vinci
La Gioconda, 1503-1506
Louvre, Paris



Summary: receptive & perceptive fields


General Reading:

Specific References:


to download a pdf copy of lecture slides from moodle, click here


back to course outline
last update 28-01-2015
Johannes M. Zanker