PS1061: Sensation and Perception 2014-15
Term 2,    Thursday 11 am - 1 pm (Windsor Auditorium)

Lecture 3: Illusions as Key to Reality

Course co-ordinator: Johannes M. Zanker, j.zanker@rhul.ac.uk, (Room W 246)


Lecture Topics


representing the outside world

the guiding question towards the understanding of the visual system is:
what are the fundamental steps of information processing to convert the outside (physical) world into internal (psychological) events

an early description of the fundamental principles of optical projection, (neural) transmission and (cortical) mapping can be found in Descartes (1664)

questions to ask:

answers to these questions can be given by PSYCHOPHYSICS (defined as discipline by Fechner, 1898)

some hints come from looking at visual illusions: what you see is not what you have


enjoying everyday illusions

illusions play a major role in the scientific study of sensory systems
illusions are most favourite tools in a painter’s bag of tricks


identify various depth cues in Tiepolo’s (1758) ‘An allegory with Venus and Time’ (National Gallery London)

animated arts of the 21st century: cinema creating big, fast, loud illusions
(e.g. Harry Potter, Lord of the Rings, The Golden Compass, ...)

Exclusive behind the scenes video that features the creatures of Middle-earth and interviews with Richard Taylor and the cast
Click here to View the Quicktime Video

Illusions and other visual puzzles are frequently used in advertising: have a look at the Droste effect



so what can we learn from illusions about visual information processing ?



Minimal illusions in science

 

    

in science, minimal configurations are essential to study visual processing : the Kanizsa triangle

there are two fundamental approaches to understand illusions:

>> Gestalt Psychology interpretation: the whole is more than the sum of parts - the perceptual system chooses the best, simplest and most stable shape : ‘Praegnanz’ (Gestalt ‘laws’ in next lecture)  >> computational approach : illusions tell us about mechanisms of visual information processing - corresponding phenomena can be observed in the responses of  single neurons in the visual system


a brief revision of the visual system

the modern picture of visual processing looks a bit different than the sketch of Descartes and contains much more detail, but still is driven by similar organisational principles:

flow of visual information along a complex optical and neural pathway, which involves massively parallel and hierarchical processing
   

the 3D environment is projected to flat two-dimensional (2D) images in the two eyes >>> from there the information is

in the brain an internal 3D representation of objects & events is formed

processing in the visual stream

From the visual field to the retinotopic map ('cortical images'): the 3D environment is captured as flat two-dimensional (2D) images in two eyes and then the information is processed along the visual stream.


visual scene
> left and right eye
> optic nerve > optic chiasm > optic tract
> LGN (lateral geniculate nucleus)
> optic radiation
> striate cortex
> association cortex

the approach here is to look at the visual stream with the eyes of an engineer, in the framework of cognitive psychology, to understand the mechanisms of visual information processing - the key processing concepts are: parallel and serial data analysis (feature extraction) and mapping in a complex network of brain regions

serial processing   –   one processing step is taken after the other
information processing hierarchy

parallel processing   –   the same processing step is taken simultaneously on a complete data set  (such as mapping through a broad data bus)

 

 

  • sequential data distribution and compression in successive brain regions (serial processing)
  • parallel data processing and mapping in each brain region
  • parallel feature extraction in neighbouring brain regions (e.g. colour and motion)
  • data fusion is needed in the final scene analysis to create a unitary percept: the problem problem is to 'bind' features together that belong together (e.g. moving coloured object)

 


For the rest of this lecture, we simplify the question of representation : spatial vision
(two-dimensional, static: stimulus properties brightness & colour) … we just look at a flat and static world ...


“Primrose's field”
Akiyoshi Kitaoka "Trick eyes" Tokyo: KANZEN 2002

perceiving bright and dark

white to black: how many shades of grey can you discriminate ? (8 bit computer screen: 256 grey levels > 5000 perceptual)

spatial resolution > coarse to fine texture: what is the finest pattern you can resolve ?  
(your comouter screen typically shows 1028 x  768 pixels : can you see them? at which viewing distance?)

fundamental concept: filtering

the action of sensory neurons is often described as filter - what does this mean ?

the visual system is considered as parallel sets of pattern analyzers (filter banks) which are operating in the visual stream

the size of receptive fields determines the optimum size of encoded stimulus: filter mechanism

through this process the retinal image is represented in different spatial frequency channels : from fine-grain to coarse versions of an image are treated (processed, stored, used) in separate neurons !
see Campbell & Maffei 1974

the filter mechanism is interpreted as a property emerging from the receptive field structure of sensory neurons
parallel sets of filters for other properties: colour, orientation, velocity, …


encoding: contrast enhancement


what you have


what you see


he opponent receptive field organisation can be interpreted as coding strategy : luminance differences across contours are amplified - contrast enhancement

at each level of the luminance steps the grey seems to be brighter at the left boundary than at the right boundary

- why is this happening ? -


at each luminance step the receptive field (the simplified light counter model) works like this:


no response in dark areas: excitation and inhibition cancel each other

negative response at dark side of boundary: more  inhibition than excitation


positive response at bright side of boundary: more excitation than inhibition

no response in bright areas: excitation and inhibition cancel each other

opponent filtering (centre-surround receptive fields) :
generates minimum and maximum at luminance boundaries and discards average luminance


another brightness illusion …




can you see grey spots in the white intersections of lines of this ‘Hermann Grid’ ?

… and a neural explanation

receptive fields (spatial filters) can account for this illusion !

these grey spots are believed to be the result of opponency filtering (centre-surround receptive fields, leading to contrast enhancment)
at the intersections larger parts of the inhibitory surround are stimulated, leading to an apparent reduction of brightness:

inside black squares: no stimulation of excitatory centre and inhibitory surround : no overall excitation >> perceived as dark

white bars: only small parts of the inhibitory surround are stimulated: excitation dominates inhibition >> perceived as bright

white intersections: larger parts of the inhibitory surround are stimulated: small overall excitation >> apparent reduction of brightness

obviously this is a very idealised picture, but the main result is essential: because the intersection regions would generate a lower score than the bar regions, they would be perceived as darker !

also have a look at the Scintillating Grid Illusion, and compare various versions here
for some explanations see Schrauf et al.1997.


encoding: image compression

when we revisit the simple receptive field outputs at different locations of the black squares in the Hermann grid we make the following observation:
central regions should be perceived as brighter than boundary regions,  but each square looks plain black !

encoding of spatial changes through opponent processes removes 'redundant' image components (i.e. those which do not contain information): image compression
  • no signal is transmitted for regions without luminance change
  • only meaningful information (luminance boundaries) is encoded

what you have

what you should see

however, the world does not look like the output of opponency filters (closed areas of grey are perceived homogenous in brightness)

this raises another question:

how is the average intensity level reconstructed in the brain ?

to reconstruct the average intensity in regions without intensity change, there are special cortical mechanisms by which regions enclosed by clear boundaries assume the same intensity (e.g. Grossberg & Pessoa 1998)

a cortical mechanism called ‘filling in’ reconstructs redundant image regions by assuming that regions enclosed by clear boundaries originally have the same intensity


The Craik-Cornsweet illusion

image compression and reconstruction leads to the prediction that certain luminance profiles should be perceived as similar although they are physically rather different !!!!

  • this effect is nicely demonstrated by the Craik-Cornsweet illusion  (Cornsweet 1970 ): the central area appears brighter than the regions on the left and right

  • this illusion demonstrates ‘filling in’ : surfaces between boundaries are apparently covered with uniform brightness !!

physical luminanceprofile

perceived brightness profile

for an impressive demonstration of this effect, click here


A zoo of illusions

Brightness illusions
(please note that these images give only a rough idea, because digital image compression like JPEG and other interferes with the actual luminance)


  simultaneous contrast          Mach Band          grating induction

the perceived brightness at a given location depends on its neighbourhood (simultaneous contrast effects)


  closed Koffka ring        open Koffka ring       Craik-Cornsweet
the perceived brightness of a clearly outlined surface tends to be uniform (filling in)


colour information


how do you identify objects ??


colour adds a lot of information to an image !!



how to describe colour ...

physical

spectrum of a ligth source = wavelength + relative intensity
two independent dimensions are needed to describe light (apart from overall intensity)


perceptual

  •   hue (dimension1): what we usually would refer to as perceived colour, like red, green, blue etc.
  •   saturation (dimension2): intensity (richness) of colour sensation: compare inner and outer ring
  •   note that contrast in the colour domain is more complicated than in the brightness domain: changes which generate contrast are possible in two directions - hue and saturation

spectral composition is to be converted into hue & saturation

some Historical Systems of colour description (Hans Irtel)


mixing colours: theory 1

many observations in colour vision are based on the fact that a colour is defined by 3 components (e.g. RGB in colour monitors)

trichromatic theory of colour vision advanced by Thomas Young and H. v. Helmholtz: (Young-Helmholtz trichromatic theory)



additive colour mixing:
superposition of coloured spotlights produces new colours (and can generate colour shadows)

 

note: subtractive colour mixing in paints is quite a different thing, because pigments in paints remove comonents from the light spectrum

  •   each colour can be matched by a mixture of three components
  •   it is easy to generate metamers (same appearance from different components)
  •   primaries: blue green red correspond to S M L photoreceptors
response curves for three neural mechanisms: short-, medium-, long-wave sensitive photoreceptors: this is the physiological basis of trichromatic processes


simultaneous contrast

simultaneous contrast is the enhancement of colour differences in space, as consequence of opponent processing similar to that in the perception of brightness (see Jameson & Hurvich 1964)


the cross lines appear grey in front of the yellow and yellow in front of the grey field, but they are the same colour (look at the top centre where they touch!)
Josef Albers
, The Interaction of Color, 1963

… strawberries look most tempting when embedded in green leaves …

such observations lead to another theory of colour vision:

opponent colours : theory 2

opponent-process theory of colour vision, advanced by E. Hering, is suggesting that two pairs of opponent colours that exclude each other (red-green and blue-yellow) deternmine the perceptual space of colour vision.

NOTE: these ‘opposing’ theories do not exclude each other !!!
they reflect different coding strategies at different stages of the visual stream ... 

so can you see true colours?

Munker-White (1979) effect : assimilation of colour! (this is a change of perceived colour in the opposite direction of simultaneous contrast - in this demonstration coloured stripes are getting closer to their surrounding: assimilation instead of contrast)

physically, the two red squares have the same colour
superposition of yellow and blue strips makes the red to change its appearance dramatically

There are also longlasting effects that affect perceived colour, such as the McCullough effect (Vladusich and Broerse, 2002)

Other factors that influence perceived colour include: illumination (red sunlight, blue artificial light), tinted glass, colour blindness.

For instance, colour constancy further affects our perception of colour: surface colours are often perceived independent of illumination !


successive colour contrast …


.

fixate the black dot in the centre for 60 seconds ...

… and then look at a the black dot in the right panel !

what do you see?

afterimages are the consequence of encoding stimulus change in time:

the logic of aftereffects

(1) adaptation :

As a natural property of sensory neurones, after the onset of the green-red stimulus the neural response jumps to a maximum value and then gradually returns to resting levels.
NOTE: this 'adaptation' means that only changes are encoded (remove redundancy in time)

(2) opponency :

According to the opponent theory of colour vision, stimuli of opposite quality (red-green) are subtracted from each other, and perceived colour is determined by the difference between green and red activation.
NOTE: this is conventional feature contrast enhancement (<< this also means that there is some kind of temporal filling in because colours are not fading with time >>)

(3) aftereffect :

At the end of the adaptation period locations stimlated by green light are less sensitive to green than to red, and vice versa. After stimulus offset, a grey region (grey = red + green light) appears red where the green light was shown before (and vice versa), because a larger neural signal is generated in the non-adapted red channel than in the adapted green channel: this imbalance leads to a preceived opponent colour as result of the opponency processing.
NOTE: this creates contrast enhancement in the time domain

(for more about this, see Jameson & Hurvich 1964, Anstis et al. 1978)


contrast in space and time


finally - have a look at some geometrical illusions

there are similar ways to interpret illusions of position, size, orientation, etc...
try to think about possible solutions - for many more examples and explanations click here


summary: brightness & colour


Specific References:

main textbook : Zanker, J.M. (2010) Sensation, Perception, Action - an evolutionary perspective. Palgrave (152.1 ZAN) : chapter1

additional textbook : Goldstein, E.B. (2007) Sensation and Perception (7th ed.) Wadsworth-Thompson (152.1 GOL), (in particular first pages of chapter1)


to download a pdf copy of lecture slides, click here


back to course outline
last update 28-01-2015
Johannes M. Zanker