Do we really understand how moving images work?
Moving beyond the theory of "persistence of vision" into the vast mysteries of the motion picture illusion
You may or may not have heard of the concept of “persistence of vision”. If you have even a passing fascination with film, you’ve likely heard the term invoked and (briefly) explained. This article will cover this concept as well as some lesser known aspects, such as “apparent motion”, “the stroboscopic effect”, and “beta movement”. I’m sorry if this article is a bit dense, but there’s a lot of ground to cover here.
So, how do moving images work?
Well, you’re presented a series of still images in rapid succession, and even though no real movement happens, the still images all played in sequence, at the right speed, takes advantage of something called persistence of vision. Just as images stay in your mind after you see them, leaving an impression, (like after you stare at a light source and it leaves a blob in your vision), so too do film frames leave an impression and so each frame after the next stays in our mind, and we’re able to comprehend it as one constant stream.
Right?
Well, not exactly.
Take a basketball game for instance. It’s the end of the game, the teams are tied, and a player goes for that desperate 3 pointer, and it miraculously makes it. When he puts that basketball through that hoop, you see the ball go through the hoop. You really do. But there’s not a single frame that captures it. The ball never goes through the hoop. So your mind filled it in, it filled in the movement between frames. Except, persistence of vision doesn’t explain this. The fact that we retain images doesn’t change the fact that we never see that ball go through the hoop. So how do we see it? How does the ball seem to move?
“Everyone, it would seem, knows that moving pictures are made by projecting a series of stationary frames on a screen in rapid succession. Yet few people seem to be curious about the basis of this effect and those who are, seem to be satisfied with an incorrect explanation... The fact of the matter is that we do not know why movement is perceived.”
- Irvin Rock: An Introduction to Perception
1
In this day and age, visual media surrounds us and is essentially part of all of our everyday lives. Yet, despite the proliferation and near-omnipresence of this thing we call video, very few little knowledge is widespread about how it works.
When it comes to explaining how the illusion of film and video operates, there is only that one wide-spread explanation: the persistence of vision. This term floats around in the popular consciousness as an acceptable and comprehensible explanation for visual media. If you need an example, look no further than this General Motors ad.
Although most people associate this term with a type of illusion which seems almost miraculous, it nonetheless underplays how complex the illusion is, and oversimplifies our current understandings of human psychology and the nature of perception itself.
When people talk about “persistence of vision” it’s really a type of shorthand for a whole world of strange and mysterious interactions between our minds and optical illusions. Persistence of vision, as it is understood today, is really only a facet of what this phenomenon really is.
Although persistence of vision may seem like a satisfactory explanation to many, once you start looking at the history of visual illusions and our understand of them, things start to become much more complicated. And once you understand how complicated this illusion really is, you can gain a greater understanding of how different forms of technology exploit different vulnerabilities of our perceptual systems. Film, video, and television are not just different types of displays, they are completely different ways of seeing.
2
In 1821, a letter called “Account of an Optical Deception” was published in the Quarterly Journal of Science, Literature, and The Arts. The author observed that sometimes while a wheel rotates, the straight fence slats appear to be curved.
The story of the theory of persistence of vision largely begins with a series of explanations in response for this strange effect by Peter Mark Roget.
The illusion, he says, is best observed when the spokes are in dark color and a bright light shines on the wheel. He equates the effect with that of “a closed circle of light caused by a luminous object being rotated quickly.” If you don’t know what he means by this, think of the way that sparklers and fire works are able to achieve the appearance of a continuous stream of light rather than just looking like random particles of light.
His claims that both of these optical effects are by the way images linger in the retina.
He continues, "an impression made by a pencil of rays on the retina, if sufficiently vivid, will remain for a certain time after the cause has ceased."
This is considered one of the first accounts of persistence of vision. And, for our purposes, the beginning of how this concept intertwines with how we think about cinema.
3
A good place to really start understanding the persistence of vision and related concepts, is the pre-cinema invention known as the Phenakistiscope or the “stroboscopic disk”. This mesmerizing device was one of the earliest examples of moving image illusionism.
It was invented by a Belgian physicist named Joseph Plateau who wanted to demonstrate the ways in which our perceptions of objects can be manipulated into creating illusions of motion. Although Plateau is not as nearly well-remembered as other pioneers of the moving image, such as Edward Muybridge, the Lumiere Brothers, or Thomas Edison, his contributions to how we understand visual media are indispensable, and much of our modern theories rely largely on his work.
Plateau’s most important work began with his dissertation called “Dissertation on some properties of the impressions produced by light on the organ of sight”. This paper laid the groundwork for much of his life work, it contained research he had done on the effects of color on the retina, perceived distortions caused by movement, and, of course, persistence of vision. He was the first scientist to determine that the duration of an impression on the human retina lasted up to 0.34 second.
After he earned his Ph.D., he put his expertise in optics to use by experimenting with illusions involving motion. Inspired by the mysterious wheel effect observed by Roget, he soon developed two unique types of moving image illusions involving spinning disks. Keep in mind this is in 1829, six decades before Muybridge’s famous horse experiment and seven before Edison manufactured the Kinetoscope. In many ways, Plateau is the unintentional and unheralded father of cinema.
The first, and lesser known of his inventions, was called the the “anarthoscope” or “anorthoscopic disk”. The device utilized anamorphsis, a form of media in which an image is only recognizable from a specific vantage point. The disks contained an anamorphic picture which appeared distorted when still, but undistorted when in continuous, rotational motion.
The project was born out of the mysterious wheel effect observed by Roget. Plateau independently noticed the optical illusion of the motionless wheel during his own experiments, and then later read Roget’s article, “Explanation of an optical deception in the appearance of the spokes of a wheel when seen through vertical apertures”. He soon developed his unique Anorthoscope, and when showing it to others, he described it as a “totally new sort of anamorphoses”. He first called it the “anothoscope” in a letter to Quetelet. After sending one of the disks to Micheal Faraday, a prominent scientist at the time, he said,
"it has wonderfully surprised many to whom I have showed it and they all refuse to believe their own eyes and cannot admit that the forms seen are the things looked at"
Plateau’s invention which would soon follow was inspired by an exchange with Faraday. Inspired by Plateau’s experimentation with spinning disks, Faraday began building off his experiments, eventually suggesting a unique viewing method which would prove revolutionary: to view a spinning disk that has regular slits through those slits while looking in a mirror. This method led to Plateau’s most important invention, what many consider to be the first real step towards cinema.
While Faraday’s experiments involved disks with small cut-outs which looked like wheels, Plateau took it one step further and attempted the mirror experiment with a repeated image, “a completely immobile image of a little, perfectly regular horse". After successfully creating a still image via an object in motion, he soon began creating short animations which featured simple, looping motions.
After publishing the results of his experimentations, Plateau soon dubbed his invention “phenakisticope” which was inspired by the Greek root words for “decieving” and “face”.’
Plateau founded his invention upon a conception of the persistence of vision, and thus he is seen by film historians as a type of grandfather, or, even, prophet of the cinema. The discovery of the concept of persistence of vision and the co-incidence with the invention of some of the first moving image technology led to the widespread film history fallacy, in which the two are seen as the same event.
4
The invention of the spinning disk illusion of motion was popular and fascinating, but many wondered how the illusion really worked. In Stampfer’s explanation for the effect, he said,
“In his 1833 patent and his explanatory pamphlet for his stroboscopic discs, Simon Stampfer emphasized the importance of the interruptions of the beams of light reflected by the drawings, while a mechanism would transport the images past the eyes at an appropriate speed. The pictures had to be constructed according to certain laws of physics and mathematics, including the systematic division of a movement into separate moments. He described the idea of persistence of vision only as the effect that made the interruptions go unnoticed.”
Here, the afterimage is not attributed the quality of perceived motion, but rather simply as the way our mind doesn’t notice, or is not bothered by the negative, blank space between images. It’s important to note that afterimage is a retinal phenomena, in overly simplistic terms: it’s actual physiological information that your eye is sending to your brain. The brain of course also plays some role in perceiving the illusion of “persistence of vision”, but in an entirely different sense than in how it perceives apparent motion.
Similarly, a physician by the name of William Benjamin Carpenter, insisted that the illusion was mental, rather than retinal. This is, perhaps, a nudge towards a more accurate naming of the actual effect of the motion picture itself.
Here it is clear that it was not the inventors of the disks nor their scientific contemporaries who invented the illusion who invented the myth of “the persistence of vision”, rather, they used the term to refer to a specific effect. The mythology perhaps, then, begins with film historians, inspired by a misinterpreted quote. Terry Ramsaye and Arthur Knight, two film historians, both write of “persistence of vision” as the founding illusion of the cinema, and they attribute the discovery of this illusion to Roget. We can see this in a quote from Terry Ramsaye’s film history writings,
“The star of motion picture destiny, always traveling westward in the course of the race, reached England with Roget. One day while he was engaged in his inquiry into the affairs of the external senses he chanced to glance from his study window to note the approach of a vehicle. It was only a baker’s cart and Roget hurriedly turned back to his papers. But, as he turned away, the line of his vision swept past the interferences of the slats of a Venetian blind. Through the slitted apertures the scientist caught the impression that the cart was proceeding by jerks. He saw it, despite its rapid motion, momentarily at rest in each slit, and, through each successive opening, he saw it in a different phase of motion.”
Referring back to the wheelbarrow illusion, Ramsay, in 1926, described the moment which Roget witnessed the effect as if it were that legendary apple falling on Newton’s head. But it was built on a fundamental misunderstanding from within Roget’s writing. He he speculates that the illusion of the curved wheel spokes is similar to another phenomenon, which he describes as:
“The illusion that occurs when a bright object is wheeled rapidly round in a circle, giving rise to the appearance of a line of light throughout the whole circumference- namely, that an impression made by a pencil of rays on the retina, if sufficiently vivid. will remain for a certain time after the cause has ceased.”
In an attack on this myth, Joseph and Barbara Fisher wrote “The Myth of the Persistence of Vision. In it they praise Roget’s explanation for the mathematical origin of the illusion of curving, but critique his inability to account for the phenomenon on the whole.
“It cannot be, as Roget suggests, so simple a process as the fusing of slowly decaying tracings upon the retina, for the curved spokes persist even when the eye is moved about over the display. To put the matter another way, when Roget speaks of tracings remaining upon the retina from a sufficiently bright object, he is apparently referring to after-images. Yet it is well known that after-images, since they are in fact tracings of stimulation left upon the retina, yield stabilized images (an image that is on the retina itself and therefore moves as the eye is moved). If after-images were involved in the spoked wheel illusion, the result would be a plethora of images resulting from the tracings scattered about the retina according to each separate fixation of the eye. This is, of course, not what one sees; what is seen is a single unified, symmetrical pattern of curved spokes, no matter how one moves his eyes about. Thus we are forced to conclude that while Roget explained very well the mathematical origin of the curved paths of the stimulus itself, he was unable to account properly for the phenomenal experience as a whole.
It is at the level of accounting for human processing of the stimulus array that he falls short. No psychologist today would attempt to explain the phenomenon solely in terms of processing occurring at the level of the retina. Unfortunately, modern psychologists have not attempted to explain this phenomenon at all. Direct references to Roget’s observation are nonexistent in contemporary literature on perception. [...] The phenomenon which he describes has little, if anything, to do with filmic illusion. Roget has described a case in which a series of moving points results in the perception of a static image. In cinema a series of static images results in the illusion of motion. Nevertheles, Roget’s explanation of ’an optical deception in the appearance of the spokes of a wheel seen through vertical apertures" has been accepted by a generation of film scholars as the valid explanation of the perceptual combination of successive frames of a motion picture.”
So, even if the myth of persistence of vision was later rejected by psychologists and other scientific fields, the question still stands, what is going on? Why do repeated images at a certain speed, under the right conditions, create the illusion of motion?
5
Throughout the first decade of the 1900s and into the second, motion picture film was rapidly becoming more wide spread, and by 1912, buildings dedicated to showing films were accessible to the public in many major cities. But while the illusion founded upon Plateau’s invention was thriving and becoming a whole artform unto its own, some scientists still sought to explain the strange phenomenon, and, in fact, to explain some more fundamental things about the operations of the human mind. These men founded the Gestalt school of psychology.
One of the founders of the Gestalt school, Max Wertheimer, began his studies right around the time of the birth of film, and around 1912, Wertheimer began to adopt the term “Gestalt quality” which was coined by his professor, Christian von Ehrenfels, to refer to the essential nature of a perceptual experience. An example of the Gestalt quality the school was named after is what “allows a tune to be transposed to a new key, using completely different notes, while still retaining its identity.” The Gestalt school saw the tendency to break psychological phenomena down into smaller parts as misguided. They opposed the then-dominant Structuralist school of thought, which used an atomistic approach, in which “all knowledge, even complex abstract ideas, is built from simple, elementary constituents”. The Gestalt school of thought considered perceptual experiences to be more than the sum of its sensory components. Correspondingly, they insisted that psychologists should study the way the perceptual experience appears first as whole, and only then can be divided into parts, “"what is given me by the melody does not arise ... as a secondary process from the sum of the pieces as such. Instead, what takes place in each single part already depends upon what the whole is." The alleged evidence for this concept was largely founded on an experiment by Wertheimer, in which he observed objectless motion, or, as he dubbed it, phi phenomenon.
“The development of his ideas and experiments on apparent motion sound genuinely anecdotal: during a train ride from Vinna to the Rheinland– a long trip which gave him time to think – Wertheimer supposedly, very spontaneously, got off the train in Frankfurt/Main to purchase a toy stroboscope. The following morning he asked the head of the Frankfurt Institute, Firedrich Schumann, if he would be allowed to work in the lab there. The psychologist Viktor Sarris [...] believes to have seen a typical approach of Wertheimer which can be understood as a pattern: first the thought experiment in the train ride, then the reflection with the toy model, and finally the systematic experimentation with the tachistoscope.”
Wetheimer soon began experimenting with projection techniques inspired by the stroboscope and discovered the phi phenomenon. The phi phenomenon, in the broadest possible terms, is a visual illusion in which stationary objects appear to move when presented in a rapid sequence, the illusion has been described as a “disembodied perception of motion”, or a a diffuse, amorphous shadowlike something seems to jump in front of the stimuli and occlude them temporarily.
I have a hard time putting some of this stuff into words. So, allow me to quote the New World Encyclopedia:
“The Gestalt psychologists believed that the apparent movement in the phi phenomenon is due to an electrical charge passing across the brain giving a sense of movement. This idea, and the simple design of experimental apparatus to test it, led to significant research into visual perception. Today, however, the electric charge in the brain is no longer considered a valid explanation of phi phenomena. Scientists do not yet fully understand what causes the illusion of apparent motion with beta movement or phi phenomenon. Still, most theories involve a physiological, rather than psychological explanation, and have to do with the various ways the brain and optic nerves communicate.”
It’s important to understand what this effect is, as it is often misused as a synonym for “beta movement”, which is a separate, yet similar phenomenon, but because “the viewer also perceives two distinct lines” and “not the continuous motion of objects” it is considered a separate phenomenon.
Ultimately the “phi phenomenon” didn’t end up affecting the history of film as much as it did the history of psychology, as it initiated a shift in the way perception was studied. Although phi phenomenon surely has an occasional effect on film viewing experiences, it does not serve as an explanation for how the mind sees apparent motion.
Although the phi phenomenon remains largely unexplanaible, one model for the human mind is able to account for the illusion: the “Hassentein-Reichardt model”. In a series of experiments by two scientists named “Bernhard Hassenstein” and “Werner Reichardt” measured the visual reflexes or “optomotor response” of a beetle.
“This response is the animal's tendency to follow the movement of the visual surround to compensate for its mistaken perception of self-motion in the opposite direction. The beetle was glued to a rod so it could not move its body, head or eyes relative to the surround, but could express its behavior at decision points by rotating a 'Y-maze globe' under its feet [...] Their results led to the development of a model for motion detection that became known as the 'correlation-type motion detector', the 'Hassenstein-Reichardt model' [...] The core computation in this model is a delay-and-compare mechanism: delaying the brightness signal as measured by one photoreceptor by a low-pass filter and comparing it by multiplication with the instantaneous signal derived from a neighboring location. Doing this twice in a mirror-symmetrical fashion and subtracting the output signals of both subunits leads to a response that is fully directionally selective. The strict mathematical treatment of this model led to many counterintuitive predictions, which nevertheless were experimentally verified in many species' behavior and in many types of neurons. For example, the model predicted that the response, unlike a speedometer, should not increase continuously with increasing velocity; instead, going beyond an optimum velocity should decrease the response.”
What Hassensetein and Reichardt were suggesting, in more simplistic terms, was a model of visual system which detects motion by way of correlation. The mind perceives motion not by simply seeing things move, by by detecting “a cross-correlation” of light intensities from neighboring points. In short, suppose a detector that is perceiving motion and thus two visual inputs are received, and two signals correspond. The detector compares the signals and generates a direction sensitive response. This concept of the visual system is thus creating the perception of motion by way of correlation between point
This would potentially explain the reason why the mind supposes a relationship, or motion occurring between the two flashing points in Wertheimer’s experiments. But the field of motion perception is still being intensely researched and while the Reichardt-Hassenstein model has seemed to be proven to be accurate in certain experiments, it has not been proved on the level of anatomy of anatomy or physiology and remains just that: a model. Therefore it is perhaps still too early to lay to rest the mysteries of the phi phenomenon.
6
So, just to catch up, when the human eye is exposed to a series of images with strobing light at a frequency of 60 flashes per second, the mind utilizes something called persistence of vision to essentially remove the strobe from your vision. But this technique could be just as easily be the appearance of a perfectly still image as an apparently moving one, so what is this other thing which appears to make it move?
The answer, some claim, is Beta Movement.
Before the Gestalt school, an Austrian scientist named Friedrich Kenkel began experimenting with light, and noticed that he could create an uncanny optical illusion. In 1875, about fourty years after Plateau invented the phenakistiscope, Kenkel discovered an optical effect caused by electric light. He found that, “under the right conditions, people will see two quick, spatially separated but stationary electrical sparks as a single light moving from place to place, while quicker flashes were interpreted as motion between two stationary lights.” What’s significant here is that viewers aren’t seeing that same type of objectless motion they were with the phi phenomenon, they’re actually seeing what appears to be the spark itself moving from one place to another. It’s also highly significant that it’s an illusion being created via electricity, and is therefore considered by some to be the foundation of video
During Wertheimer’s experimentation, he created a similar illusion, which at the time he seems to have referred to as “optimal motion”. This effect was created by using a tachistoscope, a projection device whose functionality is focused on displaying images for specific amounts of time, and usually used slides in combination with camera-style shutter systems. In the experiment, two figures in two different positions were shown, and when shown at a medium speed, the figure seemed to be moving from one position to the next. It was at the higher speeds, he found the aforementioned phi phenomenon.
In 1913, a scientist by the name Kurt Koffka, who had been one of Wertheimer’s test subjects, begin doing subsequent experiments, working with another scientist Friedrich Kenkel. It was Kenkel who finally made the designation of “beta movement” as that specific phenomenon where the viewer perceives the illusion of an object moving from one place to the next. And it is this “beta movement” which is now attributed as the reason for the central illusion of the motion picture, and of video.
But what really distinguishes the two, on a physical level? What’s actually happening in our visual systems? It’s important to reiterate that the process is still largely mysterious, but that doesn’t mean that there aren’t plenty of theories and research going on still to this day.
But much of this subject is buried within dense scientific and academic texts, to the point where I find it difficult to find accessible and palatable resources. So, again, it is important to re-iterate that beta movement is still a largely mysterious phenomenon.
7
“We read a student paper, and we cringe. We attend the lecture of a seasoned film scholar, and we cringe. We cringe not only because they have chosen to perpetuate the notion that persistence of vision but they apparently, even at this late date, do not understand it’s implications.
By this time most film scholars seem to have heard that the term ‘persistence of vision’ is inadequate [...] why are film people so reluctant to let go of this notion? Those engaged in film study cling to persistence of vision because they need it. For film scholars, it is our myth of creation.
It answers our central question of origin: why, when we look at a succession of still images on the film screen or TV set, are we able to see a continuous moving image? We answer ‘Persistence of vision’.
Persistence of vision is the name given to the miracle by which the silver halide dust of photography is transformed into palpable, living motion. And just as the story of Adam and Eve explains not only the mechanism by which people originated and reproduced, but also the relationship of human beings to God, the myth of creation for the motion picture explains not only the mechanism for the origin of motion but the relationship of the film to the viewer.
The viewer implied by the Myth of Perisistence of Vision is a passive viewer upon whose sluggish retina image pile up.”
- Joseph Anderson and Barbara Fisher, “The Myth of Persistence of Vision Revisited” (1993)
The goal of this article was to make you understand motion pictures less rather than more. Because motion pictures really are still a mysterious thing. And although “beta movement” seems to be the more accurate description of the impetus behind the motion picture illusion, it doesn’t really “explain away” much of anything at all. It’s just a more precise term for this strange phenomenon which surround us now. The idea of motion picture film creating the believable illusion of motion when shot at and presented at the rate 24 frames per second was one which came through practical trial-and-error rather than an inventor stumbling upon some type of monolithic understanding of human vision.
So, to be perfectly clear, persistence of vision is an incredibly important, if overemphasized, concept. It does indeed explain a fundamental concept of motion picture film. The shutter in a film projector which covers the screen in darkness while one frame is “out of place” while transitioning to the next is totally unnoticeable to the human eye. This is due to persistence of vision.
Persistence of vision probably impacts your daily perceptual experience in ways one doesn’t really take note of. But digital video really doesn’t make much use of it, as there is no shutter, and no need to fill the screen with darkness while moving from one frame to the next. Nowadays, frames are represented by data which tells machinery how to arrange tiny lights in a pattern which resembles said frame, then the data simply tells the machinery what lights need to be changed to represent the next frame. So video really solely relies on beta movement to achieve the affect of the motion picture illusion. Which means beta movement is all around us. Yet still, it seems as if we have a long way to go before we can obtain a comprehensive knowledge of this phenomenon.
I think there’s a bit of a wider lesson to be learned here. We humans fancy that when we’ve created a piece of technology that we have, in some way, mastered the phenomenon which that technology exploits. But in reality, an inventions can be created with some knowledge and can easily inspire some more knowledge, without the inventor ever really picking up some true understanding along the way. Technology, more than anything else, is centered around usage. And if we can get usage out of a motion picture medium, we will. We won’t wait to truly understand it before using it. I think a similar analogue can be found in the hot-button topic of Artificial Intelligence. We’ll keep improving this technology until it’s supremely usable, and a century later there will still be some people trying to wrap their heads around really understanding how the thing works.
Maybe we (as a society, as a planet) should try to understand the things we use a bit more before we use them, and I mean seriously understand them, down to the physics.
Or maybe, we should just be more honest with ourselves. We’re seeing if things work first, and trying to understand them second. And the understanding part takes far longer.
Ultimately, it’s important to remember that even our ways of trying to understand things is based on effectiveness and usage, and truly understanding something (anything!) beyond it’s usage is likely an experience few human beings ever have.
8
The philosopher W.V.O. Quine wonderfully addresses the ways in which the observation really operates in practice. He is wonderfully honest about the limitations of the human mind, and, consequently, science itself. I’d like to quote him here, as I think we all need to adopt a bit of his kind of humility into our quests for knowledge and understanding.
“As an empiricist I continue to think of the conceptual scheme of science as a tool, ultimately, for predicting future experience in the light of past experience.
Physical objects are conceptually imported into the situation as convenient intermediaries—not by definition in terms of experience, but simply as irreducible posits comparable, epistemologically, to the gods of Homer.
For my part I do, qua lay physicist, believe in physical objects and not in Homer's gods; and I consider it a scientific error to believe otherwise.
But in point of epistemological footing the physical objects and the gods differ only in degree and not in kind.
Both sorts of entities enter our conception only as cultural posits.
The myth of physical objects is epistemologically superior to most in that it has proved more efficacious than other myths as a device for working a manageable structure into the flux of experience.”
- W.V.O Quine "Two Dogmas of Empiricism"
9
Let’s not forget this simple thing.
That art is an incredible gift. And that it’s a type of miracle that art forms even exist.
That if they’re not miraculous, then they’re at least deeply mysterious.
We don’t know the why of art, music, film, or expression, because knowledge can never really catch up to experience.
"What we regard as expression consists of incalculability."
- Ludwig Wittgenstein