The GeeksverseLeaving Proof 164 | Storytelling 101: How “too much realism” can undermine immersion in comics art

Leaving Proof 164 | Storytelling 101: How “too much realism” can undermine immersion in comics art
Published on Tuesday, December 18, 2012 by
Is there such a thing as too much realism in comics art? Join us as we explore the “uncanny valley” and related locales in this week’s Leaving Proof. ALSO: The latest trade and hardcover reviews.
Peter Jackson's decision to shoot "The Hobbit: An Unexpected Journey" in HFR has split audiences and critics.

Jackson’s decision to shoot “The Hobbit: An Unexpected Journey” in HFR has split audiences in their opinions of the film.

Peter Jackson’s The Hobbitt: An Unexpected Journey went into wide release last week and not surprisingly, the director’s decision to shoot the film in HFR (High Frame Rate)—specifically, at 48 FPS (frames per second)—has divided audiences and critics, with the prevailing sentiment online at the time of this writing seeming to be that HFR detracts from the viewing experience. I do wonder how many of the people complaining on the Web actually saw the film in HFR—there are only 450 theaters in North America equipped to play films at 48 FPS—and how many are simply extrapolating their misgivings from their experiences with the so-called “soap opera effect” that can sometimes be seen in HDTVs equipped with motion interpolation technology, which introduces a similar artifact of viewing as that associated with HFR films. Whatever the case, the following remarks by Grantland‘s Zach Baron in his review of the film echoes many of the common issues people have found with Jackson’s use of HFR:

Jackson pioneered a new technique in filming the movie, shooting at 48 frames per second, twice the usual speed, which lends an unwanted, disorienting clarity to the proceedings. Much of the film looks like a video-game cut scene; or, more accurately, a movie set on which actors are acting, since you can see with terrible precision the costumes and the makeup and wigs and the fake rocks. High definition has been a miracle for sports and a largely unresolved catastrophe for nearly everything else. Anyone who has ever been on a movie or TV set knows just how much artifice and trickery and elaborate lighting and angles go into making it all look real. Filming in 3-D, and with a higher frame rate, doesn’t enhance that artifice; it exposes it. You can see the paycheck sticking out of Sir Ian McKellen’s beard.

We’ve discussed in some detail part of the reason why footage shot in 48 FPS looks so jarring in this space before, in the days immediately after Jackson screened early footage of the film at the Las Vegas Cinemacon earlier this year. It is my aim in today’s column to expound on those ideas and how they affect our subjective impressions of realism and believability in contemporary representative art, particularly the art of comics.

Sensory perception is a complex, active process.

Sensory perception is a complex, active process.

Sensory perception involves so much more than just “sensing” stimuli. That is, when we perceive something, we aren’t simply taking in features and elements of the stimuli (in the case of vision, the electromagnetic radiation that is visible light) and unifying and interpreting them as a single “percept.” When we perceive things, we actually do what is called “top-down” processing: sensory information is checked in fractions of fractions of a second, consciously or unconsciously, against preexisting knowledge and expectations before an interpretation of the stimuli is formulated. (Keep that part about “preexisting knowledge and expectations” in mind, as it will be important in the later part of our discussion.) There is a strong evolutionary grounding for this phenomenon. We almost always operate in situations where we have incomplete and imperfect information about our environment—our early primate ancestors needed to develop the capacity to fill in the blanks, so to speak, when confronted with, say, a partially obscured figure or muffled footfalls in the thick underbrush of the savanna. Is it a predator? A rock-wielding rival? Prey? Without top-down processing, reasonably accurate identification of the potential threat from incomplete sensory data would be impossible, as would a subsequent intelligent (not instinctual) reaction appropriate to the situation.

The late Will Eisner had groundbreaking ideas about the relationship between sequential art and natural language.

The late Will Eisner had groundbreaking ideas about the relationship between sequential art and natural language.

This is an overly-simplified account of the development and workings of the apparatus of perception of course, but it is detailed enough for our purposes. We mentioned previously that expectations—so-called perceptual set—play an important role in how sensory information is interpreted. The existence and demonstration of perceptual set also implies that to some degree, sensory interpretations are shaped by previous experiences, or that they can be learned. Another implication of perceptual sets and the top-down processing model of perception is that they suggest that we hold abstracted conceptions of objects—what are referred to in cognitive neuroscience and related fields as perceptual categories—in our mind against which we verify and classify perceived stimuli. The late comics legend Will Eisner theorized that what we can now call the perceptual set and categories that influence how readers “read” the sensory information provided by a comic book panel are partially descended from the same underlying structure that governs natural language, an intriguing claim that I considered with some depth in a previous article. Whether or not a visuo-spatial syntax of comics and sequential art actually exists, a number of researchers (Huttenlocher et al, 2000; Feldman et al, 2009; Moore, 2012) have used Bayesian statistics to model the categorical effects in stimulus perception and judgment and their findings point to the possibility that these effects exist in diverse domains, from linguistic processing to discrimination between colors, to the differentiation of faces, and perhaps more.

So what does this all have to do with comics—or representational art, for that matter? Well, if you play video games or watch a lot of CGI-based TV shows and feature films, you’ve probably heard of the concept of the “uncanny valley” introduced in 1970 by Japanese roboticist Masahiro Mori. The following video from Penny Arcade‘s Daniel Floyd does an excellent job of breaking down the phenomenon and how it affects our reaction to representative art, in this case, the art of character design and animation in video games:

Mori also posited that the uncanny valley effect can be seen in still, non-moving near-human artifacts—not as pronounced as that seen in moving robots but still evident—as seen in the graph below:

Response curve showing the uncanny valley dip with regards to still near-human artifacts. Adapted from MacDorman, Minato, et al (2005) from the original by Mori (1970), as presented in Moore (2012).

Familiarity response curve showing the uncanny valley dip with regards to non-moving, near-human artifacts. Adapted from MacDorman, Minato, et al (2005) as presented in Moore (2012), from the original by Mori (1970).

Various explanations have been proposed for the uncanny valley effect, ranging from the existential to the socio-cultural to the evolutionary to ones rooted in categorical perception. While it is likely that no single explanation can completely account for the uncanny valley effect, it is only the elucidation grounded in categorical perception—which says the discomfort and unease (i.e., “creepiness”) of the uncanny valley effect is a manifestation of categorical mismatch and conflicting perceptual cues—that can satisfactorily explain similar effects seen in non-humanoid representations, such as the “so-real-it-looks-fake” sets and props of The Hobbit: An Unexpected Journey and non-animated photorealistic depictions of humans such as those seen in some comics and sequential art.

Consider the following improvised triptych showing, from left to right, a photograph of lingerie model Allessandra Ambrosio, a “photorealistic” sketch of the DC superheroine Donna Troy drawn by comics artist Greg Land that is clearly based on Ambrosio’s picture, and a highly-stylized drawing of Donna Troy:

AllessandraAmbrosio_vs_DonnaTroy

Let us make the assumption that for the viewer, the leftmost and rightmost images belong to different, conflicting perceptual categories—say, “photograph” and “stylized drawing” or “lingerie model” and “superhero”—and that these categories present different perceptual cues and perceptual sets. The first category (photograph/lingerie model), let us designate as the “background category,” and the second category (stylized drawing/superhero), let us designate as the “target category.” The middle image, let us call the stimulus that shares features of both the background and the target categories, giving it uncertain membership in either category.

Saygin et al (2011) were actually able to show via an fMRI study that the human brain shows area-specific, measurable physiological responses to visual stimuli that present categorical mismatches. Moore (2012), working off of Feldman et al‘s (2009) Bayesian statistical model of categorical perception, plotted the relationship between the conflicting perceptual cues given off by a stimulus that is a marginal member of two different categories and the resultant “perceptual tension” and affinity generated by the stimulus. Applied to our example, the effects of Greg Land’s Donna Troy drawing looking both like a lingerie model/photo and a superhero/stylized drawing on perceptual tension and affinity are described by the following graphs:

Differential distortions arising from conflicting perceptual cues in terms of (a) perceptual tension and (b) affinity. Adapted from Moore (2012).

Differential distortions arising from conflicting perceptual cues in terms of (a) perceptual tension and (b) affinity. Adapted from Moore (2012).

The fact that the Land drawing is so obviously based on a background category that carries with it different cues compared with that of the target category gives rise to an increase in perceptual tension and a decline in affinity which can in turn manifest in certain viewers vague feelings of unease or “wrongness,” (i.e., the Land illustration seems to belong to a different context than the one it is being associated with, even to viewers who have never seen the original Ambrosio photograph the illustration is based on). Categories are partially shaped by experience of course, and to viewers who do not make a categorically significant distinction between lingerie models and superheroes or photographs and stylized drawings, or viewers who are unfamiliar with one or the other category, the increase in perceptual tension and decline in affinity may be ameliorated or even non-existent. The key lesson here isn’t that realism in comics art is bad, the lesson is that too much realism and not enough stylization can plunge a comic’s artwork into uncanny valley territory. Sure, an artist can claw his way out of the valley in the forward direction by going for even more extreme realism as predicted by Mori’s graph, but at that point, it wouldn’t be all that different from doing a fumetti-style photo-comic.

In one of the very first articles I wrote for this site, I posted a scan of one of the pages from artist David Mazzucchelli’s afterword to the 2005 edition of Batman: Year One. In light of what we have just discussed, Mazzucchelli’s words and drawings, reproduced again below, take on renewed significance, showing that many of the medium’s best and most experienced artists have a profound intuitive understanding of how the mind processes art.

themorerealisticsuperheroesbecome

The latest trade paperback and hardcover reviews

I’ve spent the past two weeks collating my notes on the trade paperbacks and hardcovers that, for one reason or another, I wasn’t able to post full-length reviews for. I try to write as many full-length reviews of graphic novels and bound comic collections as I can within a reasonable time frame around books’ release dates, but as I’d mentioned in this space before, we are a modest operation here and it is inevitable that some books will fall through the cracks as we schedule readings and reviews. Anyway, I’ve posted “capsule reviews” for these various publications based on the notes I’d written when I first read them, and posted them in the listed articles below. I offer my sincerest apologies to the creators, publishers, and PR reps who provide these books to us for review for the tardiness of the feedback. In the coming year, it is my plan to focus on more concise, multiple item review articles to allow for a wider coverage of releases without sacrificing timeliness or information value.

Trades and Hardcovers released January to May, 2012:

  • Fearless, Vol. 1 (Image Comics)
  • Between Gears (Image Comics)
  • Avalon Chronicles, Vol. 1: Once in a Blue Moon (Oni Press)
  • Abe Sapien, Vol. 2: The Devil Does Not Jest and Other Stories (Dark Horse Books)
  • Adventures into the Unknown! Archives Vol. 1 (Dark Horse Books)
  • Channel Zero: The Complete Collection (Dark Horse Books)
  • Empowered, Vol. 7 (Dark Horse Books)
  • The Foot Soldiers, Vol. 1 (Image Comics)
  • Marksmen, Vol. 1 (Image Comics)
  • Netherworld (Image Comics)

Trades and Hardcovers released June to August, 2012

  • The Activity, Vol. 1 (Image Comics)
  • Baltimore, Vol. 2: The Curse Bells (Dark Horse Books)
  • House of Night: Legacy (Dark Horse Books)
  • After the Fire 100-Page Spectacular (IDW Publishing)
  • Artifacts (Image Comics)
  • Batula (Image Comics)
  • Freaks of the Heartland (Dark Horse Books)
  • The Monolith (Image Comics)
  • Prophet, Vol. 1: Remission (Image Comics)

Trades and Hardcovers released September to November, 2012:

  • The Manhattan Projects, Vol. 1: Science. Bad. (Image Comics)
  • Rex Mundi Omnibus, Vol. 1 (Dark Horse Books)
  • Thief of Thieves, Vol. 1: I Quit (Image Comics)
  • Dancer (Image Comics)
  • Epic Kill, Vol. 1 (Image Comics)
  • Once Upon a Time Machine (Dark Horse Books)
  • Saga, Vol. 1 (Image Comics)
  • Valentine, Vol. 1: The Ice Death (Image Comics)
  • Marked Man (Dark Horse Books)
  • Ragemoor (Dark Horse Books)
  • Trigun Maximum Omnibus, Vol. 1 (Dark Horse Manga)

Note that many of these books are likely to still be on the shelves at your local comic shop or comics-carrying bookstore, and those that aren’t can be backordered via the same or bought through the many online retailer options.

I also have one new full-length review:

Discuss this article below or contact the author via e-mail
6 Responses
Advertisements

Connect With Us!
The Geeksverse on Instagram
Recent Comments