NYU 9th Floor Talks: The Cutting Edge of Game Research

Ken Perlin, a Computer Science professor at NYU, presented some of his cutting edge research in procedural animation and education games.

Presentation

• Ken is the founding director of NYU's Media Research Lab and the Director of the Games for Learning Institute. He is a mathematician who has won an Academy Award for his noise and procedural texturing techniques that have been used in film and television.
• Ken started his career in computer graphics. Being in the industry for so long, he has experienced the power of Moore's law, seeing computers get twice as faster every two years.
• He has developed several hundred Java applets. Some of his work can be found on his NYU homepage (http://mrl.nyu.edu/~perlin/).
• When researching something, you have to boil it down to the simple possible version.
• One of Ken's biggest research interests has been conveying emotions through artificial characters.
• In Polly's World (http://mrl.nyu.edu/~perlin/experiments/polly/track.html), Ken tries to convey emotions with the least possible number of vertices and polygons. The procedurally animated character is made up of only 6 vertices, but displays a wide range of emotions through its movements. Our brain maps simple animations to emotions.
• Why do we care about animated characters? Ken researches the human body as an instrument to understand psychological emotions.
• In Responsive Face (http://mrl.nyu.edu/~perlin/experiments/facedemo/), Ken researches what's the simplest emotive face possible. At the very least, an emotive face requires movements of the eyebrows, lid, gaze, head, and mouth. A person has n-points in the face like a keyboard has a number of notes and facial expressions are like chords.
• Micro noise movements and procedural variations makes the emotions more realistic. Without these micro movements, the face is just a lifeless still picture.
• Autistic children have trouble looking at other human's faces and thus grow up not knowing how to read emotions. However, they don't have that problem with artificial, computer-generated faces. Many autistic children used Responsive Face to learn human emotions.
• Plan 9 From Outer Space, often regarded as the worst movie ever made, is almost unwatchable because the acting is so bad. The actors' faces are so emotionless that it's impossible to identify with the characters.
• In an attempt to create an "interactive Pride and Prejudice," Ken developed an applet with five birds with disembodied feet. The birds showcase procedural walking animation, which is tons better than canned animations.
• We should think of avatars the same way a director thinks about actors in a live play. We shouldn't communicate actions to the avatar (ie. "jump here", "punch that", "open this door"), but communicate emotions (ie. "you love her"). Likewise, actors shouldn't indicate (ie. "I am feeling happy!", "I feel very sad."), but actually emote.
• With Fish Tales (http://cims.nyu.edu/~perlin/fishtales2/), players can record a one-minute story using Bob the fish and two bouncing balls.
• In Candy Circle (http://mrl.nyu.edu/~perlin/candycircle/), players make procedural music using a spinning wheel filled with candy. It researches the rule of the fifths; when the further away the chords are, the more dissonant the sounds are.
• Ken also experimented with spiral escalators (http://mrl.nyu.edu/~perlin/escalator/). Spiral staircases always go right going upwards because they were designed solely for battle. Since most knights were right-handed, it would give the advantage to the person who's above.
• The barrier of entry to making games is so high. Ken would like to have more people coming into the arts that are enabled by programming.
• There are "fake languages" such as Code Do It and Scratch that enable people to jump into programming. Ken is researching how to make programming more integral to education.
• Ken's New Line Fractals (http://mrl.nyu.edu/~perlin/newlinefractal/) show mathematical visual examples of fractals. Students can move vertices around to experiment with creating their own fractals.
• Using a real-time interpretive version of Java, Ken created Flower, in which students can directly edit the code and immediately affect the flower. In Musical Rubber Ducky, students can affect the ducky but also the music too using live code. Can we invite them to the narrative as well?
• Ken developed a Pride and Prejudice reader. Before the invention of books, literature used to be written on scrolls. Someone later introduced the concept of pages, a fixed length record that changes per edition. But writers wrote and thought in terms of sentences and paragraphs, not in pages.
• In this interactive map of Pride and Prejudice, you can see the entire book's topology. You can easily identify the key chapters of the book using filters and highlights.
• Asteroid is a game that topologically and mathematically exists on a torus or a doughnut.
• The second book to use the interactive map is Winnie the Pooh. This version also includes a procedurally animated bear that users can play around with using live coding. The code editor includes tools that enable learning programming easier. When you insert a 2D array, a map will appear that lets you set dots and automatically generate the points in the array for you.
• Everyone should be able to program. It should be as accessible as painting, filming, and writing. People need immediate feedback to instigate learning.
• Ken is really interested in how to power up this maker culture.

Question and Answer

• Learning tools should be like Google Docs in that they are collaborative, shared documents. People should have conversations with each other about creation. Learning is a performative activity.
• Just like how jazz is to music composition and improvisation is to acting, we need live perfomative structures for coding to learn programming.
• What's the difference between games and simulation? Games have goals.
• We need to increase programming literacy and convey programming as a liberal art. Kids need to look at code in their early stages.
• Ken remembers reading a children's book when he was a child and at the end were the advanced notes for the teacher to read. Although Ken couldn't read the notes as a child, he understood that that's what he was learning and that he'll eventually be able to read the advanced text. Kids shouldn't be using a fake language, but rather be looking at what the grown ups are using. They should be exposed to C++ or Java-like languages.
• Kids are learning reading through other subjects such a history and science. We should teach coding gradually in the same way.
• Many people are reluctant to learn coding, but that's okay. We should focus on teaching the kids. We don't currently live in a society that requires that level of technical knowledge, but this might change ten years in the future.
• Excel is a programming environment that many people use, despite not being programmers. Max/MSP is a programming language that non-programmers such as visual and sound artists use. Both of these programs are always running live and give immediate feedback. There's no intermediate building or compiling stage.
• People don't necessarily need to learn C++ or Java the same way that not all writers have to linguists. Guitarists don't need to learn how to make guitars.
• Programmers have a sense of being macho. Music, on the other hand, is more inclusive. We need a real-time sharing structure and a more inclusive environment for non-programmers.
• Will books become non-linear in the future? This may very well happen, but it won't replace the books as we know them. Different media can coexist the same way that different instruments can coexist. It's not all about hacking and recombinations. Technology is the wrong word to describe media; these are instruments.
• Will the secondary functions of the interactive reader overshadow the literary work itself? The work lives on if they're great. Beethoven lived on when the Beatles came around and the Beatles didn't go away when Lady Gaga came around.

NYU 9th Floor Talks: From Boticelli's Venus to Super Mario

Chris Solarski, author of Drawing Basics and Video Game Art, gave a talk at NYU about how video games can learn from the foundations of art.

Presentation

• Chris was a 3D artist at Sony but always wanted to be a concept artist. He quit his job and studied traditional art and painting for two years. He later worked at Gbanga.
• He was selected by the Swiss Arts Council to present at the Game Culture Conference and argue for games as an art form. It was the first time for him to think about game art and fine art as the same.
• His book, Drawing Basics and Video Game Art, covers character design, environments, fundamentals of art, and many other topics. The constant thread throughout is emotions.
• The fundamentals of art such as classical composition, volume, light, perspectives, and proportions can all be applied to video games.
• Many classical artworks have hidden composition lines that dominate our eye flow. For example, Johannes Vermeer's Diana and her Companions uses a circular composition to guide our eyes towards the soft nature of the work. On the other hand, Peter Paul Rubens' Massacre of the Innocents uses very angular and diagonal lines to convey aggressiveness. Emotions are communicated by displaying composition over details.
• Edgar Degas's Waiting shows a good contrast between a circular composition and an angular one.
• "Reality can be reduced to simple shapes."
• Games, unlike fine art, have dynamic composition. While fine art captures a moment in time, the composition of a video game is constantly changing based on the player's path.
• There are many ways emotions can be communicated.
• By level design and pathways
• Gears of War is a very aggressive and angular game. The way the characters move, the sudden movements of the camera, and the level design and pathways are all very angular. The level design include many corners and sharp turns.
• By character movement
• Vanquish is a very angular game in its character movements.
• Journey, however, uses circular composition in its open canvas and the way the characters move.
• By character and environment design
• Sword and Sworcery uses an art style that's halfway between round and angular. The use of rectangles communicate tranquility.
• Mario uses a sphere concept to denote that he's a good guy. His head, nose, stomach, hands, feet, and even his mustache are composited using circle shapes. Wario, on the other hand, has sharpening details to make him a more aggressive character. In the Mario games, many enemies have spikes and triangles to indicate their evil nature.
• By the relationship between character and environment
• In Chris's own game, MORF (http://www.solarskistudio.com/games.html), there are only two levels with no technical differences between them. There are only aesthetic differences in that the first level, everything is rounded, and in the second level, everything is angular. However, playtesters treated the second level as dangerous and even said "ouch" whenever they run into the spikes.
• By narrative and character development
• Link in The Legend of Zelda: Ocarina of Time always stands and moves in the same way the entire game. There's nothing about his movement that shows you he's becoming more of a hero. However, his heroic progression is shown through the UI icons. As he becomes stronger, he will have more options and icons in his inventory.
• Eugene Delacroix has said that we're not all flat, we're always changing every minute.
• We're too focused on creating one experience (aggressive vs. passive) that we're not focusing on character depth.
• Journey does a decent job of conveying emotions through character movement (walking vs. running vs. walking against the snowstorm).
• By player's movements
• Motion controllers, camera, and touch use the physical body language to control the game. This revolution bring artist collaboration between the developer and the players.
• Mario Kart has smooth turns while Tron Revolution has sharp, abrupt turns.
• A game designer is like a music composer.
• Immediate actions require immediate communication.
• The three most basic shapes are circle (representing softness), squares (representing stability), and triangles (representing danger and aggressiveness).
• In The Lord of the Rings, the Hobbit homes are very circular, the castles and the forests are very rectangular, and the towers and Sauron himself are very angular. Even the volcano is the shape of a triangle.
• Industrial design of cars also take into account of these three simple shapes. Compare a Volkswagon beetle vs. a minivan vs. a Ferrari.
• We do judge books by its cover. We think of speed, stability, or softness just by looking at its shape.
• GlaDos from Portal is a circular character, yet she turns out to be evil. She is a very deep and complex character. Her design was inspired by Boticelli's Venus when turned upside down. You can see the zig-zag hair and the opposing curves.

Question and Answer

• Lines, shapes, and composition all have powerful influences on emotions. So do complementary and conflicting colors. Mario's colors are near complementary, while Wario's are super contrasting.
• Halo's characters move in circular, slow arcs, while the enemies are more angular and aggressive.
• Pixar's Wall-E has a slow, lonely introduction and a large chase at the end of the film. Games are often the opposite with the chase at the beginning and the slow grind afterwards.
• The Art of Journey is a wonderful book that covers level design and color composition of Journey.
• In-game photography is not a surprising trend to come out of video games, since many game environments and composition are very beautiful.
• The talk reduced emotions to three fundamental shapes, but reality is much more complex. Designers should find inspiration from outside world and real experiences to drive their design.

Anna Anthropy's Book Release and Lecture

Anna Anthropy, also known as Auntie Pixelante in the blogosphere, is the creator of video games such as Calamity Annie, Mighty Jill Off, Lesbian Spider Queen of Mars, and most recently, Dys4ia. She recently released her book, Rise of the Videogame Zinesters: How Freaks, Normals, Amateurs, Artists, Dreamers, Drop-outs, Queers, Housewives, and People Like You Are Taking Back an Art Form. As part of her book tour, she held a talk at the NYU Game Center curated by Frank Lantz, where she discusses her advocacy of game creation by anyone, the games that are most interesting to her, and her identity as a queer transsexual game author. 

Conversation with Frank Lantz

• She refers to the game industry as an engine that speaks only to itself. It's a male-dominated, market-driven, violence-focused industry that continues making the same types of games. Since it's mainly gamers themselves who enter the industry, it becomes a perpetual cycle of recycling game ideas and engaging in the same conversations.
• Current game education such as those in Digipen, Full Sail, and more specifically, Guildhall at Southern Methodist University in Texas are "essentially useless." They reinforce the industry myth that getting a job in a big developer or publisher is the only avenue to get into games. People can definitely make games on their own or in a small group, without much technical knowledge. Anna herself left Guildhall and made Calamity Annie on her own.
• Game development is not monolithic. Everyone should dabble in games even if they're not familiar with gaming at all. There are so many game editors and tools that allow people to create their own games without any technical knowledge. These include Gamemaker, RPG Maker, Knytt Story, and ZZT (a text adventure creator).
• Frank attributes NYC's punk gaming scene to Messhof and Anna Anthropy, two very distinguished creators and voices in the community. But both moved out to the west coast, much to Frank's dismay.
• Games that are less polished and less thought out are more interesting and speaks more about the creator himself. Games that digress from the mainstream tend to be more weird and engaging. For example, Cactus Block is a platformer where the player places the platforms with the mouse. However, in quite possibly a design decision that took 5 minutes, a cactus could randomly appear instead of a platform. This design choice, however, makes the game and the dynamics a lot more interesting. It becomes a game of chance and the player has to second guess every platform that he places. The player has to plan carefully to accommodate for potentialities, maybe clicking in a less optimal place so that a random cactus wouldn't appear and block a crucial location.
• There is a story to Tetris, whereby errors compound into more errors until the builder is overwhelmed. This is a story authored by its mechanics.
• The current state of gaming requires an advanced gaming literacy. An XB360 controller has 10 buttons and 3 directional controls, which requires many years of experience to pick up or a high learning curve. People should forget about hardcore games and just pick up something easy and responsive (casual games, indie games).
• Anna as a creator wants to do everything herself during development of a game (art, music, programming, design, writing), but finds it increasingly hard to do as games get bigger.
• She sold Dys4ia to Newgrounds for a rather low price, because she thought she would not be able to make money off this game otherwise. However, it became her biggest hit. Fans were so receptive of it and sent her completely positive feedback (although there was some share of hate mail and death threats as well, which is "par for the course"). Dys4ia is a Wario Ware style game that chronicles her experience of taking hormones as a transgender. This game is unique in that it is quite possibly the first purely autobiographical game made, or as some people say, a diary entry in the form of a game.
• There was a larger presence of outsider artists at GDC. Pirate Kart is a compilation of over 1000 indie games and was submitted as an IGF entry to bypass the submission fees. Games were being made at GDC and were added to the Pirate Kart compilation on the spot.
• She calls IGF awards an "absurd and grotesque spectacle." Games don't fit comfortably in an annual format, since games take years to make. The IGF award ceremony itself was so bizarre. Once, Jessica Chobot ran a skit where she made fun of indie developers, not in a laughing-with-them kind of way, but quite viciously.
• Frank mentions that the awards given at the end of a Global Game Jam event are his least favorite part, but he understands the need that people have for recognition.
• Anna says that the way that Ludum Dare handles competition is the best. If you participate in the jam, you can get awards and you can give awards. Those that don't participate don't get to vote or win. The awards can be anything; Anna herself got an award for "Best Monsters."
• "Games industry is haunted by the spectre of monetization."
• Indie Game the Movie had a weird part in the beginning where game journalists were interviewed. They each gave praise to the games in the movies, ie. "Braid was the best indie game of the year. It made so much money." It's weird that we always want discrete outcomes. We want to be told "This game is quantifiably better than that game." But scores and grades are holding us back as an industry.
• Anna advocates the creation of games by everyone and anyone. Frank, playing devil's advocate, asks if there is any value to that. If someone who's never listened to music before says, "I have an idea for a great band," it's definitely a fruitless endeavor and the end result would be terrible. Anna argues that we shouldn't fall into video game tropes. Games should be and can be anything.
• One triple A game that Anna likes is Saints Row 2. The game has the best character customization, where the player can do anything and be any gender. The player can be a masculine guy in a dress or a hot girl with a deep voice. This is closer to the real world where anyone can be any gender as well. Developers shouldn't lock out players from certain choices, whether it be fashion, demeanor, voice, etc., based on the avatar's gender. The developers of Saints Row 2 consciously decided to never gender the player. In one scene of the game, during a heist, an abstracted look of the map layout was shown. All the enemies were shown on the map as an icon of their faces, but the player was shown as a chess piece, thus being genderless.

Q&A

• How do you make personal games in a team? As a creator, you have to describe your vision to the team members. It's not bad to merge collaborator's personalities. It becomes a conversation between collaborators. Anna, however, admits that she likes hierarchy and likes to boss her collaborators.
• There is an expressive power of creation as an act in and of itself. Indie games are more personal and shows more of the author.
• There is a divide between games made for the player (like Street Fighter or Starcraft) and games made for the creator (like Jason Rohrer's games).
• Portal antiseptic theme is interesting because it expresses how Valve as a developer makes the game. They put testers in a lab, test the game repeatedly, and change everything that they're doing wrong. This is analogous to Portal's story.
• Anna previously worked on a multiplayer game called Mindfuck. It's a two-player game where each player has one button. Points increase as long as neither player presses their buttons, but the first person to press it wins all the points. It is a game about gambling and predicting your opponent's moves.
• Anna is presenting a new game at Babycastles at the Project Secret Robot location. It is a 4 player game where 3 players have cyanide pills and the other player plays as a vampire. The 3 players have to predict who the vampire will attack next. The 3 players wear bras with a button hidden inside them. At the end of the round, if they've pressed the button an odd number of times, they don't take the cyanide pills, but if they've taken it an even number of times, they take the cyanide pills and kill themselves before the vampire gets to them. The point is to overload the vampire with information to confuse him in attacking a suicidal victim.
• Anna's definition of game is "an experience created by rules." She wants the definition to be broad and all encompassing as much as possible. We shouldn't divide digital games from physical games, entertaining games from those that offer negative emotions.
• "PAX is a place where the audience and industry meet to confirm that the status quo is good and to keep with the status quo."
• Anna wants more discussion about gender or about queers in games.
• She wants games to be more like Youtube. With Youtube, everyone and anyone can make a video. They didn't need film education or knowledge of film history. They just needed a webcam. Although most of Youtube is crap, there are gems that pop up from the sea. We don't know what games would look like at the hands of a large group of creators.