Teaching the Language and Grammar of Film

The past few years, I have gotten back into teaching filmmaking. When I was doing graduate work in media with a focus on film and video, I came to believe that films can be treated as "texts" and that they can be "read" and analyzed, as I had done in my undergraduate studies in literature.

If films can be read like texts, then the language that films use must also have a kind of grammar that can explain its structures.

Roger Ebert used to do "shot at a time" workshop where he would examine a film closely. A film, like a novel, is very controlling. I think a film is even more controlling than a novel. When I read The World According to Garp, I had an idea about how Garp looked. My original sense was he looked like the author, John Irving. But after I saw the film version, Garp became - and still is - Robin Williams. Atticus Finch is Gregory Peck. When you watch a film, you only see what the camera’s eye shows us. The director, editor, cinematographer, actors, set designers, costumers and many others control (and at times manipulate) viewers. 

Knowing about the grammar of film allows you understand how that is done and can give you back some control over the way the film works. Part of the grammar is knowing the reasons why a long shot, medium shot, close up, or an extreme close up was chosen. Studying the language and grammar of the shot, the grammar of the edit will make you consider whether a high angle, a low angle, or eye level is used. Is the camera being objective or subjective? When the camera is subjective, we become one of the characters, and that can be like reading a first-person narrated novel. How does the pace of the edit affect an audience?

Citizen Kane
Citizen Kane offers many opportunities to illustrate film grammar

Every language teacher talks about composition. Every film teacher talks about the composition of shots and scenes. Look at how the director has arranged actors, objects and lighting.

Besides showing and discussing films, to teach the grammar of film you should really have students make films. Otherwise, you are teaching grammar in isolation. I learned through decades of teaching writing that grammar should be taught along with writing. Teaching grammar in isolation is not only boring, it is not effective.

You can start to teach students to make films on paper. Not every teacher has access to filmmaking gear - although today, many students are carrying a video camera in their pocket that is many times more powerful than the Super8 film cameras and video camcorders I first used in classes when I started teaching. Then and now, I have students use storyboarding as a way to really think about shots and angles and building a scene.

A wonderful "side effect" of teaching how to read a film and make a film is that it fosters critical thinking.  

I recently discovered Pixar in a Box which is a behind-the-scenes look at how Pixar artists do their jobs. It allows you to animate bouncing balls, build a swarm of robots, and make virtual fireworks explode. The program connects to math, science, computer science, and humanities in very natural ways. The project is a collaboration between Pixar Animation Studios and Khan Academy and is sponsored by Disney. 

One part of the Art of Storytelling section is on the grammar of film

Basic Shot Types

 

Get Deeper Into This

How To Read a Film

Film Studies

Film Analysis

 

ELIZA and Chatbots

sheldonI first encountered a chatterbot, it was ELIZA on the Tandy/Radio Shack computers that were in the first computer lab in the junior high school where I taught in the 1970s.

ELIZA is an early natural language processing program that came into being in the mid-1960s at the MIT Artificial Intelligence Laboratory. The original was by Joseph Weizenbaum, but there are many variations on it.

This was very early artificial intelligence. ELIZA is still out there, but I have seen a little spike in interest because she was featured in an episode of the TV show Young Sheldon. The episode, "A Computer, a Plastic Pony, and a Case of Beer," may still be available at www.cbs.com. Sheldon and his family become quite enamored by ELIZA, though the precocious Sheldon quickly realizes it is a very limited program.

ELIZA was created to demonstrate how superficial human to computer communications was at that time, but that didn't mean that when it was put on personal computers, humans didn't find it engaging. Sure, kids had fun trying to trick it or cursing at it, but after awhile you gave up when it started repeating responses.

The program in all the various forms I have seen it still uses pattern matching and substitution methodology. She (as people often personified ELIZA), gives canned responses based on a keyword you input. If you say "Hello," she has a ready response. If you say "friend," she has several ways to respond depending on what other words you used. Early users felt they were talking to "someone" who understood their input.

ELIZA was one of the first chatterbots (later clipped to chatbot) and a sample for the Turing Test. That test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human, is not one ELIZA can pass by today's standards. ELIZA fails very quickly if you ask her a few complex questions.

The program is limited by the scripts that are in the code. The more responses you gave her, the more variety there will be in her answers and responses. ELIZA was originally written in MAD-Slip, but modern ones are often in JavaScript or other languages. Many variations on the original scripts were made as amateur coders played around with the fairly simple code.

One variation was called DOCTOR and was made to be a crude Rogerian psychotherapist who likes to "reflect" on your questions by turning the questions back at the patient.  This was the version that my students when I taught middle school found fascinating and my little programming club decided to hack the code and make their own versions.

Are chatbots useful to educators?  They have their uses, though I don't find most of those applications to be things that will change education in ways I want to see it change. I would like to see them used for things like e-learning support and language learning

If you want to look back at an early effort, you can try a somewhat updated version of ELIZA that I used in class at my NJIT website. See what ELIZA's advice for you turns out to be.

 

It's Not the Singularity Just Yet

There were some alerts this past summer that made it sound like an artificial intelligence (AI) system being developed at Facebook was taking over the world. At least that is what some anti-AI folks seemed to be saying. 

The story seemed to be that the AI had started its own conversation between two AI agents developed inside Facebook. They were speaking to each other in plain English. The revelation to the researchers was that because of a mistake in programming the AI had created its own language. It developed a system of code words to make communication more efficient. The researchers shut the system down when they realized it was no longer using English - and they didn't understand what the two agents were saying.

The "singularity" (at least the tech one, not the mathematical or gravitational versions) is the point hypothesized when an upgradeable artificial intelligence will enter a "runaway reaction" of self-improvement cycles. It improves itself to the point of being a superintelligence that surpasses human intelligence. It's when the machines are smarter than us. John von Neumann first used the term "singularity" back in the 1950s i talking about technological progress causing accelerating change.

Why is Facebook messing around with this? For one thing, they want to build chatbots that can have conversations and negotiate with humans in a way that mimics human responses so that they can then make decisions on their own.

Does that scare you?

Facebook was trying to get the chatbots working with a "partner" to divide up several objects that had different numerical points value. That requires negotiation to work out the best way to divide the objects and accumulate the highest possible number of points.

The event is not the first example of AI diverging from its training in English to develop its own language. The new language is nonsense to humans but has semantic meaning when interpreted by AI agents.

chatbotsA chatbot (like the ones shown conversing above) repeating "to me" five times might mean to run a routine five times. It's shorthand. A + B = C is the kind of unsophisticated math we can easily understand, but to the computer the “A” could mean thousands of line of code and that is when we are lost.

It's not that the Facebook chatbots gave up on using English in order to hide from the human observers, it was just more efficient to use another language.

The scary factor is that when Bob the chatbot says "I can can I I everything else” and chatbot Alice replies “Balls have zero to me to me to me to me to me to me to me to me to” we really don't know what they are saying. 

At the OpenAI artificial intelligence lab founded by Elon Musk, they experimented with letting AI bots learn their own languages and it worked. This strikes fear in the hearts of many people, but there's not enough evidence to determine whether AI presents a real threat that could enable machines to overrule their operators.

The team that works on Google Translate believes that the AI created the most efficient solution to some problems.

The singularity is not here yet, but it is coming.

The Information Literacy of Fake News

fake news

Pre- and post-election last fall, there were many stories in all types of media about "fake news." An article in The Chronicle asks "How Can Students Be Taught to Detect Fake News and Dubious Claims?" but I would say that non-students need even more education in this area. Of course, the real question is whether or not this is a teachable skill.

If you had asked me last January to define "fake news" I would have said it was a kind of satire or parody of mainstream journalism. The Onion online, or Saturday Night Live's news segment would fit that definition. Satire always has a bit of truth in it or it doesn't really work.

The Daily Show and Last Week Tonight with John Oliver and other shows and sites have blurred the line. They use real news and sometimes parody it, but sometimes they are closer to investigative journalism. They can edit together clips of a persons inconsistencies in views over the years and create a montage that shows someone who either has a terrible memory or is a liar. It may frighten some to hear it, but many young people and adults list shows like these as their main source for news.

The fake news that is really the focus of attention now are ones (almost exclusively online) that produce wholly fictionalized news stories. Those non-journalistic entities have a very powerful delivery system via social media like Facebook and twitter.

A Stanford University report published last year concluded that many students could not detect fake or misleading information online. They gave students from middle school to college tasks to see how well they could tell a native advertisement from a news article or identify a partisan website as biased or separate a verified social-media account from an unauthenticated one

A larger conclusion I see here is that faculty often assume that young people are fluent in or savvy about n social media in the same way that it is assumed that digital natives know how to use smartphones, websites, photos, video and other digital technology. Bad assumption or expectation.

I remember teaching lessons on determining the veracity of research sources before there was an Internet and after. That has been a part of literacy education since the time when books became more common. I'm sure it was a teachable moment pre-print when a parent told a child to ignore gossip and stories from certain people/courses.

The Stanford researchers said that we need to teach "civic online reasoning" which is something that goes beyond its need in academic settings.

In whose purview is this teaching? English teachers? Librarians? I would say it would only be effective if, like writing in the disciplines, it is taught by all teachers with a concentration on how it occurs in their field.

The science instructor needs to teach how to determine when science is not science. An easy task? No. Look at teaching the truth of climate science or evolution. It is controversial even if the science seems clear.

Napoleon Bonaparte is credited with saying that "History is a set of lies agreed upon." If that is true, how do we teach the truth about history past and the history that is unfolding before our eyes?

But we can't just say it's impossible to teach or assume someone else will take care of it. Information literacy is still a critical, difficult and overlooked set of skills to teach.


Coding as a (second) Language

languages



There is global interest in teaching programming in schools. Initiatives that come from outside education, like Code.org, which is backed by Mark Zuckerberg and Bill Gates, are trying to get more students learning a second (or third) language, but it's not one that is spoken. But I also see a backlash of those who say that writing code is a terrible way for humans to instruct computers and that newer technology may render programming languages "about as useful as Latin."

I support some middle ground. Teaching some coding as part of regular language study in English and world language classes.

This week I am giving a presentation at the NJEDge.Net Faculty Best Practices Showcase that I titled "Code as a (second) Language." It's not about becoming a programmer. Learning about code, like learning about grammar, is about understanding how a system of communication works below the surface.

There are several "computer science, meets humanities" programs. One is at Stanford University, which offers a new major there called CS+X  which is a middle ground between computer science and any of 14 disciplines in the humanities, including history, art, and classics. 

What are the cognitive advantages to learning a second language? Learning any system of signs, symbols and rules used to communicate improves thinking by challenging the brain to: recognize & negotiate meaning, work within structures and rules, and master different language patterns.

As a longtime language teacher - and shorter term coder - I know that code-switching (and that is the term) occurs when a speaker alternates between two or more languages, or language varieties, in the context of a single conversation. That can be done between English and French, but also between English and Java. 

Whether you are working in a traditional language class or a programming class, memorizing rules and learning new vocabulary strengthens overall memory. Multilingual people are better at remembering lists or sequences. Language study & coding forces a focus on knowing important information & excluding extraneous information. We have all heard and read beautiful” and elegant language, such as in a Shakespeare play or great poem, but programmers and mathematicians also talk about beautiful and elegant code and equations.

logoThe conference this week is about STEAM -- STEM plus the arts, including language arts.

Engineering and other STEM subjects are appealing to students in part because they often include hands-on, real-world applications. Many students also feel that these majors lead to better job prospects. Of course, learning to think like an engineer could be useful no matter what students decide to pursue. An increasing number of high schools offer introduction to engineering courses that are project-based, an inquiry-centered. 

There is a Code as a Second Language National Initiative that brings tech professionals and software engineers into schools to introduce students to coding in classes, but also in after-school sessions and events like coding jams. 

This is all great, but my interest here is bring the coding found in STEM courses into languages classes. 

How is a programming language comparable to a spoken language?

My idea is not without precedents. Natural language processing looks at syntax, semantics and models of language analysis, interpretation & generation. Human language technology continues to grow. On a large scale, products like Google and other search tools and Apple's Siri and speech drive commercial uses. The field of computational linguistics is one that grew out of early machine translation efforts and generated mechanized linguistic theories.

There are many programming languages we might use, depending on the grade level and applications. Although JAVA is the most popular programming language, and the AP computer science exam uses a Java subset, it is more than many students will have time to learn. There are coding options that I have written about here for using simpler languages (such as SCRATCH) and tools to aid in writing programs

Although Java might not be the coolest language to use these days, you can do many things with it - including tapping into the current interest by young people for Minecraft. Using mods for Minecraft makes Java more beginner-friendly.  

Language teachers can work with STEM teachers, especially in K-12 schools, to show students the connections between concepts like syntax and help bridge student knowledge of the two fields and also understand commonalties in communications.





The 2016 NJEDge.Net Faculty Best Practices Showcase is a venue to showcase faculty work, work-in-progress or posters to the New Jersey Higher Ed and K-12 communities. Registration and Information on the presentations at NJEDge.net/activities/facultyshowcase/2016/

View the "Coding as a (second) Language" slides via Slideshare by Kenneth Ronkowitz