Introduction to Computational Media Taught Fall 2007, Fall 2011, and Fall 2012 at NYU/ITP

What can computation add to human communication? Creating computer applications, instead of just using them, will give you a deeper understanding of the essential possibilities of computation. The course focuses on the fundamentals of programming the computer (variables, conditionals, iteration, functions, and objects) and then touches on some more advanced techniques such as text parsing, image processing, networking, computer vision, and serial communication. The Java-based ‘Processing’ programming environment is the primary vehicle for the class, however at the end of the semester, the course offers a peek behind the Processing curtain and directly into Java. The course is designed for computer programming novices. Although experienced coders can waive this class, some programmers use ICM to acclimatize to the ITP approach and for the opportunity play further with their project ideas. Weekly assignments are required throughout semester. The end of the semester is spent developing an idea for a final project and implementing it using computer programming.

Frame By Frame: Creation and Manipulation of the Moving Image – Taught Fall 2008, Fall 2009, and Fall 2010 at NYU/ITP

Thanks to modern day computers and software, we now have a very high degree of control over digital images and video. Non-Linear editors allow us to easily assemble sequential images on the frame level while image manipulation programs give us the power to change images on the pixel level. By using techniques from animation, special effects, video editing, and programming, we break images apart and reassemble them into new moving imagery. Our primary tool is Adobe After Effects but we also explore the algorithms behind image manipulation so that students might integrate the techniques into their own code. Student’s assignments can either be pre-rendered animation, or real- time/ interactive animation. Grades are based on weekly assignments, a midterm project, and a final project. Class participation and discussion are also required. No previous knowledge of After Effects is necessary, but students should be relatively comfortable with Photoshop. Experience with non- linear editing is a plus, but not required. Students must have completed either one animation class, or one post-ICM programming class

Redial: Interactive Telephony – Taught Spring 2010- Fall 2014 at NYU/ITP

New technologies such as Voice over IP, and open source telephony applications such as Asterisk, have opened the door for the development of interactive applications that use telephony for it’s traditional purpose — voice communications. This course explores the use of the telephone in interactive art, performance, social networking, and multimedia applications. Asterisk and low cost VoIP service are used to develop applications that can work over both telephone networks and the internet. Topics include: history of telephony, plain old telephone service (POTS), voice over IP (VoIP), interactive voice response systems (IVR), audio user interfaces, voice messaging systems (voicemail), text to speech and speech recognition, phreaking (telephone hacking), VoiceXML, conferencing and more. This course involves programming with PHP, Ruby or Java. Some proficiency with one of those languages is required.

Do-it-yourself VR (DIY-VR) – Taught Fall 2015 at NYU/ITP

The field of Virtual Reality is changing almost daily. New products and techniques now give us incredible control over the immersive VR experience, and the vocabulary for VR communication is now being written. Powerful game engines like Unity3D have lowered the bar for VR hobbyists and artists, and modern-day cell phones have all of the components necessary to render convincing VR in real-time. This class will focus on experimental and narrative VR projects. We will look at the history of visual storytelling and art, as well as critical analysis of media, as a starting point for conceiving our VR projects. We will continuously compare older mediums to VR in order to best understand what works well in a VR experience. We will also discuss the history of VR, and look at recent VR projects.

The class will cover 3 broad conceptual themes, which will give focus to the projects. The themes are recontextualization, symbolism, and mythology. Some techniques that we will explore are 2D media in a 3D space, 3D world building, VR GUI, and external networked interfaces.

Our primary tool will be Unity3D. Unity allows us to combine a variety of media- images, audio, 3D models, and programming- into a single real-time VR experience. Although VR creation has never been easier, there will be a learning curve for most students. Be prepared to spend time outside of class learning new skills. Because of the diversity of skills required, students are encouraged to collaborate. Success in the class is based on weekly assignments, a midterm project, and a final project. Class participation and discussion are also required. No previous knowledge of Unity3D is necessary, but students should be motivated to explore and learn on their own. Ideally, students will have completed either one animation class, or one post-ICM programming class.