News

Please make an account on the Spring’17 CS 523 Piazza account: https://piazza.com/uic/spring2017/cs523.

Description

This course explores the creative application of new media technologies, including graphics, visualization, audio, machine learning, and interaction technologies. We will focus especially on the use of generative models with which to simulate, accentuate, and ponder the complex systems in our lives. New media artists have traditionally been involved in exploring the creative, cybernetic potentials of new technologies and their cultural ramifications, using creative outputs to question how our awareness and thinking change when we introduce technology into our lives. We are currently in the midst of an explosion of new ideas related to automatic and interactive information processing. Advances in data mining and machine learning are rapidly finding their way into all corners of our lives. In addition to better understanding the nature of these advances (and learning how to make use of them), we will explore how to think critically about their transformative power. We specifically will focus on a class of machine learning algorithms that learn generative models. Students will become familiar with the TensorFlow library for machine intelligence, as well as any other software tools and languages that will help express and document projects (i.e., video software, Matlab, javascript frameworks, etc). In addition to the completion of weekly readings and assignments, students will be responsible for a series of projects that involve the creation, demonstration, and documentation of a series of three projects. Coursework will be organized around the development of these projects, to be completed individually or in small groups.

Schedule

Week 1

— Course overview & policies
— Introduction to generative systems
— Eulerian Video Magnification; lambda vue
— Visual Microphone
— Slides: Week 1

Assignments:

— Read Gardner‘s “Five Minds for the Future”
— Process data of your choice using Eulerian Video Magnification
— Download TensorFlow and associated environment / tools to get it setup on your laptop

---

Week 2

— Survey of creative ML projects
— Slides: Week 2

Assignments:

— Two short assignments due next week: Assignments 1 and 2
— Make sure to have TensorFlow installed and go through first MNIST database tutorial
— Watch first part of Tensorflow and Deep Learning - Without a PhD for a thorough walkthrough of this tutorial
— Read McDonald’s “A Return to Machine Learning”
— Read Amos’s “Image Completion with Deep Learning in TensorFlow”

---

Week 3

— Short project presentations
— Overview of neural networks + different types neural networks
— TensorFlow lab
— Slides: Week 3

Assignments:

— The first project is due in two weeks: Project 1
— Read Alammar’s A Visual and Interactive Guide to the Basics of Neural Networks
— Read LeCun, Bengio, and Hinton’s Deep Learning
— Read Gatys, Ecker, and Bethge’s A Neural Algorithm of Artistic Style

---

Week 4

— Convolutional Neural Networks
— TensorFlow lab
— Slides: Week 4

Assignments:

— Read Karn’s An Intuitive Explanation of Convolutional Neural Networks
— Read Krizhevsky, Sutskever, and Hinton’s ImageNet Classification with Deep Convolutional Neural Networks
— Read Zeiler and Fergus’s Visualizing and Understanding Convolutional Networks

---

---

Week 6

— Recurrent Neural Networks
— Slides: Week 5

Assignments:

— The second project is due in three weeks: Project 2
— Read Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks
— Read Oleh’s Understanding LSTM Networks
— Read Graves’s Generating Sequences With Recurrent Neural Networks
— Read Atken’s Realtime Control of Sequence Generation with RNN Ensembles

---

Week 7

— Generative Adversarial Nets
— Slides: Week 7

---

Week 8

— Project 2 updates
— Applications of Generative Adversarial Nets
— Slides: Week 8

Assignments:

— Project 2 is due next week

---

---

Week 10

— Overview of Project 3
— Slides: Week 10

Assignments:

— Read Goodfellow et al.’s Generative Adversarial Nets
— Watch Goodfellow’s talk at AIWTB

---

Week 11

— Slides: Week 11

Assignments:

— Read Nguyen et al.’s Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
— Read Radford et al.’s Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

---

Week 12

— In-class GAN programming exercises
— Literature survey reports

---

Week 13

— Report on progress of final project
— Slides: Week 13

---

Week 14

— Meet with groups to work on final project

Assignments:

— All deliverables for Project 3 are due next week, along with a summary report of all assignments and projects completed during the semester.

---

---

Required materials & texts

— There are no required texts for this course, but students are encouraged to bcome familiar with a wide range of machine learning techniques. (A list of resources is posted toward the bottom of this webpage.)

— Bring your laptop to class.

— Use a git repo to back-up your files and make your projects/assignments available online.

— Set up a user account on Piazza and join CS 523: https://piazza.com/uic/spring2016/cs523.

Evaluation & requirements

Your final grade will be based on your performance on the group project, evaluations of team member performance (peer evaluation), attendance and participation.

On-time class attendance is mandatory. It is not possible to make up or compensate for missed class sessions. More than two unexcused absences will result in a reduction of the final grade by one letter grade; with every additional unexcused absence, the final grade will drop by an additional grade. Some of the discussions and exercises are done and graded in-class so you must attend class to receive these points.

A = Consistent growth as well as excellent work that consistently goes above and beyond what is required.
B = Above average growth as well as above average work.
C = Average growth as well as average work.
D = Dissatisfactory growth and incomplete work.
E = Dissatisfactory growth, incomplete work, and poor attendance.

The numeric breakdown for the final grade follows:

— Project 1: 10%
— Project 2: 20%
— Project 3: 30%
— Shorter assignments: 20%
— Participation: 20%

Policies

No cell phone usage in the lab. No eating during class (coffee or gum, etc, is fine though).

Any individuals with learning disabilities or special needs must make the instructor aware of them prior to the due date of the first major assignment. Those who require accommodations for access and participation in this course must be registered with the UIC Disability Resource Center.

If you wish to observe your religious holiday that is in the conflict with mandatory academic attendance, you must notify the instructor in advance.

You are responsible for understanding what constitutes academic dishonesty. Academic dishonesty is an extremely serious offense. All cases of academic dishonesty will be dealt with in accordance with the policies of the University.

Resources

Python resources

— Official Python2 and Python3 documentation
— Codeacademy's Python tutorials
— Learn Python the Hard Way
— scikit-learn
— SciPy, NumPy
— TensorFlow, An open-source software library for Machine Intelligence
— Keras Deep Learning Library for TensorFlow

Machine learning resources

— Awesome TensorFlow
— A Visual Introduction to Machine Learning
— Flow Machines
— Learning to Generate Chairs, Tables, and Cars with Convolutional Networks
— How do Convolutional Neural Networks work?
— DeepMind
— Andrej Karpathy's blog, The Unreasonable Effectiveness of Recurrent Neural Networks
— Neural Networks and Deep Learning online textbook
— Machine Learning for Software Engineers
— CHI 2016 Workshop on Human Centered Machine Learning
— Piekniewski's blog, Can A Deep Net See A Cat?, Just How Close Are We To Solving Vision?
— Google Research, Google Brain
— Hubel and Wiesel’s Visual Cortex Cell Recording, 1959
— Visual Analysis for Recurrent Neural Networks
— A.I. Experiments with Google
— Machine Learning is Fun!, parts 1-6
— OpenAI Gym
— Overview of GANs, KD Nuggets
— Deep Advances in Generative Modeling
— Fresh Machine Learning, Sirajology
— Machine Learning Recipes
— Decoding The Thought Vector
— Image Kernels Explained Visually
— A Guide to Convolution Arithmetic for Deep Learning
— The Great AI Awakening
— NIPS papers, Code repos for NIPS’16
— 2016: The Year AI Got Creative
— A Visual and Interactive Guide to the Basics of Neural Networks
— TensorFlow in a Nutshell, Pts 1,2, and 3
— CreativeAI.net
— ML@B blog
— List of GAN Papers and Projects
— Abusing Generative Adversarial Networks to Make 8-bit Pixel Art
— 40 Ways Deep Learning is Eating the World
— Understanding LSTM Networks
— How to build a Recurrent Neural Network in TensorFlow, parts 1 through 7
— Conv Nets: A Modular Perspective
— Understanding Neural Networks Through Deep Visualization
— PixelCNN & PixelRNN in TensorFlow
— Minimal ML Algorithms
— A Gentle Introduction to Backpropagation
— A Step-by-Step Backpropagation Example
— Anyone Can Learn To Code an LSTM-RNN in Python
— Recurrent Neural Network Tutorial for Artists (p5.js)
— Sentdex’s RNN Example in Tensorflow
— StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, code
— Reddit's Machine Learning channel
— Reddit's Machine Learning Questions channel

Creative projects

— Deep Dream: blog, code, gallery
— Neural Photo Editor: article, code, video
— DeepBach: article, code, blog
— Flow Composer: article, AI makes pop music, Composing with AI
— EQ-radio: project page
— Eulerian Video Magnification: project page
— Visual Microphone: project page
— Generating Faces with Deconvolution Networks: blog, code
— Image Compression with Neural Networks: article, blog
— Image Completion with Deep Learning in TensorFlow: blog, code
— Wekinator: project page
— Image Synthesis from Yahoo's open_nsfw: blog,
— Face2Face: Real-time Face Capture and Reenactment of RGB Videos: project page", article
— Nightmare Machine: project page
— AlphaGo: article, project page
— DQN: article, project page, DQN-TensorFlow, video
— WaveNet: article, project page, Tensorflow code
— DeepRhyme: blog
— DeepBeat / DopeLearning: project page, article
— PlaNet: Photo Geolocation with Convolutional Neural Networks: article
— A Neural Algorithm of Artistic Style: article, article, code, Keras code, gallery
— Generating Videos with Scene Dynamics: project page (TinyVideo), article
— Quiver: Interactive Deep Convolutional Networks Features Visualization: project page, code
— One-Shot Generalization in Deep Generative Models: article, video clip, video clip
— Image Inpainting : Overview and Recent Advances: article
— Automated Inference on Criminality using Face Images: article
— Image-to-Image Translation with Conditional Adversarial Nets: project page, article
— Zero-Shot Translation: blog, article
— Generative Visual Manipulation on the Natural Image Manifold: video, project page, article
— Magenta TensorFlow: blog
— Neural Doodle: code, blog, article, article, video
— Four Experiments in Handwriting with a Neural Network: project page + references
— Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks : video, project page, article
— Curated AI: A Literary Magazine Written by Machines, for People: homepage, http://curatedai.com/tags/#deep-gimble-ii
— Traffic Sign Recognition with TensorFlow: blog, BelgiumTS dataset
— Project Dreamcatcher: project page, blog
— Big-Data Poetry: http://bdp.glia.ca/, a crumb of friction milks god
— Plug and Play Generative Networks: project page, article, code
— Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis: article
— Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (DCGANs) : project page, article, code, code, Neural Face
— DRAW: A Recurrent Neural Network For Image Generation: article, video, blog, code, more code

Artist and researcher pages

— Kyle McDonald: A Return to Machine Learning, Style Studies
— Rebecca Fiebrink: website
— Memo Akten: The Dawn of Deep Learning, Algorithmic Decision Making, Machine Bias, Creativity and Diversity, other articles on Medium
— Embodied AudioVisual Interaction: EAVI at Goldsmiths
— Christopher Oleh: website
— Big Picture Group: website
— Jessica Rosenkrantz + Nervous System: talk, studio
— Tom White: Neural Facegrids, Transfers
— Artists and Machine Intelligence: Blog postings

Datasets / APIs

— YouTube8M
— Google Knowledge Graph
— Yahoo's open_nsfw
— Affectiva Emotion SDK (Affdex)
— Norvig’s Ngrams data
— Kaggle Academic Machine Learning Competitions
— ImageNet
— MSRA-CFW: Data Set of Celebrity Faces on the Web
— Flickr8k, images with captions
— WordNet
— MNIST handwritten digits dataset
— UCI Machine Learning Repository datasets — YouTube-BoundingBoxes
— An extensive dataset of eye movements during viewing of complex images, MIT Saliency Benchmark datasets
— Model Zoo
— Google BigQuery Public Datasets
— KDnuggets list of public datasets
— Wikipedia List of Datasets for Machine Learning Research

Related courses

— Parag Mital’s Creative Applications of Deep Learning with TensorFlow
— Udacity's Deep Learning by Google
— Andrew Ng's Coursera Machine Learning
— Philippe Pasquier's Generative Art and Computational Creativity
— Brad Knox’s and Cynthia Breazeal’s Interactive Machine Learning
— Jeremy Howard’s Practical Deep Learning For Coders
— Standford’s Deep Learning Tutorial