ECE 401 Final Project 2017

The final project in spring 2017 is to create a video, using at least FOUR SPECIAL EFFECTS based on at least FOUR DIFFERENT LABS from this semester. Possible special effects you could use include: (lab 1) a cartoon drawing based on a sine wave or a moving circle, (lab 2) a melody synthesized from pure tones, (lab 3) an audio recording matched to its own spectrogram, (lab 4) an image upsampled by hand, e.g., you might get interesting visual effects by gradually increasing the upsampling rate without doing any interpolation at all, (lab 5) a cartoon showing a signal getting convolved with some impulse response, (lab 6) audio reverberation, (lab 7) a cartoon showing the magnitude spectrum or level spectrum, e.g., you might show the level spectrum of an audio signal changing from frame to frame, or you might show how it changes with different types of filtering, (lab 8) effects that result from adding two sinusoids together, e.g., beating of one tone against another; or, effects that result from windowing a signal, e.g., show differences in the sensitivity of the DTFT with different windowing methods, (lab 9) gradual reduction of pure-tone noise, e.g., of 60Hz hum, by notch filtering the signal, (lab 10) lowpass or highpass filtering of an audio signal using a one-tap Butterworth filter. You might need to filter multiple times to get an audible difference. If you compare the results of filtering ten times forward in time, versus five forward+five backward, you might be able to hear the difference: one might sound differently reverberant from the other. Feel free to come up with special effects that are completely different from those I've specified in this list, but in each case, be sure that the special effects section of your final project report specifies the lab from which each special effect was drawn.

Deliverables

By May 9, 2017, upload to Compass a zip file containing the following:

The Video: Your video, in mp4 format (check with me if you want to use any other format). The video should be at least one minute, and no longer than seven minutes. It can be a documentary on the subject of the special effects themselves, a documentary on any other subject, or a fictional or nonfictional narrative of any kind; it's up to you. Here's a partially completed example.
The Shot List: A plaintext file in a sort of tabular format, giving me your detailed edit list. This should be a list of the audio components that were added into your soundtrack, and a list of the image components that were added into your video. It should specify the frame rate of your video (frames/second). For each audio and/or video component that is separately generated and/or separately sourced in your code, it should then specify: (1) the time at which that audio or video component begins, in frames or in seconds, (2) duration (in frames or in milliseconds),(3) the line numbers, in your source code, at which that component audio or video file is read into Python, processed, and copied to the output audio buffer or to output image files, and (4) the name of the audio or image special effect applied to that component, if any. Here's a partially completed example.
Source Code: the code you used to create and sequence all of the audio and video components. This code should read in whatever sources you used, process them, and it should write out a single audio file of the appropriate duration, and a sequence of separate image frames and/or a video file. Here's a partially completed example.
Final Project Report: at least two pages, and not more than six pages. This should contain the following sections:
1. Abstract: describe the content of this report.
2. Plot Synopsis: summarize the content of your movie from an artistic or documentary point of view. Give the key plot elements.
3. Special Effects: provide one paragraph for each of your four special effects. (1) First sentence: specify from which lab the special effect is being drawn. (2) Second sentence: give a qualitative description of the type of visual or audio effect you were trying to achieve, and how you thought the equation(s) in this paragraph would help you to achieve it --- even if it didn't succeed, please tell me what you were trying to accomplish. (3) Provide at least one equation, centered on its own line, describing how the special effect was implemented, in equation format. The equation should use single-letter symbols for each of the signals involved, do not use whole-word or multi-character symbols, because that will be marked as wrong (but single-letter symbols with subscripts and superscripts are OK). (4) The text immediately before or immediately after the equation should give a definition, in words, of each of the single-letter symbols used in the equation, and of each subscript or superscript. (5) Finally, provide at least one sentence that gives a description, in words, of what the equation computes. Do these five things for each of your four special effects.

Suggested Methods

OpenCV and/or ffmpeg make video processing easy on some computers; other computers will have lots of trouble with codec incompatibilities. In order to avoid those troubles, I recommend the following procedure:

Use your cell phone to record any source video that you want to use, or download souces from the web. Note 1: feel free to use copyrighted materials if you wish, but be sure to tell me about it. Note 2: you don't have to use any source materials at all; you can synthesize everything in Python if you wish.
If you have any source video, chop it into individual image frames, e.g., using ffmpeg as shown in this tutorial. If you also want to extract the audio from your video, you'll probably want to convert it to a WAV file. I think you can do that using: ffmpeg -i inputvideo.mp4 -vn -acodec pcm_s32le outputwav.wav.
Process the image frames and the audio file using your own source code.
Recombine the processed image frames and processed audio into a finished movie using VirtualDub as shown in this tutorial video, or using ffmpeg as shown in this tutorial.

ECE 401 Final Project 2017

Deliverables

Suggested Methods

Examples from Fall 2014

Project Definition, Fall 2014

Example Student Projects, Fall 2014