Determine color ranges
All single frames of the film sequence are analyzed as regards their colors. Each pixel of a single frame has a certain rgb color value, which is determined and can be changed.
Color Ranges: Highlights, Midtones, Shadows
The color values that have been determined for each frame are divided into three color ranges (Highlights, Midtones, Shadows). Then each color range of the frame is being copied, displaying only the pixels of the respective range. All color values outside the range are transparent. In this way, the former rectangular set of frames turns into a sculpture-like figure, so that shapes and compressions emerge and details of the film’s content are displayed roughly.
The white or black color components of the frame are deleted (transparent), while all other ranges remain in their original color, i. e. they are extracted. Changes in these extracted elements, such as tracking shots or movements, become comparable in this mode.
Arrangement according to color ranges
During the color analysis the number of pixels of a color range is also determined. Thus, each frame exhibits a certain number of Highlights, Midtones and Shadows. In the arrangement mode the frames are moved to the right-hand side depending on this number. Example: Arrangement: Highlights: the lighter the frame is, the more it is moved to the right-hand side. This mode allows for a quick overview of how individual color ranges change in the course of the film. Preferred colors and rhythms in color design become visible.
The arrangement mode can be used for displaying various types of information. The following example deals with various possibilities of displaying film cuts in a clear arrangement. The cutting positions of the film sequence have been defined manually in advance. This allows placing the respective film cuts relative to the set value either to the side or to the upper border. The quick overview of the cutting process presents the first frame of each cut side by side on one level.
The volume level of the film`s audio track can be exactly measured in each frame. Thus, there are frames with a high and with a low volume level. By transferring the value to height, width or both proportionally, each frame is scaled according to its volume level, i. e. volume allocation can be displayed directly.
This is a very experimental form of display, in which the frequency is transferred to the rotation of the single frames. Thus, interesting effects can be created when playing the film sequence in this mode.
ID tags – single frame metadata
Similar to MP3 music files each single frame contains certain additional information (metadata), so-called ID tags. However, these ID tags do not contain the usual information such as author, genre etc., but mainly data that can be measured by the computer. The existing values regarding the number of Highlights, Midtones, Volume etc. are used as metadata. The exact value of each frame can be obtained by rolling the cursor over a frame (Preview Mode).
In addition, there is a search mode that makes it possible to quickly locate similar frames with the help of their ID tags.
The ID tags can be displayed as a bar in three-dimensional space, the width corresponding to the respective value. They can be positioned above or within the sequence of frames.
User ID tags are an experimental approach. They are based on the assumption that certain measured values related to the viewer are captured in real-time. These values are then stored directly as ID tags. With the help of devices like e. g. pulse monitors or a camera that records the viewer`s expression, it would be possible to draw conclusions regarding the viewer`s habits or emotional conditions, which in turn might influence the further course of the film. The values for the example shown below have been simulated for this purpose.
A camera on the monitor records the viewer’s exact eye movements. For evaluating the recorded data, the image is laid on a matrix. The red circle marks the segment that the camera has captured. The value (C5 in the example) is stored as ID tags. When displayed in three-dimensional space only the segments of the circle are visible. If the same film sequence is loaded twice, the original version and the recorded expression can be faded in simultaneously, so that a direct relation to the original frame is obtained.