What is DataMoshing?

DataMoshing is exploiting compression by altering frames to create mind melting glitch effects. Just play like you understand I will explain below. There are three types of frames that can be manipulated. The P-Frame, The B-Frame and the I-Frame. DataMoshing could not be possible without compression. In the real world you can see Moshing when you skip ahead a poorly compressed DVD rip, occasionally streaming a movie when the bandwidth fluctuates or bench pressing at a gym looking up at the T.V. and seeing Arnold Schwarzeneggers face morphing into hemorrhoid cream. Then excitedly pointing at the T.V. to gym goers as they look back and forth from the T.V. to my shit eating grin and concluded that I am insane.True story. Not the insanity part of course….Humm…

In order to have an understanding of DataMoshing we need to understand compression. Compression at its basics is making a file smaller. A movie, text document, encoded music can be compressed. That’s the amazing thing about ones and zeros. Limiting bandwidth also can compress size. Telephones and cellphones. Are a good example. There are cool exploitations you can do with these also but it is not DataMoshing. I digress.

With DataMoshing we exploit certain aspects of compression called frames to create wonderful kinds of glitches. The beauty about this style of glitch is the variation and also its otherworldly nature. How does compression work? Before we delve into frames let us jump into this example so we can begin to see the full picture. ( Yes I know I am annoying my brain is non linear. So for all of you that are linear thinkers I apologize in advance. )

Hold on to this right here in yer brain ——> Lets look at a 10 second compressed and uncompressed file. A 10 second, 30 fps uncompressed movie would be composed of essentially 300 still images. A 10 second compressed video would use only 30 still images (I-Frames) and the rest would be just the information (P-frames) that changes from still to still. 

OK got it? It’s ok if you don’t. Let’s blow this taco stand….

So what are these frames thingies, motion compensation and how do they relate to compression?

P-Frames : The granddaddy of compression. Frames in succession have a high similarity. P-frames are small reference files that inform the video player of changes in the image since the previous frame. Only information about areas of motion or changes in lighting values are contained in a P-frame. The large frames that are redundant can be thrown out reducing the file size. The valuable information of the thrown out video frames are stored in the small P-frames. These essentially re-create the video but lower resolution. This is why a smaller file has more P-Frames. Essentially you can think of P-Frames as small bits of information tracking what pixels change. 

Another example if that was confusing. Imagine rather than storing 24 frames per second you only need to store 2 frames because the rest of the information is redundant. Now multiply 2 frames by the movie length in seconds and you have a much smaller digital file. Rather than gigabytes you have megabytes depending on how intense the compression is. More intense compression equals shittier looking videos. 

Incase you didn’t know A pixel is the smallest controllable element of a picture represented on the screen. The word pixel is a combination of pix and el for element. The word pix appeared in variety in 1932, as an abbreviation for the word pictures, in reference to movie! 

I-Frames :  The big bloat frame. I-frames are “inter” frames also referred to as keyframes. They are full representations of a single frame of the video. They are still images containing all the color and light information of that frame. They contain all the information. This frame is what makes a video file large in size. P-Frames rely on this for prediction of what will happen and what to throwout or store.

B-Frames : This frame predicts both directions or forward and backward. Basically this is the interrelation between P-Frames and I-Frames. Got it? Haha you may need to read these again. Kind of mind bending. 

Motion Compensation : More compression. A good example is panning. Rather than storing all of the frames of the panning it only stores the motion positions further compressing and making the file smaller. So you can think of it this way. Someone gives you a big book of pictures how to get to your friends house as opposed to a paper with the street names and the rights and lefts associated with them. Or if you are old school someone hands you a Thomas guide as opposed to an iPhone using Waze. Yes I just dated myself. One makes your arm fall off and one doesn’t. Ah the good old days of being a dyslexic pizza delivery man. Fun times. 

So what exactly is a frame rate?

Motion in video is created by changing of image frames in succession. This is called a frame rate. The frame rates are calculated per second! So in a typical movie there are 24 frames per second! Multiple 24 by the amount of seconds in a movie and you have a shitload of frames! The frames need to be plentiful and fast so you do not perceive the film as a 1950’s stop motion movie. You perceive the film as fluid and realistic movement. The more frames the more perceived fluidity. The more frames the longer the moshing will take.

Another way to visually understand frame rates is refresh rates. Refresh rates are to monitors as frame rates are to film. Wave your hand across your computer screen and you will see your hand stuttering. Those stutters are our eyes perceiving the gaps in the refresh rates or the gaps in the refresh frames (I just made that up) . The higher the refresh rate is the less stuttering you will see and you will have more perceived fluid motion. Eventually in the future refresh rates will be so fast there will be no stutters. 

The point?

Here is the secret sauce. Compression happens when you get rid of redundant information. Predictive frames make that happen. When you rearrange the predictive frames you get the crazy effects that transcend space and time shifting forward, backward, morphing, and jumping in time. Video time travel. Woh…

There are lots of ways to mess with the frames. Duplicating them, deleting them, swapping them etc…Or adding an I-Frame back to stop the madness for a bit. Each yields a different result from blooming, painterly effects, freakish morphing and others. DataMoshing Deluxe has other tricks under the hood as you will see it has millions of possibilities of DataMoshing. DataMosh Reactive relies on your fingers to do the duplicating and rearranging. Stay tuned for more apps and updates….

Also if any of you Glitch artists want to respond to this have at it. I might have missed things or fucked up some information.