Datamosh

2022-02-26 22:23

I saw this one on tiktok. It's a technique that takes advantage of the H.264 file format. (detailed here: https://sidbala.com/h-264-is-magic/) H264 achieves tiny video file sizes by only saving data for a keyframe when the image has changed a lot since the last frame. In between all of those, only the movement of pixels is saved. It uses the fact that most frames of h264 video do not store the full image - instead they just store motion related to the last image. These common frames are called 'P' frames. Each frame which stores the full image is called an I frame. If the image changes a lot on the screen, the encoder will use an I frame. - You can data mosh a video by either deleting I frames or adding excess P frames. - You can take advantage of this by deleting or inserting keyframes, making the video player apply movement from one clip to a different one. For example, I made a video of my cats face superimposed on the movement of my own. It was terrifying. - The most creative of datamoshing I've seen is to use it to create 3D data of a clip. Better explained here https://www.youtube.com/watch?v=VLGKFk3wIL4 - Resources - https://blogs.umass.edu/Techbytes/2017/12/07/datamoshing-what-it-is-and-how-it-works/