My original plan for this post was to comment on how Disentanglement is the Next Deepfake Revolution using CGI to fill in the “blanks” where data for the target identity is missing, and to point to a DYI guide to Deepfakes. When the site became unavailable when I first wrote this post, I feared it had been censored, but it is now back.
If you watch a lot of deepfakes, you’ll probably notice they’re usually directly front on. Once you want to move into more extreme angles, or side views, where no sample data exists, the deepfake falls apart. The new work builds an underlying CGI models that the sampled (original images of the target of the deepfake) is mapped on to. As the CGI model moves and warps, the deepfake follows along.
This is not mere deepfake head-puppetry, a technique more suitable for avatars and partial-face lip-synching, and which has limited potential for full-fledged deepfake video transformations.
Rather, this represents a way forward for a fundamental separation of instrumentality (such as ‘change the angle of the head’, ‘create a frown’) from identity, offering a path to a high-level rather than ‘derivative’ image synthesis-based deepfake framework.
It’s a deep article that can be summarized as “deepfakes are going to get better, more flexible and easier,” which leads to the much greater issues of how society will deal with the ramifications.
When deepfakes were a lab phenomena, and required huge datasets, the potential for nefarious use was low. As they became easier, and more widespread it’s becoming harder to spot the real and the fake personalities in video. But, they always show that flat, front view because that’s what’s available for input data. The more flexible deepfakes become, as they will with the implementation of this new research, the more likely it is that someone will pout one in a political ad.
Will we spot it? Will it matter after the damage has been done? After all, “I’ve seen him say that with my own eyes and ears” is going to be difficult to counter.
I don’t have the foggiest clue on how to solve these issues, but I do know we will have to solve them.