Investigative Study | Digital Sketchbook

Week 1 – Introduction to the module & deep dive into deepfakes

For our first week were introduced to the module and given and overview of what would be expected. The main goal of the module is for us to write an investigate study answering a question in an area of VFX that interests us.

To get us thinking we were split into groups and told to come up with ideas for proposals that we could use. Here are the ideas we came up with

Production

Race to the bottom

In the context of VFX race to the bottom comments on the idea that the cheapest bin on a project wins, this means the VFX may be inferior in quality but overall it is cheaper to produce. With this idea a writer could comment on whether this is in fact true to life and whether it is a good way for the VFX industry to operate.

VFX over promises and under delivers

This follows in the same vein as race to the bottom but this could draw on how VFX is perceived in relation to budget or studio name as well as audiences expectations. The focus here would be on the culture of the VFX industry

Things to look at that are based around these topics:

Life of Pi case study. The studio ran out of money making the movie
A bugs life and the coincidental creation of the water simulation
George Lucas discovers computer graphics by Alvy Ray Smith

AI in the VFX industry

This proposal would look at how AI is impacting the industry, whether there are benefits to it’s inclusion such as automation or whether benefits are outweighed by negatives such as loss of entry level jobs (rotoscoping, clean up etc.), audiences being tuned to an “Ai look” and Ais impact on creativity

Ethics

Resurrecting actors, de-aging, storage of actors’ data to be used in other projects

The study could be on an ethical issue as there’s a lot to talk about here. Actor resurrection is a big topic with a prominent example being in the Star Wars franchise. It could be discussed how the audience perceived it and how impactful it’s inclusion was to the media

Similarly de-aging is becoming more common, is this deception? Should younger actors be hired for this work instead of using computers to de-age people?

Data protection is an interesting topic in VFX as it is a relatively new worry for actors and performers in this industry. Is their data safe and could it be sold or used for AI modelling? Could actors lose some of their personal identity if extensive capture of their voice or motion capture performance was made available publicly?

An example of this is Disney’s handling of Robin Williams in Aladdin.

AI is also something that has an ethical angle that could be explored.

Uncanny aesthetics in VFX

Is uncanny valley just a technical hurdle that can be overcome?

Animal usage in movies. Inability to capture accurate data of most animals means now they are created entirely through CGI. Is this noticeable?

Examples of this “bad” CGI animals include – Andy Serkis’ Mowgli and animals in Those Who Are About to Die

This could also link to my previous essay about ways of creating animation, the differences between keyframe and motion capture, specifically drawing on our inability to capture animal movement accurately

Photorealism

This proposal would focus on photorealism and the immersion of the audience. How important high quality graphical imagery has become for retaining audiences in a story. This could also talk about the need to degrade or imperfect the images to make them more believable for an audience

Motion Capture and Keyframe (using same research as I did in my year 2 essay)

Another topic I could use for my investigative study is to use the same topics and research as I did for my year 2 essay which was comparing performance capture methods of motion capture and keyframe animation. I would need to come up with an original question and idea to explore to make sure I do not reuse any content however I could use the same papers and research in the piece.

In addition to talking about the methods of performance capture I could also talk about how that data is protected and stored. Using the same points explored above. When actors provide data for a performance is that data stored, could it be used maliciously at some point. I could explore data protection of performance capture.

Week 1 Main topic – Deepfakes

This week we dove into the topic of deepfakes.

Deepfakes are manipulated content where and actors face is replaced by and digitally made alternative. Often used to misrepresent people or spread misinformation.

Genres where deepfakes are used – comedy, politics, fake news, drama, horror, sci-fi, pornography

A prominent example of a deepfake is a deepfake produced by Channel 4 of Queen Elizabeth the second. Channel 4 say it has two intentions, one for comedy and the other to inform consumers of deepfakes and show how they can be created with relative ease.

There were quite a lot of complaints over this as some viewers found it offensive and upsetting. It asks the question of whether it was necessary or what the intention of Channel 4 was. Another aspect is that they cannot have had permission from the queen to use her data in this way. Which links into the ethical question brought up earlier.

Deepfake queen prompts 200-plus complaints to Ofcom – BBC News

A more recent example is of Taylor Swift’s political leanings in the 2024 election. One of the reasons she came out publicly with her endorsement for Kamala Harris is because Donald Trump had been sharing manipulated images of her that implied, she was endorsing him. This led to her using her platform to stop the misinformation that people can just create now that so much imagery and technology is available.

Trump posts image of fake Taylor Swift endorsement | CBC News

An article in the guardian () comments on a famous Obama deepfake and says it was “a small-scale demonstration of how this technology might be used to threaten our already vulnerable information ecosystem – and perhaps undermine the possibility of a reliable, shared reality”.

You thought fake news was bad? Deep fakes are where truth goes to die | Technology | The Guardian

It is important to not that this article is from 2018, and technology is only improving in quality and accessibility.

The camera never lies

An interesting topic to think about is even without the manipulation that deepfakes involve, do cameras lie? Historically in areas like journalism photographs have been used as compounding evidence which provide information and context that people trust. However, these can always manipulated in post-production. Furthermore, even basic cameras lie, due to the photographer having the power to choose the angle of the shot and what to frame means they can control how the image is perceived.

One morally grey area where deepfakes are used is to incorporate actors who are no alive back into movies. A prominent example of this is Carrie Fisher in Star Wars. It could be discussed whether this is ethical or whether it does mislead the audience in some way

Finding academic sources for deepfakes

When searching the university library database I found a few academic journals discussing the ethics of deepfakes.

This paper interesting as it talks about how actors can protect themselves from losing their jobs to deepfakes

Preminger, A. and Kugler, M.B. (2024) ‘The Right of Publicity Can Save Actors from Deepfake Armageddon’, Berkeley Technology Law Journal, 39(2), pp. 783–840. doi:10.15779/Z383R0PV4H.

This article talks about how deepfakes should be regulated. It discusses the advancement and impacts of deepfakes and how they impact our culture

Weiner, M.D. (2024) ‘Destined to Deceive: The Need to Regulate Deepfakes with a Foreseeable Harm Standard’, Michigan Law Review, 122(4), pp. 771–807. doi:10.36644/mlr.122.4.destined.

This article talks about a specific case study of the Russian Ukraine war and how deepfakes of the events have disrupted trust in media and reports.

Twomey, J. et al. (2023) ‘Do deepfake videos undermine our epistemic trust? A thematic analysis of tweets that discuss deepfakes in the Russian invasion of Ukraine’, PLoS ONE, 18(10), pp. 1–22. doi:10.1371/journal.pone.0291668.

My personal thoughts on deepfakes – Do I think it is okay to replace an actors face using deepfake technology?

Overall I do not think it’s okay to replace an actors face using deepfake technology as I do not think it abides by where the industry should stand on asking permission and data protection. In the example used earlier with the Queen, I personally do not think this should have been made as I don’t think the comedic value was high enough as far-reaching enough to outweigh the fact that the production misrepresented the queen without her consent. Although some are made for entertainment purposes, I believe most are made to spread misinformation and damage the reputation of whoever they are representing. The aspect of deepfakes that I am less impacted by are the examples like Carrie Fisher in Star Wars as here I know in the production the studio wanted to honour her legacy and have her be a part of the modernisation of the franchise. They were created tastefully and with respect and they didn’t involve her saying or doing anything that went against her beliefs however I still wonder if they were necessary for the film, I don’t know if audiences would have been impacted if the scene/the deepfake was excluded given that her scene wasn’t integral to the plot of the film.

Week 2 – Advancements of AI in the VFX industry

Article – Photographer admits prize-winning image was AI-generated

Photographer admits prize-winning image was AI-generated | Sony world photography awards | The Guardian

Boris Eldagsen a german artist entered a photography contest using an image he created using AI.

This was the first win of a high profile contest by an AI generated image
Eldagsen refused to accept the award, acknowledging that it wasn’t his photo
The artists believes after this incident there should be separate categories for images that use AI and those who don’t

Something that I found interesting is how the photograph industry was unprepared to receive AI submissions, they were potentially too trusting?

It is impressive that an AI image won and proves how strong AI art can be.

This sparked a movement over prompt-photography versus photography

Overall in class our discussion leant toward prompt photography not being similar to photography at all due to a lack of skill and creativity being used. However that are positives of it like time spent and conveninece.

It is worth noting that in history there have been forms of photography that havent used camera. Such as rayographs and Thomas Ruff’s photograms which are renders from Maya

Examples of photograms by Thomas Ruff. These are made in and rendered out of Maya. Often creating and hiding objects in the software to render their shadows

Looking at Wonder Studio

Wonder Studio by wonder dynamics is a service which allows actors to be replaced by modelled characters using AI. In class we discussed how we feel about this as VFX artists as it takes a lot of our work and atomizes it.

I think that from an audience perspective the technology is very impressive. Being knowledgeable about VFX means that we can see imperfections in it and know that it is not production ready. However to the general audience it is very convincing and realistic.

Technologies like this are a concern as they will only keep improving over time

Ideas for topics I could write a proposal on

At the moment I am most leaning toward an ethical question, particularly around protection of actor’s data. Alternatively, I could also see myself writing about deep fakes as I feel there is a lot of content here that I could draw from

Case studies

Carrie Fisher Star Wars
- Guardian article – Carrie Fisher: Princess Leia stole my identity
- Hollywood reporter – Carrie Fisher, ‘Star Wars’ and the Legal Issues of Dead But In-Demand Actors

Possible questions

Are deepfakes a good asset to the VFX industry?

Are deepfakes ever are a good idea? Examining the legal, ethical and creative concerns.

Week 3

This week we were tasked with making a PowerPoint presentation on our essay question for the module.

This outlined a brief explanation of what I want to discuss and case studies and sources I can use

PowerPoint

After presenting my ideas I had a clear plan on what I wanted to write about and how to go about it. My question will likely be refined and I also will likely not focus on de-ageing and instead only look at bringing back actors who are now deceased.

Additionally through class discussion I was alerted to some new case studies.

The first being Christopher Reeve who died in 2004. He played superman in the 90s and was not interested in playing the role again. However he was digitally replicated and given a cameo in The Flash (2023). His family had no say in whether or not his likeness was used therefore this is unethical.

Christopher Reeve’s Family Had No Involvement With His CGI Superman Cameo in The Flash – IGN

The Flash’s Christopher Reeve cameo is reigniting the CGI dead actor debate | British GQ (gq-magazine.co.uk)

Christopher Reeve’s Children Never Watched The Flash Cameo (variety.com)

Another case study is Ian Holmes being brought back for Alien: Romulus. Conversely to Christopher Reeve’s story. Holmes’ family were delighted for him to receive a cameo in the movie as they see it as his legacy being honored.

Ian Holm’s Role In Alien: Romulus Explained (screenrant.com)

Week 4 – Working on my proposal

Proposal document for week 4

Proposal Formative Submission

Proposal for investigative study

Keywords

Deepfakes
Rights of Publicity
Digital replicas
Artificial Intelligence
Privacy
Consent
Entertainment

Research Question

Are deepfakes ever a good idea? Examining the legal ethical and creative concerns of this new form of VFX.

Looking at the ethics of resurrecting actors using AI/machine learning and the legal framework around this.

Introduction

For my investigative study I have chosen to look into deepfakes, more specifically how they are used to bring back to screen actors who have passed away. Recent example of these include Albert Einstein in Smart Energy GB’s advertising and Ian Holm’s reappearance in Alien: Romulus (2024), with a long-standing example being Carrie Fisher in the Star Wars franchise. The aim of this essay is to discuss the way digital replications of actors are made, how they are or are not clearly explained in actor’s contracts, and the dangers that they can cause to the wider VFX industry and its workers.

This is an important issue to investigate as this use of artificial intelligence is becoming more and more common and as spoken about in a lot of the research I will bring up, laws around this may need to be updated and clarified to protect both studios and actors in the future

Methodology

My study will be entirely theoretical as I will examine a variety of sources including academic journals, books and articles from reputable websites.

In my study I will look at a few academic journals that focus on the legalities of these deepfakes, they comment on how the law may need to change and why this is a pressing issue currently in our culture. I will also look at broader academic journals and texts which highlight the history of deepfakes, how they are made and lightly on ethics and audience response.

The third type of source will be web interviews that I have collated from either an actor themselves or relatives speaking on their behalf on their opinions of their own deepfakes and more broadly how the actor in questions’ data has been handled by the studio

Sources

This article talks about how celebrities are replicated and how easy to make and manipulative they can be, whilst also acknowledging the invasion of privacy to the original person

Bode, L. (2019) ‘Deepfaking Keanu: YouTube deepfakes, platform visual effects, and the complexity of reception’, Convergence: The International Journal of Research into New Media Technologies, 27(4), pp. 919–934

The is an article where Carrie Fisher herself talks about how she sold the rights to her portrayal of Princess Leia without knowing the implication and how she wanted to raise awareness of this for future generations of actors

Child, B. (2011) Carrie Fisher: Princess Leia stole my identity, The Guardian. The Guardian. Available at: https://www.theguardian.com/film/2011/sep/15/carrie-fisher-princess-leia-lucas (Accessed: 16 October 2024).

This is a law review paper which talks about how the Right of Publicity should be updated to protect actors and how this would create a better industry.

Curren, A. (2023) ‘Digital Replicas: Harm Caused by Actors’ Digital Twins and Hope Provided by the Right of Publicity’, Texas Law Review, 102(1), pp. 155–183. Available at: https://research.ebsco.com/linkprocessor/plink?id=4773b6af-9ef7-3c68-b309-94eae199dd35 (Accessed: 14 October 2024).

This is about the rise of AI and most importantly the rise of accessibility to AI and programs that can make deepfakes, how this is changing the landscape of media, especially to impressionable audiences

Koski, R.M. (2024) ‘Warhol, Drake, and Deepfakes: Monetizing the Right of Publicity in the Generative Ai Era’, Georgia State University Law Review, 40(4), pp. 981–1027. Available at: https://research.ebsco.com/linkprocessor/plink?id=8cf9a227-f584-3fe6-b8c8-fa4cb7846bc0 (Accessed: 14 October 2024).

This is a book that has a lot of good information on how deepfakes are made and the process involved. It likely touches on the risks and how laws may need to change to protect actors and audiences

Kalpokas, I, & Kalpokiene, J 2022, Deepfakes : A Realistic Assessment of Potentials, Risks, and Policy Regulation, Springer International Publishing AG, Cham. Available from: ProQuest Ebook Central. [14 October 2024]

Bibliography

Carter, J. (2023) Robin Williams’ Aladdin Agreement Proves How Great His Genie Was, ScreenRant. Available at: https://screenrant.com/aladdin-robin-williams-genie-disney-agreement-great-reason/.

Cullins, A. (2017) Carrie Fisher, ‘Star Wars’ and the Legal Issues of Dead but In-Demand Actors, The Hollywood Reporter. Available at: https://www.hollywoodreporter.com/movies/movie-news/carrie-fisher-star-wars-legal-issues-dead-but-demand-actors-997335/ (Accessed: 16 October 2024).

Giardina, C. (2024). How Ian Holm in ‘Alien: Romulus’ Relates to SAG-AFTRA’s Digital Replicas Work. [online] Variety. Available at: https://variety.com/2024/artisans/artists/ian-holm-alien-romulus-sag-aftra-digital-replicas-1236126922/ [Accessed 23 Oct. 2024].

Kahn, E.W. and Lee, P.-I. “Bonnie” (2016) ‘Delebs’ and Postmortem Right of Publicity, Americanbar.org. Available at: https://www.americanbar.org/groups/intellectual_property_law/publications/landslide/2015-16/january-february/delebs_and_postmortem_right_publicity/ (Accessed: 16 October 2024).

Parkin, S. (2022) Who owns Einstein? The battle for the world’s most famous face, The Guardian. Available at: https://www.theguardian.com/media/2022/may/17/who-owns-einstein-the-battle-for-the-worlds-most-famous-face (Accessed: 16 October 2024).

Preminger, A. and Kugler, M.B. (2024) ‘The Right of Publicity Can Save Actors from Deepfake Armageddon’, Berkeley Technology Law Journal, 39(2), pp. 783–840. doi:10.15779/Z383R0PV4H

Smart Energy – Einstein Knows Best (no date) The Mill. Available at: https://themill.com/the-mill/work/smart-energy-einstein-knows-best/.

Velasquez, S.J. (2023) How AI is bringing film stars back from the dead, www.bbc.com. Available at: https://www.bbc.com/future/article/20230718-how-ai-is-bringing-film-stars-back-from-the-dead (Accessed: 16 October 2024).

Weiner, M.D. (2024) ‘Destined to Deceive: The Need to Regulate Deepfakes with a Foreseeable Harm Standard’, Michigan Law Review, 122(4), pp. 771–807. doi:10.36644/mlr.122.4.destined.

Westerlund, M. (2019) ‘The Emergence of Deepfake Technology: A Review’, Technology Innovation Management Review, 9(11), pp 39-52.

Figures

A recreation of Einstein for “Smart Energy Einstein Knows Best” campaign by The Mill VFX Studio

Screenshots of Carrie Fisher deepfakes used in Star Wars

Ian Holm recreated for Alien Romulus (2024) (Giardina, 2024)

Week 5 – Refining sources

Left – recreation of Ian Holm in Alien Romulus using VFX

Right – his original portrayal in Alien (1979)

Looking in to the WGA’s writers strike and how it affected UK VFX

The WGA writers strike which started in May 2023 and ended that September had a major impact on the UK VFX industry. That is because a lot of the work in the UK is for big US streaming clients. As an example, at Pinewood studios, normally a national hub of prestigious work there was 7 films completely stopped and put on hiatus during the duration of the strike.

“Over the last five years, US studio-backed productions have consistently funded more than two thirds of total UK film spend, according to the British Film Institute”

Many freelancers were let go as a result and their jobs were unprotected.

Spencer MacDonald, national secretary of BECTU, a union representing production workers said thousands of jobs were in limbo and he expected many talented people with transferrable skills to leave the industry

Seal, T. and Hughes-Morgan, C. (2023) ‘Hollywood Strikes Threaten UK’s US-Dependent Film Industry’, Bloomberg.com, p. N.PAG. Available at: https://research.ebsco.com/linkprocessor/plink?id=ebf465e2-f13f-3f3c-9f05-d15d2835e3bf (Accessed: 28 October 2024).

Starting my annotated bibliography

Source:

Digital Replicas: Harm Caused by Actors’ Digital Twins and Hope Provided by the Right of Publicity by Alexandra Curren

Notes:

“A digital replica is a computer-generated image that reproduces the likeness of a person and then can be manipulated to create a “performance” without the involvement of that person”

I really like this concise overview of deepfakes as I think it very clearly explains the hesitation around this technology

This article talks less about using deepfakes to resurrect actors and more about the consequences for current actors and how their jobs are at risk. Actors could be out of a job while they are still alive and able to perform as their digital replicas are good enough substitutes which are a lot cheaper for studios to work with and they are easier to manipulate.

This article also notes that technology disrupting and/or taking jobs has been a growing trend that has been impacting many industries, and it poses the question of whether acting will be the next profession to be taken by technology.

The difference when actors get replaced by technology is that the result is more invasive to the individuals. This is because when making a digital replica the technology also take data on actors faces, voices and mannerisms

In terms of how attainable the scans are, Digital Domain the visual effects company that worked on Avengers: Endgame report they can scan individuals in minute detail in a matter of seconds. As well as this Netflix and Disney are hiring AI engineers. Lucasfilm has implemented a policy of scanning all lead actors. Can the offer of a role be conditional on agreeing to be scanned?

The right of publicity

In the only written work dealing formally and systematically with the subject of the right of publicity, J. Thomas McCarthy and Roger E. Schechter define the right of publicity as “the inherent right of every human being to control the commercial use of his or her identity”

The most straightforward examples of right of publicity claims come in the context of advertising. When an advertisement features an individuals identity or likeness it leads consumers to believe that the individual endorses the product.

In terms of digital replicas there is a grey area here in which there are commercial uses and expressive uses for likenesses. Commercial uses are more widely appreciated by law as this is a clearer line to draw.

In 2021, New York passed a right of publicity statute that specifically addresses digital replicas. It allows claims against unauthorised use of digital replicas. The statute says anyone who uses a deceased performers digital replica in a scripted audiovisual work as a fictional character or for the live performance of a musical work shall be liable for any damages sustained by the person or persons injured as a result thereof if the use occurs without prior consent from the person

In the statute a digital replica is defined as a newly created, original, computer-generated, electronic performance by an individual in a separate and newly created original expressive sound recording or audiovisual work in which the individual did not actually perform that is so realistic that a reasonable observer would believe it is a performance by the individual being portrayed and no other individual.

This statute was the result of the negotiations between SAG-AFTRA and the Motion Picture Association. New York is doing well with the statute however more states need to follow suit. New York made this move because of the influence of the SAG-AFTRA strikes.

Deepfaking Keanu: YouTube deepfakes, platform visual effects, and the complexity of reception by Lisa Bode

This article looks a specifically at a case study where VFX YouTube Channel Corridor Digital made a deepfake of famous actor Keanu Reeves. In the video it is made out that Reeves stops a robbery however the entire scenario is fictional.

Much of the article isn’t of particular relevance for my study as I aim to focus on the use of deepfake technology to resurrect deceased actors however it does bring up some interesting on the reception of the technology and how it is used.

The article argues that “the dominant issue with deepfakes is not the act of image manipulation itself but the potential for such manipulation to deceive or mislead viewers”. The case study analysed in the article is an interesting one as this example reinforces the positive perception of Keanu Reeves instead of undermining or damaging the actors reputation.

The Emergence of Deepfake Technology: A Review by Mika Westerlund

Westerlund, M. (2019) ‘The Emergence of Deepfake Technology: A Review’, Technology Innovation Management Review, 9(11), pp 39-52.

This article talks about the emergence of deepfakes and how they are a threat to society, political system and businesses and how they can be regulated.

It highlights how accessible the software and technology is as well as how good the outcomes achieved can be (almost indistinguishable from authentic media) Another point is how much freedom deepfakes give their creators, the consent of the person who image and voice is being used isn’t needed at present.

The phrase “deepfake” is a combination of “deep learning” and “fake”. Deepfakes are hyper realistic videos digitally manipulated to depict people saying and doing things that never actually happened

“Deepfakes rely on neural networks that analyse large sets of data samples to learn to mimic a persons facial expressions, mannerisms, voice and inflictions”. The process involves feeding footage of two people into a deep learning algorithm to train it to swap faces

Often deepfakes target social media platforms, where conspiracies, rumours and misinformation spread easily and users tend to go with the crowd

The film industry can benefit from deepfake technology in multiple ways. For example, it can help in making digital voices for actors who lost theirs due to disease, or for updating film footage instead of reshooting it. Movie makers will be able to recreate classic scenes in movies, create new movies starring long dead actors.

Deepfake technology also allows for automatic and realistic voice dubbing for movies in any language. (Example given of David Beckham being multilingual in 2019 malaria awareness campaign).

Deepfakes can be used to spread misinformation and manipulate public opinion and undermine confidence in a given country’s institutions. The journalism industry is going to face a massive consumer trust issue due to deepfakes.

Another issue is that deepfakes are likely to hamper digital literacy and citizens’ trust toward authority-provided information, as fake videos showing government officials saying things that never happened make people doubt authorities

My methods and methodology for my essay

My essay is primarily theoretical as I will be looking at academic journals and secondary sources surrounding my topic

Additionally I will include case studies one on Carrie Fisher and another on Albert Einstein. I think these will enhance my essay by providing real life examples and some insights on the creation and reception to them. This data will mainly be collected from websites and interviews online

For my essay I have set out to find literature that discusses the ethics of bringing back actors that are no longer with us via the medium of deepfakes, as well are more broadly what deepfakes are and how they are used in the creative industries.

In terms of finding relevant academic journals I have utilized the universities library search to find relevant texts using keywords like ‘deepfakes’ and ‘right of publicity’

Due to the topic I have chosen and the fact that many argue it should be under regulation I have found a couple of law review journals that have been useful for my research, this is a further example of why theoretical/secondary research is the best fit for my proposal

Week 11

Presentation of material within the study