Week 1 – What is trending in the world of VFX?

When it comes to the current trends of digital effects and its relationship with practical filmmaking, the technology has evolved exponentially in terms of taking a step forward in accurate lighting, real-time rendering backgrounds and accurate facial and motion capture that increases the immersion for certain cast and crew. This allows them to achieve what used to be challenging situations such as sets and consistent colours what would be considered time consuming to recreate practically.

The Volume (Stagecraft) – Virtual Production by ILM

Ref. https://www.youtube.com/watch?v=gUnxzVOs3rk

Ref. https://www.youtube.com/watch?v=2kQzfng264w

Real-time Facial and motion capture – Unreal Engine etc.

3D scanning & Photogrammetry

---

Lens effects – Sense of scale & isolation

The wide range of lens has broaden cinemtographers’ ability to capture beyond what they see and also the stillness in now much higher detail. Combining it with digital effects we’re able to convince the audience how they view the world presented to us in ways we can only imagine.

• Stillness
• Distortion
• Wide & narrow
• Close ups
• High motion

---

Harold Edgerton’s influence to the world of art forms

---

 Dr. James Fox’s ‘The Age of the Image’

https://learningonscreen.ac.uk/ondemand/index.php/prog/158E7D18?bcast=131408235

What is meant by James Fox saying we’re in the ‘The Age of the Image’?

James Fox mentions the different ages that defined the the 18th and 19th century, with the introduction of photography, the ability to capture one’s movement had instantly attracted everyone around the world to analyse and be mesmerised of moments that were previously possible by just memories or drawings. I think what James means is that an image spans across numerous methods of communication, even beyond the means of entertainment. For scientific purposes, there are now records that can be in layers such as x-ray scans and transferring the image to screen for doctors and scientists to observe and bring out an accurate hypothesis, leading to potential groundbreaking results. An image can change one’s opinion as proof of an event, as well as being held as with sentimental value that billions of us would cherish with their life with smartphones and digital cameras dominating the industry over traditional usage of films. Every frame alters our interpretation and gives us new perspectives, the power of filmmaking and photography has demonstrated just that.

Every word we speak can now be translated into the medium of an image, especially for artists as they transform images into many genres with immense skill and years of practice that generates even more original work that inspires and influence the next generation. Saying now is ‘The Age of the Image’ might even be considered an understatement, the creation of an image has become the new form of expression that defines not only this generation but also helps redefine the previous ‘Ages’, a retelling of an ancient transcript or a story you’ve heard that came from your ancestors can be recreated for us to look at.

---

Week 2 – The Photographic Truth Claim: Can we believe what we see?

Plato’s The Republic: Allegory of the cave

Plato’s philosophical approach of how people are easily deceived by their every-day surroundings, in this case using a dark cave for prisoners as an example to express the isolation and dread they go through when constant appearances of a shadow figure becomes their ideal approach in perceiving anything they might have the fortune to encounter if they can escape the cave. What they look at everyday and their behavior towards it has become their reality, their belief and their system of operation.

The Divided Line

---

Semiotics

The study of signs that communicates and defines a meaning, branching out to indicating metaphors, analogies and symbolism.

Icon

Index

Symbol

---

Film examples using deception to redefine a reality

---

What’s the Point of an Index? or, Faking Photographs by Tom Gunning

what is meant by the theory: The Photographic Truth-Claim?

Gunning mentions the impact in which the old and new technologies of visual media has gave audiences an introduction to new horizons of filmmaking and photography, widening our understanding of what’s real and what’s fake. But the rapid development of such artistry has cost us little time to perceive and determine that whatever we see that is considered sorcery or impossible, we believe it to be not necessarily real every time but becomes a ongoing topic that none of us can avoid. Nowadays, we question the integrity of an image that is posted on any media, any sort of manipulation within the image generates arguments and even conflicts that has the power to shift the narrative to all sorts of directions, according to what the general public believes in, their voices can overpower any narrative, giving yet another option for the public to choose whose side their on.

The high fidelity and fast transfer rate of the image now has allowed us to compare the quality and method of producing an image centuries ago but people tend to do this in a negative manner, dismissing and ignoring the context and history that created this new curiosity and popularity that people have towards the versatility of an image because of how superior and easy capturing an image is now. With news, we see examples of blackmailing, artificially generated images and out of context moments captured from camera that corporations use to their advantage, victims have no voices nor control in telling the truth, that’s how poetic and dangerous an image can be, sparking cultural divides across the world.

---

Week 3 – Faking Photographs: Image manipulation and computer collage

“Faced with an image on a screen, we no longer know if the image testifies to the existence of that which it depicts or if it simply constructs a world that has no independent existence” – Casetti, 2011

---

A definition of vfx compositing

Blade Runner, The Dark Knight and Dune. Three films in the span of 50 years has demonstrated exceptional compositing techniques utilised from green screen shots and detailed, concise special 3D effects outsourcing from different softwares and perhaps real life images and videos that combines into a epic scaled spectacle with Los Angeles in 2049, mystical dunes from the world of Arrakis and consistently era-accurate architecture in the Gotham City.

There are multiple ways to create and maintain a good composited sequence such as rotoscoping, essentially cutting the object from a frame and keeping it in frame to accompany the other combined shots or effects. The invention of a rotoscope machine came from Max Fleischer, who pioneered the development of cartoon animation, bringing iconic characters like Betty Boop and Popeye to the screen. In order to create an impression of reality, the position of the background doesn’t necessarily have to move due to the scale and depth of field but whatever color and lighting the atmosphere gives off, anything like frame rate and accurate rotoscoping that’s been put into the composition layer by layer in front of the ‘camera’ will have to match to become believable for the audience as they start view and be immersed into the screen.

---

Week 4 – trends of Photorealism

Photorealism in VFX

In VFX, photorealism is produced by carefully combining technical innovation, artistic talent, and attention to detail. The objective is to produce digital pictures that are identical to actual photos, allowing computer-generated elements to be smoothly incorporated into live-action film. It may seem like a recent achievement of such technology can fool the audience’s eyes but 2014’s Ex Machina, directed by Alex Garland, paid extra attention to detail in combining the practical filmmaking in its dull lighting and subtle cinematography to further allow the light to accurately interact with the surface of the skin of Ava, the AI.

Proper lighting and shading are crucial components of photorealism. The lighting in the live-action video is examined by VFX artists, who then recreate it in the virtual world. This factors in producing realistic shadows, taking reflections and refractions into consideration, and comprehending how light interacts with various surfaces. More complex rendering methods such as ray tracing and mimicking light’s behavior to increase the final image’s realism. 

Achieving photorealism requires careful texturing. High-resolution textures are used by artists to give fine details to surfaces, simulating flaws. 

---

Week 5 – Digital Index: Bringing indexicality to the capture of movement

Motion capture & key-frame animation

In computer animation, there are two different techniques used to create lifelike figures and objects: motion capture and key-frame animation. Using sensors or cameras on live actors, motion capture captures real-world movements and converts them into digital form. On the other hand, key-frame animation requires precise control through the manual creation of key postures and the filling in of intermediate frames. Matt Reeves’ War for the Planet of the Apes exceled (2017) in both methods, all thanks to actor Andy Serkis’ performance both emotionally and physically that aided immensely in telling Caesar’s story with dramatic expressions that echoes to those of humans, by having Serkis literally ‘be’ Caesar, all responses we see on screen are considered raw rather than programmed or scripted.

Motion capture is a realistic and stylish method of capturing natural motions from live performances, making it ideal for realistic scenes in movies and video games. Key-frame animation, which aims for artistic or fanciful approaches in animated films, TV shows, and video games, enables more stylized and exaggerated movements.

Motion capture offers great realism in terms of control and artistic freedom, but it may also restrict control over artistic features.

---

week 6 – Reality Capture (LIDAR) and VFX

Innovative 3D scanning from ScanLAB Projects

Ideas & concepts of 3D from Thinking 3D

---

What 3D scanning needs to exist?

• A virtual 3D space: To imitate or simulate 3D objects
• A 3D scanner: Taking millions of measurements

What does perspective needs to function?

• Vanishing points:
• Digital perspective:
• Non-digital perspective:

Depth-based scanning

Laser scanning (LIDAR)

Photogrammetry

---

Case study – Reality Capture

Reality capture has opened up numerous opportunities to bring inspirations from real world environments and pre-existing architecture to create a brand new atmosphere by using LIDAR technology and 360 degree photogrammetry technique that captures raw data such as colour, texture and depth as reference. In Ghost in the Shell (Sanders, 2017), its source material from the manga and anime series has demonstrated a vibrant and detailed cyberpunk-themed city, covered with holograms and robust architecture that would barely allow pedestrians to view the sky. In order to make a futuristic Hong Kong feel even more lively, Epic Games’ RealityCapture came up the idea of ‘solograms’, which are basic photogrammetry but in motion, all synchronization to be automated across the cameras that will bring the volumetric data of skin, face and clothes and by transferring the 80 images for each frame in a 24 fps format, the result comes with light particles grouping up to be moved and augmented in the Z space, which will be versatile in transparency and the freeflowing movement of said holograms shown in the film.

I really appreciate how they expand the technique that was built upon the already capturing not just the outside of raw data as reference, but now the inside to add more elements that helps compliment the world that VFX artists built. By using new digitally augmented data we are able to give new perspectives on how the world or city works, in this case these solograms have massively immersed us to the city of Hong Kong in Ghost in the Shell, which I thoroughly enjoyed.

---

Week 7 – Reality capture (Photogrammetry) & Vfx

The Veronica Scanner | Exhibition | Royal Academy of Arts

Mimesis test

Versirimilitude & Hyperrealism

Verisimilitude is the combined context of visualisation that aids you to realising the relatability of a realistic situation or event.

Hyperrealism combines different realities from opposite spectrums that occurs in the real world, resulting in a what we view are unusual behaviours from any medium that utilises such approach.

The digital michelangelo Project

---

Week 8 – Simulacra, Simulation and the Hyperreal

Simulation in VFX, plays a crucial role in recreating realistic scenarios based on real life physics via a computer programed with a set of rules e.g. destruction, explosions, water, air particles etc. Often times this method reduces the time consumption in attempting a practical effect on set that involves many man-made and planning.

Flame simulation in 1917 (Mendes, 2019)

 

 

 

 

 

 

 

 

 

 

 

 

Water and light simulations in Blade Runner 2049 (Villeneuve, 2017)

---

Jean Baudrillard presents the sign order: Phases of the image

Here we use Joi, a simulated AI from Blade Runner 2049, played by Ana De Armas, which I think perfectly demonstrates how simulations defines its own reality and how it interacts with the real world we live in.

The reflection of a profound reality

Masking and denaturing a profound reality

Masking the absence of a profound reality

No relation to any reality whatsoever; it is its own pure simulacrum

---

Week 9 – Virtual Filmmaking

Arguably the best example right now in Unreal Engine, the evolution in being a traditional game engine to becoming a real-time production tool that creates a new collaboration method for filmmakers to capture accurate lighting from a virtual environment. The fast rendering time has enabled artists to be on set and instantly alter even the smallest details such as foliage and background objects according to their script and creative vision, which helps reduce time in telling how the story the way its meant to be told.

 

The eras of filmmaking according to Sam Nicholson

• Chemical Era

• Digital Era

• Virtual Era

---

Assignment 2 Essay – What are Spectacular, Invisible and seamless effects?

I’ve chosen the third topic option that deals with the invisible effects that we often don’t realise and question in films and shows, I’ll be also combining this topic with a sub-question that will expand the exploration of these effects and how visual effects have pushed the power of photorealism that further enhances these fictional realities, helping the immersion and emotional response for the audience.

Essay title – HOW THE INVISIBLE EFFECT HAVE ENHANCED A PHOTOREALISTIC APPROACH FOR HYPER-REALITIES

Current Trends of VFX A2 presentation.pptx

Essay

The term hyper-reality is used to describe an exaggerated version of reality, which is often unique from our reality. It is the idea that an environment or scenario can feel more real than reality, challenging traditional representations of reality. Visual effects are a key part of the hyper-reality process. The invisible effect is one of the most important techniques used to achieve hyper-reality, it involves seamlessly blending computer-generated components with real-life footage. This technique enhances the photorealism approach by making sure that digital elements blend seamlessly with the real environment. Some of the most well-known examples of hyper-reality in films include James Cameron’s Avatar (2009), in which otherworldly landscapes blend seamlessly into a live-action performance, and Christopher Nolan’s film Inception (2010), in which overwhelming scenes blend seamlessly into the narrative.  

Invisible effects blend CGI and live-action footage to create a visual narrative that seems like an extension of reality. The “photorealistic approach” is all about adding a personal touch to visuals, aiming for a level of detail and precision that blurs the boundaries between digitally created and real elements to create an immersive experience. When we dive into hyper-realities in film, it’s as if we’re entering a realm of heightened personal feelings where the familiar is challenged, we’re transported to environments or scenarios that seem more real than our day-to-day life. These terms become our accompaniment to leading us on a personal odyssey through the complex world of filmmaking and storytelling, where each frame tells a personal story.  

On a personal level, artists struggle with the line that separates authenticity from the uncanny valley on achieving photorealism in hyper-realities. It becomes a psychological tightrope walk to achieve a realism that emotionally resonates without it feeling forcefully artificial. In addition to testing technical capabilities, the need for computational power to produce realistic textures and nuances tests the endurance and patience of artists working behind the scenes. 

The growth of hyper-reality is linked to the impact invisible effects have on immersion. Invisible effects not only enhance photorealistic approaches but shape a new generation of virtual environments that blur the line between reality and simulation. Photorealistic hyper-realities transport viewers to a world where the real and the imaginary are blurred, greatly influencing their beliefs on a psychological and emotional level. The audience interact with these immersive experiences through complex visuals that are like reality, evoking a raw response that inspires awe and wonder, and occasionally even disorientation. The heightened realism taps into our deepest emotional reserves and brings out a more intimate connection because it evokes emotions that are like those we experience in our daily lives. Setting out to achieve photorealism in hyper-realities presents a variety of difficulties, much like traversing an emotional terrain in Avatar (2009). The intricate details of replicating reality need a careful balancing act between artistic creativity and technological mastery.  

From the way light plays on surfaces to the subtle facial expressions of characters, every detail demands attention, stretching the limits of imagination and computational power. Ray tracing and high-resolution textures all contribute to a more realistic visual experience, resulting in a wide range of detail that feels almost tangible. Ray tracing, by simulating the behaviour of light in a scene, improves the quality of shadows, reflections, and ambient lighting, simulating the complexities of the real world. This dynamic interplay of light and shadow adds layers of realism to the scene, casting it in a more nuanced and natural light. With their wealth of detail, high-resolution textures bring surfaces to life, from the roughness of a stone pavement to the fine grooves on a wooden table. These textures increase immersion, making the visual experience more captivating and convincing. Seeing the impact of these technologies personally feels like glancing through a window into alternate realities, where the line between the virtual and the real becomes beautifully and remarkably blurred. 

Across a wide range of industries, photorealistic techniques and invisible effects play a crucial role in forming deeply impactful experiences. Invisible effects in filmmaking enables filmmakers to create cinematic worlds that captivate audiences by skillfully fusing the fantastical with the real. Awe and immersion are sparked in viewers when they see characters like Tony Stark and Rocket Raccoon interact with photorealistic CGI in films such as Avengers: Endgame (2019). Experiments done in virtual reality have proven that users who are immersed in environments that closely resemble their real-world surroundings feel more present and emotionally engaged. For example, users who can explore virtual representations of real buildings through simulations experience a strong sense of familiarity. For example, the popularity of open-world games with lifelike settings, like Grand Theft Auto V (Rockstar Games, 2013), shows how gamers are drawn to vast virtual worlds that imitate actual cities. The recreation of urban landscapes with attention to detail enhances player immersion and heightens the gaming experience. Cyberpunk 2077 (CD Projekt Red, 2020) immerses players in the expansive, visually stunning streets of Night City that enhance emotional engagement with futuristic but realistically depicted environments that respond to the players’ actions. They are submerged in stories thanks to a sense of presence and immersion by heavily drawing inspirations from the nightlife of Hong Kong and Tokyo. Photorealistic techniques improve the visualisation of products in advertising. To help consumers visualise the practical uses and advantages of their products, brands use invisible effects to present their goods in realistic environments. A more intimate and captivating marketing story is fostered by this individualized connection made possible by the images. Car ads often use photorealistic CGI in the automotive industry. By seamlessly blending computer-generated visuals with real footage, manufacturers create realistic scenes of cars in dynamic settings. This helps consumers feel the car’s appearance and features in lifelike situations, strengthening their emotional connection and affecting their decisions to buy. On a personal level, these applications prove the transformative power of invisible effects and photorealistic approaches, not only in entertainment but also in how we perceive, interact with, and innovate across industries. The ability to integrate digital elements seamlessly into our physical environment creates a dynamic interplay between reality and imagination, utilising a new era of storytelling and visual communication. 

In the last 40 years, filmmaking has seen incredible progress in creating realistic visuals using VFX. From basic CGI to today’s advanced technologies, the blend of live-action and digital elements has transformed. This seamless integration lets filmmakers craft visually stunning and immersive stories. The level of detail in making things look real has reached new heights, making photorealistic effects a crucial part of storytelling. This innovation not only enhances cinema’s visual appeal but also gives filmmakers more creative possibilities, taking audiences to worlds that seem just like the real thing. The ongoing evolution in VFX technology has not only changed how films are made but also envisions a future where reality and digital creations become even harder to tell apart. Within this 40 year-period, there were 4 films in each decade that perfectly represented the evolution of achieving a photorealistic hyper-reality. 

Ridley Scott’s Blader Runner (1982) through its pioneering photorealistic approach to visual effects, achieves profound immersion. The dystopian cityscape depicting the city of Los Angeles in 2019 feels oddly authentic at the time, with its towering skyscrapers and atmospheric nuances. The seamless integration of practical effects and detailed miniatures, accompanied by groundbreaking visual effects for the time, immerses viewers in a futuristic world resonating with depth and complexity. The film’s overall photorealistic aesthetic is enhanced using matte paintings, practical lighting techniques and atmospheric elements, which make the cyberpunk universe tangible and takes audiences into a visually mesmerizing and emotional experience. 

The groundbreaking use of photorealistic visual effects in Lana and Lily Wachowski’s The Matrix (1999) achieves immersive storytelling. The film’s iconic “bullet time” sequences create a surreal experience through the innovative use of multiple cameras and CGI. The seamless integration of real and virtual elements, combined with attention to detail in crafting the Matrix’s digital world, immerses viewers in a reality-bending narrative. The use of the green coding symbols and visually stunning martial arts scenes in the film adds to the film’s sense of otherworldliness. The film not only shows its era’s technological magnus opus, but it also sets up a standard for immersive storytelling through photorealistic effects, leaving an incredible impression on the cinematic landscape. 

Even for its time, Christopher Nolan’s Inception (2010) achieves immersive storytelling through cutting-edge photorealistic visual effects. The film’s manipulation of gravity-defying architecture and mind-bending landscapes blends practical effects with CGI in a seamless manner. Notably, scenes such as the folding cityscape and the hotel hallway fight demonstrate the film’s dedication to the blend of photorealism and hyper-reality. The complexity of the dreamscapes with realistic detail, such as shifting environments within dreams, captivates audiences and adds to the overall sense of immersion. The film’s use of invisible effects, such as physics manipulation, contributes to a seamless integration of digital elements with live-action footage, resulting in a visually and emotionally engaging experience that challenges the narrative’s boundaries of reality. The film remains considered a testament to its ability to push the envelope. 

Denis Villeneuve’s Dune (2021) immerses viewers in its world through breathtaking photorealistic visual effects. Everything in the film’s depiction of Arrakis, from realistic desert landscapes to futuristic technology, feels genuine. The film creates a cohesive and immersive experience by seamlessly blending practical elements with advanced CGI, such as the lifelike sandworms and replacing green screen with light brown backdrops to further immerse the actors with the already detailed production design in an impressive scale. The attention to detail in both natural and otherworldly elements adds to the sense of presence in the story of main character Paul Atreides’ relationship with the sand dunes, ultimately captivating audiences by creating a visually rich universe that feels incredibly real and tangible. 

Invisible effects and photorealism appear to have a promising future. Artificial intelligence and machine learning advancements may result in more efficient creation processes, allowing artists to focus on creativity rather than technical complexities. Virtual and augmented reality are likely to become more immersive, literally putting the user in the virtual world, blurring the distinction between the digital and physical worlds. Furthermore, greater access to high-quality rendering tools such as Epic Games’ Unreal Engine allows a broader range of creators to experiment with photorealistic storytelling. Looking ahead, the future of hyper-realistic experiences is full of exciting possibilities. Further advancements in AI machine learning may refine the complexities of rendering, allowing for even more lifelike details in digital environments. Real-time ray tracing integration could result in a paradigm shift, providing unparalleled realism in lighting and reflections. Haptic feedback technology advancements may improve the tactile aspect of experiences, increasing immersion. Furthermore, the advancement of 3D scanning and photogrammetry techniques may allow for the creation of digital assets with accuracy that mimic real-world objects and environments. However, ethical concerns about the potential misuse of hyper-realistic technologies may appear influenced by social and political means. 

In conclusion, the journey through the evolution of hyper-reality, driven by invisible effects and photorealistic approaches, has brought us to new horizons of understanding and perceiving what’s real and what could be real. It encourages us to imagine and embrace a future in which the extraordinary becomes the ordinary and the line between reality and simulation blurs into a bigger gallery of storytelling possibilities. Personally, I expect a gradually closer merging of these technologies, fostering a future in which hyper-realistic experiences captivate not only visually but also engage all senses, pushing the boundaries of what is possible in the digital realm. I envision a future in which invisible effects blend seamlessly into our daily lives, enhancing storytelling across multiple mediums and sparking new areas of creativity. 

REFERENCE 

• Blade Runner (1982)  

 Blade Runner (1982). Scott, R. (Director). [Feature Film]. Warner Bros. 

• The Matrix (1999)  

 The Matrix (1999). Wachowski, L. & Wachowski, L. (Directors).  [Feature Film]. Warner Bros. 

• Avatar (2009)  

 Avatar (2009). Cameron, J. (Director). [Feature Film]. 20th Century Studios. 

• Inception (2010)  

Inception (2010). Nolan, C. (Director). [Feature Film]. Warner Bros. Pictures. 

• Avengers: Endgame (2019) 

 Avengers: Endgame (2019). Russo, A. & Russo, J. (Directors). [Feature Film]. Walt Disney Studios Motion Pictures. 

• Dune (2021) 

Dune (2021). Villeneuve, D. (Director). [Feature Film]. Warner Bros. Picture. 

• Grand Theft Auto V (2013)  

Grand Theft Auto V (2013).  [Video Game]. Rockstar North. 

• Cyberpunk 2077 (2020)  

Cyberpunk 2077 (2020).  [Video Game]. CD Projekt Red. 

• Unreal Engine (1998) 

Unreal Engine (1998). [Application] Epic Games.

---