Venez(IA), where cinemas meets Artificial Intelligence

HAI and EDI Effetti Digitali Italiani: ‘With “The Last Image”, we are showing the future of AI generated in cinema.’

The benefits of the new workflow with all the hallmarks of cinema: creative control, savings of up to 30–50% and more space for new directors.

The future of cinema (also) lies in artificial intelligence, as explained by Francesco Grisi, Francesco Pepe and Stefano Leoni, the team behind EDI Effetti Digitali Italiani, leaders in the VFX sector and newly founded HAI Human & Artificial Imagination, at a meeting held at the Casa della Critica in Venice 82 entitled ‘Venez(IA), where cinema meets Artificial Intelligence’, presenting a technical and creative document that explains how generative AI is changing the way cinema is made, without distorting the role of the artist.

Until now, AI has been discussed as a tool that eliminates the need for human involvement. Our approach is to redefine the use of AI in filmmaking. It has always been a collaborative effort to achieve perfection. AI is not a substitute for human labour, it is an amplifier of creativity and efficiency. Our task is to govern it: to choose when to use it, why and with what guarantees of quality, budget and rights”, says the team.

After six years of research, since the first tools were developed, HAI believes that it is now possible to work with artificial intelligence while maintaining control and integrating it into the filmmaking workflow without compromising its principles. ‘AI helps us to create new tools and assist producers and authors in finding new solutions.’

The advantages are: simplification and cost optimisation; being able to maximise creativity at a sustainable cost; being able to enhance and improve preparation (pitching, previz, storyboards); automating repetitive technical processes.

More streamlined VFX, less full CG when not needed

AI reduces the need for expensive and slow full CGI pipelines, focusing resources where they really make a difference. It speeds up or automates tasks such as camera tracking, masks, animation, rotoscoping, colour, upscaling, and the production of complex or costly drone shots/establishing shots on set. It also opens up new ways of interpreting historical or anachronistic locations.

‘Whereas before we aimed to do everything in post-production, the future is AI first. Combining it with VFX is a very powerful tool’.

AI allows you to control camera movement based on simple photographs, enabling you to transform existing locations into completely different settings, such as a fishing village or a fantasy setting. This gives you control over the geometry and geography of the location, making it easier to preview and assess the feasibility of projects.

Managing crowd scenes, for example, has always been costly, but AI, particularly machine learning, now offers new solutions for generating crowds, even from green screens. This allows for the creation of crowd situations with a high degree of control, even if the crowd is not CGI, optimising costs and production times.

Furthermore, AI can be used to speed up previewing, finalisation and compositing processes, drastically reducing the number of versions required for approval. This leads to significant cost savings, as the director can preview the results earlier and make decisions more quickly.

Enhanced pre-production, automated processes (Auto Lab), accessibility and a wide range of options

With pitching, previewing, storyboarding and concept art, HAI makes decision-making processes more robust and rapid for directors and producers, helping projects get off the ground with greater visual and budgetary clarity.

Proprietary pipelines accelerate key technical steps: tracking, animation, colour grading, rotoscoping, upscaling. The result: more time spent on creative value, less on repetitive work.

AI enables similar quality on different budgets, gives space to new emerging directors, and allows you to do more for the same cost or spend less while maintaining quality. In the cases analysed by HAI, savings can range from 30% to 50%, paving the way for projects that would otherwise be stalled due to being over budget.

‘It’s a democratic tool that gives more space to emerging directors and new authors. We help producers understand how to spend their money and promote projects with more visual effects’

To demonstrate how AI can unlock highly complex scenes without compromising quality or project scope, HAI highlights several sequences that, if shot live, would involve costs and risks that could jeopardise the entire budget. AI allows you to protect your creative vision while keeping your project sustainable.

For example, de-aging, which used to be extremely expensive and is now, thanks to machine learning, accessible at a very low cost, allowing actors to be rejuvenated or aged in a short time. In addition, it will be possible to animate characters at low cost, retiring motion capture suits and sensors, thus facilitating the creation of animated films or films with digital characters.

‘We are not in conflict with creativity, but we enhance performance; we are an empowerment tool.’

Another application of AI is Autolab, a highly automated video lab that delegates all non-creative tasks, such as transcoding, conforming, rendering and archiving, to the machine. Creative tasks are simplified for professionals, and the system manages the workflow from shooting to final master, integrating audio, graphics and colour.

Workflow: the example of “The Last Image”

To demonstrate this, the team has created a short film, the first to be made using generative artificial intelligence while maintaining control over the entire production process. It will be screened for the first time on 15 October in Rome and on 16 October at the Anteo cinema in Milan.

The title is ‘The Last Image’, directed by Frankie Caradonna and set in 13th-century England. The protagonist is Adam, a young farmer with a pure heart, who is locked in a prison carriage bound for the gallows. As he awaits execution, his thoughts intertwine with memories of the woman he loves, Charlotte, and his best friend, Clarence. Together they had dreamed of escaping a miserable life, planning a robbery to secure a future elsewhere.

To do this, they created a new workflow, a work entirely generated but with direction, photography, and set design, not simply a prompt for its own sake. A new way of thinking about and structuring a short film or a film scene to help producers.

Starting from a subject to develop the screenplay, then studying the characters, the storyboard, then a visual breakdown with instructions from all department heads, including photography, costumes, set design and location managers. The first images created are static and, once approved, are generated in motion. Meanwhile, the editor gathered information to pass on to post-production, a crucial activity for refining the product generated by AI, as the raw image needed to be carved out to achieve superior quality and stylistic definition.

‘If we had actually had to do it, we would have needed a budget ten times higher. The goal is not to cut jobs, but to enable the same people to carry out more projects, focusing on activities with greater creative value.’

Tax Credit and copywrights

Everything that is post-produced has no legal issues, but authors and actors who have to transfer machine learning services must be contracted. ‘What we will do is completely protected because it will be done with internally trained models, so the material will not be sent to various artificial intelligence models.’ And once these guidelines are followed, there are no difficulties in obtaining the tax credit.