The recent rise of AI art generators has brought to the fore major questions about the role of artificial intelligence in the creative process. We asked production designers to share their thoughts on the current conversation in our industry, and on using AI in their design practice.
Anu Schwartz
"The first step to harnessing AI is to recognize it as a tool, like any other tool we use to create our work."
Alexander Whittenberg
"It is important that we continually adapt to an ever-changing world and rapidly evolving technological realm."
Sara Millán
"Our creative power comes from adding intention, meaning and emotional depth into our creations."
Rose Lagacé
"Compensation, clearances, and government regulation seem to be inevitable outcomes for the use of artificial intelligence"
April 1st, 2023
Should we use artificial intelligence in production design?
Anu Schwartz
Anu Schwartz is a production designer living and working in New York City. His approach to production design is informed by his classical training and education in art history, architecture and fine art painting. Credits include the TV adaptation "Dear Edward", "The Endgame" pilot, "Bridge and Tunnel S1", "The Politician S2", "The History of Future Folk" dir. John Mitchell and Jeremy Kipp Walker. Notable credits as supervising art director include the TV shows "Fosse Verdon", "Maniac" and "Gotham".
Artificial Intelligence or AI, as it is commonly known, has recently been dominating the public discourse. It is seemingly reaching all corners of our collective dialogue. Despite the recent attention, AI has been in our computing systems for decades. The simplest definition of AI is the simulation of human intelligence processes by computing systems. A Google search is a form of AI. Every time we ask Alexa the weather, AI. Auto-correction and spell check are also AI. The seismic shift toward a world dominated by AI is a simple factor of computer processing power and Moore’s Law, computing power doubles every two years. As more of our world is controlled by computing, the closer we move toward machine intelligence.
New technologies are often seen as a threat to labor. In many cases that is true. Automation of the assembly line left a generation of workers behind, and some communities are still feeling the effects of that transition. Artificial Intelligence is no different. The automation of the creative arts will inevitably influence and effect the labor force in those industries. The question becomes how we, as professionals, harness the technology of AI to our advantage. Despite the fears, there are advantages.
The first step to harnessing AI is to recognize it as a tool, like any other tool we use to create our work. A designer’s process for inspiration can include searching the internet and sifting through books and magazines, as well as experiencing art, dance, live theatre, and film. Although highly effective, these are derivative approaches to finding a conclusion. An AI powered prompt is an algorithmic search engine, supercharged to sift through the collective digital mind of society and spit out an image or text from key words that we prompt the tool to search. This too is a derivative process which still requires our participation. The aspect of our work that AI cannot replace is our own knowledge and keen ability to recognize good design, architecture, art and spatial relationship. This comes from training, education and passion for craft.
Education in art, architecture and design has never been so critical to the creation of our work as it is today. No matter what AI produces for us, if we don’t have context, understanding and knowledge for what we seek, we are no better than a computer.
The craft of production design relies heavily on the ability to edit. We respond to visual cues, sorting and combining inspirations into our final work. AI can be just another tool to finding our inspiration and leading us to our final outcomes. AI also has the capability of filling in visual gaps, assembling and reordering pixels to create a new image. This is a new form of art.
Another technology that emerged in the mid 19th century was photography. This was seen by portrait artists at the time as the end of their craft, and in many ways it was. Photography, however, simply altered the way we view art. It forced us to look beyond the image, to visually deconstruct what we see through our eyes, helping to invent a new, dynamic visual language in painting. This new perspective facilitated the creation of some of humanity's most prized works of art. It also democratized the portrait, enabling many people to own a photograph of their loved ones. Prior to that, only the wealthy could afford a portrait painting.
The potential benefits and dangers of AI have captured our imagination and is a manifestation of our fears and dreams for the future. Humanity has a unique knack to always adapt and evolve. Like all systems in nature, evolution and adaptation is key to survival.
Artificial Intelligence is here - adapt or be left behind.
Sara Millán
Sara Millán is a Colombian production designer and art director based in LA. She has worked with a variety of talented directors throughout her career, including Bret Easton Ellis, Brad Furman, Lea Thompson and Mark Boal, among others. Her design credits include: “The Year of Spectacular Men”, “The Deleted”, “East of la Brea”, “Greetings to the Devil” and “Unidentified Objects”.
I find AI fascinating and challenging at the same time. As technology keeps advancing, we are in awe of how rapidly our lives are changing. Everything seems possible, and yet we are unable to anticipate the cost and the impact of these technological advances. The extension of their impact will affect not only one field but many including education, healthcare, agriculture, politics, economics, films, art and more. AI feels like an unstoppable force that has unlocked a world of wonders and challenges for us, pushing us to keep moving forward, to adapt quickly, and to embrace the unknown.
As designer, it is exciting to see how our imagination has expanded with generative AI models like Midjourney and DALL-E 2. Now we can visualize ideas in seconds, explore in minutes multiple possibilities about an specific topic and generate images with higher speed and efficiency. The most interesting aspect for me is when AI generates unexpected results, which allows us to create new connections in our brains and think outside the box. I see AI models as a powerful tool for artists, even though sometimes we feel that our creative process is being reduced to the generation of prompts and the automation of tasks.
While AI models can analyze data and patterns to produce pleasing and high quality images, true creativity involves the capacity to generate new and original ideas. As humans, we are creative by nature. We don’t only create images but art, and that involves various cognitive processes such as divergent and associative thinking as well as problem-solving skills. Our creative power comes from adding intention, meaning and emotional depth into our creations, and these qualities allow us to transcend the technical or merely aesthetic aspects in our work. In this context, AI presents an opportunity for us to learn about its strengths and limitations, and to use it as a tool to create works of art that are truly unique and meaningful.
Change leads to evolution, and AI is changing the way we think, communicate and work. It introduces new methods and tools that allow us to build new paradigms and elevate our craft to a higher level. In the area of art, these new technologies are transforming the traditional mediums and processes that artists have historically focused on. This shift in the status quo presents a window for artists to explore new territories that AI cannot follow us into. Perhaps the future of art lies more in the process of exploration and the generation of new and innovative ideas than in a final result or the pursuit of a technical achievement. Nothing is written, and that leap into the unknown is an opportunity for us to expand our horizons.
The main challenge of these technological innovations is to ensure they are used for the benefit of humanity and not against it. While generative AI models can democratize art by allowing anyone to create their own images without the need to hire a designer or an artist, we should be mindful that there have been ethical controversies around the IP of thousands of artists’ images that were used to train some of these AI models without consent. What this reveals are not the dangers of AI itself, but the intentions of some of those behind these technologies. There are many ethical challenges in the near future and our responsibility is to be informed and to participate in those conversations.
Alexander Whittenberg
Alexander Whittenberg is an Atlanta based designer and art director working in film, television, theatre and theme parks. He is a member of the Art Directors Guild - Local 800. His credits include “Headless: A Sleepy Hollow Story”, “RISE” , “The Mirror Game”, “Love, Courage and The Battle of Bushy Run”, “Bear With Us”, “The Speed of Time” and “Them (S2)”. He recently designed a family comedy called “Popular Theory” which is slated for a theatrical release later this year. Alexander has also worked on a number of projects for QVC, Livesight Creative and Disney Live Entertainment.
The Designer’s toolbox is vast. Sure, the toolbox contains an itemized list of all the hardware, software, books and art supplies that we lug around from job to job, but it also contains all of our memories, thoughts, beliefs and all things unquantifiable. I’ve always believed that, as designers, our work is never done. We must constantly be taking stock of where we are, what we are doing and how we instinctively respond to what is happening around us/to us. I call it listening to the world. To that end, it is important that we continually adapt to an ever-changing world and rapidly evolving technological realm. Artificial Intelligence is at the nexus of this conversation and it’s best to have our fingers on the pulse.
It was mid-2022 when I first fell down the AI rabbit hole. I started my explorations with a program called DALL-E 2. This program markets itself as an AI system that can create realistic images and art from a description in natural language. I spent several sleepless nights diving deep and submitting endless abstract prompts into the engine that would turn around seconds later with a set of images. Typing a short visual description of my dreams into the program became a routine morning exercise. Sometimes these images spoke to me for one reason or another while, other times, the results yielded were utterly soulless. Nonetheless, my curiosity was piqued.
In October, I noticed that my friend Gabriel Aronson, an animator and designer for stage and screen, had started a personal project via Instagram. ARTOMATON (@artomaton) is your Dream Window to the Future Fair of Yesteryear! and it is entirely created using the more advanced and stylistically savvy engine, Midjourney (currently on version 5). The images were mesmerizing and I had to know more about how it was done. Gabriel sent me an example of a prompt that he used to coax a particular image out of the engine. I was instantly struck by how dramatically the prompt differed from the natural language I had used to speak to the DALL-E engine. For all intents and purposes, he had learned a nuanced language in order to best collaborate with the program.
I recently spoke with Gabriel over the telephone to get a better understanding of the proverbial man behind the curtain. “It’s kind of like improv” he said. This instantly resonated with me as I spend a fair bit of my time off-set studying theatrical improvisation (this is a topic for another day but I can’t recommend it enough). In an improv performance, a player will ask the audience for a suggestion (location, relationship, activity) to inspire a scene. From there, two or more players will spontaneously begin to establish characters and dialogue created through a series of back-and-forth offers and rooted in the tradition of “yes, and…”. The offers given by a scene partner are frequently surprising and could quickly move the narrative in an unexpected direction but the wise improviser will listen and support their scene partner’s offer opening up a whole new world of ideas not previously considered. In the case of AI, the engine becomes your scene partner.
Another AI program that I find important to include in this conversation is ChatGPT which interacts with the user in a verbally conversational manner. Whether you need to quickly assemble of list of prominent Mexican architects from the mid 1900’s, or you’re looking for a poetic description of what it feels like to take in the sunset on a Grecian island, or you need the basics of Quantum Mechanics described to you in a single paragraph, this program will provide you with detailed results in a matter of seconds. Using a streamlined approach, this app delivers a starting point from which you can embark on your research journey. It differs from search engines like Google in its effective consolidation of information. Naturally, taking the time to fact-check remains of the utmost importance.
With all that’s been discussed above, the question of morals/ethics/authorship is important to address. Like with all tools, what really matters is the context of how it’s being used. As designers, we are constantly collecting ideas and riffing on our influences. If I were to recreate a structure designed by Eero Saarinen and claim it as my own creation…that’s plagiarism and there’s never a place for that. However, if I were to use the influence of Saarinen and combine that with the influence of Le Corbusier to create a completely unique new structure, that’s a perfect example of channeling inspiration to arrive at something wholly original. In the world of AI, it’s important not to take credit as the creator of the images that are rendered from your prompts. Although there is a certain skill set used to conjure these prompts, we mustn’t forget that the computer is simply drawing from a pre-established dataset (LAION is the dataset used by Midjourney).
The AI train is moving fast and it’s not slowing down anytime soon. It’s a frightening new world and if you look at how the results have evolved in detail and quality over just the past year, it’s rather remarkable. However, it’s worth noting that the dataset used by programs like Midjourney is not updated in real time and draws only from a limited sample of information available on the web. If you pair this information with the fact that only a relatively small number of books/artwork/images have ever been scanned and uploaded to the world wide web, it becomes evident that the seemingly infinite output of unique visual combinations is drawing from a finite fountain of sources. If you’re looking for something you’ve never seen before, traveling, whether overseas or down the street to your local library, will awaken your senses in a manner that a computer never could.
If you know me, you know I feel most at home with a Blackwing pencil, a set of watercolor paints, a sketchbook and a stack of books from the library. I’m as old school as it gets. However, when it comes to using AI in the Production Design process, there are a few methods that I’ve found rather useful:
- During the early phase of research and conceptual exploration on a project, I will sometimes turn to AI to create a series of what I call “visual tone poems”. If nothing else, one of these images might open my eyes to a shape, color or composition that could inspire the next phase of design. Or it might create an image that I’ll include on a mood-board along with other images from other forms of research.
- As stated before, I’ll turn to ChatGPT to assist in my early research efforts mainly as a tool to consolidate information before diving deep at a library, on a field research trip or at a research archive.
- If I’m working on a set that is very heavy with artwork and/or graphics, I’ll use AI as a tool to provide my graphic designers and set decoration team with a foundational layer upon which they can build their own work. The amount of effort and energy spent on clearing artwork can be drastically minimized if you use this technique properly and the volume of deep background graphics can greatly increase.
I leave you with the closing paragraph from the response I received from ChatGPT upon asking, “How will AI affect the role of a Production Designer in film?”
“Overall, AI has the potential to make the role of a production designer in film more efficient and effective by providing new tools and technologies that can assist in the design process. However, it is important to note that AI is not a replacement for human creativity and expertise, and it will be up to production designers to determine how to best incorporate these new technologies into their work.”
You heard it here first folks, straight from the horse’s mouth…AI will never replace the human’s capacity for creativity, expertise and, well, being human.
Rose Lagacé
Rose Lagacé is a production designer based in Toronto, Canada. She has had the privilege of designing "TallBoyz", helmed by Kids in the Hall alum Bruce McCulloch, the "Baroness von Sketch Show", and "Blood and Water: Fire + Ice". Rose is also the founder and editor of Art Departmental, a website which celebrates the art and craft of production design.
At some point over the years, I decided to try my best to be an early adopter of new technologies when I could easily (or even not so easily) adopt the new technology into my workflow. I learned early on that too often those who refuse to accept the future of technology, even the most talented of individuals, can fall behind the times and become what some might deem less “employable.” After all, most new technology is just a new tool or technique to get the job done more efficiently in such an increasingly fast-paced industry like film and television.
In the late Summer and early Fall of 2022, I learned how to use Midjourney and DALL-E as best as I could. In my art department in September, we needed dozens of turn-of-the-century photographs of frightening, doll-like children. Our graphic designer created 60 of these images quickly using Midjourney which turned out perfectly for what we needed. We also created 60 images from what we could source and manipulate from iStock. Not surprisingly, the AI images were far superior. However, as we suspected, legal would not clear the AI images for use so we went with the iStock manipulations that worked well enough. Recently I know more and more legal teams are allowing AI images to be cleared for on-screen use despite rising ethical concerns.
Since then, I’ve used AI to help create mood boards and pitch decks. It’s incredibly easy to create new textures or patterns using the technology. It can quickly pump out dozens of versions of rust or marble on various surfaces, for example, that you just can’t find anywhere else. I can also create very believable imagery of a set with AI tools and a little bit of Photoshop. I’ve also started playing around with ChatGPT which can be used to create text when needed rather than using Lorem Ipsum in early design phases. I am also excited to see how AI might work with Unreal Engine in the future for increased ease of use within the Volume.
More importantly, this brings me to my growing concerns over the ethics of AI. When I started playing around with it, I didn’t give much thought or consideration as to how the technology had been trained to create, nor did I know that they had illegally scraped data to create these images. I don’t feel comfortable with the fact that the hard work of artists was used to create these applications and these artists didn’t permit this use and were not compensated. I also don’t want to see concept artists, graphic designers, or any other careers in our industry decimated by this technology.
I suspect Getty will win their lawsuit over Stable Diffusion and compensation, clearances, and government regulation seem to be inevitable outcomes for the use of artificial intelligence. We simply can’t afford to lose the number of jobs AI could potentially poach if it is not well-regulated, similar to the Napster to Spotify trajectory.
The Concept Art Association has been invaluable in helping me sort out my feelings about the technology and its effects. They also led me to the Human Artistry Campaign which recently unveiled its Core Set of Principles for Artificial Intelligence in Support of Human Creativity and Accomplishment. I am now more comfortable using these principles to guide my ethical use of AI technology.