AI & Design
Tools:
Midjourney: Render Ideation, Background Generation
Stable Diffusion: Car Design Learning, Ideation, and Visualiation
Plasticity 3D: 3D Modeling
Keyshot: Render
Photoshop: Post Processing
(Apr 21 update: yea, I still modeled it.)
(It’s not a formal process recording, but more like a blog)
How will AI change our design workflow? Will they replace us? It’s not a design project, but my exploration, experiments and learning 🌞
What AI image generator I’ve tried?
- Midjourney
- Stable Diffusion
- DALL.E 2
I mainly use SD (Stable Diffusion) locally because it’s open-sourced and customizable. You can have most control with the image With SD and train your own model with minimum worry about privacy (only when running locally)
It’s my first attempt to design with AI. I love cars but I’m not a car designer, so this sketch stays in my iPad for almost a year (it’s one of my thesis proposal but apparently there’s a huge gap between industrial designer and car designer😑).
👈So, everything started with this sketch.
What I (or maybe you too) expected for an AI image generator in the very beginning:
Yes, that’s a general image generation workflow, and it do generates amazing pictures.
However, they’re nothing (at least mostly) like my sketch. I used Midjourney at very beginning to help with my design, but since Midjourney provides less control than SD, it’s hard to produce what I really want.
Some Irrelevant Stuff:
Hang on a sec, it’s what happened in the background (from my own understanding). Behind the magic is a gigantic, colossal database born after billions times of training. AI generates images based on its database, but if what you want is not in the database, or the prompts are vague, AI models can not give what you want.
The image training process is, let’s imagine, similar to force you (you are now nearsighted) to guess what object is in distance without glasses in the beginning. Then you guess the object with a pair of clearer glasses each time until you guess it right or see it clearly. After times of training, you’ll be able to guess the object without glasses and portrait it out with what you learned (the database)
So I switched to Stable Diffusion, a diffusion model with more controllable options and can runs locally (runs locally!).
What I’m expecting the AI to help me with:
- Add details
- Produce Variation
- Visualize
Generative isometric based on sideview(almost impossible to realize)Photorealistic render(still learning and trying)
Steps Overview
I won’t talk too much details about every step because tools and models are evolving fast. However, I’m shocked by how effective artificial intelligence can aid my design process. Following are some high lights:
My first attempt with default preset parameter:
The first attempt, though the output is not my desire, demonstrated how capable AI is. It produces visualization with prompts at the speed my hands can not reach. Also, the irritation shows how image generation works, and you can spot many traits of existing cars on those renderings:
A Latter Attempt
After some research and experiment, I can produce sketches with minor or some variation while keeping the original design language.
I also tried to train my first ‘add-on’ model (Hypernet) and tested its effects on the output with different steps trained (150 and 1000 in the image, generally it takes about 10000 steps to create a usable model) and weight. By applying appropriate parameters of the AI model and Hypernet, SD creates convincing visualization with detailed shadow and light.
A random line was generated by mistaken, but SD produced design variation with a hard line on the door in the visualization, giving the car more fidelity. So I went on with the ‘mistake’.
I also generated the design with corrected sketch (without messy lines), but I like the ‘mistaken’ one better.
3rd (maybe it’s the 4th or 5th or higher) Attempt
After many experiments, I handled the process of making variation for this project (only this project): I hand-drew a vector-like image with clean lines and simple colors to let ControlNet (an extra tool to ground the generated design in the prompted shape) to better interpret the shape, and SD model to understand the CFM color. After several irritations, it produced amazing result:
Next: Really Shines
The previous visualization is pretty decent, but still need some revise. So I created another illustration with reduced details to leave more freedom for SD, and lower the prompts’ accuracy.
This is genuinely fabulous
The output stunned me, the AI produced something over my expectation, even imagination. The visualization follows my original design’s CMF and color, but created a more futuristic design with some retro styling, which is exactly what I want. As a car design hobbyist, it’s definitely something I can not achieve in a short time.
I generated more images for more possibilities. By accident, it gave me 100 variations🙃:
At this time, I find AI is actually doing permutation and combination, which…is kinda boring.
Still, I picked two images that match my vision. Together there are three design for me to take further step:
The three variations are truly inspiring, and each has details I like and dislike, so I tweaked the images based on the reality and my preference:
At this stage, I paused my experiment because there’s so much thinking I need to organize and digest, and I need to find a job now 🥲. How will AI impact design, design process, and industrial design?
My Thinking and Learning
First, the question most designers care about: ‘will Artificial Intelligence replace designers’?
My answer is ‘no in the near future’. Design is much, much, much more than a visual presentation. For me, an industrial designer, design is a process to physicalize a solution, whereas visual presentation is just one of the multiple stages. Below is the Double Diamond design model which, from my own experience, applies to most of the design process:
Artificial Intelligence, specifically talking about image generator, majorly impacts the Develop and Deliver stage. Most of those impacts, I propose, will be positive. The diffusion model actually helps me learn a new design field faster, and saves tons of time on form visualization. It is especially a great learning tools for the beginner, like me, a newbie in car design who probably will never dive into this field as a career. AI’s fast visualization enables me to experiment with my design and learn car design efficiently.
AI seems intimidating because its output looks amazing, but it actually never produces ‘new things’. The final visualization of my car design is largely based on my proactive thinking, and it will never design ‘a silver and black futuristic 3-door sport hatchback with retro styling‘ on its own without tons of guide(prompts). Narrowly speaking, AI image generator is like a fast, automation sketch renderer. Designer will still dominate the Design section in the Double Diamond model:
- You’ll never know what prompts to give unless you know what to design.
- Human makes drawing because it’s sometimes much more effective than textual expression.
Now let’s think broadly: how will image generation technology change industrial design?
Upon my short experience on this project, my summarize is (up to Apr 13rd):
The technology allows new designers to learn style effectively, provides more variation for brainstorming, and visualizes sketch rapidly. AI helps us learn and design faster while human designer is still a major, indispensable role.
Pause
(Apr 14th) I asked GPT4 to propose a design solution to let drivers change vehicle’s automation without touching the steering wheel and the center screen (my thesis project). Here is GPT4’s answer:
As soon as the machine learns all our wisdom (output), then what’s the definition of ‘new’? Something more interesting is happening.
🙃 I’m hoping AI can takeover works on boring, meaningless, no-audience, visual-only ‘industrial design’, like this.
Kuangming Qin 