Animated Video Creation Workflow with Midjourney and Runway

TIPS FOR CREATING ANIMATED VIDEOS WITH MIDJOURNEY AND RUNWAY The first thing I want to make clear is that I am not a guru, just someone who has created two short films so far and is preparing the next one. I thought my experience might be useful to those who are just starting out. This is not and is not intended to be a tutorial, just a guide with recommendations. CHOOSE A GOOD IDEA It all starts there. It must be a feasible idea, and if possible, an original one. To make it easier to maintain visual coherence (this is fundamental), I recommend choosing a cinematographic style (science fiction, horror, comic, etc.) or the style of an author, as I did with Dalí, and use it in the prompt to generate the images. To find out if the idea is feasible and the problems you might encounter, you can download a couple of images and test them in Gen-2 of Runway. If they turn out badly, it's probably best to discard the idea and move on to another one. I'm sure you know that Runway works on a per-credit ('seconds') basis, so unless you have an Unlimited Plan, I recommend you be very selective. STORYBOARD Once you have chosen a viable idea, it would be good to make a storyboard, even a mental one. Try to visualise the story you want to tell. Maybe it's a story that already exists, like, in my case, The Wizard of Oz. That will make it easier for you. When you have this clear, start generating images in Midjourney. Here you will have no problem creating as many as you need. Important: images must have a 16:9 aspect ratio in order to be displayed full screen. Remember what we talked about visual coherence? This is where it comes into play. Choose images that maintain a visual line. Keep in mind that many of them won't work when you animate them, so it's a good idea to have a lot to choose from. You can also generate more as you edit the video. ANIMATE IMAGES WITH RUNWAY I'm not going to kid you, Gen-2 is a lottery. No matter how careful and selective you are, some images will come out wrong: without movement or distorted by excessive movement. There is no control. However, I have drawn a few conclusions: - Landscape images tend to work well most of the time. It tends to make a very smooth tracking shot that looks really good. - Frontal shots of the characters fail a lot of the time. With Dorothy I had real problems. If you want to be sure and it doesn't look bad, it's better to have the back turned. - The eyes are usually the worst. They have ruined many animations for me. - Note the type of composition of the image. If it is a character on a path, the animation will most likely show the character walking. Also, if there are dynamic elements such as balloons, it is easier for it to animate (although it can get out of control). I signed up for the Standard Plan, and it was too short. I finished the credits in two or three days, so then I had to buy more credits. If you want to create a test video, it will work, but if your idea is to create two or more videos, maybe you should sign up for another subscription. Yes, Runway is expensive. VIDEO EDITING WITH CAPCUT If you have some experience in video editing (Flash, Edge, Animate, Avid, Premiere), this free application will be very easy to use. If you know what Keyframes are, that's enough. The rest is exploring effects, transitions and choosing audio tracks. A good number of them are included, but you can add whatever you want. The soundtrack is very important when creating a short film. When you see your images animated to the beat of the music, it's as if you've entered another dimension. Be careful also when choosing fonts for credit titles, that they keep in line with the visual style. For me this is the worst thing about CapCut. It's not very well done. There is no predetermined length for a short film, but I like it to be around 01:20. If it is too long, you may lose the user's attention, which in social media is limited. Keep in mind that each animated image is 4 seconds, so you'll need 15 images to get to one minute long, and 20 for 01:20. You can also use a static image in the intro or in the closing. So that's my recommendations. I'm sure I'm forgetting a lot of things, but we can discuss them in this post (the longest I've written).