Name: Generative Image Workflow
Uploaded: 2023-06-07T14:32:40.575Z
Description: Learn how to create generative images using various AI Magic Tools.

Use this tutorial to learn the basics of using Gen-2 in Runway.

- Use Gen-2 via web or mobile (iOS)
- Navigate to Gen-2, and simply type in what you want to generate
- Adjust settings like seed value, interpolation, upscaling, and watermark
- Click “Generate”
- Rate your generation to help the Runway team improve Gen-2
- Generation will automatically save to your Runway assets folder
- If you’re feeling stuck or need inspiration, view Gen-2 inspiration feed below the prompt bar to remix existing prompts
- Gen-2 also works with an image reference
- Helpful to structure prompts like this:
    - base prompt
    - “in the style of”
    - description


Gen-2 Basics

Gen-2

Use this quick tutorial to learn how to use Runway's mobile app for iOS.

- Download the Runway app from the App Store app
- Sign into Runway using the same account you use on the web
- Circular purple button will initiate a new generation
- 1 second of video generation = 14 credits
    - To top up credits, click profile icon in the top right corner of the Home page and tap “Credits”
- To create a video, simply hold down the record button and let go when you’re finished
- Three main style settings:
    - Presets by Runway
    - Reference image: click “Add” to choose an image from your library; adjust style strength and seed number here
    - Text prompt
- Hit “Generate” — video will be queued and show a percentage status circle
- Generate new options in the meantime; videos will continue generating in the background
- Generations will be stored in library tab


Using the Mobile App

Runway Mobile

Learn the basics of using Gen-1 in Runway.

- Head into AI Magic Tools in Runway to find Gen-1
- Select an asset
- Three style reference options:
    - Image (choose your own image)
    - Presets by Runway
    - Prompt (enter a text prompt)
- Keep settings simple or dive into advanced settings: https://help.runwayml.com/hc/en-us/articles/15161225169171-Gen-1-Gen-2-Advanced-Settings
- Find recommendations for settings by hovering over tool tips


Gen-1 Basics

Keep your Gen-1 results sharp and polished with this tutorial.

- Use this tutorial to get your Gen-1 results to be clean and as close to the original video as possible
- Upload the video to your Runway account, and then open it in Gen-1
- If you’ll be using a reference image, it’s helpful to use a tool like Image to Image to generate the look you’re going for, so the image and videos line up 1:1
- Open “Advanced” in Gen-1
    - Turn “Structural consistency” to 0
    - Turn “Weight” to a lower number (between 3-7 works well)
    - Turn “Frame consistency” to 1.2
- Click “Generate”


Clean Results with Gen-1

Use Gen-1 to turn your videos into animated sketchbooks.

- Upload you video and open Gen-1
- Find or generate a reference image that has a hand-drawn, sketch-type feel to it
    - It doesn’t need to resemble the source video; the art style is most important
- Open “Advanced”
    - Turn “Structural consistency” to 0
    - Turn “Weight” to 5 or 6
    - Turn “Frame consistency” to .6
- Click “Generate”


Sketchbook Look with Gen-1

Looking for Gen-1 inspiration? Use this tutorial and experiment with unexpected results.

- Upload the video you want to use as your driving source
- Incorporate a prompt that is wildly different or opposite from the source video
- Open “Advanced” in Gen-1
    - Turn “Structural consistency” to 6
    - Turn “Weight” to 4 or 5
    - Turn “Frame consistency” to 1.2
- Click “Generate” and expect the unexpected


Unexpected Results in Gen-1

Learn how to take your Gen-1 workflow to the next level by combining multiple AI Magic Tools.

- Head to Text to Image in AI Magic Tools to generate images to use as a reference image in Gen-1
- Edit the generated image in Erase and Replace or iterate in Image to Image
- Use Inpainting to remove subject from the scene, then use Green Screen to extract the subject by itself, for complete foreground and background separation
    - Affect the background with one set of prompts, and affect the foreground with another
    - Combine them back together for unique results!


Gen-1 Workflow w/ AI Magic Tools

Turn simple comps into compelling videos with Gen-1.

- Gen-1 source video doesn’t have to be a “normal” straight out of camera video; try creating a rough composite instead!
- Compositing = combining multiple visual elements for a cohesive final output
- Take rough mockups into Gen-1 with an image prompt for surprising results


Using Composites in Gen-1

Gen-1

Use this tutorial to learn the basics of AI Training in Runway.

- Gather your images… here are some tips:
    - Use at least 15 images
    - Subject in focus
    - Different lighting/context
    - Only contain subject you’re aiming to train on
- Give model unique and simple keyword
- Train your model — it should take approximately 30 minutes depending on number of images, image sizes, etc.
- Try using your model in Text to Image for further generations!


Introduction to AI Training

Learn how to continue using your model in Text to Image for hyper-customized images.

- You aren’t limited to the first 100 images — continue using your model in other mediums, like Text to Image
- Important tip to keep in mind: include the name of your model in the text prompt in addition to selecting the model from the dropdown
- Add in-depth description to fine-tune results


Using Your Model

Learn how to create custom art styles with AI Training.

Step 1: Upload at least 15 pieces of artwork (make sure the style is similar across the board).
Step 2: Give your Custom Style a unique keyword. Then, train your model.
Step 3: Head over to Text-to-Image to start creating. Make sure you include your keyword in your prompt and that the model is set to your custom keyword.
Step 4: Go wild generating new images in your own unique style.
Step 5: Want to get really wild? Try using your new custom style model with Image-to-Image to transform existing images into your unique style.


Creating Custom Art Styles

AI Training

Learn how to create generative images using various AI Magic Tools.

- Learn how to create a generative image using AI Magic Tools
- AI Magic Tools used:
    - Text to Image: creates an original generated image from a text-based prompt
    - Erase and Replace: eliminates a specific part of an image and replaces it seamlessly with a new image generated from a new text based prompt
    - Image to Image: uses an image as the input, and generates a brand new image based on both the original and a descriptive text based prompt
- When writing prompts, be descriptive. Include object colors, orientations, and artistic styles in your prompt.
- Refine as you go — the initial output does not have to be perfect. You can slowly work and refine your image with tools like Erase and Replace and Image to Image to gradually get your content to where you want it.


Generative Image Workflow

Learn how to use AI Magic Tool Infinite Image.

- Find Infinite Image in the AI Magic Tools section. Start with a text prompt to generate an original image, or upload your own image.
- Move the generation frame to desired area. Make sure it’s overlapping the original image!
- Be descriptive with your text prompt and click “Generate.” You’ll then be presented with 4 initial generated options to choose from.
- Repeat this process as many times as you’d like to continue extending your image.
- Use adjustment button to adjust prompt weight; lower for more surprising and creative results, higher to stay more true to the prompt.
- Try opening your final image in Erase and Replace to clean up any elements in your image you’d like to change.


Infinite Image

Use this lesson to learn tips and tricks for using Text to Color Grade.

- The main thing to keep in mind: you can never go wrong with natural language.
- Typing in a movie title alone might be counterproductive sometimes; add in a description to make the movie title interpretation less literal.
- Same goes for specific film stocks — add additional descriptors.
- Try different seed numbers for different options.


Text to Color Grade

This short video goes over how to use Runway’s Blur Faces AI Magic Tool, to automatically censor your videos.

- Find Blur Faces in AI Magic Tools section of Runway
- Import a video and Runway will get to work scanning the video for faces
- A square blur will appear over the faces and be tracked throughout the video
- Control Blur and Padding, or adjust View Mode to toggle between views
- Use “Remove Face” to refine face selection
- If you’re not happy with the result, try scrolling to a different part in the video, and try re-detecting


Blur Faces

Learn how to add a bokeh effect to your videos using the Bokeh AI Magic Tool.

- Bokeh = aesthetic from blur, derived from shallow depth of field
- Add a video and Runway will automatically add a bokeh effect
- In the settings panel, toggle to switch view modes
- In “Editing” mode, hover over different parts of the video to apply the effect to certain areas
- Control focal distance, amount, shapes, and depth of field adjusters to refine further


Bokeh

Automatically split your video scenes using the Scene Detection AI Magic Tool.

- Scene Detection takes an edited video with multiple scenes, and splits it into individual clips
- Upload your video file and Runway will automatically start detecting clips and break them out into individual shots
- Export All Scenes — find your assets neatly organized in a folder


Scene Detection

Learn to use the Backdrop Remix AI Magic Tool, and remix your images into anything you want them to be.

- Backdrop Remix changes the background of your photos while keeping the subject in tact
- Upload your photo, and choose a preset or a custom prompt
- Click “Generate,” and Runway will get to work replacing the background automatically — that’s it!


Backdrop Remix

Learn how to seamlessly transition images into one another by using our Frame Interpolation AI Magic Tool.

- Frame Interpolation works well to create videos with subtle movement from still images
- The more frames you have to work with, the better
- Keep the differences between photos subtle if possible for realistic effect
- Try using Erase and Replace to create new images to interpolate between!


Frame Interpolation

Bring still images to life and learn how to create a living portrait in Runway.

Step 1: Brush out and replace the parts of your image you'd like to change using either Erase & Replace or Infinite Image.
Step 2: Head over to Frame Interpolation to animate your various images into a video.
Step 3: Behold the beauty of your new living image.


Create a Living Portrait

Learn new Image to Image techniques in this short tutorial.

Step 1: Select you source video then upload it to Runway. Green Screen your subject. Then export as a PNG sequence.
Step 2: Using the first frame from your PNG sequence, head to Image to Image and start to define the style you like.
Step 3: Using the same settings, seed number and prompt, repeat Step 2 for every other frame in your sequence.
Step 4: Load all of the results from Step 2 and 3 into a new timeline, cutting each one down to a single frame. Make sure you're sandwiching them all together without any gaps.
Step 5: Export the sequence as an MP4 then take it back into Green Screen. Once again, remove the background.
Step 6: Either repeat the steps above with a different asset to create a background you like or combine your results with existing footage.
Step 7: Keep experimenting. Finding the look you like can take time so don't be afraid of trying different things. The more you play around, the more you'll unlock creatively.


Image to Image Techniques

Use this tutorial to learn how to use the Text to Image AI Magic Tool in Runway.

- Head to AI Magic Tools to find Text to Image
- Enter a simple text prompt
- Open up “Advanced” under “Settings” sidebar to explore refinement options
    - Size, style, medium, mood, prompt weight
- Prompt weight will determine how much the text prompt is taken into consideration; higher number = most consideration, lower number = least
- Experiment with more complex prompts
- “Add to assets” to save to your Runway asset library
- “Download” to download locally to your computer


Text to Image

Learn how to use AI Magic Tool Erase and Replace to easily modify images.

- Head to AI Magic Tools to find Erase and Replace
- Choose an image asset from your computer or asset library
- Adjust brush size using the slider on the right sidebar
- Brush over the portion of the image you’d like to replace
- Enter a text prompt and click “Replace”
- “Add to assets” to add to your Runway asset library
- “Download” to download locally to your computer
- Use “Clear” to generate something new with a new prompt


Erase and Replace

Transform a low resolution image into a high resolution image with Upscale Image.

- Use Upscale Image to add more detail to increase image resolution
- Upload an image asset
- Select desired size under “Scale” and click “Process”
- That’s all there is to it!
- Download the image or add to your assets by hovering over it and clicking the icons on the top right


Upscale Images

AI Magic Tools

Learn what compositing is and how to unlock new editing skills.

- Compositing doesn’t have to be intimidating; it isn’t just for big Hollywood projects or visual effects studios.
- Editing tools are becoming more accessible and easy to use.
- Compositing: taking parts of multiple images and combining them to create one image.


What is Compositing?

Walk through a full compositing example in Runway using Green Screen, Inpainting, and Motion Tracking.

- We’re going to use a combination of tooling in Runway to composite this example: Green Screen, Inpainting, and Motion Tracking.
- Drag desired background layer into the timeline below your subject layer.
- To remove the background from the top subject layer, select Green Screen to mask the subject out. Refine as needed.
- Track the background layer using Motion Tracking to emulate the movement in the foreground layer. Tip: select an area of high contrast for best results.
- Rescale or adjust position of tracked background layer to fill the frame.
- Use Inpainting to paint out any undesired objects or people in your background layer.
- Correct color of background to match foreground layer by using effects and LUTs. In this example, we use a LUT, the Hue/Saturation/Lightness effect, and a small amount of blur.


How to Composite

Compositing Made Easy

Learn blend mode fundamentals in this introductory lesson.

- An understanding of blend modes is an understanding of how colors from different clips interact with each other under different conditions.
- Simply put: base color, blending color, and result.
- This course will take you through what each blend mode does, and how they work best in different scenarios.


What Are Blend Modes?

Walk through various scenarios for using blend modes to learn realistic and useful use cases.

- Add texture with blend modes: give text, shape, or background an added layer of life or character
    - Try: Linear Dodge when blending a texture layer and a text layer for a unique visual effect
- Use blend modes for color grading: improve color of an image or jumpstart color correction
    - Try: Multiply to decrease brightness of the highlights while decreasing darkness of shadows
- Blend modes can be used to composite picture-in-picture videos
    - Try: Darken to ignore the lighter value and keep the darker values
    - Use Motion Tracking (coming soon!) for bonus precision


Using Blend Modes

Working With Blend Modes

Learn the best tips and tricks for shooting and editing vertical videos.

- Vertical video is ubiquitous due to the popularity of smartphones and social media.
- This course will show best practices and tips and tricks for shooting and editing vertical videos.


Intro to Vertical Videos

Learn to make the right decisions for shooting videos to ensure they will work in a vertical or portrait setting.

- Have an understanding of the larger goal at hand. For example, will the video be vertical only once published? Use these foresights to inform your shooting decisions.
- Shoot in 4K wide if your video will be shown in multiple aspect ratios; more pixels = more flexibility. This will provide more cropping options in the editing room.


Shooting Vertical Videos

Get creative by styling your frame to fit the vertical setting. 

- Sometimes you’ll be working with video that was not originally intended to be formatted vertically; you’ll need to style your frame.
- Create dividers, add text, additional images, etc. to avoid having to stretch the video clip.
- Scale your video and blur for a tried and true visual effect.
- Try out Runway templates for inspiration!


Editing Vertical Videos

Creating Vertical Videos

Sound is a crucial element in creating a comprehensive and easy to interpret storyline. Learn more on sound design and sound mixing in this course!

- This course will teach you about sound design and sound mixing in video.
- Sound adds just as much value as any other visual element. Sound can add necessary context for audience understanding.


Intro to Editing With Sound

Editing With Sound

Learn a beginner-friendly workflow for creating a product tutorial or demo from scratch.

- Teaching others through video is becoming increasingly popular for growing businesses, creating online educational content, or sharing information.
- This course will take you through how to record and edit your own tutorial or product demonstration.


Creating a Product Tutorial

Watch this short lesson to learn about the options for recording your screen.

- Some suggestions for screen-recording softwares:
    - Loom (what we use here — simple and easy to use)
    - CloudApp
    - Camtasia
    - OBS
- Select screen section or app, or record entire screen.
- Double check you’re using the intended webcam.
- Don’t stop recording! We can remove the unwanted portions later.
- Finally, download the file to bring into Runway for editing.


Recording a Product Tutorial

Pick up on some handy shortcuts to make cutting down your footage a breeze.

- Upload and drag your recorded file into the timeline.
- Play back video to begin cutting out unwanted footage. Continue throughout the video until you’re happy with the raw recording; we’ll polish it all up in the next lesson!


Assembling a Product Tutorial

Finish off your tutorial presentation by adding in extra effects and intro slides to make things engaging.

- Try adding a title to the beginning of your composition as an introduction. Create additional space by dragging footage to the right in the timeline, add text, and a solid background.
- Check out the effects library to enhance your edits.
- Add music to add ambience, and take advantage of AI Magic Tools like Clean Audio or Beat Snapping.
- Add subtitles and customize.
- Export your video, download, and share!


Polishing a Product Tutorial

Showcasing Your Product

This short course will teach you how to create a stunning product advertisement from start to finish — no big budget or special equipment required.

- Product ads are made to intrigue, inform, and capture attention.
- Have a target or vibe to aim at before you begin creating. Collect references to create a mood board, and refine until you have a clear vision.


Product Ad Basics

This lesson will teach you how to make the most of the filming equipment you already have at home, with some extra tips on shooting.

- Don’t get caught up in special equipment; use what you have! Your phone as a camera, your lamp as studio lighting. Editing will help you out.
- Make a storyboard comprised of still images and notes to depict the actions in you video. Then, try your best to recreate those shots from your references.
- Shoot more than you think you need. It’s better to have too much footage than not enough.
- Natural light is your best friend.


Shooting a Product Ad

Learn the fundamental steps for editing down your footage into a complete product ad creation.

- Start by pulling “selects” from your footage; choose your best shots that you’d like to incorporate into your final edit.
    - Rename your shots and be intentional for even better organization.
- Find music to help with overall pace, mood, and tone. Utilize convenient Runway features like Beat Snapping to make editing easy.
- Create a rough cut. Get your shots in the general order you want them to appear in, and try to match them up loosely to the music. Get a feeling for if the edit is going to work and make adjustments as needed.
- Use Runway AI Magic Tools like Green Screen to enhance your edits.
- Bring your edit to life with added animation.
- Make a “beauty” pass; tweak fine adjustments, color grading, and effects to make your edit feel more professional and polished.
- Time to make a final pass! Add text, animations, sound effects, additional video clips, etc. to totally complete your composition.


Editing a Product Ad

Creating a Product Ad

Get an overview of what rhythm is, and how it affects video editing.

- Rhythm affects cohesion, and influences how your audience may receive a message.
- Rhythm is comprised of a series of beats.
- The pace of your visual cuts help to define the rhythm of your overall edit. For example: shots that are longer between cuts = slow pacing; shots that are shorter between cuts = fast pacing.


What is Rhythm?

This lesson will teach you three simple steps on how to express feeling through rhythm and pacing. 

- Use music for the foundation of establishing pace. The beats can act as a great starting point for cutting down your footage.
- Observe, and adjust pacing accordingly. Think about the story you want to tell, and how you want viewers to feel.
- Create an X-factor to inject uniqueness into your edits. For example: surprising the viewer by breaking a rhythm that you previously established.
- Get inspired by paying attention to videos and films, and how pacing contributes to the feelings evoked.


Editing With Rhythm

Understanding Rhythm

This course will teach you the basics of motion design, and how to implement your learnings to create engaging content of your own.

- Motion graphics and motion design simply boil down to designing movement.
- The lessons in this course will explore motion design fundamentals, keyframes, and easing.


What is Motion Design?

Learn the fundamentals of motion design and pick up some tips on how to make animation decisions.

- Make sure to spend time on design. A great design can tolerate basic motion, but a poor design is tougher to enhance.
- Add to your designs with complementary animation choices. Think about how your design interacts with different movements.
- Fundamental properties of motion design:
    - Position: the X and Y coordinates of your media
    - Scale: how big or small your media is
    - Rotation: the circular movement of your media around a pivot point
    - Opacity: how opaque your media is
- Make your designs more realistic by incorporating easing to express anticipation and overcorrection.
- Don’t forget to keep it simple! Even the most subtle motion can make a significant impact.


Motion Design Fundamentals

Learn the basic functions of keyframes, and how to use them in Runway.

- What are keyframes?
    - Markers that video editors use to indicate how they want their video to look at a specific point in time.
    - Can control properties such as position, scale, rotation, opacity, and effects.
    - Transitions are automatically created by video editing softwares between each keyframe.
- Learn how to create and use basic keyframes in Runway.


How To Use Keyframes

Learn how to use motion easing and animation smoothing in Runway.

- Prevent your animations from looking stiff and robotic by incorporating easing.
- Utilize Runway’s animation velocity presets or adjust curves manually for custom movements.
    - Ease in: affects how the graphic leaves point A
    - Ease out: affects how the graphic arrives at point B
    - Ease both: affects how the graphic both leaves point A and arrives at point B
- Pull easing curve handles outside of bounding box to express anticipation or overcorrection.
- Time to put it into practice!
