Introduction

Writing prompts for generative image and video models is a new skill that builds on communication abilities you already have. Just like giving creative direction to a colleague, prompting requires you to articulate your vision clearly.

The key difference is that generative models interpret your words more literally and lack the shared context that a colleague might have. If you prompt "a beautiful landscape," the model doesn't know whether you envision mountains at sunset or a tropical beach at noon—both interpretations are technically correct, but may not match your vision.

This comprehensive guide explains how to write effective prompts for both Text to Video and Image to Video modes by starting simple and adding detail strategically, using positive language, and embracing iteration as part of the creative process.

Iteration as a process

Before diving into techniques, understand that getting the perfect result may not happen on the first try. Creative work—whether you're writing, designing, or filming—involves drafting, collaborating, and refining. The same is normal and expected when working with generative media.

Think of prompting as a conversation with the model: You make a request, review the response, then clarify or expand your request based on what you see. Each generation teaches you something new about how the model interprets your words.

This iterative approach is often intentionally leveraged in workflows by seasoned creators, as it lets you refine your vision while simultaneously exploring possibilities you hadn't imagined.

Prompting iteration strategies

There are two main approaches when writing an initial prompt. Each has distinct advantages depending on your workflow:

• Starting simple lets you add one element at a time and see what each change does. 
• Starting detailed can reduce the total steps in your iteration process, but can be more challenging to iterate.

Extremely complex, multi-paragraph prompts reduce the room for creative freedom a model has, constraining it to operate within tightly defined parameters. This over-specification can paradoxically lead to unexpected or unnatural results, as the model struggles to honor every detail simultaneously.

Core prompt elements

This section dives further into specific elements you may prompt for in either generation mode.

Text to Video

Text to Video models transform written descriptions of scenes into video. Effective prompts describe what appears in the frame and how those elements move through the scene using direct, clear language. You're building everything from written text prompts—composition, subjects, environment, lighting, style, and motion.

Effective text to video prompts contain at least two essential elements:

• Visual descriptions — Describes what we see, where, and how it looks
• Motion descriptions — Describes how the scene moves and behaves

These elements may encompass multiple components:

Image to Video

Image to Video models transform images into videos with a text prompt to guide motion. When using this mode, you upload an image to define composition, subject matter, lighting, and style that guide the video. Your prompt's role is to describe what should happen—the motion, camera work, and temporal progression you want to see using clear, direct language.

Effective image to video text prompts focus almost exclusively on motion. Rather than describing elements present in the image, use your prompt to describe the motion of the scene.

Motion Components:

• Subject action
• Environmental motion
• Camera motion
• Motion style & timing
• Direction & speed

To control individual elements from your image, refer to characters and objects with general language to isolate them and define motion.

Do I need to include every component in my prompt?

No, you do not. Omitting certain components grants the model creative freedom to produce your video. We recommend starting with a simple prompt that focuses on the most critical motion components and then adding more detail to refine as needed.

This approach to iterating helps you understand how additions and changes may affect your results.

Are there situations where I should describe visual components?

Yes, there are cases where visual descriptions can be helpful:

• Introducing an element not present in the image
• Dramatic changes from the starting image
• Specifying transformation details
• Specifying interactions between two (or more) elements

Image Prompt

Your input image acts as the first frame and provides the model with the composition, subject matter, lighting, and style information for the video.

For best results, ensure that the input image is high quality and free of visual artifacts. Visual artifacts, such as blurry hands or faces, may be intensified once your image is transformed into a video.

Best Practices

Use positive phrasing

Describe what you want to see rather than what you don't want. Most models respond better to positive instructions.

Avoid ambiguous or conceptual language

Be specific and concrete. Instead of abstract concepts, use clear, observable descriptions.

Avoid conflicting instructions

Make sure your prompt elements don't contradict each other. Conflicting instructions can confuse the model and produce unexpected results.

Prompt Structure & Organization

You don't need to follow a strict formula to generate great results. Structure and order are far less important than clearly conveying an idea and reducing ambiguity.

However, establishing an organization method can assist with effectively conveying ideas and make future iteration and adjustments easier. We recommend trying this structure if you're new to generative media:

For Text to Video

[Camera] shot of [a subject/object] [action] in [environment]. [Supporting component descriptions]

For Image to Video

The camera [motion description] as the subject [action]. [Additional descriptions]

Below are examples of this structure in practice.

Advanced Techniques

Sequential Prompting

Sequential prompting provides an order of events for temporal control. This can be done through natural language, or by providing rough timestamps for an action to occur:

• Natural language: X occurs, then Y occurs. Finally, Z occurs.
• Timestamps: [00:01] X occurs. [00:03] Y occurs. [00:04] Z occurs.

For best results, consider if the requested sequences make sense with the selected duration. You may opt for higher durations for more complex sequences.

Creating Longer Sequences

Create longer sequences by extracting the last frame of a completed generation and using that as the image input for a new video.

To extract the last frame:

1. Move the playback scrubber to the very end of the completed video
2. Select Use from beneath the video
3. Select Use current frame

This will load in the selected frame into the current model. Once the generation completes, you can combine both clips in a video editor to adjust timing and remove the shared frame.