Reference to video: how it works and how to do it well

Reference-to-video vs. text-to-video and image-to-video

Text-to-video starts from words only. The model decides what the subject looks like, so two clips with the same prompt show two different people.

Image-to-video starts from one picture and animates it. The first frame matches your image. Everything the image does not show, such as the other side of a face, is invented as the camera moves, and every new scene needs a new starting image.

Reference-to-video separates the subject from the scene. The references say who. The prompt says where, what happens and how the camera moves. The model can place the subject in a scene none of the images show, from an angle none of them show, and still keep the identity. That is why it suits series, ads and short stories.

How the model reads your references

Most reference models let you point at each input in the prompt. Kling's Omni models group the images of one subject into an element and address it as @Element1. Other models address images by position. Naming matters when a shot has two subjects: it tells the model which picture is the person and which is the product.

The model treats everything in a reference as part of the subject unless told otherwise. A watermark, a busy background or a strong color filter can leak into the output. Clean references beat many references.

Which models support it (as of October 2026)

Kling 3.0 Omni takes up to seven reference images, or one reference video of 3 to 10 seconds plus up to four images, according to Kling's own guide. It generates up to 15 seconds per request at 720p or 1080p and is available through Kling's API and providers such as fal.ai.

Kling 4.0 launched in Kling's own app on October 15, 2026. Kling's guide lists up to 15 reference inputs per request, including up to 10 images, and clips of 3 to 30 seconds. Its developer API is still marked as coming soon.

Google Veo 3.1 calls the mode Ingredients to Video. Google's documentation allows up to three reference images of a single person, character or product, and generations with references are always 8 seconds long.

ByteDance's Seedance 2.5 and MiniMax H3 also offer multi-reference modes. Their limits differ between API providers, so check the provider you plan to use.

Choosing references: a short checklist

For a person: a clear front view, a three-quarter view, a profile, and one close-up of anything distinctive, such as glasses or a tattoo. Same outfit and hairstyle in every image.

For a product: the hero angle, the back or side, a sharp close-up of the logo or label, and one shot that shows scale, such as the product in a hand.

Light should be soft and even, backgrounds plain, and images sharp at full size. Skip filters, text overlays and heavy compression. Three to five strong images usually beat ten mixed ones. If a model caps at three, pick the views that show the most identity per image.

Writing the prompt around the references

Keep the subject out of the prompt and the scene in it. Skip hair color and clothing. Describe location, action, camera and light instead. For example: "She stands on a windy beach at golden hour, looks out to sea, then glances at the camera. Slow push-in from a medium shot to a close-up."

Do not contradict the references. If the images show brown hair and the prompt says blonde, the model has to choose, and you get a blend. If you want a change, make new references.

One action per shot works better than three. A clip that has to fit a walk, a turn, a pickup and a laugh into five seconds tends to rush and lose identity in the fast parts.

Common failures and what fixes them

The face changes when the subject turns away. Add a profile reference. The model is guessing an angle it was never shown.

The product label is redrawn. Frame the shot so the label is not the focus, or add a sharper close-up. Small print is still the weakest point of current models.

The background of a reference shows up in the scene. Replace that image with one on a plain backdrop.

One clip in a batch is off while the others are fine. That is usually chance. Regenerate with the same inputs first.

Trying it on RefVid

RefVid is built around this mode. You sign in, create a character or product, and upload 1 to 5 reference images: JPG, PNG or WebP, at least 300 px and up to 10 MB each, with the main view first. Then you write a scene prompt, pick 3, 5 or 10 seconds, and generate. Clips run on Kling 3.0 Omni and take about 2 to 4 minutes. RefVid will switch to Kling 4.0 when its API opens.