Photo to Video with AI: What Works, What Breaks, and How to Keep a Real Face Real
We are making a tribute film this month for a family: their late parents, restored from one black-and-white studio portrait, walking out of a temple to bless a couple on the husband's sixtieth year. The whole premise of the film is that these are these people. That premise is exactly what most photo-to-video tools break. We ran the same portrait through twelve image models and the same shot through four video models before the family looked at a frame and said that is them. This post is what we learned, in the order you would hit it yourself.
Key takeaways
- Photo to video is three jobs, restoration, animation and generation, done by different tools in that order, with the family approving the face before each next step.
- Character and portrait generators invent; only a dedicated restoration model kept the real face in our twelve-model test, and the family rejected two rounds before the third was them.
- Reference-to-video models drift in consistent directions; two leading ones aged a couple by about fifteen years, so choose the model by whether it holds faces, and give action shots with invented people to a different model.
- Colourise as a separate decision, keep both parents consistent in the same scene, and ask the family for real clothing colours before locking.
- Lock the approved portraits as references for every shot, people first and location last; reword a blocked prompt for free rather than abandoning a model; edit a near-right take before generating again.
Photo to video is three different jobs wearing one name
When someone searches for photo to video AI they usually mean one of three things, and the tools that do them are not the same tools. Restoration takes a damaged or faded photograph and gives you back the same photograph, sharper. Animation takes a still and adds small motion: a breath, a blink, a slow push of the camera, the so-called living photo. Generation makes a new scene that the photograph was never in, with the person from the photograph inside it, by feeding the photo to a video model as a reference.
Each job has its own way of failing, and the failures compound. A generated scene built on a restoration that already changed the face gives you a stranger doing the right thing. So the work has to happen in that order, restoration first, and each step has to be approved by someone who knew the person before the next step starts.
| Job | What good looks like | How it fails |
|---|---|---|
| Restoration | The same face, sharper, with the damage gone | A cleaner face that is subtly a different person |
| Animation (living photo) | Breath, a blink, a slow camera move; the photo stays the photo | Motion that warps the features, especially the mouth and eyes |
| Generation (reference to video) | The person, recognisably, in a scene that was never photographed | Aged, slimmed or beautified into someone the family does not know |
The three jobs inside photo to video
Where it breaks: the face that is not the person
The first thing we learned is that a model built to generate characters must never be used to restore a photograph. Fed the parents' portrait, one such model returned two completely different, younger people, confidently. It was doing its job, which is inventing. A restoration model does the opposite job: it sharpens what is there and adds nothing. Only one of the twelve image models we tried behaved as a true restorer, and it was the only one the family accepted.
The second thing is that the failure has a direction. Given identical references and an identical prompt, two of the leading reference-to-video models both aged the couple by roughly fifteen years, greying the man's temples and giving the woman a different face shape. Two models from the same family, the same drift, so it is a trait, not a bad seed you can re-roll. A third class of video model cannot take a reference photograph at all; it animates from a starting frame only, so it cannot hold two faces and a location together. The two models that did hold the approved faces were, as it happens, also the cheapest of everything we tried.
The third thing is quieter. Models slim faces and beautify them by default, so the prompt has to say keep the exact face shape, in those words. From a black-and-white source they keep the clothing monochrome, which is honest but may be wrong, so you ask the family what colour the saree really was before you lock anything. And the family rejected our first two rounds of restoration as not realistic. They were right. A restoration that impresses a stranger and fails the daughter is a failed restoration, and the daughter is the only judge there is.
A restorer sharpens. A generator invents.
If the tool you are using can make a person who never existed, it will also make a person who did exist into someone else. Use a restoration model for the face, and only then hand the approved result to anything that generates.
The order of operations that worked
Start from the best-preserved source, even if it is black and white. A sharp studio portrait from 1975 beats a faded colour print from 1990 every time, because restoration can add sharpness and colour but cannot recover a face that was never captured cleanly. Restore it with a model whose only job is restoration, with face enhancement on. Then stop, and show it to the family. Nothing else happens until they say yes.
Colourising is a separate decision and a separate step, and it is the family's decision. Once you have two approved portraits, check that they match: one parent in colour and the other in black and white cannot stand side by side in the same scene. Either both go to colour, or the parents' scenes become monochrome by design, which is period-correct and often more moving. Decide it per film, not per photograph.
Only now do the approved files become references. Pass exactly those files, unchanged, into every shot that shows the person; put the people first in the reference list and the location plate last. Choose the video model for the job: one that holds faces for any shot with the real people in it, and one that obeys blocking for scenes with invented people, because in our tests the model that followed the written action best was also the one that aged the faces. Those are two different shots and two different tools.
Two small habits saved us hours. A generation the platform blocks costs nothing, and our temple doorway was blocked once for what the filter took to be something else; rewording the prompt to say an elderly married couple instead of describing them physically cleared it, at no cost. And when a take is nearly right, cut it in the edit before you generate again: trim, hold, split. Every new generation is a fresh roll of the dice on continuity, so it usually creates a new mismatch while fixing the old one.
| Job | Use | Never use |
|---|---|---|
| Bring back the real face | A dedicated restoration model, face enhancement on | Any character or portrait generator |
| Add colour | A colouriser run on the approved restoration, as its own step | Restoring and colourising in one pass |
| A living photo | A motion model fed the approved still | Anything that redraws the face to move it |
| The person in a new scene | A reference-to-video model that holds faces, references locked | Models that take only a start frame, or that age from references |
| An invented crowd or place | The model that obeys blocking best | Real faces in the same shot |
Which class of model for which job
What we tell a family before we start
You will approve the faces before any scene is made; that is the gate, and it does not move. The first restoration may not be them, and if it is not, say so plainly, because that is useful information and not a complaint. We will ask you the real colours of clothes you may never have seen in colour. If the film uses songs from films, it stays a private film for the family and the function; anything posted publicly needs licensed music. And the calendar runs faces first, scenes second, so the week you spend finding the best photograph is the week that decides everything.
The rest of the method, how the story is told, what the film costs and how long it takes, is in our post on the Sashtiapthapoorthi film and the rate card. The principle underneath all of it is the same one we apply to a client's product in a commercial: AI draws the world, never the thing that has to be true. A face is the most true thing there is.
A restoration that impresses a stranger and fails the daughter is a failed restoration. The daughter is the only judge there is.
Frequently asked questions
Can AI turn a photo into a video?
Yes, in three different ways: restoring the photograph, animating it with small motion such as a breath or a blink, or generating a new scene with the person in it from the photo as a reference. The tools for each are different, and the order matters: restore first, get the face approved, then animate or generate.
Can AI restore an old photo without changing the face?
Only with a model built for restoration rather than generation. In our test of twelve image models on one portrait, a single restoration-only model kept the real face; the character generators invented different, younger people. Ask the model to keep the exact face shape, and have someone who knew the person approve the result before any further step.
Why does the AI-generated face look younger, older or slightly different?
Because most video and image models are built to invent, and they drift in consistent directions: some age a face by years from a reference, most slim and beautify by default. Use a dedicated restorer for the face, lock the approved portraits as references, and choose a video model that is known to hold faces for any shot with the real person in it.
Can I animate a photo of a parent who has passed away?
Yes, and it is one of the most common reasons families ask. Start from the sharpest source photograph even if it is black and white, restore it, get the family's approval on the face, and only then animate or place the person in a scene. Keep the animation small and dignified; the photograph stays the photograph.
Should a black-and-white photo be colourised?
It is the family's decision, made as a separate step after restoration. Ask for the real colours of the clothes before locking anything, and keep both parents consistent: one in colour and one in black and white cannot appear in the same scene. Monochrome by design is often the more moving choice.
How do I keep the same face across many AI video shots?
Restore once, get the portrait approved, and pass exactly that file as a reference into every shot, people first and the location last. Use a reference-to-video model that holds faces for those shots, and keep invented people and crowds in separate shots generated by whatever model follows action best.
How much does a photo-to-video film cost?
At our public rate card, AI video is ₹6,000 per 30 seconds of finished film with a ₹40,000 project minimum, one revision round included, quoted in writing with a GST invoice. Restoration of the source photographs is part of the work.
Have a project in mind?
We design, build, and ship software end-to-end — with a fixed, written quote after a free scoping call.