Depth-Based Action Reenactment: A Beginner's Guide to Motion Transfer

Depth-based action reenactment separates the parts of a video that beginners often mix together. A character image controls appearance. An optional environment image controls the scene. A tracked-depth video controls the movement. This separation lets an agent transfer a dance or other human action to a new subject without treating the original performer as the new subject. The ArtArch CLI page shows this workflow as an agent-led creation path.

What the depth step actually provides

ArtArch's action-reenactment workflow prepares a grayscale relative-depth video from a human-action reference. Depth Anything V2 Small estimates relative depth, so the result describes near and far relationships in the frame. It is a motion guide, not a measurement of physical distance in meters.

The guide carries the selected performer's action order, timing, body silhouette, screen position, and weight shifts. These are the details a new character needs to follow a dance. The guide does not own the subject's face, clothing, species, or visual identity.

The input map is easier than it sounds

Think of the workflow as three roles:

• The subject image tells the final video who or what to show.
• The optional environment frame tells it where the scene is and how it is framed.
• The person-tracked depth video tells it how the selected performer moves.

The tracked-depth pattern uses exactly one video reference for the generation step. The original dance video stays separate, so its appearance, wardrobe, background people, and unrelated movement do not become extra instructions for the new clip.

This is useful for a beginner because each correction has a clear place. Change the character image when the subject looks wrong. Change the environment frame when the location or composition is wrong. Change the motion reference when the dance timing or body action is wrong.

Why there are two depth previews

The strict grayscale depth file is the evidence artifact. It preserves the relative-depth result without adding visual labels. The colored person-track preview is easier to inspect because each tracked person receives a stable temporary color across nearby segments.

Those colors are motion IDs, not identity evidence. A color means “this tracked foreground subject” inside the preparation output. It does not mean the system recognized a person or proved that a performer remains the same after a hard cut.

How the preparation is organized

The action workflow samples compressed representative frames before model processing. This gives the agent a chance to check whether the video is decodable, whether human movement is the main subject, and how many people appear together.

The processor works in frame-aligned segments near 15 seconds. It keeps the source display width and height, average frame rate, frame count alignment, and aspect ratio for the depth and people-track variants. The output includes source segments, grayscale depth segments, colored people-depth segments, and a manifest with the verification results.