From Video to Publication: A Subtitle Localization Pipeline

From video to publication, building a subtitle localization workflow for technical videos with transcription, cleanup, translation, proofreading, glossary management, and metadata generation.

Lab

Subtitle Localization Pipeline

Recently, while translating technical talks from Vercel, I gradually consolidated a collection of standalone scripts into a complete subtitle localization pipeline.

The goal of the project isn't simply to generate subtitles. Instead, it automates most of the work required to transform a video into a publication-ready package, including transcription, translation, proofreading, and metadata generation.

Today, the workflow looks like this:

video.mp4
    │
    ▼
FFmpeg
    │
    ▼
video.wav
    │
    ▼
Whisper
    │
    ▼
en.draft.srt
    │
    ▼
GPT Cleanup
    │
    ▼
en.srt
    │
    ▼
DeepSeek Translation
    │
    ▼
zh.draft.srt
    │
    ▼
GPT Proofread
    │
    ▼
zh.srt
    │
    ▼
Generate meta.md

The pipeline takes a single video directory as its input and produces both localized subtitles and the metadata required for publishing.

video.mp4
video.wav
en.srt
zh.srt
meta.md

Instead of relying on one large prompt, each stage has a well-defined responsibility, making the workflow easier to maintain and improve over time.

Starting with Audio, Not Subtitles

The first step is FFmpeg, not Whisper.

Every video is converted into a normalized audio file before transcription:

  • PCM WAV

  • 16 kHz

  • Mono

Normalizing the input provides two benefits.

First, every video is transcribed under the same conditions.

Second, the extracted audio becomes a reusable intermediate artifact. If transcription needs to be rerun, there's no need to process the video again.

Whisper Handles Transcription, Not Cleanup

Speech recognition is powered by whisper.cpp with the large-v3-turbo model.

Its output is always saved as en.draft.srt.

Keeping the draft stage is intentional.

Although Whisper produces high-quality transcripts, raw ASR output still contains artifacts such as:

  • repeated words

  • false starts

  • filler words

  • inconsistent punctuation

  • unstable recognition of technical terminology

For that reason, transcription is treated as an intermediate result rather than the final English subtitles.

Cleaning English Before Translation

The cleanup stage is handled by GPT.

Its responsibility is not translation, but improving the English subtitles.

This includes:

  • correcting spelling and punctuation

  • removing repeated words and filler phrases

  • normalizing official terminology

  • preserving timestamps and subtitle numbering

One design decision here is worth mentioning.

Cue merging, splitting, and renumbering are handled entirely by scripts before the subtitles reach the language model.

The model focuses only on language quality, while the subtitle structure remains deterministic throughout the pipeline.

Translation and Proofreading Are Separate Stages

Once the English subtitles are cleaned, translation begins.

DeepSeek is currently used to generate zh.draft.srt.

The translation stage focuses on preserving the relationship between English and Chinese subtitles.

Each English cue maps to exactly one Chinese cue while preserving timestamps, API names, commands, filenames, and technical terminology.

After the draft is generated, GPT performs a separate proofreading pass using both en.srt and zh.draft.srt.

This stage checks:

  • mistranslations

  • missing content

  • terminology consistency

  • natural Chinese expressions

  • awkward cue boundaries introduced by line-by-line translation

Separating translation from proofreading makes each stage simpler and easier to control.

A Shared Glossary

Technical talks often reuse the same terminology across multiple videos.

Relying entirely on language models makes it difficult to maintain consistent translations over time.

To address this, the project maintains a standalone glossary.

Instead of embedding the entire glossary into every prompt, only the entries relevant to the current batch are injected.

This keeps prompts concise while ensuring consistent terminology.

The glossary is shared across cleanup, translation, and proofreading instead of belonging to any single stage.

Publishing Metadata Is Part of the Pipeline

Generating subtitles is only part of the publishing process.

A typical video also requires:

  • title

  • keywords

  • cover text

  • chapters

  • introduction

  • source information

  • copyright disclaimer

These are generated into a single meta.md file.

The generator can also reuse an existing meta.md, making updates incremental instead of regenerating everything from scratch.

As a result, a video directory already contains most of the assets required for publication.

Scripts Own the Rules, Models Own the Language

One principle gradually became clear while building the project:

Scripts handle deterministic rules. Language models handle language.

Tasks such as:

  • subtitle parsing

  • cue merging

  • timestamps

  • caching

  • validation

  • file organization

are implemented as scripts.

Language models are responsible only for:

  • English cleanup

  • Chinese translation

  • Chinese proofreading

This separation also keeps prompts relatively small.

Most rule changes happen in scripts or validators rather than inside prompts.

Validation

The final stage validates both subtitles and meta.md.

Validation covers things such as:

  • required metadata fields

  • glossary compliance

  • subtitle formatting

  • publishing structure

The validator has also been useful for identifying historical inconsistencies, such as outdated terminology rules or subtitle formatting that no longer matches the current standards.

Rather than assuming model output is always correct, the workflow verifies it before considering the pipeline complete.

What's Next

The current pipeline already automates most of the work between a source video and a publication-ready package.

Future improvements include:

  • refining glossary rules for plurals and capitalization

  • expanding automated test coverage

  • reducing false positives in subtitle validation

  • generating validator summaries at the end of each run

  • continuously improving glossary entries and publishing templates

The project wasn't designed around a single all-purpose model.

Instead, it is built around small, well-defined stages with clear inputs and outputs.

Rules live in scripts.

Language lives in language models.

Validation ensures the final output meets the expected constraints.

That separation has made the workflow both more predictable and easier to evolve.