Skip to content

Validate whisper beginning_timestamp_token_id bounds - #31636

Open
apsonawane wants to merge 2 commits into
mainfrom
msrc/whisper-timestamp-token-bounds
Open

Validate whisper beginning_timestamp_token_id bounds#31636
apsonawane wants to merge 2 commits into
mainfrom
msrc/whisper-timestamp-token-bounds

Conversation

@apsonawane

Copy link
Copy Markdown
Contributor

This pull request improves validation for the Whisper model's beginning_timestamp_token_id parameter in both runtime and test code. The main focus is to ensure that invalid values for this parameter are caught early, preventing out-of-range errors during inference and processing.

Validation improvements for Whisper timestamp token ID:

  • Added a check in BeamSearchParameters::SetSubgraphParameters to enforce that beginning_timestamp_token_id is within the valid range [0, vocab_size) for Whisper models, throwing an error if not.
  • Added a similar runtime check in TimestampLogitsProcessor to ensure beginning_timestamp_token_id_ is within bounds before processing logits.

Unit test enhancements:

  • Introduced new tests in beam_search_test.cc to verify that SetSubgraphParameters correctly rejects negative or out-of-range beginning_timestamp_token_id values and accepts valid ones.

- reject out-of-range beginning_timestamp_token_id for whisper logits processor once vocab size is known

- add defense-in-depth runtime check in TimestampLogitsProcessor

- add regression tests for invalid/valid boundary values
@apsonawane
apsonawane requested review from tianleiwu and a lite review from Copilot August 4, 2026 20:09
@apsonawane apsonawane changed the title [security] Validate whisper beginning_timestamp_token_id bounds Validate whisper beginning_timestamp_token_id bounds Aug 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds early validation for Whisper’s beginning_timestamp_token_id to prevent out-of-range indexing during timestamp logits processing, and extends unit tests to cover invalid/valid values.

Changes:

  • Add bounds checks for beginning_timestamp_token_id during subgraph-parameter setup for Whisper generation.
  • Add a runtime guard in TimestampLogitsProcessor to fail fast if beginning_timestamp_token_id is outside the logits width.
  • Add unit tests to ensure negative/out-of-range values are rejected.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
onnxruntime/test/contrib_ops/beam_search_test.cc Adds new unit tests covering invalid/valid Whisper beginning_timestamp_token_id values.
onnxruntime/contrib_ops/cpu/transformers/logits_processor.h Adds runtime enforcement that beginning_timestamp_token_id is within the logits vector bounds.
onnxruntime/contrib_ops/cpu/transformers/beam_search_parameters.cc Adds Whisper-specific validation of beginning_timestamp_token_id during SetSubgraphParameters.
Suppressed comments (2)

onnxruntime/test/contrib_ops/beam_search_test.cc:67

  • Like the previous Whisper validation test, initialize parameters.vocab_size explicitly to avoid using an indeterminate value in SetSubgraphParameters's vocab_size-override logic.
TEST(BeamSearchParametersTest, SetSubgraphParametersRejectsWhisperBeginningTimestampTokenIdEqualToVocabSize) {
  contrib::transformers::BeamSearchParameters parameters;
  parameters.model_type = contrib::transformers::IGenerationParameters::kModelTypeWhisper;
  parameters.logits_processor = contrib::transformers::IGenerationParameters::kLogitsProcessorTypeWhisper;
  parameters.beginning_timestamp_token_id = 128;

  EXPECT_THROW(parameters.SetSubgraphParameters(128, 2, 4, 6), OnnxRuntimeException);

onnxruntime/test/contrib_ops/beam_search_test.cc:78

  • beginning_timestamp_token_id = 0 is accepted by the test, but the Whisper timestamp logits processor assumes beginning_timestamp_token_id > 0 (it computes a max over the non-timestamp prefix [0, beginning_timestamp_token_id)). Use a positive in-range id here, and also set parameters.vocab_size explicitly to avoid indeterminate default state.
TEST(BeamSearchParametersTest, SetSubgraphParametersAcceptsValidWhisperBeginningTimestampTokenId) {
  contrib::transformers::BeamSearchParameters parameters;
  parameters.model_type = contrib::transformers::IGenerationParameters::kModelTypeWhisper;
  parameters.logits_processor = contrib::transformers::IGenerationParameters::kLogitsProcessorTypeWhisper;
  parameters.beginning_timestamp_token_id = 0;

  EXPECT_NO_THROW(parameters.SetSubgraphParameters(128, 2, 4, 6));
  EXPECT_EQ(parameters.vocab_size, 128);
}

Comment thread onnxruntime/contrib_ops/cpu/transformers/logits_processor.h Outdated
Comment on lines +158 to +163
if (model_type == IGenerationParameters::kModelTypeWhisper &&
logits_processor == IGenerationParameters::kLogitsProcessorTypeWhisper) {
ORT_ENFORCE(beginning_timestamp_token_id >= 0 && beginning_timestamp_token_id < vocab_size,
"beginning_timestamp_token_id is out of range, it is ", beginning_timestamp_token_id,
", vocab_size is ", vocab_size);
}
Comment thread onnxruntime/test/contrib_ops/beam_search_test.cc
- require beginning_timestamp_token_id > 0 and < vocab_size

- initialize vocab_size in new tests for deterministic branch behavior

- update valid test case to beginning_timestamp_token_id=1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants