Mapping multimodal embeddings to Good PIDs and Video/Ad PIDs

[trannhuthuat96]

Thank you for open sourcing the dataset and materials. I have a few questions regarding the released multimodal embeddings and semantic IDs (SID), and would appreciate your clarification.

1. Regarding the multimodal embeddings released on Hugging Face:
   openonerec_multimodal_embedding

• Are these embeddings for goods, video ads, or both?
• Is there a way to map PID of each embedding row back to its corresponding PID (e.g., goods PID aligned with hist_goods_pid, or video/ad PID aligned with hist_video_pid)?

2. I noticed there is overlap between PIDs in goods_pid2sid.parquet and video_ad_pid2sid.parquet. I assume this is because the two files use different PID spaces.
   To reproduce the SIDs in these files:

   • How should we obtain the corresponding embeddings for goods and video/ad PIDs?
   • Which embeddings were used to train the RQKMeans tokenizer?
   • Why does the total number of rows in the two pid2sid files differ from the number of released embedding rows?

3. In the paper/codebase, you mention sampling a subset of item embeddings to train the tokenizer. Could you share more details about the training setup? E.g.,

   • Approximate training set size
   • Whether the training data mixes goods and video/ad embeddings, or uses them separately
   • Any important preprocessing details

4. Does pid2caption.parquet contain video descriptions/captions?
   If so:

   • Are the PIDs in the same ID space as video_ad_pid2sid.parquet?
   • Why does pid2caption.parquet contain ~12.6M rows while video_ad_pid2sid.parquet contains ~15M rows?
   • If we encode these captions using Qwen-8B, should the resulting embeddings approximately reproduce the released video embeddings?

5. Finally, I would appreciate your advice on reproducing goods_pid2sid.parquet and video_ad_pid2sid.parquet.
   Specifically, what embedding setup was used for SID generation?

   • text embeddings only
   • vision embeddings only
   • multimodal fusion
   • or another setup

Thank you again for releasing the resources and supporting reproducible research.