How to get the output of features from MViT?

[GeorgeHu6]

Could anyone give me an example.

Here is the code and yaml config file:

MVIT:
  ZERO_DECAY_POS_CLS: False
  NUM_HEADS: 1
  EMBED_DIM: 96
  PATCH_KERNEL: (3, 7, 7)
  PATCH_STRIDE: (2, 4, 4)
  PATCH_PADDING: (1, 3, 3)
  MLP_RATIO: 4.0
  QKV_BIAS: True
  DROPOUT_RATE: 0.0
  DROPPATH_RATE: 0.2
  NORM: "layernorm"
  MODE: "conv"
  DEPTH: 16
  POOL_Q_STRIDE: [[1, 1, 2, 2], [3, 1, 2, 2], [14, 1, 2, 2]]
  DIM_MUL: [[1, 2.0], [3, 2.0], [14, 2.0]]
  HEAD_MUL: [[1, 2.0], [3, 2.0], [14, 2.0]]
  POOL_KV_STRIDE_ADAPTIVE: [1, 8, 8]
  POOL_KVQ_KERNEL: [3, 3, 3]
  SEP_POS_EMBED: True
  CLS_EMBED_ON: True
  FREEZE_PATCH: False

cfg = get_cfg()
cfg.merge_from_file("/path/to/yaml/file")

fake_imgs = torch.randint(0, 255, (24, 3, 16, 224, 224)).float().cuda()
model = build_model(cfg)
model(fake_imgs)

and here is the error message:

Traceback (most recent call last):
  File "/opt/data/private/code/tapis-based-surgicalc2c/test_tapis.py", line 32, in <module>
    model(fake_imgs)
  File "/opt/data/private/virtualenv/torch_grasp/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/data/private/code/tapis-based-surgicalc2c/tapis/models/video_model_builder.py", line 856, in forward
    x = torch.cat((cls_tokens, x), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 96 but got size 8 for tensor number 1 in the list.