Skip to content

Use new Python API of OpenVINO - #345

Open
jiwaszki wants to merge 1 commit into
nebuly-ai:mainfrom
jiwaszki:jiwaszki/update_ov_inference
Open

Use new Python API of OpenVINO#345
jiwaszki wants to merge 1 commit into
nebuly-ai:mainfrom
jiwaszki:jiwaszki/update_ov_inference

Conversation

@jiwaszki

@jiwaszki jiwaszki commented Jun 5, 2023

Copy link
Copy Markdown

Updating OpenVINO to use new capabilities of 2023.0 release.

Applied changes:

  • Remove InferRequest use in favor of direct CompiledModel calls. New class behavior create InferRequest on demand when inference is run for the first time and holds it internally. More info here.
  • Use of new shared_memory mode to limit copies of input data. If project is implemented in a manner to run sequentially, it can be left as True. If not, the new mode can be exposed as additional flag to the wrapper method. More on the feature here.
    (note to maintainers: requires benchmarking on project's side)
  • Address resulting dictionary directly with names. More on OVDict here.

Thanks in advance for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant