OpenVINO
 | |
| Developer(s) | Intel Corporation |
|---|---|
| Initial release | May 16, 2018 |
| Stable release | 2023.3
/ January 2024.[1] |
| Repository | github |
| Written in | C++, Python |
| Operating system | Cross-platform |
| License | Apache License 2.0 |
| Website | docs |
| As of | January 2024 |
OpenVINO is an open-source software toolkit for optimizing and deploying deep learning models. It enables programmers to develop scalable and efficient AI solutions with relatively few lines of code and to target a variety of hardware: CPU, GPU, NPU, FPGA, and Intel GNA.[2] The supported models come from several popular frameworks and fall into many different categories, including Large Language Models, computer vision, and generative AI, including text-to-image models.[3]
Actively developed by Intel, it prioritizes high-performance inference on Intel hardware but also supports ARM/ARM64 processors[4] and encourages contributors to add new devices to the portfolio.
OpenVINO is cross-platform and free for use under Apache License 2.0.[5]
Overview
The high level pipeline of OpenVINO consists of two parts: generate IR (Intermediate Representation) files via Model Optimizer using your trained model or public one and execute inference on Inference Engine on specified devices.
The toolkit’s Inference Engine is a C++ library for inferring input on devices and getting results. To better understand OpenVINO API there are a lot of samples, that demonstrate how to work with OpenVINO.
OpenVINO has different sample types: classification, object detection, style transfer, speech recognition, etc. It is possible to try inference on public models. There are a variety of models for tasks, such as:
- classification
- segmentation
- object detection
- face recognition
- human pose estimation
- monocular depth estimation
- image inpainting
- style transfer
- action recognition
- colorization
Along with the primary components of model optimization and runtime within Intel® Distribution of OpenVINO toolkit, the toolkit also includes a user-friendly web browser interface called the Deep Learning Workbench to aid in model analysis and experimentation; a tool called the Post-Training Optimization Tool to accelerate inference by converting models into low-precision and that do not require re-training (e.g., post-training quantization); and, additional add-ons, such as the Deep Learning Streamer to aid in streaming analytics pipeline interoperability, the OpenVINO Model Server to enable scalability via a serving microservice, Training Extensions like the Neural Network Compression Framework, and the Computer Vision Annotation Tool, an online interactive video and image annotation tool.
OpenVINO has two webpages: one for documentation another for downloads.
OpenVINO model format
OpenVINO IR[6] is the default format used to run inference. It is saved as a set of two files, *.bin and *.xml, containing weights and topology, respectively. It is obtained by converting a model from one of the supported frameworks, using the application's API or a dedicated converter.
Models of the supported formats may also be used for inference directly, without prior conversion to OpenVINO IR. Such an approach is more convenient but offers fewer optimization options and lower performance, since the conversion is performed automatically before inference.
The supported model formats are:[7]
- PyTorch
- TensorFlow
- TensorFlow Lite
- ONNX (including formats that may be serialized to ONNX)
- PaddlePaddle
Programming language
OpenVINO is written in C++ and Python.
OS support
OpenVINO runs on the following desktop operation systems: Windows, Linux and MacOS.
OpenVINO also runs on Raspberry Pi.[8]
See also
References
- ^ "Release Notes for Intel Distribution of OpenVINO toolkit 2023.3". January 2024.
- ^ "What is OpenVINO and what are the hardware requirements?". onlogic.com. Retrieved 9 February 2024.
- ^ "Text-to-Image Generation with Stable Diffusion and OpenVINO™". Intel. Retrieved 9 February 2024.
- ^ "OpenVINO Compatibility and Support". OpenVINO Documentation. 24 January 2024.
- ^ "License". OpenVINO repository. 16 October 2018.
- ^ "OpenVINO IR". www.docs.openvino.ai. 2 February 2024.
- ^ "OpenVINO Model Preparation". OpenVINO Documentation. 24 January 2024.
- ^ Wilbur, Marcia. "Use the Model Downloader and Model Optimizer for the Intel® Distribution of OpenVINO™ Toolkit on Raspberry Pi*".
- Agrawal, Vasu (2019). Ground Up Design of a Multi-modal Object Detection System (PDF) (MSc). Carnegie Mellon University Pittsburgh, PA. Archived (PDF) from the original on 26 January 2020.
- Driaba, Alexander; Gordeev, Aleksei; Klyachin, Vladimir (2019). "Recognition of Various Objects from a Certain Categorical Set in Real Time Using Deep Convolutional Neural Networks" (PDF). Institute of Mathematics and Informational Technologies Volgograd State University. Archived (PDF) from the original on 26 January 2020. Retrieved 26 January 2020.
{{cite journal}}: Cite journal requires|journal=(help) - Nanjappa, Ashwin (31 May 2019). Caffe2 Quick Start Guide: Modular and scalable deep learning made easy. Packt. pp. 91–98. ISBN 978-1789137750.
| Open source | |
|---|---|
| Proprietary |
|
