Apache ORC
 | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
| Apache ORC | |
| Initial release | 20 February 2013[1] |
|---|---|
| Stable release | 1.6.0
/ 3 September 2019[2] |
| Repository | ORC Repository |
| Operating system | Cross-platform |
| Type | Database management system |
| License | Apache License 2.0 |
| Website | orc |
Apache ORC (Optimized Row Columnar) is a free and open-source column-oriented data storage format of the Apache Hadoop ecosystem. It is similar to the other columnar-storage file formats available in the Hadoop ecosystem such as RCFile and Parquet. It is compatible with most of the data processing frameworks in the Hadoop environment.
In February 2013, the Optimized Row Columnar (ORC) file format was announced by Hortonworks in collaboration with Facebook.[3] A month later, the Apache Parquet format was announced, developed by Cloudera and Twitter.[4]
Comparison
Apache ORC is comparable to RCFile and Parquet file formats---all three fall under the category of columnar data storage within the Hadoop ecosystem. They all have better compression and encoding with improved read performance at the cost of slower writes.
Based on Hadoop computing environments, a comprehensive study to understand and compare several table storage formats is conducted and the results are published in a paper in VLDB 2013[5]. The basic structure of a table storage format consists of three core operations: row reordering, table partitioning, and data packing. This study also gives the rationale of ORC design with performance comparisons based on the setups for the core operations.
References
- ^ "The Stinger Initiative: Making Apache Hive 100 Times Faster". Retrieved Jan 1, 2019.
- ^ "Releases".
- ^ Alan Gates (February 20, 2013). "The Stinger Initiative: Making Apache Hive 100 Times Faster". Hortonworks blog. Retrieved Dec 31, 2018.
- ^ Justin Kestelyn (March 13, 2013). "Introducing Parquet: Efficient Columnar Storage for Apache Hadoop". Cloudera blog. Archived from the original on September 19, 2016. Retrieved May 4, 2017.
- ^ Yin Huai, Siyuan Ma, Rubao Lee, Owen O'Malley, Xiaodong Zhang, "Understanding insights into the basic structure and essential issues of table placement methods in clusters". Proceedings of the VLDB Endowment, Vol. 6, No. 14, 2013.
