Jump to content

Draft:BGZF

From Wikipedia, the free encyclopedia
  • Comment: The article uses a large amount of technical jargon and does not provide a general introduction to the subject. I am not sure if the topic meets GNG. Caleb Stanford (talk) 20:40, 10 May 2025 (UTC)
  • Comment: Publications by the developers of BGZF are not independent. jlwoodwa (talk) 23:28, 1 April 2025 (UTC)


Blocked GNU Zip Format (BGZF) is block compression implemented on top of the standard gzip file format used by several bioinformatics file formats such as BAM, Binary VCF, or compressed FASTA files[1] for efficient storage, random access with indexed queries,[2] and parallel processing.[3][4] It can be decompressed by any gunzip compatible utility. BGZF was developed as part of SAM/BAM specification.

Schema

[edit]

A BGZF file consists of a series of concatenated BGZF blocks. Each block, whether in its compressed or uncompressed state, is limited to a maximum size of 64 kilobytes. Each BGZF block is itself a fully compliant gzip archive, adhering to the specifications outlined in RFC 1952.[5]

Schema of a single compressed block that constitutes the basic unit of a BGZF compressed file.

Each BGZF block contains a standard gzip file header with the following standard-compliant extensions:

  1. The F.EXTRA bit in the header is set to indicate that extra fields are present.
  2. The extra field used by BGZF uses the two subfield ID values 66 and 67 (ASCII ‘BC’).
  3. The length of the BGZF extra field payload (field LEN in the gzip specification) is 2 (two bytes of payload).
  4. The payload of the BGZF extra field is a 16-bit unsigned integer in little-endian format. This integer gives the size of the containing BGZF block minus one.

This block design allows use of an associated index file (storing offsets of each BGZF block) to fetch and decompress only the block of data that pertains to the query, thus avoiding the computational overhead of reading and decompressing all BGZF blocks.[5] However, due to block design, BGZF files tend to be larger than equivalent GZIP compressed files.[6]

Random access

[edit]

EOF marker

[edit]

End-of-file marker for BGZF enables detection of erroneously truncated files and generate warnings or errors for the user. The EOF marker block is an empty (data block of length zero) BGZF block encoded with the default zlib compression level settings, and consists of the following 28 hexadecimal bytes:

1f 8b 08 04 00 00 00 00 00 ff 06 00 42 43 02 00 1b 00 03 00 00 00 00 00 00 00 00 00

The presence of an EOF marker by itself does not signal an end of the file, however, an EOF marker present at the end of a BGZF file indicates that the immediately following physical EOF is the end of the file as intended by the program that wrote it.[7]

Further reading

[edit]

For detailed information on BGZF, see "The BGZF compression format" section in the SAM file specification.

References

[edit]
  1. ^ Bonfield, James K; Marshall, John; Danecek, Petr; Li, Heng; Ohan, Valeriu; Whitwham, Andrew; Keane, Thomas; Davies, Robert M (2021-02-01). "HTSlib: C library for reading/writing high-throughput sequencing data". GigaScience. 10 (2): giab007. doi:10.1093/gigascience/giab007. ISSN 2047-217X. PMC 7931820. PMID 33594436.
  2. ^ Yamada, Taiju (2020-04-01). "7bgzf: Replacing samtools bgzip deflation for archiving and real-time compression". Computational Biology and Chemistry. 85: 107207. doi:10.1016/j.compbiolchem.2020.107207. ISSN 1476-9271. PMID 32092548.
  3. ^ Carpentieri, Bruno (2019-11-01). "Next Generation Sequencing Data and its Compression". IOP Conference Series: Earth and Environmental Science. 362 (1): 012059. Bibcode:2019E&ES..362a2059C. doi:10.1088/1755-1315/362/1/012059. ISSN 1755-1307.
  4. ^ Li, Heng; Handsaker, Bob; Wysoker, Alec; Fennell, Tim; Ruan, Jue; Homer, Nils; Marth, Gabor; Abecasis, Goncalo; Durbin, Richard; 1000 Genome Project Data Processing Subgroup (2009-08-15). "The Sequence Alignment/Map format and SAMtools". Bioinformatics. 25 (16): 2078–2079. doi:10.1093/bioinformatics/btp352. ISSN 1367-4803. PMID 19505943.{{cite journal}}: CS1 maint: numeric names: authors list (link)
  5. ^ a b Li, Heng (2011-03-01). "Tabix: fast retrieval of sequence features from generic TAB-delimited files". Bioinformatics. 27 (5): 718–719. doi:10.1093/bioinformatics/btq671. ISSN 1367-4803. PMID 21208982.
  6. ^ Cock, Peter (2011-11-08). "Blasted Bioinformatics!?: BGZF - Blocked, Bigger & Better GZIP!". Blasted Bioinformatics!?. Retrieved 2025-04-10.
  7. ^ "HTS format specifications". samtools.github.io. Retrieved 2025-04-01.