Jump to content

Talk:Bfloat16 floating-point format

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by Spintendo (talk | contribs) at 05:47, 25 July 2023 (→Reply 18-JUL-2019: Reply). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.
WikiProject iconComputing C‑class Low‑importance
WikiProject iconThis article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ComputingWikipedia:WikiProject ComputingTemplate:WikiProject ComputingComputing
CThis article has been rated as C-class on Wikipedia's content assessment scale.
LowThis article has been rated as Low-importance on the project's importance scale.

Error in bit layout illustration

The file Bfloat16_format.svg shows the wrong bit indices (first field is 0-7 when it should be 0-6). I tried uploading a newer version with that fixed. It shows up. But the SVG itself is the old version.

Maybe it's because I used the exact same filename. I still believe this behavior is incorrect. Tried changing the filename and reuploading. That version is not even showing up. Now I can't even upload newer versions. Maybe this was flagged for moderation.

I don't know. I give up. Uploading experience is really bad. --nachokb (talk) 09:21, 27 September 2018 (UTC)[reply]

It seems to be correct, now. I suppose that your issue was due to caching. Vincent Lefèvre (talk) 09:49, 27 September 2018 (UTC)[reply]

History

It would be good to have a section explaining where this format was introduced and a history of its adoption. — Steven G. Johnson (talk) 20:04, 3 June 2021 (UTC)[reply]

Add additional information about rounding

  • Specific text to be added or removed

1. Use Shortened instead of Truncated

To avoid confusion with the rounding mechanism, the first change is to replace “truncated” with “shortened” when it comes to describing bfloat16. “The bfloat16 format, being a shortened IEEE 754 single-precision 32-bit float, …”. In such a way, the text won’t be misunderstood with the truncation rounding in conversion.

2. Add a Format Conversion Section Add a format conversion section to detail the rounding mechanism. The proposed text is as follows:


Rounding and Conversion

The most common use case is the conversion between binary32 and bfloat16. The following section describes the conversion process and its rounding scheme in the conversion. Note that there are other possible scenarios of format conversions to or from bfloat16. For example, int16 and bfloat16.

a. From IEEE754 binary32 (32-bit floating point) to bfloat16

When bfloat16 was first introduced as a storage format[1], the conversion from IEEE754 binary32 (32-bit floating point) to bfloat16 is truncation (round-to-zero). Later on, when it becomes the input of matrix multiplication units, the conversion can have various rounding mechanisms depending on the hardware platforms. For example, for Google TPU, the rounding scheme in the conversion is round-to-nearest-even[2]; for ARM, only one rounding mode is supported, the non-IEEE Round to Odd mode[3]; for NVIDIA, it supports converting float number to bfloat16 precision in round-to-nearest-even mode[4].

b. From bfloat16 to IEEE754 binary32 (32-bit floating point)

Since IEEE binary32 can represent all exact values in bfloat16, the conversion simply pads 16 zeros in the mantissa bits.


  • Reason for the change: To reflect the recent status that the industry vendors have supported various rounding mechanisms other than truncation.
  • References supporting change: NVIDIA, ARM

2A00:79E1:ABC:12B:BDC7:5327:147A:E174 (talk) 22:09, 18 July 2023 (UTC)[reply]

Disclosure: the requestor works at Google, and some content of the edit request is related to Google's product. 2A00:79E1:ABC:12B:BDC7:5327:147A:E174 (talk) 22:09, 18 July 2023 (UTC)[reply]

Reply 18-JUL-2019

  Unable to review  
Your edit request could not be reviewed because the request is not formatted correctly.

  1. The citation style predominantly used by the bfloat16 floating-point format article appears to be Citation Style 1. The citation style used in the edit request consists of bare URL's.[a] Any requested edit of yours which may be implemented will need to resemble the current style already in use in the article – in this case, CS1. (See WP:CITEVAR.)
  2. Citation ref tags have not been placed within the requested text indicating which portions of the text the source is referencing. (See WP:INTEGRITY.)
  3. The ref note for the Dillon source is only used once, and it is placed in a part of the sentence which confirms no information apart from the fact that bloat16 was once a storage format: "When bloat16 was first introduced as a storage format, ....." The reference entry for Dillon (the only one out of four from the request that uses citation style #1) is also missing the |page= parameter.
  4. One of the web references provided, from Google, confirms 12% of the information provided. The other two web sources do not confirm the text provided. At this link you will see text from two websites The text on the right side of the pane is from the Google source that you provided as a reference. The text on the left side of the pane is from your edit request proposal. Besides the 12% which does match, kindly point out the text from the right side of the pane that matches the text in your proposal on the left side of the pane. The comparison of your edit request proposal with the nvidia source can be viewed here. The comparison of your request"s text with the community.arm source can be viewed here.
  5. Significant portions of text have no ref tags.

In the collapsed section below titled Request edit examples, I have illustrated two: The first shows how the edit request was submitted; the second shows how requests should be submitted in the future.

Request edit examples
INCORRECT


The Sun's diameter is 864,337.3 miles, while the Moon's diameter is 2,159 miles. The Sun's temperature is 5,778 Kelvin.

https://www.booksource.com
https://www.journalsource.com
https://www.websource.com

In the example above there are three URL's provided with the claim statements, but these URL's have not been placed using Citation Style 1, which is the style predominantly used by the bfloat16 floating-point format article. Additionally, ref tags have not been placed within the text at the exact positions where the information they reference resides. Using the correct style and the correct positioning of the ref tags, the WikiFormatted text should resemble the following:

CORRECT


The Sun's diameter is 864,337.3 miles,<ref>{{cite book|last1=Sjöblad|first1=Tristan|title=The Sun|url=http://www.booksource.com|publisher=Academic Press|date=2020|page=1}}</ref> while the Moon's diameter is 2,159 miles.<ref>{{cite journal|last1=Harinath|first1=Prisha|title=Size of the Moon|journal=Science|issue=78|volume=51|url=http://www.journalsource.com|date=2020|page=46}}</ref> The Sun's temperature is 5,778 Kelvin.<ref>{{cite book|last1=Uemura|first1=Shu|title=The Sun's Heat|url=http://www.websource.com|publisher=Academic Press|date=2020|page=2}}</ref>

Which displays as:


The Sun's diameter is 864,337.3 miles,[1] while the Moon's diameter is 2,159 miles.[2] The Sun's temperature is 5,778 Kelvin.[3]

References

  1. ^ Sjöblad, Tristan. The Sun. Academic Press, 2020, p. 1.
  2. ^ Harinath, Prisha. "Size of the Moon", Science, 51(78):46. 2020.
  3. ^ Uemura, Shū. The Sun's Heat. Academic Press, 2020, p. 2.

In the example above the references have been formatted according to Citation Style 1, which shows the author, the source's name, date, etc. Also, the ref tags are placed in the exact location where the text which they reference resides. As Wikipedia is a volunteer project, edit requests such yours are generally expected to have this formatting done before the request is submitted for review.

Kindly rewrite your edit request so that it aligns more with the second example shown in the collapsed section above, and feel free to re-submit that edit request at your earliest convenience. If you have any questions about this formatting please don't hesitate to ask myself or another editor. Regards,  Spintendo  22:53, 18 July 2023 (UTC)[reply]

Thanks for the advices. I've addressed the comments:
For #3 regarding the Dillon source, it is also used in the original article, so the final change will be using the already existing citation (and the original article doesn't have the |page= parameter. 2620:15C:2D1:206:95E1:3368:FE14:B48E (talk) 21:24, 19 July 2023 (UTC)[reply]
@2620:15C:2D1:206:95E1:3368:FE14:B48E Thank you for your reply. To add the information, please provide the source that you stated was already in the article that can be used as a reference for the proposed information here on the talk page. Also, if no page number is provided, please use the |quote= parameter to give the information from the source that confirms the proposed information here. Thank you! Regards,  Spintendo  05:47, 25 July 2023 (UTC)[reply]

Notes

  1. ^ The use of bare URLs as references is a style which is acceptable for use in Wikipedia. However, general practice dictates that the style already in use for an article be the one that is subsequently used for all future additions unless changed by editorial consensus.
  1. ^ Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, Rif A. Saurous (2017-11-28). TensorFlow Distributions (Report). arXiv:1711.10604. Bibcode:2017arXiv171110604D. Accessed 2018-05-23. All operations in TensorFlow Distributions are numerically stable across half, single, and double floating-point precisions (as TensorFlow dtypes: tf.bfloat16 (truncated floating point), tf.float16, tf.float32, tf.float64). Class constructors have a validate_args flag for numerical asserts{{cite report}}: CS1 maint: multiple names: authors list (link)
  2. ^ "The bfloat16 numerical format". Google Cloud. Retrieved 2023-07-11. On TPU, the rounding scheme in the conversion is round to nearest even and overflow to inf.
  3. ^ Stephens, Nigel (2019-08-29). "BFloat16 processing for Neural Networks on Armv8-A". community.arm.com. Retrieved 2023-07-11. Only one rounding mode is supported, the non-IEEE Round to Odd mode [round-odd], sometimes known as sticky rounding.
  4. ^ "1.3.5. Bfloat16 Precision Conversion and Data Movement". docs.nvidia.com. Retrieved 2023-07-11. Converts float number to nv_bfloat16 precision in round-to-nearest-even mode and returns nv_bfloat16 with converted value..
Retrieved from "https://en.wikipedia.org/w/index.php?title=Talk:Bfloat16_floating-point_format&oldid=1167020410"