Improve subnormal expectations

[honno]

Currently the spec doesn't mention subnormal floats, so assumedly the IEEE 754 behaviour stands.

For typical CuPy builds subnormals are flushed to zero, as explained by @leofang in numpy/numpy#18536 (comment).

Just FYI. In CuPy we turn flush-to-zero on by default (via setting a compiler flag -ftz=true), see floating.py#L56-L58. The reason is there could be significant computational cost on accelerators like GPU. So, I suspect that CuPy is not the only array library that does this.

It does seem bad if IEEE 754 subnormal behaviour is expected but will be seemingly forever violated by one of its adopters (when compiled with default settings). So I wonder if subnormal behaviour could just be specified as out-of-scope. What would be the ramifications?

Notably in #131 and numpy/numpy#18536 there was discussion of a smallest_subnormal property for finfo, but it was seen as a pretty awkward fit. It might be useful if some kind of information could tell the user that subnormals are not supported, e.g. smallest_subnormal == smallest_normal.

[rgommers]

+1 for making behavior on subnormals implementation-defined.

Not sure we should add any feature to determine this behavior. It may indeed make sense to add to finfo, but then I'd rather have something more direct than smallest_subnormal == smallest_normal.

Are there more options than "handle subnormals correctly" or "flush them to zero"?

[leofang]

Are there more options than "handle subnormals correctly" or "flush them to zero"?

Quick drive-by comment: On NVIDIA GPUs, no. See this section: https://docs.nvidia.com/cuda/floating-point/index.html#compiler-flags

[leofang]

I support leaving the behavior on subnormals implementation-defined too. Perhaps, in finfo we could add a new field "is_subnormal_zero"?

Are there more options than "handle subnormals correctly" or "flush them to zero"?

For HIP/OpenCL the answer is also no, at least for LLVM-based compilers: https://clang.llvm.org/docs/ClangCommandLineReference.html

[leofang]

@oleksandr-pavlyk What's the subnormal behavior of oneAPI on Intel CPUs/GPUs?