Let’s say you’re writing an image processing program. The program takes in an image, converts it to floating point, does some processing and finally saves the modified pixels to disk as 8-bit colors. The question today concerns how exactly the integer-to-float conversion should be done. There are two approaches which, written in Python and NumPy, look like this: Standard division by 255 Alternative division by 256 pixels = img / 255.0 result = process(pixels) output = np.trunc(result * 255 + 0.5 ) pixels = (img + 0.5 ) / 256.0 result = process(pixels) output = np.trunc(result * 256 ) I assume that in both cases the output values are clamped before the final typecast: # Clamp and cast to 8 bits output_8bit = output.clip( 0 , 255 ).astype(np.uint8) The standard approach maps the integer 0 to 0.0 and 255 to 1.0. It works perfectly fine and is how GPUs do it . The alternative adds a 0.5 bias and divides by 256 instead, so the integer 0 gets mapped to 0.5/256=0.001953125 .…