From what I can tell, most of the time the reason is “the necessary hardware acceleration isn’t enabled/installed” or “the video decode logic seems to only have been tested on Windows”.

Firefox enabled hardware video decoding on Linux only very recently and it’ll only work if you install a bunch of packages that don’t come with every Linux distro.

Without GPU acceleration, everything is done on CPU, and CPU decodes are incredibly slow and inefficient.