Floating Point Errors and Large Scale Worlds

Definition of floating point

Floating Point Error Demonstration

Floating-Point Numbers – Real Python

Floating-Point Representation Error – Real Python

Java: Floating Point Numbers

Glitchified on Instagram: "Modern GPUs perform 10^12–10^15 arithmetic operations per second depending on class and precision. Explanation and examples: FLOPS = floating-point operations per second. 1 TFLOP = 10^12 FLOPS. Midrange/mobile GPUs: hundreds of GFLOPS to a few TFLOPS (10^9–10^12 FLOPS). High-end consumer desktop GPUs: tens of TFLOPS. Example: an RTX 4090 ≈ 82.6 TFLOPS (≈8.26×10^13 FLOPS) for FP32 peak. Data-center AI/HPC GPUs using low-precision matrix engines: hundreds to nearly a tho

Artificial Intelligence | AI on Instagram: "Neural scaling laws describe how a model’s performance improves (predictably) as we increase the amount of training data, the size of the model (in terms of parameters), or the total compute used (measured in FLOPs or floating point operations per second). The scaling laws are power laws, meaning that as we scale things up, the performance increases proportionally. For instance, as we increase the size of the model, we should expect better performance