We fine-tuned a 235B model on 4 GPUs. Then we didn't need it
What happened
Four training setups, three of which failed before a single step ran, and why quantization and sharding do not compose.
Summary assembled by rule from the sources below
Four training setups, three of which failed before a single step ran, and why quantization and sharding do not compose.
Summary assembled by rule from the sources below