Running AI models without floating point matrix math could mean far less power consumption.

Running AI models without matrix math means far less power consumption—and fewer GPUs?

  • bitfucker@programming.devEnglish
    61·
    3 days ago

    Good

    Edit: Oh shit nvm. It still requires dedicated HW (FPGA). This is no different than say, an NPU. But to be fair, they also said the researcher tested the model on traditional GPU too and reduce memory consumption.

    • Pennomi@lemmy.worldEnglish
      2·
      3 days ago

      Only for maximum efficiency. LLMs already run tolerably well on normal CPUs and this technique would make it much more efficient there as well.