Reply to thread

Message: [QUOTE="Techmeme, post: 8828"] [URL='https://venturebeat.com/ai/huaweis-new-open-source-technique-shrinks-llms-to-make-them-run-on-less'][IMG]http://www.techmeme.com/251004/i18.jpg[/IMG][/URL] [URL='http://www.techmeme.com/251004/p18#a251004p18'][IMG]http://www.techmeme.com/img/pml.png[/IMG][/URL] Carl Franzen / [URL='https://venturebeat.com/']VentureBeat[/URL]: [B][URL='https://venturebeat.com/ai/huaweis-new-open-source-technique-shrinks-llms-to-make-them-run-on-less']Huawei's Zurich Lab unveils SINQ, an open-source quantization method that it claims can reduce LLM memory use by 60-70% without significant quality loss[/URL][/B] — - Dual-Axis Scaling: Instead of using a single scale factor for quantizing a matrix, SINQ uses separate scaling vectors for rows and columns. [/QUOTE]