IEEE

Approximate DCT and Quantization Techniques for Energy-Constrained Image Sensors

Abstract: Recent expansions in multimedia devices for many applications, such as surveillance, self-driving cars, and healthcare, gather enormous amounts of real-time images for processing and ...
IEEE

Randomized Quantization for Privacy in Resource Constrained Machine Learning at-the-Edge and Federated Learning

Abstract: The increasing adoption of machine learning at the edge (ML-at-the-edge) and federated learning (FL) presents a dual challenge: ensuring data privacy as well as addressing resource ...
GitHub

Python implementation of the TurboQuant and QJL vector quantization algorithms.

turboquant-py implements the TurboQuant and QJL vector quantization algorithms from Google Research (ICLR 2026 / AISTATS 2026). It compresses high-dimensional floating-point vectors to 1-4 bits per ...
Microsoft

Advances to low-bit quantization enable LLMs on edge devices

Large language models (LLMs) are increasingly being deployed on edge devices—hardware that processes data locally near the data source, such as smartphones, laptops, and robots. Running LLMs on these ...
InfoWorld

What is model quantization? Smaller, faster LLMs

Reducing the precision of model weights can make deep neural networks run faster in less GPU memory, while preserving model accuracy. If ever there were a salient example of a counter-intuitive ...
Techopedia

What is Quantization?

Quantization is a process aimed at simplifying data representation by reducing precision – the number of bits used. This process involves approximating a continuous range of values with a smaller set ...
GitHub

KV cache quantization fails with GGML_ASSERT

I'm using llama-cpp-python==0.2.60, installed using this command CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python. I'm able to load a model using type_k=8 and type_v=8 (for q8_0 cache).
unite

Comparing Quantization Techniques for Scalable Vector Search

Imagine looking for similar things based on deeper insights instead of just keywords. That’s what vector databases and similarity searches help with. Vector databases enable vector similarity search.