MIT nyitóoldal

   Complete list

The improvement of the theory of quantization, modelling of errors in computers

OTKA project
2000 - 2003

Local supervisor: Kollár István
Official project supervisor: BME MIT

Roundoff (or, in more general terms: quantization) occurs in every digital equipment. Quantities are represented by finite bit number words, so the true values need to be quantized. The statistical theory of quantization is based on the additive white noise model of the quantization error.
The conditions of good approximation have been recently analyzed in a doctoral thesis [1], and a book written in USA-Hungarian cooperation [2]. In these, the existing theory has been extended by new models, describing floating point number representation. These results opened new fields for research:

  • further analysis of roundoff, associated with floating point number representation
  • special dither signals (auxiliary signals which help to decrease the bias of quantization)
  • generation of deterministic dither signals, advantages and disadvantages
  • investigation of test procedures of A/D converters on the basis of the statistical theory of quantization, development of novel error limits
  • characterization of the roundoff in IEEE double precision computers
  • development of new and effective simulation procedures
The results expected from this research:
  • improved understanding of the phenomena helps to avoid false scientific predictions and implications. These will be published in the form of a book [2].
  • analysis methods and suggestions for improvement will be elaborated for floating point computers. The aim is to decrease the errors by using the existing hardware.
The research is partly conducted in USA-Hungarian cooperation. We have also joined to the work of the EUPAS (European Project for ADC-based devices Standardisation) initiative.

[1] Kollár I., ?Quantization Noise?. Doctoral Thesis at the Hungarian Academy of Sciences, Budapest, 1996. 416 p.
[2] B. Widrow and I. Kollár, ?Quantization Noise?. In preparation.


  • Stanford University

Further information about the project:
Official homepage:
Department homepage:
Official email address: