• Mingyi Kang's avatar
    Hyperloglog avoid allocate more than 'server.hll_sparse_max_bytes' bytes of... · f8ac5a65
    Mingyi Kang authored
    Hyperloglog avoid allocate more than 'server.hll_sparse_max_bytes' bytes of memory for sparse representation (#11438)
    
    Before this PR, we use sdsMakeRoomFor() to expand the size of hyperloglog
    string (sparse representation). And because sdsMakeRoomFor() uses a greedy
    strategy (allocate about twice what we need), the memory we allocated for the
    hyperloglog may be more than `server.hll_sparse_max_bytes` bytes.
    The memory more than` server.hll_sparse_max_bytes` will be wasted.
    
    In this pull request, tone down the greediness of the allocation growth, and also
    make sure it'll never request more than `server.hll_sparse_max_bytes`.
    
    This could in theory mean the size of the hyperloglog string is insufficient for the
    increment we need, should be ok since in this case we promote the hyperloglog
    to dense representation, an assertion was added to make sure.
    
    This PR also add some tests and fixes some typo and indentation issues.
    f8ac5a65
hyperloglog.tcl 8.57 KB