1. 28 Feb, 2023 1 commit
    • Oran Agra's avatar
      Make sure that fork child doesn't do incremental rehashing (#11692) · 789f6a95
      Oran Agra authored
      Turns out that a fork child calling getExpire while persisting keys (and
      possibly also a result of some module fork tasks) could cause dictFind
      to do incremental rehashing in the child process, which is both a waste
      of time, and also causes COW harm.
      
      (cherry picked from commit 2bec254d)
      (cherry picked from commit 3e82bdf7)
      (cherry picked from commit 4803334cf6cb1eccdd33674a72a215ed6cd10069)
      789f6a95
  2. 17 Jan, 2023 1 commit
    • Ozan Tezcan's avatar
      Some fixes to undefined behaviour bugs taken from (#9601) · cb6beec2
      Ozan Tezcan authored
      **Signed integer overflow.** Although, signed overflow issue can be problematic time to time
      and change how compiler generates code, current findings mostly about signed shift or simple
      addition overflow. For most platforms Redis can be compiled for, this wouldn't cause any issue
      as far as I can tell (checked generated code on godbolt.org).
      
      UB means nothing guaranteed and risky to reason about program behavior but I don't think any
      of the fixes here worth backporting. As sanitizers are now part of the CI, preventing new issues
      will be the real benefit.
      
      partial cherry pick from commit b91d8b28
      The bug in BITFIELD seems to affect 12.2.1 used on Alpine
      
      (cherry picked from commit 4418cf166e025e7d0d2c965e75ad57c05ecff43f)
      cb6beec2
  3. 22 Feb, 2021 1 commit
    • Greg Femec's avatar
      Fix random element selection for large hash tables. (#8133) · cde69883
      Greg Femec authored
      
      
      When a database on a 64 bit build grows past 2^31 keys, the underlying hash table expands to 2^32 buckets. After this point, the algorithms for selecting random elements only return elements from half of the available buckets because they use random() which has a range of 0 to 2^31 - 1. This causes problems for eviction policies which use dictGetSomeKeys or dictGetRandomKey. Over time they cause the hash table to become unbalanced because, while new keys are spread out evenly across all buckets, evictions come from only half of the available buckets. Eventually this half of the table starts to run out of keys and it takes longer and longer to find candidates for eviction. This continues until no more evictions can happen.
      
      This solution addresses this by using a 64 bit PRNG instead of libc random().
      Co-authored-by: default avatarGreg Femec <gfemec@google.com>
      (cherry picked from commit 266949c7)
      cde69883
  4. 27 Oct, 2020 1 commit
    • Oran Agra's avatar
      Squash merging 125 typo/grammar/comment/doc PRs (#7773) · 2ae7f491
      Oran Agra authored
      List of squashed commits or PRs
      ===============================
      
      commit 66801ea
      Author: hwware <wen.hui.ware@gmail.com>
      Date:   Mon Jan 13 00:54:31 2020 -0500
      
          typo fix in acl.c
      
      commit 46f55db
      Author: Itamar Haber <itamar@redislabs.com>
      Date:   Sun Sep 6 18:24:11 2020 +0300
      
          Updates a couple of comments
      
          Specifically:
      
          * RM_AutoMemory completed instead of pointing to docs
          * Updated link to custom type doc
      
      commit 61a2aa0
      Author: xindoo <xindoo@qq.com>
      Date:   Tue Sep 1 19:24:59 2020 +0800
      
          Correct errors in code comments
      
      commit a5871d1
      Author: yz1509 <pro-756@qq.com>
      Date:   Tue Sep 1 18:36:06 2020 +0800
      
          fix typos in module.c
      
      commit 41eede7
      Author: bookug <bookug@qq.com>
      Date:   Sat Aug 15 01:11:33 2020 +0800
      
          docs: fix typos in comments
      
      commit c303c84
      Author: lazy-snail <ws.niu@outlook.com>
      Date:   Fri Aug 7 11:15:44 2020 +0800
      
          fix spelling in redis.conf
      
      commit 1eb76bf
      Author: zhujian <zhujianxyz@gmail.com>...
      2ae7f491
  5. 22 May, 2020 1 commit
  6. 08 May, 2020 2 commits
  7. 31 Mar, 2020 1 commit
  8. 05 Mar, 2020 1 commit
    • Oran Agra's avatar
      RM_Scan disable dict rehashing · fff6b26a
      Oran Agra authored
      The callback approach we took is very efficient, the module can do any
      filtering of keys without building any list and cloning strings, it can
      also read data from the key's value. but if the user tries to re-open
      the key, or any other key, this can cause dict re-hashing (dictFind does
      that), and that's very bad to do from inside dictScan.
      
      this commit protects the dict from doing any rehashing during scan, but
      also warns the user not to attempt any writes or command calls from
      within the callback, for fear of unexpected side effects and crashes.
      fff6b26a
  9. 19 Feb, 2019 1 commit
  10. 18 Feb, 2019 2 commits
  11. 17 Jul, 2018 1 commit
  12. 16 Jul, 2018 2 commits
  13. 03 Jul, 2018 1 commit
  14. 08 May, 2018 1 commit
  15. 29 Mar, 2018 1 commit
  16. 05 Dec, 2017 2 commits
  17. 20 Feb, 2017 2 commits
    • antirez's avatar
      84fa8230
    • antirez's avatar
      Use SipHash hash function to mitigate HashDos attempts. · adeed29a
      antirez authored
      This change attempts to switch to an hash function which mitigates
      the effects of the HashDoS attack (denial of service attack trying
      to force data structures to worst case behavior) while at the same time
      providing Redis with an hash function that does not expect the input
      data to be word aligned, a condition no longer true now that sds.c
      strings have a varialbe length header.
      
      Note that it is possible sometimes that even using an hash function
      for which collisions cannot be generated without knowing the seed,
      special implementation details or the exposure of the seed in an
      indirect way (for example the ability to add elements to a Set and
      check the return in which Redis returns them with SMEMBERS) may
      make the attacker's life simpler in the process of trying to guess
      the correct seed, however the next step would be to switch to a
      log(N) data structure when too many items in a single bucket are
      detected: this seems like an overkill in the case of Redis.
      
      SPEED REGRESION TESTS:
      
      In order to verify that switching from MurmurHash to SipHash had
      no impact on speed, a set of benchmarks involving fast insertion
      of 5 million of keys were performed.
      
      The result shows Redis with SipHash in high pipelining conditions
      to be about 4% slower compared to using the previous hash function.
      However this could partially be related to the fact that the current
      implementation does not attempt to hash whole words at a time but
      reads single bytes, in order to have an output which is endian-netural
      and at the same time working on systems where unaligned memory accesses
      are a problem.
      
      Further X86 specific optimizations should be tested, the function
      may easily get at the same level of MurMurHash2 if a few optimizations
      are performed.
      adeed29a
  18. 02 Jan, 2017 1 commit
  19. 30 Dec, 2016 1 commit
  20. 20 Sep, 2016 1 commit
    • antirez's avatar
      dict.c: fix dictGenericDelete() return ASAP condition. · 67058671
      antirez authored
      Recently we moved the "return ASAP" condition for the Delete() function
      from checking .size to checking .used, which is smarter, however while
      testing the first table alone always works to ensure the dict is totally
      emtpy, when we test the .size field, testing .used requires testing both
      T0 and T1, since a rehashing could be in progress.
      67058671
  21. 14 Sep, 2016 3 commits
    • antirez's avatar
      dict.c: dictReplaceRaw() -> dictAddOrFind(). · 09a50d34
      antirez authored
      What they say about "naming things" in programming?
      09a50d34
    • antirez's avatar
      Apply the new dictUnlink() where possible. · a636aeac
      antirez authored
      Optimizations suggested and originally implemented by @oranagra.
      Re-applied by @antirez using the modified API.
      a636aeac
    • oranagra's avatar
      dict.c: introduce dictUnlink(). · afcbcc0e
      oranagra authored
      Notes by @antirez:
      
      This patch was picked from a larger commit by Oran and adapted to change
      the API a bit. The basic idea is to avoid double lookups when there is
      to use the value of the deleted entry.
      
      BEFORE:
      
          entry = dictFind( ... ); /* 1st lookup. */
          /* Do somethjing with the entry. */
          dictDelete(...);         /* 2nd lookup. */
      
      AFTER:
      
          entry = dictUnlink( ... ); /* 1st lookup. */
          /* Do somethjing with the entry. */
          dictFreeUnlinkedEntry(entry); /* No lookups!. */
      afcbcc0e
  22. 12 Sep, 2016 1 commit
  23. 07 Sep, 2016 6 commits
  24. 25 Apr, 2016 1 commit
  25. 01 Oct, 2015 1 commit
  26. 23 Aug, 2015 1 commit
  27. 14 Jul, 2015 1 commit
    • antirez's avatar
      DEBUG HTSTATS <dbid> added. · 0f64080d
      antirez authored
      The command reports information about the hash table internal state
      representing the specified database ID.
      
      This can be used in order to investigate rehashings, memory usage issues
      and for other debugging purposes.
      0f64080d
  28. 27 Mar, 2015 1 commit