1. 15 Oct, 2022 1 commit
    • filipe oliveira's avatar
      optimizing d2string() and addReplyDouble() with grisu2: double to string... · 29380ff7
      filipe oliveira authored
      optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (#10587)
      
      All commands / use cases that heavily rely on double to a string representation conversion,
      (e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ),
      could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the
      equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures
      100% coverage of conversion.
      
      This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries
      ( fmtlib ) that use the optimized double to string conversion underneath.
      
      
      The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on
      https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). 
      
      test suite changes:
      Despite being compatible, in some cases it produces a different result from printf, and some tests
      had to be adjusted.
      one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000`
      instead of 5e+9, which sounds like a bug?
      In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding
      issues (`expr 0.8 == 0.79999999999999999`) 
      29380ff7
  2. 28 Feb, 2022 1 commit
  3. 05 Jan, 2022 1 commit
    • filipe oliveira's avatar
      Added INFO LATENCYSTATS section: latency by percentile distribution/latency by... · 5dd15443
      filipe oliveira authored
      
      Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462)
      
      # Short description
      
      The Redis extended latency stats track per command latencies and enables:
      - exporting the per-command percentile distribution via the `INFO LATENCYSTATS` command.
        **( percentile distribution is not mergeable between cluster nodes ).**
      - exporting the per-command cumulative latency distributions via the `LATENCY HISTOGRAM` command.
        Using the cumulative distribution of latencies we can merge several stats from different cluster nodes
        to calculate aggregate metrics .
      
      By default, the extended latency monitoring is enabled since the overhead of keeping track of the
      command latency is very small.
       
      If you don't want to track extended latency metrics, you can easily disable it at runtime using the command:
       - `CONFIG SET latency-tracking no`
      
      By default, the exported latency percentiles are the p50, p99, and p999.
      You can alter them at runtime using the command:
      - `CONFIG SET latency-tracking-info-percentiles "0.0 50.0 100.0"`
      
      
      ## Some details:
      - The total size per histogram should sit around 40 KiB. We only allocate those 40KiB when a command
        was called for the first time.
      - With regards to the WRITE overhead As seen below, there is no measurable overhead on the achievable
        ops/sec or full latency spectrum on the client. Including also the measured redis-benchmark for unstable
        vs this branch. 
      - We track from 1 nanosecond to 1 second ( everything above 1 second is considered +Inf )
      
      ## `INFO LATENCYSTATS` exposition format
      
         - Format: `latency_percentiles_usec_<CMDNAME>:p0=XX,p50....` 
      
      ## `LATENCY HISTOGRAM [command ...]` exposition format
      
      Return a cumulative distribution of latencies in the format of a histogram for the specified command names.
      
      The histogram is composed of a map of time buckets:
      - Each representing a latency range, between 1 nanosecond and roughly 1 second.
      - Each bucket covers twice the previous bucket's range.
      - Empty buckets are not printed.
      - Everything above 1 sec is considered +Inf.
      - At max there will be log2(1000000000)=30 buckets
      
      We reply a map for each command in the format:
      `<command name> : { `calls`: <total command calls> , `histogram` : { <bucket 1> : latency , < bucket 2> : latency, ...  } }`
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      5dd15443
  4. 03 Jan, 2022 1 commit
    • chenyang8094's avatar
      Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) · 87789fae
      chenyang8094 authored
      
      
      Implement Multi-Part AOF mechanism to avoid overheads during AOFRW.
      Introducing a folder with multiple AOF files tracked by a manifest file.
      
      The main issues with the the original AOFRW mechanism are:
      * buffering of commands that are processed during rewrite (consuming a lot of RAM)
      * freezes of the main process when the AOFRW completes to drain the remaining part of the buffer and fsync it.
      * double disk IO for the data that arrives during AOFRW (had to be written to both the old and new AOF files)
      
      The main modifications of this PR:
      1. Remove the AOF rewrite buffer and related code.
      2. Divide the AOF into multiple files, they are classified as two types, one is the the `BASE` type,
        it represents the full amount of data (Maybe AOF or RDB format) after each AOFRW, there is only
        one `BASE` file at most. The second is `INCR` type, may have more than one. They represent the
        incremental commands since the last AOFRW.
      3. Use a AOF manifest file to record and manage these AOF files mentioned above.
      4. The original configuration of `appendfilename` will be the base part of the new file name, for example:
        `appendonly.aof.1.base.rdb` and `appendonly.aof.2.incr.aof`
      5. Add manifest-related TCL tests, and modified some existing tests that depend on the `appendfilename`
      6. Remove the `aof_rewrite_buffer_length` field in info.
      7. Add `aof-disable-auto-gc` configuration. By default we're automatically deleting HISTORY type AOFs.
        It also gives users the opportunity to preserve the history AOFs. just for testing use now.
      8. Add AOFRW limiting measure. When the AOFRW failures reaches the threshold (3 times now),
        we will delay the execution of the next AOFRW by 1 minute. If the next AOFRW also fails, it will be
        delayed by 2 minutes. The next is 4, 8, 16, the maximum delay is 60 minutes (1 hour). During the limit
        period, we can still use the 'bgrewriteaof' command to execute AOFRW immediately.
      9. Support upgrade (load) data from old version redis.
      10. Add `appenddirname` configuration, as the directory name of the append only files. All AOF files and
        manifest file will be placed in this directory.
      11. Only the last AOF file (BASE or INCR) can be truncated. Otherwise redis will exit even if
        `aof-load-truncated` is enabled.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      87789fae
  5. 10 Oct, 2021 1 commit
  6. 23 Sep, 2020 1 commit
  7. 24 Aug, 2020 1 commit
  8. 06 Aug, 2020 1 commit
  9. 12 Mar, 2020 1 commit
  10. 10 Jan, 2020 1 commit
  11. 01 Jan, 2020 1 commit
  12. 18 Sep, 2018 1 commit
  13. 06 Jul, 2016 1 commit
    • antirez's avatar
      Generate Makefile.dep at every build. · 10361829
      antirez authored
      Normally we used to update it from time to time. Too fragile... better
      to generate dependencies at every run and delete them on 'make clean'.
      10361829
  14. 03 Feb, 2015 1 commit
    • antirez's avatar
      Handle redis-check-rdb as a standalone program. · 7d1e1580
      antirez authored
      This also makes it backward compatible in the usage, but for the command
      name. However the old command name was less obvious so it is worth to
      break it probably.
      
      With the new setup the program main can perform argument parsing and
      everything else useful for an RDB check regardless of the Redis server
      itself.
      7d1e1580
  15. 28 Jan, 2015 1 commit
  16. 07 Oct, 2014 1 commit
  17. 18 Apr, 2013 1 commit
    • antirez's avatar
      Redis/Jemalloc Gitignore were too aggressive. · d04afd62
      antirez authored
      Redis gitignore was too aggressive since simply broken.
      
      Jemalloc gitignore was too agressive because it is conceived to just
      keep the files that allow to generate all the rest in development
      environments (so for instance the "configure" file is excluded).
      d04afd62
  18. 17 Sep, 2012 1 commit
  19. 23 Jul, 2012 1 commit
  20. 21 Nov, 2011 1 commit
  21. 15 Nov, 2011 1 commit
    • Pieter Noordhuis's avatar
      Rebuild deps/ and src/ when ARCH changes · 4b8a6394
      Pieter Noordhuis authored
      This change moves the build instructions for dependencies to a separate
      Makefile in deps/. The ARCH environment variable is stored in a
      .make-arch file in the same directory as the Makefile. The contents of
      this file is read and compared to the current ARCH, and, on a mismatch
      triggers rebuilding the entire source tree.
      
      When file .make-arch exists and matches with ARCH from the environment,
      the dependencies are assumed to already be built.
      
      The new "clean" target only cleans the Redis source tree, not its
      dependencies. To clear the dependencies as well, the "distclean" target
      can be used.
      4b8a6394
  22. 23 Oct, 2011 1 commit
  23. 14 Jun, 2011 2 commits
  24. 15 Mar, 2011 1 commit
  25. 30 Dec, 2010 1 commit
  26. 29 Dec, 2010 1 commit
  27. 20 Dec, 2010 1 commit
  28. 15 Dec, 2010 1 commit
  29. 29 Nov, 2010 1 commit
  30. 15 Nov, 2010 1 commit
  31. 01 Jul, 2010 1 commit
  32. 25 May, 2010 1 commit
  33. 17 May, 2010 1 commit
  34. 05 May, 2010 1 commit
  35. 13 Mar, 2010 1 commit
  36. 03 Nov, 2009 1 commit
  37. 26 Mar, 2009 1 commit
  38. 23 Mar, 2009 2 commits