- 29 Feb, 2016 1 commit
-
-
antirez authored
-
- 26 Feb, 2016 3 commits
-
-
antirez authored
-
antirez authored
-
antirez authored
The new bitfield command is an extension to the Redis bit operations, where not just single bit operations are performed, but the array of bits composing a string, can be addressed at random, not aligned offsets, with any width unsigned and signed integers like u8, s5, u10 (up to 64 bit signed integers and 63 bit unsigned integers). The BITFIELD command supports subcommands that can SET, GET, or INCRBY those arbitrary bit counters, with multiple overflow semantics. Trivial and credits: A similar command was imagined a few times in the past, but for some reason looked a bit far fetched or not well specified. Finally the command was proposed again in a clear form by Yoav Steinberg from Redis Labs, that proposed a set of commands on arbitrary sized integers stored at bit offsets. Starting from this proposal I wrote an initial specification of a single command with sub-commands similar to what Yoav envisioned, using short names for types definitions, and adding control on the overflow. This commit is the resulting implementation. Examples: BITFIELD mykey OVERFLOW wrap INCRBY i2 10 -1 GET i2 10
-
- 27 Jul, 2015 1 commit
-
-
antirez authored
-
- 26 Jul, 2015 4 commits
- 11 Dec, 2014 1 commit
-
-
Matt Stancliff authored
Previously the string was created empty then re-sized to fit the offset, but sds resize causes the sds to over-allocate by at least 1 MB (which is a lot when you are operating at bit-level access). This also improves the speed of initial sets by 2% to 6% based on quick testing. Patch logic provided by @oranagra Fixes #1918
-
- 03 Dec, 2014 1 commit
-
-
Sun He authored
-
- 02 Dec, 2014 1 commit
-
-
Sun He authored
-
- 13 Aug, 2014 1 commit
-
-
antirez authored
-
- 30 Mar, 2014 1 commit
-
-
antirez authored
All the Redis functions that need to modify the string value of a key in a destructive way (APPEND, SETBIT, SETRANGE, ...) require to make the object unshared (if refcount > 1) and encoded in raw format (if encoding is not already REDIS_ENCODING_RAW). This was cut & pasted many times in multiple places of the code. This commit puts the small logic needed into a function called dbUnshareStringValue().
-
- 27 Feb, 2014 4 commits
-
-
antirez authored
-
antirez authored
With the new behavior it is possible to specify just the start in the range (the end will be assumed to be the first byte), or it is possible to specify both start and end. This is useful to change the behavior of the command when looking for zeros inside a string. 1) If the user specifies both start and end, and no 0 is found inside the range, the command returns -1. 2) If instead no range is specified, or just the start is given, even if in the actual string no 0 bit is found, the command returns the first bit on the right after the end of the string. So for example if the string stored at key foo is "\xff\xff": BITPOS foo (returns 16) BITPOS foo 0 -1 (returns -1) BITPOS foo 0 (returns 16) The idea is that when no end is given the user is just looking for the first bit that is zero and can be set to 1 with SETBIT, as it is "available". Instead when a specific range is given, we just look for a zero within the boundaries of the range.
-
antirez authored
It appears to work but more stress testing, and both unit tests and fuzzy testing, is needed in order to ensure the implementation is sane.
-
antirez authored
-
- 22 Jul, 2013 1 commit
-
-
antirez authored
Previously two string encodings were used for string objects: 1) REDIS_ENCODING_RAW: a string object with obj->ptr pointing to an sds stirng. 2) REDIS_ENCODING_INT: a string object where the obj->ptr void pointer is casted to a long. This commit introduces a experimental new encoding called REDIS_ENCODING_EMBSTR that implements an object represented by an sds string that is not modifiable but allocated in the same memory chunk as the robj structure itself. The chunk looks like the following: +--------------+-----------+------------+--------+----+ | robj data... | robj->ptr | sds header | string | \0 | +--------------+-----+-----+------------+--------+----+ | ^ +-----------------------+ The robj->ptr points to the contiguous sds string data, so the object can be manipulated with the same functions used to manipulate plan string objects, however we need just on malloc and one free in order to allocate or release this kind of objects. Moreover it has better cache locality. This new allocation strategy should benefit both the memory usage and the performances. A performance gain between 60 and 70% was observed during micro-benchmarks, however there is more work to do to evaluate the performance impact and the memory usage behavior.
-
- 26 Jun, 2013 1 commit
-
-
antirez authored
-
- 17 May, 2013 1 commit
-
-
YAMAMOTO Takashi authored
NetBSD-current's libc has a function named popcount. hiding these extensions using feature macros is not possible because redis uses other extensions covered by the same feature macro. eg. inet_aton
-
- 08 May, 2013 1 commit
-
-
antirez authored
This reverts commit 2c75f2cf. After further analysis, it is very unlikely that we'll raise the string size limit to > 512MB, and at the same time such big strings will be used in 32 bit systems. Better to revert to size_t so that 32 bit processors will not be forced to use a 64 bit counter in normal operations, that is currently completely useless.
-
- 07 May, 2013 2 commits
-
-
Jiahao Huang authored
-
Jiahao Huang authored
bitcount commant may return negtive integer with string length more than 256 MB
-
- 28 Jan, 2013 2 commits
-
-
antirez authored
When keyspace events are enabled, the overhead is not sever but noticeable, so this commit introduces the ability to select subclasses of events in order to avoid to generate events the user is not interested in. The events can be selected using redis.conf or CONFIG SET / GET.
-
antirez authored
-
- 19 Jan, 2013 1 commit
-
-
guiquanz authored
-
- 08 Nov, 2012 1 commit
-
-
antirez authored
-
- 05 Sep, 2012 1 commit
-
-
Haruto Otake authored
remove unsafe and unnecessary cast. until now, this cast may lead segmentation fault when end > UINT_MAX setbit foo 0 1 bitcount 0 4294967295 => ok bitcount 0 4294967296 => cause segmentation fault. Note by @antirez: the commit was modified a bit to also change the string length type to long, since it's guaranteed to be at max 512 MB in size, so we can work with the same type across all the code path. A regression test was also added.
-
- 18 Jul, 2012 1 commit
-
-
antirez authored
For the C standard char can be either signed or unsigned, it's up to the compiler, but Redis assumed that it was signed in a few places. The practical effect of this patch is that now Redis 2.6 will run correctly in every system where char is unsigned, notably the RaspBerry PI and other ARM systems with GCC. Thanks to Georgi Marinov (@eesn on twitter) that reported the problem and allowed me to use his RaspBerry via SSH to trace and fix the issue!
-
- 31 May, 2012 1 commit
-
-
antirez authored
In the issue #529 an user reported a bug that can be triggered with the following code: flushdb set a "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" bitop or x a b The bug was introduced with the speed optimization in commit 8bbc0768 that specializes every BITOP operation loop up to the minimum length of the input strings. However the computation of the minimum length contained an error when a non existing key was present in the input, after a key that was non zero length. This commit fixes the bug and adds a regression test for it.
-
- 24 May, 2012 7 commits
-
-
antirez authored
This commit adds a fast-path to the BITOP that can be used for all the bytes from 0 to the minimal length of the string, and if there are at max 16 input keys. Often the intersected bitmaps are roughly the same size, so this optimization can provide a 10x speed boost to most real world usages of the command. Bytes are processed four full words at a time, in loops specialized for the specific BITOP sub-command, without the need to check for length issues with the inputs (since we run this algorithm only as far as there is data from all the keys at the same time). The remaining part of the string is intersected in the usual way using the slow but generic algorith. It is possible to do better than this with inputs that are not roughly the same size, sorting the input keys by length, by initializing the result string in a smarter way, and noticing that the final part of the output string composed of only data from the longest string does not need any proecessing since AND, OR and XOR against an empty string does not alter the output (zero in the first case, and the original string in the other two cases). More implementations will be implemented later likely, but this should be enough to release Redis 2.6-RC4 with bitops merged in. Note: this commit also adds better testing for BITOP NOT command, that is currently the faster and hard to optimize further since it just flips the bits of a single input string.
-
antirez authored
A bug in the implementation caused BITOP to crash the server if at least one one of the source objects was integer encoded. The new implementation takes an additional array of Redis objects pointers and calls getDecodedObject() to get a reference to a string encoded object, and then uses decrRefCount() to release the object. Tests modified to cover the regression and improve coverage.
-
antirez authored
At Redis's default optimization level the command is now much faster, always using a constant-time bit manipualtion technique to count bits instead of GCC builtin popcount, and unrolling the loop. The current implementation performance is 1.5GB/s in a MBA 11" (1.8 Ghz i7) compiled with both GCC and clang. The algorithm used is described here: http://graphics.stanford.edu/~seander/bithacks.html
-
antirez authored
bitop.c contains the "Bit related string operations" so it seems more logical to call it bitops instead of bitop. This also makes it matching the name of the test (unit/bitops.tcl).
-
antirez authored
We run the array by 32 bit words instead of processing it byte per byte. If the code is compiled using GCC __builtin_popcount() builtin function is used instead.
-
antirez authored
The low level popualtion counting function is now separated from the BITCOUNT command implementation, so that the low level function can be further optimized and eventually used in other contexts if needed.
-
antirez authored
All the general string operations are implemented in t_string.c, however the bit operations, while targeting the string type, are better served in a specific file where we have the implementations of the following four commands and helper functions: GETBIT SETBIT BITOP BITCOUNT In the future this file will probably contain more code related to making the BITOP and BITCOUNT operations faster.
-