Commit 4aab50ac authored by rojingeorge's avatar rojingeorge
Browse files

Merge remote-tracking branch 'refs/remotes/antirez/unstable' into unstable

parents 646c958b f60aa4de
......@@ -13,7 +13,7 @@ endif
.SUFFIXES: .c .so .xo .o
all: helloworld.so
all: helloworld.so hellotype.so
.c.xo:
$(CC) -I. $(CFLAGS) $(SHOBJ_CFLAGS) -fPIC -c $< -o $@
......@@ -23,5 +23,10 @@ helloworld.xo: ../redismodule.h
helloworld.so: helloworld.xo
$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
hellotype.xo: ../redismodule.h
hellotype.so: hellotype.xo
$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
clean:
rm -rf *.xo *.so
Native types in Redis modules
===
Redis modules can access Redis built-in data structures both at high level,
by calling Redis commands, and at low level, by manipulating the data structures
directly.
By using these capabilities in order to build new abstractions on top of existing
Redis data structures, or by using strings DMA in order to encode modules
data structures into Redis strings, it is possible to create modules that
*feel like* they are exporting new data types. However, for more complex
problems, this is not enough, and the implementation of new data structures
inside the module is needed.
We call the ability of Redis modules to implement new data structures that
feel like native Redis ones **native types support**. This document describes
the API exported by the Redis modules system in order to create new data
structures and handle the serialization in RDB files, the rewriting process
in AOF, the type reporting via the `TYPE` command, and so forth.
Overview of native types
---
A module exporting a native type is composed of the following main parts:
* The implementation of some kind of new data structure and of commands operating on the new data structure.
* A set of callbacks that handle: RDB saving, RDB loading, AOF rewriting, releasing of a value associated with a key, calculation of a value digest (hash) to be used with the `DEBUG DIGEST` command.
* A 9 characters name that is unique to each module native data type.
* An encoding version, used to persist into RDB files a module-specific data version, so that a module will be able to load older representations from RDB files.
While to handle RDB loading, saving and AOF rewriting may look complex as a first glance, the modules API provide very high level function for handling all this, without requiring the user to handle read/write errors, so in practical terms, writing a new data structure for Redis is a simple task.
A **very easy** to understand but complete example of native type implementation
is available inside the Redis distribution in the `/modules/hellotype.c` file.
The reader is encouraged to read the documentation by looking at this example
implementation to see how things are applied in the practice.
Registering a new data type
===
In order to register a new native type into the Redis core, the module needs
to declare a global variable that will hold a reference to the data type.
The API to register the data type will return a data type reference that will
be stored in the global variable.
static RedisModuleType *MyType;
#define MYTYPE_ENCODING_VERSION 0
int RedisModule_OnLoad(RedisModuleCtx *ctx) {
MyType = RedisModule_CreateDataType("MyType-AZ", MYTYPE_ENCODING_VERSION,
MyTypeRDBLoad, MyTypeRDBSave, MyTypeAOFRewrite, MyTypeDigest,
MyTypeFree);
if (MyType == NULL) return REDISMODULE_ERR;
}
As you can see from the example above, a single API call is needed in order to
register the new type. However a number of function pointers are passed as
arguments. The prototype of `RedisModule_CreateDataType` is the following:
moduleType *RedisModule_CreateDataType(RedisModuleCtx *ctx,
const char *name, int encver,
moduleTypeLoadFunc rdb_load,
moduleTypeSaveFunc rdb_save,
moduleTypeRewriteFunc aof_rewrite,
moduleTypeDigestFunc digest,
moduleTypeFreeFunc free);
The `ctx` argument is the context that we receive in the `OnLoad` function.
The type `name` is a 9 character name in the character set that includes
from `A-Z`, `a-z`, `0-9`, plus the underscore `_` and minus `-` characters.
Note that **this name must be unique** for each data type in the Redis
ecosystem, so be creative, use both lower-case and upper case if it makes
sense, and try to use the convention of mixing the type name with the name
of the author of the module, to create a 9 character unique name.
For example if I'm building a *b-tree* data structure and my name is *antirez*
I'll call my type **btree1-az**. The name, converted to a 64 bit integer,
is stored inside the RDB file when saving the type, and will be used when the
RDB data is loaded in order to resolve what module can load the data. If Redis
finds no matching module, the integer is converted back to a name in order to
provide some clue to the user about what module is missing in order to load
the data.
The type name is also used as a reply for the `TYPE` command when called
with a key holding the registered type.
The `encver` argument is the encoding version used by the module to store data
inside the RDB file. For example I can start with an encoding version of 0,
but later when I release version 2.0 of my module, I can switch encoding to
something better. The new module will register with an encoding version of 1,
so when it saves new RDB files, the new version will be stored on disk. However
when loading RDB files, the module `rdb_load` method will be called even if
there is data found for a different encoding version (and the encoding version
is passed as argument to `rdb_load`), so that the module can still load old
RDB files.
The remaining arguments `rdb_load`, `rdb_save`, `aof_rewrite`, `digest` and
`free` are all callbacks with the following prototypes and uses:
typedef void *(*RedisModuleTypeLoadFunc)(RedisModuleIO *rdb, int encver);
typedef void (*RedisModuleTypeSaveFunc)(RedisModuleIO *rdb, void *value);
typedef void (*RedisModuleTypeRewriteFunc)(RedisModuleIO *aof, RedisModuleString *key, void *value);
typedef void (*RedisModuleTypeDigestFunc)(RedisModuleDigest *digest, void *value);
typedef void (*RedisModuleTypeFreeFunc)(void *value);
* `rdb_load` is called when loading data from the RDB file. It loads data in the same format as `rdb_save` produces.
* `rdb_save` is called when saving data to the RDB file.
* `aof_rewrite` is called when the AOF is being rewritten, and the module needs to tell Redis what is the sequence of commands to recreate the content of a given key.
* `digest` is called when `DEBUG DIGEST` is executed and a key holding this module type is found. Currently this is not yet implemented so the function ca be left empty.
* `free` is called when a key with the module native type is deleted via `DEL` or in any other mean, in order to let the module reclaim the memory associated with such a value.
Ok, but *why* modules types require a 9 characters name?
---
Oh, I understand you need to understand this, so here is a very specific
explanation.
When Redis persists to RDB files, modules specific data types require to
be persisted as well. Now RDB files are sequences of key-value pairs
like the following:
[1 byte type] [key] [a type specific value]
The 1 byte type identifies strings, lists, sets, and so forth. In the case
of modules data, it is set to a special value of `module data`, but of
course this is not enough, we need the information needed to link a specific
value with a specific module type that is able to load and handle it.
So when we save a `type specific value` about a module, we prefix it with
a 64 bit integer. 64 bits is large enough to store the informations needed
in order to lookup the module that can handle that specific type, but is
short enough that we can prefix each module value we store inside the RDB
without making the final RDB file too big. At the same time, this solution
of prefixing the value with a 64 bit *signature* does not require to do
strange things like defining in the RDB header a list of modules specific
types. Everything is pretty simple.
So, what you can store in 64 bits in order to identify a given module in
a reliable way? Well if you build a character set of 64 symbols, you can
easily store 9 characters of 6 bits, and you are left with 10 bits, that
are used in order to store the *encoding version* of the type, so that
the same type can evolve in the future and provide a different and more
efficient or updated serialization format for RDB files.
So the 64 bit prefix stored before each module value is like the following:
6|6|6|6|6|6|6|6|6|10
The first 9 elements are 6-bits characters, the final 10 bits is the
encoding version.
When the RDB file is loaded back, it reads the 64 bit value, masks the final
10 bits, and searches for a matching module in the modules types cache.
When a matching one is found, the method to load the RDB file value is called
with the 10 bits encoding version as argument, so that the module knows
what version of the data layout to load, if it can support multiple versions.
Now the interesting thing about all this is that, if instead the module type
cannot be resolved, since there is no loaded module having this signature,
we can convert back the 64 bit value into a 9 characters name, and print
an error to the user that includes the module type name! So that she or he
immediately realizes what's wrong.
Setting and getting keys
---
After registering our new data type in the `RedisModule_OnLoad()` function,
we also need to be able to set Redis keys having as value our native type.
This normally happens in the context of commands that write data to a key.
The native types API allow to set and get keys to module native data types,
and to test if a given key is already associated to a value of a specific data
type.
The API uses the normal modules `RedisModule_OpenKey()` low level key access
interface in order to deal with this. This is an eaxmple of setting a
native type private data structure to a Redis key:
RedisModuleKey *key = RedisModule_OpenKey(ctx,keyname,REDISMODULE_WRITE);
struct some_private_struct *data = createMyDataStructure();
RedisModule_ModuleTypeSetValue(key,MyType,data);
The function `RedisModule_ModuleTypeSetValue()` is used with a key handle open
for writing, and gets three arguments: the key handle, the reference to the
native type, as obtained during the type registration, and finally a `void*`
pointer that contains the private data implementing the module native type.
Note that Redis has no clues at all about what your data contains. It will
just call the callbacks you provided during the method registration in order
to perform operations on the type.
Similarly we can retrieve the private data from a key using this function:
struct some_private_struct *data;
data = RedisModule_ModuleTypeGetValue(key);
We can also test for a key to have our native type as value:
if (RedisModule_ModuleTypeGetType(key) == MyType) {
/* ... do something ... */
}
However for the calls to do the right thing, we need to check if the key
is empty, if it contains a value of the right kind, and so forth. So
the idiomatic code to implement a command writing to our native type
is along these lines:
RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
REDISMODULE_READ|REDISMODULE_WRITE);
int type = RedisModule_KeyType(key);
if (type != REDISMODULE_KEYTYPE_EMPTY &&
RedisModule_ModuleTypeGetType(key) != MyType)
{
return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
}
Then if we successfully verified the key is not of the wrong type, and
we are going to write to it, we usually want to create a new data structure if
the key is empty, or retrieve the reference to the value associated to the
key if there is already one:
/* Create an empty value object if the key is currently empty. */
struct some_private_struct *data;
if (type == REDISMODULE_KEYTYPE_EMPTY) {
data = createMyDataStructure();
RedisModule_ModuleTypeSetValue(key,MyTyke,data);
} else {
data = RedisModule_ModuleTypeGetValue(key);
}
/* Do something with 'data'... */
Free method
---
As already mentioned, when Redis needs to free a key holding a native type
value, it needs help from the module in order to release the memory. This
is the reason why we pass a `free` callback during the type registration:
typedef void (*RedisModuleTypeFreeFunc)(void *value);
A trivial implementation of the free method can be something like this,
assuming our data structure is composed of a single allocation:
void MyTypeFreeCallback(void *value) {
RedisModule_Free(value);
}
However a more real world one will call some function that performs a more
complex memory reclaiming, by casting the void pointer to some structure
and freeing all the resources composing the value.
RDB load and save methods
---
The RDB saving and loading callbacks need to create (and load back) a
representation of the data type on disk. Redis offers an high level API
that can automatically store inside the RDB file the following types:
* Unsigned 64 bit integers.
* Signed 64 bit integers.
* Doubles.
* Strings.
It is up to the module to find a viable representation using the above base
types. However note that while the integer and double values are stored
and loaded in an architecture and *endianess* agnostic way, if you use
the raw string saving API to, for example, save a structure on disk, you
have to care those details yourself.
This is the list of functions performing RDB saving and loading:
void RedisModule_SaveUnsigned(RedisModuleIO *io, uint64_t value);
uint64_t RedisModule_LoadUnsigned(RedisModuleIO *io);
void RedisModule_SaveSigned(RedisModuleIO *io, int64_t value);
int64_t RedisModule_LoadSigned(RedisModuleIO *io);
void RedisModule_SaveString(RedisModuleIO *io, RedisModuleString *s);
void RedisModule_SaveStringBuffer(RedisModuleIO *io, const char *str, size_t len);
RedisModuleString *RedisModule_LoadString(RedisModuleIO *io);
char *RedisModule_LoadStringBuffer(RedisModuleIO *io, size_t *lenptr);
void RedisModule_SaveDouble(RedisModuleIO *io, double value);
double RedisModule_LoadDouble(RedisModuleIO *io);
The functions don't require any error checking from the module, that can
always assume calls succeed.
As an example, imagine I've a native type that implements an array of
double values, with the following structure:
struct double_array {
size_t count;
double *values;
};
My `rdb_save` method may look like the following:
void DoubleArrayRDBSave(RedisModuleIO *io, void *ptr) {
struct dobule_array *da = ptr;
RedisModule_SaveUnsigned(io,da->count);
for (size_t j = 0; j < da->count; j++)
RedisModule_SaveDouble(io,da->values[j]);
}
What we did was to store the number of elements followed by each double
value. So when later we'll have to load the structure in the `rdb_load`
method we'll do something like this:
void *DoubleArrayRDBLoad(RedisModuleIO *io, int encver) {
if (encver != DOUBLE_ARRAY_ENC_VER) {
/* We should actually log an error here, or try to implement
the ability to load older versions of our data structure. */
return NULL;
}
struct double_array *da;
da = RedisModule_Alloc(sizeof(*da));
da->count = RedisModule_LoadUnsigned(io);
da->values = RedisModule_Alloc(da->count * sizeof(double));
for (size_t j = 0; j < da->count; j++)
da->values = RedisModule_LoadDouble(io);
return da;
}
The load callback just reconstruct back the data structure from the data
we stored in the RDB file.
Note that while there is no error handling on the API that writes and reads
from disk, still the load callback can return NULL on errors in case what
it reads does not look correct. Redis will just panic in that case.
AOF rewriting
---
void RedisModule_EmitAOF(RedisModuleIO *io, const char *cmdname, const char *fmt, ...);
Handling multiple encodings
---
WORK IN PROGRESS
Allocating memory
---
Modules data types should try to use `RedisModule_Alloc()` functions family
in order to allocate, reallocate and release heap memory used to implement the native data structures (see the other Redis Modules documentation for detailed information).
This is not just useful in order for Redis to be able to account for the memory used by the module, but there are also more advantages:
* Redis uses the `jemalloc` allcator, that often prevents fragmentation problems that could be caused by using the libc allocator.
* When loading strings from the RDB file, the native types API is able to return strings allocated directly with `RedisModule_Alloc()`, so that the module can directly link this memory into the data structure representation, avoiding an useless copy of the data.
Even if you are using external libraries implementing your data structures, the
allocation functions provided by the module API is exactly compatible with
`malloc()`, `realloc()`, `free()` and `strdup()`, so converting the libraries
in order to use these functions should be trivial.
In case you have an external library that uses libc `malloc()`, and you want
to avoid replacing manually all the calls with the Redis Modules API calls,
an approach could be to use simple macros in order to replace the libc calls
with the Redis API calls. Something like this could work:
#define malloc RedisModule_Alloc
#define realloc RedisModule_Realloc
#define free RedisModule_Free
#define strdup RedisModule_Strdup
However take in mind that mixing libc calls with Redis API calls will result
into troubles and crashes, so if you replace calls using macros, you need to
make sure that all the calls are correctly replaced, and that the code with
the substituted calls will never, for example, attempt to call
`RedisModule_Free()` with a pointer allocated using libc `malloc()`.
/* This file implements a new module native data type called "HELLOTYPE".
* The data structure implemented is a very simple ordered linked list of
* 64 bit integers, in order to have something that is real world enough, but
* at the same time, extremely simple to understand, to show how the API
* works, how a new data type is created, and how to write basic methods
* for RDB loading, saving and AOF rewriting.
*
* ------------------------------------------------------------------------------
*
* Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../redismodule.h"
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <string.h>
#include <stdint.h>
static RedisModuleType *HelloType;
/* ========================== Internal data structure =======================
* This is just a linked list of 64 bit integers where elements are inserted
* in-place, so it's ordered. There is no pop/push operation but just insert
* because it is enough to show the implementation of new data types without
* making things complex. */
struct HelloTypeNode {
int64_t value;
struct HelloTypeNode *next;
};
struct HelloTypeObject {
struct HelloTypeNode *head;
size_t len; /* Number of elements added. */
};
struct HelloTypeObject *createHelloTypeObject(void) {
struct HelloTypeObject *o;
o = RedisModule_Alloc(sizeof(*o));
o->head = NULL;
o->len = 0;
return o;
}
void HelloTypeInsert(struct HelloTypeObject *o, int64_t ele) {
struct HelloTypeNode *next = o->head, *newnode, *prev = NULL;
while(next && next->value < ele) {
prev = next;
next = next->next;
}
newnode = RedisModule_Alloc(sizeof(*newnode));
newnode->value = ele;
newnode->next = next;
if (prev) {
prev->next = newnode;
} else {
o->head = newnode;
}
o->len++;
}
void HelloTypeReleaseObject(struct HelloTypeObject *o) {
struct HelloTypeNode *cur, *next;
cur = o->head;
while(cur) {
next = cur->next;
RedisModule_Free(cur);
cur = next;
}
RedisModule_Free(o);
}
/* ========================= "hellotype" type commands ======================= */
/* HELLOTYPE.INSERT key value */
int HelloTypeInsert_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
if (argc != 3) return RedisModule_WrongArity(ctx);
RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
REDISMODULE_READ|REDISMODULE_WRITE);
int type = RedisModule_KeyType(key);
if (type != REDISMODULE_KEYTYPE_EMPTY &&
RedisModule_ModuleTypeGetType(key) != HelloType)
{
return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
}
long long value;
if ((RedisModule_StringToLongLong(argv[2],&value) != REDISMODULE_OK)) {
return RedisModule_ReplyWithError(ctx,"ERR invalid value: must be a signed 64 bit integer");
}
/* Create an empty value object if the key is currently empty. */
struct HelloTypeObject *hto;
if (type == REDISMODULE_KEYTYPE_EMPTY) {
hto = createHelloTypeObject();
RedisModule_ModuleTypeSetValue(key,HelloType,hto);
} else {
hto = RedisModule_ModuleTypeGetValue(key);
}
/* Insert the new element. */
HelloTypeInsert(hto,value);
RedisModule_ReplyWithLongLong(ctx,hto->len);
RedisModule_ReplicateVerbatim(ctx);
return REDISMODULE_OK;
}
/* HELLOTYPE.RANGE key first count */
int HelloTypeRange_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
if (argc != 4) return RedisModule_WrongArity(ctx);
RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
REDISMODULE_READ|REDISMODULE_WRITE);
int type = RedisModule_KeyType(key);
if (type != REDISMODULE_KEYTYPE_EMPTY &&
RedisModule_ModuleTypeGetType(key) != HelloType)
{
return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
}
long long first, count;
if (RedisModule_StringToLongLong(argv[2],&first) != REDISMODULE_OK ||
RedisModule_StringToLongLong(argv[3],&count) != REDISMODULE_OK ||
first < 0 || count < 0)
{
return RedisModule_ReplyWithError(ctx,
"ERR invalid first or count parameters");
}
struct HelloTypeObject *hto = RedisModule_ModuleTypeGetValue(key);
struct HelloTypeNode *node = hto ? hto->head : NULL;
RedisModule_ReplyWithArray(ctx,REDISMODULE_POSTPONED_ARRAY_LEN);
long long arraylen = 0;
while(node && count--) {
RedisModule_ReplyWithLongLong(ctx,node->value);
arraylen++;
node = node->next;
}
RedisModule_ReplySetArrayLength(ctx,arraylen);
return REDISMODULE_OK;
}
/* HELLOTYPE.LEN key */
int HelloTypeLen_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
if (argc != 2) return RedisModule_WrongArity(ctx);
RedisModuleKey *key = RedisModule_OpenKey(ctx,argv[1],
REDISMODULE_READ|REDISMODULE_WRITE);
int type = RedisModule_KeyType(key);
if (type != REDISMODULE_KEYTYPE_EMPTY &&
RedisModule_ModuleTypeGetType(key) != HelloType)
{
return RedisModule_ReplyWithError(ctx,REDISMODULE_ERRORMSG_WRONGTYPE);
}
struct HelloTypeObject *hto = RedisModule_ModuleTypeGetValue(key);
RedisModule_ReplyWithLongLong(ctx,hto ? hto->len : 0);
return REDISMODULE_OK;
}
/* ========================== "hellotype" type methods ======================= */
void *HelloTypeRdbLoad(RedisModuleIO *rdb, int encver) {
if (encver != 0) {
/* RedisModule_Log("warning","Can't load data with version %d", encver);*/
return NULL;
}
uint64_t elements = RedisModule_LoadUnsigned(rdb);
struct HelloTypeObject *hto = createHelloTypeObject();
while(elements--) {
int64_t ele = RedisModule_LoadSigned(rdb);
HelloTypeInsert(hto,ele);
}
return hto;
}
void HelloTypeRdbSave(RedisModuleIO *rdb, void *value) {
struct HelloTypeObject *hto = value;
struct HelloTypeNode *node = hto->head;
RedisModule_SaveUnsigned(rdb,hto->len);
while(node) {
RedisModule_SaveSigned(rdb,node->value);
node = node->next;
}
}
void HelloTypeAofRewrite(RedisModuleIO *aof, RedisModuleString *key, void *value) {
struct HelloTypeObject *hto = value;
struct HelloTypeNode *node = hto->head;
while(node) {
RedisModule_EmitAOF(aof,"HELLOTYPE.INSERT","sl",key,node->value);
node = node->next;
}
}
void HelloTypeDigest(RedisModuleDigest *digest, void *value) {
/* TODO: The DIGEST module interface is yet not implemented. */
}
void HelloTypeFree(void *value) {
HelloTypeReleaseObject(value);
}
/* This function must be present on each Redis module. It is used in order to
* register the commands into the Redis server. */
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (RedisModule_Init(ctx,"hellotype",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
HelloType = RedisModule_CreateDataType(ctx,"hellotype",0,HelloTypeRdbLoad,HelloTypeRdbSave,HelloTypeAofRewrite,HelloTypeDigest,HelloTypeFree);
if (HelloType == NULL) return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hellotype.insert",
HelloTypeInsert_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hellotype.range",
HelloTypeRange_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hellotype.len",
HelloTypeLen_RedisCommand,"readonly",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
/* Helloworld module -- A few examples of the Redis Modules API in the form
* of commands showing how to accomplish common tasks.
*
* This module does not do anything useful, if not for a few commands. The
* examples are designed in order to show the API.
*
* ------------------------------------------------------------------------------
*
* Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "../redismodule.h"
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <string.h>
/* HELLO.SIMPLE is among the simplest commands you can implement.
* It just returns the currently selected DB id, a functionality which is
......@@ -448,12 +485,70 @@ int HelloHCopy_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int a
return REDISMODULE_OK;
}
/* HELLO.LEFTPAD str len ch
* This is an implementation of the infamous LEFTPAD function, that
* was at the center of an issue with the npm modules system in March 2016.
*
* LEFTPAD is a good example of using a Redis Modules API called
* "pool allocator", that was a famous way to allocate memory in yet another
* open source project, the Apache web server.
*
* The concept is very simple: there is memory that is useful to allocate
* only in the context of serving a request, and must be freed anyway when
* the callback implementing the command returns. So in that case the module
* does not need to retain a reference to these allocations, it is just
* required to free the memory before returning. When this is the case the
* module can call RedisModule_PoolAlloc() instead, that works like malloc()
* but will automatically free the memory when the module callback returns.
*
* Note that PoolAlloc() does not necessarily require AutoMemory to be
* active. */
int HelloLeftPad_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModule_AutoMemory(ctx); /* Use automatic memory management. */
long long padlen;
if (argc != 4) return RedisModule_WrongArity(ctx);
if ((RedisModule_StringToLongLong(argv[2],&padlen) != REDISMODULE_OK) ||
(padlen< 0)) {
return RedisModule_ReplyWithError(ctx,"ERR invalid padding length");
}
size_t strlen, chlen;
const char *str = RedisModule_StringPtrLen(argv[1], &strlen);
const char *ch = RedisModule_StringPtrLen(argv[3], &chlen);
/* If the string is already larger than the target len, just return
* the string itself. */
if (strlen >= padlen)
return RedisModule_ReplyWithString(ctx,argv[1]);
/* Padding must be a single character in this simple implementation. */
if (chlen != 1)
return RedisModule_ReplyWithError(ctx,
"ERR padding must be a single char");
/* Here we use our pool allocator, for our throw-away allocation. */
padlen -= strlen;
char *buf = RedisModule_PoolAlloc(ctx,padlen+strlen);
for (size_t j = 0; j < padlen; j++) buf[j] = *ch;
memcpy(buf+padlen,str,strlen);
RedisModule_ReplyWithStringBuffer(ctx,buf,padlen+strlen);
return REDISMODULE_OK;
}
/* This function must be present on each Redis module. It is used in order to
* register the commands into the Redis server. */
int RedisModule_OnLoad(RedisModuleCtx *ctx) {
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (RedisModule_Init(ctx,"helloworld",1,REDISMODULE_APIVER_1)
== REDISMODULE_ERR) return REDISMODULE_ERR;
/* Log the list of parameters passing loading the module. */
for (int j = 0; j < argc; j++) {
const char *s = RedisModule_StringPtrLen(argv[j],NULL);
printf("Module loaded with ARGV[%d] = %s\n", j, s);
}
if (RedisModule_CreateCommand(ctx,"hello.simple",
HelloSimple_RedisCommand,"readonly",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
......@@ -515,5 +610,9 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx) {
HelloHCopy_RedisCommand,"write deny-oom",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"hello.leftpad",
HelloLeftPad_RedisCommand,"",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}
......@@ -1605,7 +1605,7 @@ void clientCommand(client *c) {
pauseClients(duration);
addReply(c,shared.ok);
} else {
addReplyError(c, "Syntax error, try CLIENT (LIST | KILL ip:port | GETNAME | SETNAME connection-name)");
addReplyError(c, "Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)");
}
}
......
......@@ -97,7 +97,7 @@ robj *createEmbeddedStringObject(const char *ptr, size_t len) {
}
/* Create a string object with EMBSTR encoding if it is smaller than
* REIDS_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
* OBJ_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
* used.
*
* The current limit of 39 is chosen so that the biggest string object
......@@ -147,7 +147,7 @@ robj *createStringObjectFromLongDouble(long double value, int humanfriendly) {
* will always result in a fresh object that is unshared (refcount == 1).
*
* The resulting object always has refcount set to 1. */
robj *dupStringObject(robj *o) {
robj *dupStringObject(const robj *o) {
robj *d;
serverAssert(o->type == OBJ_STRING);
......@@ -221,6 +221,13 @@ robj *createZsetZiplistObject(void) {
return o;
}
robj *createModuleObject(moduleType *mt, void *value) {
moduleValue *mv = zmalloc(sizeof(*mv));
mv->type = mt;
mv->value = value;
return createObject(OBJ_MODULE,mv);
}
void freeStringObject(robj *o) {
if (o->encoding == OBJ_ENCODING_RAW) {
sdsfree(o->ptr);
......@@ -281,6 +288,12 @@ void freeHashObject(robj *o) {
}
}
void freeModuleObject(robj *o) {
moduleValue *mv = o->ptr;
mv->type->free(mv->value);
zfree(mv);
}
void incrRefCount(robj *o) {
if (o->refcount != OBJ_SHARED_REFCOUNT) o->refcount++;
}
......@@ -293,6 +306,7 @@ void decrRefCount(robj *o) {
case OBJ_SET: freeSetObject(o); break;
case OBJ_ZSET: freeZsetObject(o); break;
case OBJ_HASH: freeHashObject(o); break;
case OBJ_MODULE: freeModuleObject(o); break;
default: serverPanic("Unknown object type"); break;
}
zfree(o);
......@@ -371,10 +385,10 @@ robj *tryObjectEncoding(robj *o) {
if (o->refcount > 1) return o;
/* Check if we can represent this string as a long integer.
* Note that we are sure that a string larger than 21 chars is not
* Note that we are sure that a string larger than 20 chars is not
* representable as a 32 nor 64 bit integer. */
len = sdslen(s);
if (len <= 21 && string2l(s,len,&value)) {
if (len <= 20 && string2l(s,len,&value)) {
/* This object is encodable as a long. Try to use a shared object.
* Note that we avoid using shared integers when maxmemory is used
* because every object needs to have a private LRU field for the LRU
......@@ -525,7 +539,7 @@ size_t stringObjectLen(robj *o) {
}
}
int getDoubleFromObject(robj *o, double *target) {
int getDoubleFromObject(const robj *o, double *target) {
double value;
char *eptr;
......@@ -536,7 +550,7 @@ int getDoubleFromObject(robj *o, double *target) {
if (sdsEncodedObject(o)) {
errno = 0;
value = strtod(o->ptr, &eptr);
if (isspace(((char*)o->ptr)[0]) ||
if (isspace(((const char*)o->ptr)[0]) ||
eptr[0] != '\0' ||
(errno == ERANGE &&
(value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
......
......@@ -149,7 +149,7 @@ REDIS_STATIC quicklistNode *quicklistCreateNode(void) {
}
/* Return cached quicklist count */
unsigned int quicklistCount(quicklist *ql) { return ql->count; }
unsigned int quicklistCount(const quicklist *ql) { return ql->count; }
/* Free entire quicklist. */
void quicklistRelease(quicklist *quicklist) {
......
......@@ -92,8 +92,8 @@ typedef struct quicklistEntry {
quicklistNode *node;
unsigned char *zi;
unsigned char *value;
unsigned int sz;
long long longval;
unsigned int sz;
int offset;
} quicklistEntry;
......@@ -154,7 +154,7 @@ int quicklistPopCustom(quicklist *quicklist, int where, unsigned char **data,
void *(*saver)(unsigned char *data, unsigned int sz));
int quicklistPop(quicklist *quicklist, int where, unsigned char **data,
unsigned int *sz, long long *slong);
unsigned int quicklistCount(quicklist *ql);
unsigned int quicklistCount(const quicklist *ql);
int quicklistCompare(unsigned char *p1, unsigned char *p2, int p2_len);
size_t quicklistGetLzf(const quicklistNode *node, void **data);
......
......@@ -41,11 +41,6 @@
#include <sys/stat.h>
#include <sys/param.h>
#define RDB_LOAD_NONE 0
#define RDB_LOAD_ENC (1<<0)
#define RDB_LOAD_PLAIN (1<<1)
#define RDB_LOAD_SDS (1<<2)
#define rdbExitReportCorruptRDB(reason) rdbCheckThenExit(reason, __LINE__);
void rdbCheckThenExit(char *reason, int where) {
......@@ -95,7 +90,7 @@ long long rdbLoadMillisecondTime(rio *rdb) {
/* Saves an encoded length. The first two bits in the first byte are used to
* hold the encoding type. See the RDB_* definitions for more information
* on the types of encoding. */
int rdbSaveLen(rio *rdb, uint32_t len) {
int rdbSaveLen(rio *rdb, uint64_t len) {
unsigned char buf[2];
size_t nwritten;
......@@ -110,44 +105,78 @@ int rdbSaveLen(rio *rdb, uint32_t len) {
buf[1] = len&0xFF;
if (rdbWriteRaw(rdb,buf,2) == -1) return -1;
nwritten = 2;
} else {
} else if (len <= UINT32_MAX) {
/* Save a 32 bit len */
buf[0] = (RDB_32BITLEN<<6);
buf[0] = RDB_32BITLEN;
if (rdbWriteRaw(rdb,buf,1) == -1) return -1;
len = htonl(len);
if (rdbWriteRaw(rdb,&len,4) == -1) return -1;
uint32_t len32 = htonl(len);
if (rdbWriteRaw(rdb,&len32,4) == -1) return -1;
nwritten = 1+4;
} else {
/* Save a 64 bit len */
buf[0] = RDB_64BITLEN;
if (rdbWriteRaw(rdb,buf,1) == -1) return -1;
len = htonu64(len);
if (rdbWriteRaw(rdb,&len,8) == -1) return -1;
nwritten = 1+8;
}
return nwritten;
}
/* Load an encoded length. The "isencoded" argument is set to 1 if the length
* is not actually a length but an "encoding type". See the RDB_ENC_*
* definitions in rdb.h for more information. */
uint32_t rdbLoadLen(rio *rdb, int *isencoded) {
/* Load an encoded length. If the loaded length is a normal length as stored
* with rdbSaveLen(), the read length is set to '*lenptr'. If instead the
* loaded length describes a special encoding that follows, then '*isencoded'
* is set to 1 and the encoding format is stored at '*lenptr'.
*
* See the RDB_ENC_* definitions in rdb.h for more information on special
* encodings.
*
* The function returns -1 on error, 0 on success. */
int rdbLoadLenByRef(rio *rdb, int *isencoded, uint64_t *lenptr) {
unsigned char buf[2];
uint32_t len;
int type;
if (isencoded) *isencoded = 0;
if (rioRead(rdb,buf,1) == 0) return RDB_LENERR;
if (rioRead(rdb,buf,1) == 0) return -1;
type = (buf[0]&0xC0)>>6;
if (type == RDB_ENCVAL) {
/* Read a 6 bit encoding type. */
if (isencoded) *isencoded = 1;
return buf[0]&0x3F;
*lenptr = buf[0]&0x3F;
} else if (type == RDB_6BITLEN) {
/* Read a 6 bit len. */
return buf[0]&0x3F;
*lenptr = buf[0]&0x3F;
} else if (type == RDB_14BITLEN) {
/* Read a 14 bit len. */
if (rioRead(rdb,buf+1,1) == 0) return RDB_LENERR;
return ((buf[0]&0x3F)<<8)|buf[1];
} else {
if (rioRead(rdb,buf+1,1) == 0) return -1;
*lenptr = ((buf[0]&0x3F)<<8)|buf[1];
} else if (buf[0] == RDB_32BITLEN) {
/* Read a 32 bit len. */
if (rioRead(rdb,&len,4) == 0) return RDB_LENERR;
return ntohl(len);
uint32_t len;
if (rioRead(rdb,&len,4) == 0) return -1;
*lenptr = ntohl(len);
} else if (buf[0] == RDB_64BITLEN) {
/* Read a 64 bit len. */
uint64_t len;
if (rioRead(rdb,&len,8) == 0) return -1;
*lenptr = ntohu64(len);
} else {
rdbExitReportCorruptRDB("Unknown length encoding in rdbLoadLen()");
return -1; /* Never reached. */
}
return 0;
}
/* This is like rdbLoadLenByRef() but directly returns the value read
* from the RDB stream, signaling an error by returning RDB_LENERR
* (since it is a too large count to be applicable in any Redis data
* structure). */
uint64_t rdbLoadLen(rio *rdb, int *isencoded) {
uint64_t len;
if (rdbLoadLenByRef(rdb,isencoded,&len) == -1) return RDB_LENERR;
return len;
}
/* Encodes the "value" argument as integer when it fits in the supported ranges
......@@ -179,7 +208,7 @@ int rdbEncodeInteger(long long value, unsigned char *enc) {
/* Loads an integer-encoded object with the specified encoding type "enctype".
* The returned value changes according to the flags, see
* rdbGenerincLoadStringObject() for more info. */
void *rdbLoadIntegerObject(rio *rdb, int enctype, int flags) {
void *rdbLoadIntegerObject(rio *rdb, int enctype, int flags, size_t *lenptr) {
int plain = flags & RDB_LOAD_PLAIN;
int sds = flags & RDB_LOAD_SDS;
int encode = flags & RDB_LOAD_ENC;
......@@ -206,6 +235,7 @@ void *rdbLoadIntegerObject(rio *rdb, int enctype, int flags) {
if (plain || sds) {
char buf[LONG_STR_SIZE], *p;
int len = ll2string(buf,sizeof(buf),val);
if (lenptr) *lenptr = len;
p = plain ? zmalloc(len) : sdsnewlen(NULL,len);
memcpy(p,buf,len);
return p;
......@@ -281,10 +311,10 @@ ssize_t rdbSaveLzfStringObject(rio *rdb, unsigned char *s, size_t len) {
/* Load an LZF compressed string in RDB format. The returned value
* changes according to 'flags'. For more info check the
* rdbGenericLoadStringObject() function. */
void *rdbLoadLzfStringObject(rio *rdb, int flags) {
void *rdbLoadLzfStringObject(rio *rdb, int flags, size_t *lenptr) {
int plain = flags & RDB_LOAD_PLAIN;
int sds = flags & RDB_LOAD_SDS;
unsigned int len, clen;
uint64_t len, clen;
unsigned char *c = NULL;
char *val = NULL;
......@@ -295,6 +325,7 @@ void *rdbLoadLzfStringObject(rio *rdb, int flags) {
/* Allocate our target according to the uncompressed size. */
if (plain) {
val = zmalloc(len);
if (lenptr) *lenptr = len;
} else {
val = sdsnewlen(NULL,len);
}
......@@ -393,13 +424,15 @@ int rdbSaveStringObject(rio *rdb, robj *obj) {
* RDB_LOAD_PLAIN: Return a plain string allocated with zmalloc()
* instead of a Redis object with an sds in it.
* RDB_LOAD_SDS: Return an SDS string instead of a Redis object.
*/
void *rdbGenericLoadStringObject(rio *rdb, int flags) {
*
* On I/O error NULL is returned.
*/
void *rdbGenericLoadStringObject(rio *rdb, int flags, size_t *lenptr) {
int encode = flags & RDB_LOAD_ENC;
int plain = flags & RDB_LOAD_PLAIN;
int sds = flags & RDB_LOAD_SDS;
int isencoded;
uint32_t len;
uint64_t len;
len = rdbLoadLen(rdb,&isencoded);
if (isencoded) {
......@@ -407,9 +440,9 @@ void *rdbGenericLoadStringObject(rio *rdb, int flags) {
case RDB_ENC_INT8:
case RDB_ENC_INT16:
case RDB_ENC_INT32:
return rdbLoadIntegerObject(rdb,len,flags);
return rdbLoadIntegerObject(rdb,len,flags,lenptr);
case RDB_ENC_LZF:
return rdbLoadLzfStringObject(rdb,flags);
return rdbLoadLzfStringObject(rdb,flags,lenptr);
default:
rdbExitReportCorruptRDB("Unknown RDB encoding type");
}
......@@ -418,6 +451,7 @@ void *rdbGenericLoadStringObject(rio *rdb, int flags) {
if (len == RDB_LENERR) return NULL;
if (plain || sds) {
void *buf = plain ? zmalloc(len) : sdsnewlen(NULL,len);
if (lenptr) *lenptr = len;
if (len && rioRead(rdb,buf,len) == 0) {
if (plain)
zfree(buf);
......@@ -438,11 +472,11 @@ void *rdbGenericLoadStringObject(rio *rdb, int flags) {
}
robj *rdbLoadStringObject(rio *rdb) {
return rdbGenericLoadStringObject(rdb,RDB_LOAD_NONE);
return rdbGenericLoadStringObject(rdb,RDB_LOAD_NONE,NULL);
}
robj *rdbLoadEncodedStringObject(rio *rdb) {
return rdbGenericLoadStringObject(rdb,RDB_LOAD_ENC);
return rdbGenericLoadStringObject(rdb,RDB_LOAD_ENC,NULL);
}
/* Save a double value. Doubles are saved as strings prefixed by an unsigned
......@@ -505,6 +539,24 @@ int rdbLoadDoubleValue(rio *rdb, double *val) {
}
}
/* Saves a double for RDB 8 or greater, where IE754 binary64 format is assumed.
* We just make sure the integer is always stored in little endian, otherwise
* the value is copied verbatim from memory to disk.
*
* Return -1 on error, the size of the serialized value on success. */
int rdbSaveBinaryDoubleValue(rio *rdb, double val) {
memrev64ifbe(&val);
return rdbWriteRaw(rdb,&val,8);
}
/* Loads a double from RDB 8 or greater. See rdbSaveBinaryDoubleValue() for
* more info. On error -1 is returned, otherwise 0. */
int rdbLoadBinaryDoubleValue(rio *rdb, double *val) {
if (rioRead(rdb,val,8) == 0) return -1;
memrev64ifbe(val);
return 0;
}
/* Save the object type of object "o". */
int rdbSaveObjectType(rio *rdb, robj *o) {
switch (o->type) {
......@@ -526,7 +578,7 @@ int rdbSaveObjectType(rio *rdb, robj *o) {
if (o->encoding == OBJ_ENCODING_ZIPLIST)
return rdbSaveType(rdb,RDB_TYPE_ZSET_ZIPLIST);
else if (o->encoding == OBJ_ENCODING_SKIPLIST)
return rdbSaveType(rdb,RDB_TYPE_ZSET);
return rdbSaveType(rdb,RDB_TYPE_ZSET_2);
else
serverPanic("Unknown sorted set encoding");
case OBJ_HASH:
......@@ -536,6 +588,8 @@ int rdbSaveObjectType(rio *rdb, robj *o) {
return rdbSaveType(rdb,RDB_TYPE_HASH);
else
serverPanic("Unknown hash encoding");
case OBJ_MODULE:
return rdbSaveType(rdb,RDB_TYPE_MODULE);
default:
serverPanic("Unknown object type");
}
......@@ -629,7 +683,7 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
if ((n = rdbSaveRawString(rdb,(unsigned char*)ele,sdslen(ele)))
== -1) return -1;
nwritten += n;
if ((n = rdbSaveDoubleValue(rdb,*score)) == -1) return -1;
if ((n = rdbSaveBinaryDoubleValue(rdb,*score)) == -1) return -1;
nwritten += n;
}
dictReleaseIterator(di);
......@@ -667,6 +721,22 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
serverPanic("Unknown hash encoding");
}
} else if (o->type == OBJ_MODULE) {
/* Save a module-specific value. */
RedisModuleIO io;
moduleValue *mv = o->ptr;
moduleType *mt = mv->type;
moduleInitIOContext(io,mt,rdb);
/* Write the "module" identifier as prefix, so that we'll be able
* to call the right module during loading. */
int retval = rdbSaveLen(rdb,mt->id);
if (retval == -1) return -1;
io.bytes += retval;
/* Then write the module-specific representation. */
mt->rdb_save(&io,mv->value);
return io.error ? -1 : (ssize_t)io.bytes;
} else {
serverPanic("Unknown object type");
}
......@@ -1005,8 +1075,8 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
long long llval;
sds sdsele;
if ((sdsele = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS)) == NULL)
return NULL;
if ((sdsele = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL))
== NULL) return NULL;
if (o->encoding == OBJ_ENCODING_INTSET) {
/* Fetch integer value from element. */
......@@ -1026,7 +1096,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
sdsfree(sdsele);
}
}
} else if (rdbtype == RDB_TYPE_ZSET) {
} else if (rdbtype == RDB_TYPE_ZSET_2 || rdbtype == RDB_TYPE_ZSET) {
/* Read list/set value. */
size_t zsetlen;
size_t maxelelen = 0;
......@@ -1042,9 +1112,14 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
double score;
zskiplistNode *znode;
if ((sdsele = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS)) == NULL)
return NULL;
if (rdbLoadDoubleValue(rdb,&score) == -1) return NULL;
if ((sdsele = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL))
== NULL) return NULL;
if (rdbtype == RDB_TYPE_ZSET_2) {
if (rdbLoadBinaryDoubleValue(rdb,&score) == -1) return NULL;
} else {
if (rdbLoadDoubleValue(rdb,&score) == -1) return NULL;
}
/* Don't care about integer-encoded strings. */
if (sdslen(sdsele) > maxelelen) maxelelen = sdslen(sdsele);
......@@ -1075,10 +1150,10 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
while (o->encoding == OBJ_ENCODING_ZIPLIST && len > 0) {
len--;
/* Load raw strings */
if ((field = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS)) == NULL)
return NULL;
if ((value = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS)) == NULL)
return NULL;
if ((field = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL))
== NULL) return NULL;
if ((value = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL))
== NULL) return NULL;
/* Add pair to ziplist */
o->ptr = ziplistPush(o->ptr, (unsigned char*)field,
......@@ -1103,10 +1178,10 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
while (o->encoding == OBJ_ENCODING_HT && len > 0) {
len--;
/* Load encoded strings */
if ((field = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS)) == NULL)
return NULL;
if ((value = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS)) == NULL)
return NULL;
if ((field = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL))
== NULL) return NULL;
if ((value = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL))
== NULL) return NULL;
/* Add pair to hash table */
ret = dictAdd((dict*)o->ptr, field, value);
......@@ -1124,7 +1199,8 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
server.list_compress_depth);
while (len--) {
unsigned char *zl = rdbGenericLoadStringObject(rdb,RDB_LOAD_PLAIN);
unsigned char *zl =
rdbGenericLoadStringObject(rdb,RDB_LOAD_PLAIN,NULL);
if (zl == NULL) return NULL;
quicklistAppendZiplist(o->ptr, zl);
}
......@@ -1134,7 +1210,8 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
rdbtype == RDB_TYPE_ZSET_ZIPLIST ||
rdbtype == RDB_TYPE_HASH_ZIPLIST)
{
unsigned char *encoded = rdbGenericLoadStringObject(rdb,RDB_LOAD_PLAIN);
unsigned char *encoded =
rdbGenericLoadStringObject(rdb,RDB_LOAD_PLAIN,NULL);
if (encoded == NULL) return NULL;
o = createObject(OBJ_STRING,encoded); /* Obj type fixed below. */
......@@ -1201,6 +1278,27 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
rdbExitReportCorruptRDB("Unknown encoding");
break;
}
} else if (rdbtype == RDB_TYPE_MODULE) {
uint64_t moduleid = rdbLoadLen(rdb,NULL);
moduleType *mt = moduleTypeLookupModuleByID(moduleid);
char name[10];
if (mt == NULL) {
moduleTypeNameByID(name,moduleid);
serverLog(LL_WARNING,"The RDB file contains module data I can't load: no matching module '%s'", name);
exit(1);
}
RedisModuleIO io;
moduleInitIOContext(io,mt,rdb);
/* Call the rdb_load method of the module providing the 10 bit
* encoding version in the lower 10 bits of the module ID. */
void *ptr = mt->rdb_load(&io,moduleid&1023);
if (ptr == NULL) {
moduleTypeNameByID(name,moduleid);
serverLog(LL_WARNING,"The RDB file contains module data for the module type '%s', that the responsible module is not able to load. Check for modules log above for additional clues.", name);
exit(1);
}
o = createModuleObject(mt,ptr);
} else {
rdbExitReportCorruptRDB("Unknown object type");
}
......@@ -1255,7 +1353,7 @@ void rdbLoadProgressCallback(rio *r, const void *buf, size_t len) {
}
int rdbLoad(char *filename) {
uint32_t dbid;
uint64_t dbid;
int type, rdbver;
redisDb *db = server.db+0;
char buf[1024];
......@@ -1328,7 +1426,7 @@ int rdbLoad(char *filename) {
} else if (type == RDB_OPCODE_RESIZEDB) {
/* RESIZEDB: Hint about the size of the keys in the currently
* selected data base, in order to avoid useless rehashing. */
uint32_t db_size, expires_size;
uint64_t db_size, expires_size;
if ((db_size = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
goto eoferr;
if ((expires_size = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
......
......@@ -38,16 +38,17 @@
/* The current RDB version. When the format changes in a way that is no longer
* backward compatible this number gets incremented. */
#define RDB_VERSION 7
#define RDB_VERSION 8
/* Defines related to the dump file format. To store 32 bits lengths for short
* keys requires a lot of space, so we check the most significant 2 bits of
* the first byte to interpreter the length:
*
* 00|000000 => if the two MSB are 00 the len is the 6 bits of this byte
* 01|000000 00000000 => 01, the len is 14 byes, 6 bits + 8 bits of next byte
* 10|000000 [32 bit integer] => if it's 01, a full 32 bit len will follow
* 11|000000 this means: specially encoded object will follow. The six bits
* 00|XXXXXX => if the two MSB are 00 the len is the 6 bits of this byte
* 01|XXXXXX XXXXXXXX => 01, the len is 14 byes, 6 bits + 8 bits of next byte
* 10|000000 [32 bit integer] => A full 32 bit len in net byte order will follow
* 10|000001 [64 bit integer] => A full 64 bit len in net byte order will follow
* 11|OBKIND this means: specially encoded object will follow. The six bits
* number specify the kind of object that follows.
* See the RDB_ENC_* defines.
*
......@@ -55,12 +56,13 @@
* values, will fit inside. */
#define RDB_6BITLEN 0
#define RDB_14BITLEN 1
#define RDB_32BITLEN 2
#define RDB_32BITLEN 0x80
#define RDB_64BITLEN 0x81
#define RDB_ENCVAL 3
#define RDB_LENERR UINT_MAX
#define RDB_LENERR UINT64_MAX
/* When a length of a string object stored on disk has the first two bits
* set, the remaining two bits specify a special encoding for the object
* set, the remaining six bits specify a special encoding for the object
* accordingly to the following defines: */
#define RDB_ENC_INT8 0 /* 8 bit signed integer */
#define RDB_ENC_INT16 1 /* 16 bit signed integer */
......@@ -74,6 +76,8 @@
#define RDB_TYPE_SET 2
#define RDB_TYPE_ZSET 3
#define RDB_TYPE_HASH 4
#define RDB_TYPE_ZSET_2 5 /* ZSET version 2 with doubles stored in binary. */
#define RDB_TYPE_MODULE 6
/* NOTE: WHEN ADDING NEW RDB TYPE, UPDATE rdbIsObjectType() BELOW */
/* Object types for encoded objects. */
......@@ -86,7 +90,7 @@
/* NOTE: WHEN ADDING NEW RDB TYPE, UPDATE rdbIsObjectType() BELOW */
/* Test if a type is an object type. */
#define rdbIsObjectType(t) ((t >= 0 && t <= 4) || (t >= 9 && t <= 14))
#define rdbIsObjectType(t) ((t >= 0 && t <= 6) || (t >= 9 && t <= 14))
/* Special RDB opcodes (saved/loaded with rdbSaveType/rdbLoadType). */
#define RDB_OPCODE_AUX 250
......@@ -96,12 +100,19 @@
#define RDB_OPCODE_SELECTDB 254
#define RDB_OPCODE_EOF 255
/* rdbLoad...() functions flags. */
#define RDB_LOAD_NONE 0
#define RDB_LOAD_ENC (1<<0)
#define RDB_LOAD_PLAIN (1<<1)
#define RDB_LOAD_SDS (1<<2)
int rdbSaveType(rio *rdb, unsigned char type);
int rdbLoadType(rio *rdb);
int rdbSaveTime(rio *rdb, time_t t);
time_t rdbLoadTime(rio *rdb);
int rdbSaveLen(rio *rdb, uint32_t len);
uint32_t rdbLoadLen(rio *rdb, int *isencoded);
int rdbSaveLen(rio *rdb, uint64_t len);
uint64_t rdbLoadLen(rio *rdb, int *isencoded);
int rdbLoadLenByRef(rio *rdb, int *isencoded, uint64_t *lenptr);
int rdbSaveObjectType(rio *rdb, robj *o);
int rdbLoadObjectType(rio *rdb);
int rdbLoad(char *filename);
......@@ -115,5 +126,10 @@ robj *rdbLoadObject(int type, rio *rdb);
void backgroundSaveDoneHandler(int exitcode, int bysignal);
int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val, long long expiretime, long long now);
robj *rdbLoadStringObject(rio *rdb);
int rdbSaveStringObject(rio *rdb, robj *obj);
ssize_t rdbSaveRawString(rio *rdb, unsigned char *s, size_t len);
void *rdbGenericLoadStringObject(rio *rdb, int flags, size_t *lenptr);
int rdbSaveBinaryDoubleValue(rio *rdb, double val);
int rdbLoadBinaryDoubleValue(rio *rdb, double *val);
#endif
......@@ -171,7 +171,7 @@ static int processTime(int type) {
return 0;
}
static uint32_t loadLength(int *isencoded) {
static uint64_t loadLength(int *isencoded) {
unsigned char buf[2];
uint32_t len;
int type;
......@@ -190,10 +190,16 @@ static uint32_t loadLength(int *isencoded) {
/* Read a 14 bit len */
if (!readBytes(buf+1,1)) return RDB_LENERR;
return ((buf[0] & 0x3F) << 8) | buf[1];
} else {
} else if (buf[0] == RDB_32BITLEN) {
/* Read a 32 bit len */
if (!readBytes(&len, 4)) return RDB_LENERR;
return (unsigned int)ntohl(len);
return ntohl(len);
} else if (buf[0] == RDB_64BITLEN) {
/* Read a 64 bit len */
if (!readBytes(&len, 8)) return RDB_LENERR;
return ntohu64(len);
} else {
return RDB_LENERR;
}
}
......@@ -230,7 +236,7 @@ static char *loadIntegerObject(int enctype) {
}
static char* loadLzfStringObject() {
unsigned int slen, clen;
uint64_t slen, clen;
char *c, *s;
if ((clen = loadLength(NULL)) == RDB_LENERR) return NULL;
......@@ -254,9 +260,9 @@ static char* loadLzfStringObject() {
/* returns NULL when not processable, char* when valid */
static char* loadStringObject() {
uint32_t offset = CURR_OFFSET;
uint64_t offset = CURR_OFFSET;
uint64_t len;
int isencoded;
uint32_t len;
len = loadLength(&isencoded);
if (isencoded) {
......@@ -269,7 +275,8 @@ static char* loadStringObject() {
return loadLzfStringObject();
default:
/* unknown encoding */
SHIFT_ERROR(offset, "Unknown string encoding (0x%02x)", len);
SHIFT_ERROR(offset, "Unknown string encoding (0x%02llx)",
(unsigned long long) len);
return NULL;
}
}
......@@ -344,8 +351,8 @@ static int processDoubleValue(double** store) {
}
static int loadPair(entry *e) {
uint32_t offset = CURR_OFFSET;
uint32_t i;
uint64_t offset = CURR_OFFSET;
uint64_t i;
/* read key first */
char *key;
......@@ -356,7 +363,7 @@ static int loadPair(entry *e) {
return 0;
}
uint32_t length = 0;
uint64_t length = 0;
if (e->type == RDB_TYPE_LIST ||
e->type == RDB_TYPE_SET ||
e->type == RDB_TYPE_ZSET ||
......@@ -384,7 +391,8 @@ static int loadPair(entry *e) {
for (i = 0; i < length; i++) {
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element at index %d (length: %d)", i, length);
SHIFT_ERROR(offset, "Error reading element at index %llu (length: %llu)",
(unsigned long long) i, (unsigned long long) length);
return 0;
}
}
......@@ -393,12 +401,14 @@ static int loadPair(entry *e) {
for (i = 0; i < length; i++) {
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element key at index %d (length: %d)", i, length);
SHIFT_ERROR(offset, "Error reading element key at index %llu (length: %llu)",
(unsigned long long) i, (unsigned long long) length);
return 0;
}
offset = CURR_OFFSET;
if (!processDoubleValue(NULL)) {
SHIFT_ERROR(offset, "Error reading element value at index %d (length: %d)", i, length);
SHIFT_ERROR(offset, "Error reading element value at index %llu (length: %llu)",
(unsigned long long) i, (unsigned long long) length);
return 0;
}
}
......@@ -407,12 +417,14 @@ static int loadPair(entry *e) {
for (i = 0; i < length; i++) {
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element key at index %d (length: %d)", i, length);
SHIFT_ERROR(offset, "Error reading element key at index %llu (length: %llu)",
(unsigned long long) i, (unsigned long long) length);
return 0;
}
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element value at index %d (length: %d)", i, length);
SHIFT_ERROR(offset, "Error reading element value at index %llu (length: %llu)",
(unsigned long long) i, (unsigned long long) length);
return 0;
}
}
......@@ -428,7 +440,7 @@ static int loadPair(entry *e) {
static entry loadEntry() {
entry e = { NULL, -1, 0 };
uint32_t length, offset[4];
uint64_t length, offset[4];
/* reset error container */
errors.level = 0;
......@@ -445,7 +457,8 @@ static entry loadEntry() {
return e;
}
if (length > 63) {
SHIFT_ERROR(offset[1], "Database number out of range (%d)", length);
SHIFT_ERROR(offset[1], "Database number out of range (%llu)",
(unsigned long long) length);
return e;
}
} else if (e.type == RDB_OPCODE_EOF) {
......
......@@ -284,7 +284,6 @@ static void cliIntegrateHelp(void) {
break;
}
if (i != helpEntriesLen) continue;
printf("%s\n", cmdname);
helpEntriesLen++;
helpEntries = zrealloc(helpEntries,sizeof(helpEntry)*helpEntriesLen);
......@@ -314,8 +313,6 @@ static void cliIntegrateHelp(void) {
new->org = ch;
}
freeReplyObject(reply);
printf("%s\n", helpEntries[80].full);
}
/* Output command help to stdout. */
......@@ -2594,13 +2591,16 @@ int main(int argc, char **argv) {
else
config.output = OUTPUT_STANDARD;
config.mb_delim = sdsnew("\n");
cliInitHelp();
cliIntegrateHelp();
firstarg = parseOptions(argc,argv);
argc -= firstarg;
argv += firstarg;
/* Initialize the help and, if possible, use the COMMAND command in order
* to retrieve missing entries. */
cliInitHelp();
cliIntegrateHelp();
/* Latency mode */
if (config.latency_mode) {
if (cliConnect(0) == REDIS_ERR) exit(1);
......
......@@ -28,6 +28,7 @@
#define REDISMODULE_KEYTYPE_HASH 3
#define REDISMODULE_KEYTYPE_SET 4
#define REDISMODULE_KEYTYPE_ZSET 5
#define REDISMODULE_KEYTYPE_MODULE 6
/* Reply types. */
#define REDISMODULE_REPLY_UNKNOWN -1
......@@ -78,14 +79,29 @@ typedef struct RedisModuleCtx RedisModuleCtx;
typedef struct RedisModuleKey RedisModuleKey;
typedef struct RedisModuleString RedisModuleString;
typedef struct RedisModuleCallReply RedisModuleCallReply;
typedef struct RedisModuleIO RedisModuleIO;
typedef struct RedisModuleType RedisModuleType;
typedef struct RedisModuleDigest RedisModuleDigest;
typedef int (*RedisModuleCmdFunc) (RedisModuleCtx *ctx, RedisModuleString **argv, int argc);
typedef void *(*RedisModuleTypeLoadFunc)(RedisModuleIO *rdb, int encver);
typedef void (*RedisModuleTypeSaveFunc)(RedisModuleIO *rdb, void *value);
typedef void (*RedisModuleTypeRewriteFunc)(RedisModuleIO *aof, RedisModuleString *key, void *value);
typedef void (*RedisModuleTypeDigestFunc)(RedisModuleDigest *digest, void *value);
typedef void (*RedisModuleTypeFreeFunc)(void *value);
#define REDISMODULE_GET_API(name) \
RedisModule_GetApi("RedisModule_" #name, ((void **)&RedisModule_ ## name))
#define REDISMODULE_API_FUNC(x) (*x)
void *REDISMODULE_API_FUNC(RedisModule_Alloc)(size_t bytes);
void *REDISMODULE_API_FUNC(RedisModule_Realloc)(void *ptr, size_t bytes);
void REDISMODULE_API_FUNC(RedisModule_Free)(void *ptr);
void REDISMODULE_API_FUNC(RedisModule_Calloc)(size_t nmemb, size_t size);
char *REDISMODULE_API_FUNC(RedisModule_Strdup)(const char *str);
int REDISMODULE_API_FUNC(RedisModule_GetApi)(const char *, void *);
int REDISMODULE_API_FUNC(RedisModule_CreateCommand)(RedisModuleCtx *ctx, const char *name, RedisModuleCmdFunc cmdfunc, const char *strflags, int firstkey, int lastkey, int keystep);
int REDISMODULE_API_FUNC(RedisModule_SetModuleAttribs)(RedisModuleCtx *ctx, const char *name, int ver, int apiver);
......@@ -108,8 +124,9 @@ size_t REDISMODULE_API_FUNC(RedisModule_CallReplyLength)(RedisModuleCallReply *r
RedisModuleCallReply *REDISMODULE_API_FUNC(RedisModule_CallReplyArrayElement)(RedisModuleCallReply *reply, size_t idx);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateString)(RedisModuleCtx *ctx, const char *ptr, size_t len);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateStringFromLongLong)(RedisModuleCtx *ctx, long long ll);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CreateStringFromString)(RedisModuleCtx *ctx, const RedisModuleString *str);
void REDISMODULE_API_FUNC(RedisModule_FreeString)(RedisModuleCtx *ctx, RedisModuleString *str);
const char *REDISMODULE_API_FUNC(RedisModule_StringPtrLen)(RedisModuleString *str, size_t *len);
const char *REDISMODULE_API_FUNC(RedisModule_StringPtrLen)(const RedisModuleString *str, size_t *len);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithError)(RedisModuleCtx *ctx, const char *err);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithSimpleString)(RedisModuleCtx *ctx, const char *msg);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithArray)(RedisModuleCtx *ctx, long len);
......@@ -119,8 +136,8 @@ int REDISMODULE_API_FUNC(RedisModule_ReplyWithString)(RedisModuleCtx *ctx, Redis
int REDISMODULE_API_FUNC(RedisModule_ReplyWithNull)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithDouble)(RedisModuleCtx *ctx, double d);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithCallReply)(RedisModuleCtx *ctx, RedisModuleCallReply *reply);
int REDISMODULE_API_FUNC(RedisModule_StringToLongLong)(RedisModuleString *str, long long *ll);
int REDISMODULE_API_FUNC(RedisModule_StringToDouble)(RedisModuleString *str, double *d);
int REDISMODULE_API_FUNC(RedisModule_StringToLongLong)(const RedisModuleString *str, long long *ll);
int REDISMODULE_API_FUNC(RedisModule_StringToDouble)(const RedisModuleString *str, double *d);
void REDISMODULE_API_FUNC(RedisModule_AutoMemory)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_Replicate)(RedisModuleCtx *ctx, const char *cmdname, const char *fmt, ...);
int REDISMODULE_API_FUNC(RedisModule_ReplicateVerbatim)(RedisModuleCtx *ctx);
......@@ -150,11 +167,34 @@ int REDISMODULE_API_FUNC(RedisModule_HashGet)(RedisModuleKey *key, int flags, ..
int REDISMODULE_API_FUNC(RedisModule_IsKeysPositionRequest)(RedisModuleCtx *ctx);
void REDISMODULE_API_FUNC(RedisModule_KeyAtPos)(RedisModuleCtx *ctx, int pos);
unsigned long long REDISMODULE_API_FUNC(RedisModule_GetClientId)(RedisModuleCtx *ctx);
void *REDISMODULE_API_FUNC(RedisModule_PoolAlloc)(RedisModuleCtx *ctx, size_t bytes);
RedisModuleType *REDISMODULE_API_FUNC(RedisModule_CreateDataType)(RedisModuleCtx *ctx, const char *name, int encver, RedisModuleTypeLoadFunc rdb_load, RedisModuleTypeSaveFunc rdb_save, RedisModuleTypeRewriteFunc aof_rewrite, RedisModuleTypeDigestFunc digest, RedisModuleTypeFreeFunc free);
int REDISMODULE_API_FUNC(RedisModule_ModuleTypeSetValue)(RedisModuleKey *key, RedisModuleType *mt, void *value);
RedisModuleType *REDISMODULE_API_FUNC(RedisModule_ModuleTypeGetType)(RedisModuleKey *key);
void *REDISMODULE_API_FUNC(RedisModule_ModuleTypeGetValue)(RedisModuleKey *key);
void REDISMODULE_API_FUNC(RedisModule_SaveUnsigned)(RedisModuleIO *io, uint64_t value);
uint64_t REDISMODULE_API_FUNC(RedisModule_LoadUnsigned)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_SaveSigned)(RedisModuleIO *io, int64_t value);
int64_t REDISMODULE_API_FUNC(RedisModule_LoadSigned)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_EmitAOF)(RedisModuleIO *io, const char *cmdname, const char *fmt, ...);
void REDISMODULE_API_FUNC(RedisModule_SaveString)(RedisModuleIO *io, RedisModuleString *s);
void REDISMODULE_API_FUNC(RedisModule_SaveStringBuffer)(RedisModuleIO *io, const char *str, size_t len);
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_LoadString)(RedisModuleIO *io);
char *REDISMODULE_API_FUNC(RedisModule_LoadStringBuffer)(RedisModuleIO *io, size_t *lenptr);
void REDISMODULE_API_FUNC(RedisModule_SaveDouble)(RedisModuleIO *io, double value);
double REDISMODULE_API_FUNC(RedisModule_LoadDouble)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_Log)(RedisModuleCtx *ctx, const char *level, const char *fmt, ...);
/* This is included inline inside each Redis module. */
static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int apiver) __attribute__((unused));
static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int apiver) {
void *getapifuncptr = ((void**)ctx)[0];
RedisModule_GetApi = (int (*)(const char *, void *)) (unsigned long)getapifuncptr;
REDISMODULE_GET_API(Alloc);
REDISMODULE_GET_API(Calloc);
REDISMODULE_GET_API(Free);
REDISMODULE_GET_API(Realloc);
REDISMODULE_GET_API(Strdup);
REDISMODULE_GET_API(CreateCommand);
REDISMODULE_GET_API(SetModuleAttribs);
REDISMODULE_GET_API(WrongArity);
......@@ -190,6 +230,7 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(CreateStringFromCallReply);
REDISMODULE_GET_API(CreateString);
REDISMODULE_GET_API(CreateStringFromLongLong);
REDISMODULE_GET_API(CreateStringFromString);
REDISMODULE_GET_API(FreeString);
REDISMODULE_GET_API(StringPtrLen);
REDISMODULE_GET_API(AutoMemory);
......@@ -219,6 +260,23 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(IsKeysPositionRequest);
REDISMODULE_GET_API(KeyAtPos);
REDISMODULE_GET_API(GetClientId);
REDISMODULE_GET_API(PoolAlloc);
REDISMODULE_GET_API(CreateDataType);
REDISMODULE_GET_API(ModuleTypeSetValue);
REDISMODULE_GET_API(ModuleTypeGetType);
REDISMODULE_GET_API(ModuleTypeGetValue);
REDISMODULE_GET_API(SaveUnsigned);
REDISMODULE_GET_API(LoadUnsigned);
REDISMODULE_GET_API(SaveSigned);
REDISMODULE_GET_API(LoadSigned);
REDISMODULE_GET_API(SaveString);
REDISMODULE_GET_API(SaveStringBuffer);
REDISMODULE_GET_API(LoadString);
REDISMODULE_GET_API(LoadStringBuffer);
REDISMODULE_GET_API(SaveDouble);
REDISMODULE_GET_API(LoadDouble);
REDISMODULE_GET_API(EmitAOF);
REDISMODULE_GET_API(Log);
RedisModule_SetModuleAttribs(ctx,name,ver,apiver);
return REDISMODULE_OK;
......
......@@ -135,6 +135,9 @@ size_t rioWriteBulkString(rio *r, const char *buf, size_t len);
size_t rioWriteBulkLongLong(rio *r, long long l);
size_t rioWriteBulkDouble(rio *r, double d);
struct redisObject;
int rioWriteBulkObject(rio *r, struct redisObject *obj);
void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len);
void rioSetAutoSync(rio *r, off_t bytes);
......
......@@ -55,13 +55,13 @@ static inline int sdsHdrSize(char type) {
}
static inline char sdsReqType(size_t string_size) {
if (string_size < 32)
if (string_size < 1<<5)
return SDS_TYPE_5;
if (string_size < 0xff)
if (string_size < 1<<8)
return SDS_TYPE_8;
if (string_size < 0xffff)
if (string_size < 1<<16)
return SDS_TYPE_16;
if (string_size < 0xffffffff)
if (string_size < 1ll<<32)
return SDS_TYPE_32;
return SDS_TYPE_64;
}
......
......@@ -1910,6 +1910,7 @@ void sentinelReconnectInstance(sentinelRedisInstance *ri) {
link->cc->errstr);
instanceLinkCloseConnection(link,link->cc);
} else {
link->pending_commands = 0;
link->cc_conn_time = mstime();
link->cc->data = link;
redisAeAttach(server.el,link->cc);
......@@ -3874,11 +3875,11 @@ int compareSlavesForPromotion(const void *a, const void *b) {
return (*sa)->slave_priority - (*sb)->slave_priority;
/* If priority is the same, select the slave with greater replication
* offset (processed more data frmo the master). */
* offset (processed more data from the master). */
if ((*sa)->slave_repl_offset > (*sb)->slave_repl_offset) {
return -1; /* a < b */
} else if ((*sa)->slave_repl_offset < (*sb)->slave_repl_offset) {
return 1; /* b > a */
return 1; /* a > b */
}
/* If the replication offset is the same select the slave with that has
......@@ -3996,7 +3997,7 @@ void sentinelFailoverSendSlaveOfNoOne(sentinelRedisInstance *ri) {
/* We can't send the command to the promoted slave if it is now
* disconnected. Retry again and again with this state until the timeout
* is reached, then abort the failover. */
if (ri->link->disconnected) {
if (ri->promoted_slave->link->disconnected) {
if (mstime() - ri->failover_state_change_time > ri->failover_timeout) {
sentinelEvent(LL_WARNING,"-failover-abort-slave-timeout",ri,"%@");
sentinelAbortFailover(ri);
......
......@@ -145,8 +145,8 @@ struct redisCommand redisCommandTable[] = {
{"mget",mgetCommand,-2,"r",0,NULL,1,-1,1,0,0},
{"rpush",rpushCommand,-3,"wmF",0,NULL,1,1,1,0,0},
{"lpush",lpushCommand,-3,"wmF",0,NULL,1,1,1,0,0},
{"rpushx",rpushxCommand,3,"wmF",0,NULL,1,1,1,0,0},
{"lpushx",lpushxCommand,3,"wmF",0,NULL,1,1,1,0,0},
{"rpushx",rpushxCommand,-3,"wmF",0,NULL,1,1,1,0,0},
{"lpushx",lpushxCommand,-3,"wmF",0,NULL,1,1,1,0,0},
{"linsert",linsertCommand,5,"wm",0,NULL,1,1,1,0,0},
{"rpop",rpopCommand,2,"wF",0,NULL,1,1,1,0,0},
{"lpop",lpopCommand,2,"wF",0,NULL,1,1,1,0,0},
......@@ -165,7 +165,7 @@ struct redisCommand redisCommandTable[] = {
{"smove",smoveCommand,4,"wF",0,NULL,1,2,1,0,0},
{"sismember",sismemberCommand,3,"rF",0,NULL,1,1,1,0,0},
{"scard",scardCommand,2,"rF",0,NULL,1,1,1,0,0},
{"spop",spopCommand,-2,"wRsF",0,NULL,1,1,1,0,0},
{"spop",spopCommand,-2,"wRF",0,NULL,1,1,1,0,0},
{"srandmember",srandmemberCommand,-2,"rR",0,NULL,1,1,1,0,0},
{"sinter",sinterCommand,-2,"rS",0,NULL,1,-1,1,0,0},
{"sinterstore",sinterstoreCommand,-3,"wm",0,NULL,1,-1,1,0,0},
......@@ -250,6 +250,7 @@ struct redisCommand redisCommandTable[] = {
{"info",infoCommand,-1,"lt",0,NULL,0,0,0,0,0},
{"monitor",monitorCommand,1,"as",0,NULL,0,0,0,0,0},
{"ttl",ttlCommand,2,"rF",0,NULL,1,1,1,0,0},
{"touch",touchCommand,-2,"rF",0,NULL,1,1,1,0,0},
{"pttl",pttlCommand,2,"rF",0,NULL,1,1,1,0,0},
{"persist",persistCommand,2,"wF",0,NULL,1,1,1,0,0},
{"slaveof",slaveofCommand,3,"ast",0,NULL,0,0,0,0,0},
......@@ -688,7 +689,7 @@ int htNeedsResize(dict *dict) {
size = dictSlots(dict);
used = dictSize(dict);
return (size && used && size > DICT_HT_INITIAL_SIZE &&
return (size > DICT_HT_INITIAL_SIZE &&
(used*100/size < HASHTABLE_MIN_FILL));
}
......@@ -1959,8 +1960,9 @@ void initServer(void) {
server.repl_good_slaves_count = 0;
updateCachedTime();
/* Create out timers, that's our main way to process background
* operations. */
/* Create the timer callback, this is our way to process many background
* operations incrementally, like clients timeout, eviction of unaccessed
* expired keys and so forth. */
if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) {
serverPanic("Can't create event loop timers.");
exit(1);
......@@ -3823,7 +3825,7 @@ void setupSignalHandlers(void) {
void memtest(size_t megabytes, int passes);
/* Returns 1 if there is --sentinel among the arguments or if
* argv[0] is exactly "redis-sentinel". */
* argv[0] contains "redis-sentinel". */
int checkForSentinelMode(int argc, char **argv) {
int j;
......
......@@ -33,6 +33,7 @@
#include "fmacros.h"
#include "config.h"
#include "solarisfixes.h"
#include "rio.h"
#include <stdio.h>
#include <stdlib.h>
......@@ -164,7 +165,7 @@ typedef long long mstime_t; /* millisecond time type. */
#define PROTO_REPLY_CHUNK_BYTES (16*1024) /* 16k output buffer */
#define PROTO_INLINE_MAX_SIZE (1024*64) /* Max size of inline reads */
#define PROTO_MBULK_BIG_ARG (1024*32)
#define LONG_STR_SIZE 21 /* Bytes needed for long -> str */
#define LONG_STR_SIZE 21 /* Bytes needed for long -> str + '\0' */
#define AOF_AUTOSYNC_BYTES (1024*1024*32) /* fdatasync every 32MB */
/* When configuring the server eventloop, we setup it so that the total number
......@@ -195,33 +196,6 @@ typedef long long mstime_t; /* millisecond time type. */
#define CMD_MODULE_GETKEYS (1<<14) /* Use the modules getkeys interface. */
#define CMD_MODULE_NO_CLUSTER (1<<15) /* Deny on Redis Cluster. */
/* Defines related to the dump file format. To store 32 bits lengths for short
* keys requires a lot of space, so we check the most significant 2 bits of
* the first byte to interpreter the length:
*
* 00|000000 => if the two MSB are 00 the len is the 6 bits of this byte
* 01|000000 00000000 => 01, the len is 14 byes, 6 bits + 8 bits of next byte
* 10|000000 [32 bit integer] => if it's 10, a full 32 bit len will follow
* 11|000000 this means: specially encoded object will follow. The six bits
* number specify the kind of object that follows.
* See the RDB_ENC_* defines.
*
* Lengths up to 63 are stored using a single byte, most DB keys, and may
* values, will fit inside. */
#define RDB_6BITLEN 0
#define RDB_14BITLEN 1
#define RDB_32BITLEN 2
#define RDB_ENCVAL 3
#define RDB_LENERR UINT_MAX
/* When a length of a string object stored on disk has the first two bits
* set, the remaining two bits specify a special encoding for the object
* accordingly to the following defines: */
#define RDB_ENC_INT8 0 /* 8 bit signed integer */
#define RDB_ENC_INT16 1 /* 16 bit signed integer */
#define RDB_ENC_INT32 2 /* 32 bit signed integer */
#define RDB_ENC_LZF 3 /* string compressed with FASTLZ */
/* AOF states */
#define AOF_OFF 0 /* AOF is off */
#define AOF_ON 1 /* AOF is on */
......@@ -448,6 +422,90 @@ typedef long long mstime_t; /* millisecond time type. */
#define OBJ_ZSET 3
#define OBJ_HASH 4
/* The "module" object type is a special one that signals that the object
* is one directly managed by a Redis module. In this case the value points
* to a moduleValue struct, which contains the object value (which is only
* handled by the module itself) and the RedisModuleType struct which lists
* function pointers in order to serialize, deserialize, AOF-rewrite and
* free the object.
*
* Inside the RDB file, module types are encoded as OBJ_MODULE followed
* by a 64 bit module type ID, which has a 54 bits module-specific signature
* in order to dispatch the loading to the right module, plus a 10 bits
* encoding version. */
#define OBJ_MODULE 5
/* Extract encver / signature from a module type ID. */
#define REDISMODULE_TYPE_ENCVER_BITS 10
#define REDISMODULE_TYPE_ENCVER_MASK ((1<<REDISMODULE_TYPE_ENCVER_BITS)-1)
#define REDISMODULE_TYPE_ENCVER(id) (id & REDISMODULE_TYPE_ENCVER_MASK)
#define REDISMODULE_TYPE_SIGN(id) ((id & ~((uint64_t)REDISMODULE_TYPE_ENCVER_MASK)) >>REDISMODULE_TYPE_ENCVER_BITS)
struct RedisModule;
struct RedisModuleIO;
struct RedisModuleDigest;
struct redisObject;
/* Each module type implementation should export a set of methods in order
* to serialize and deserialize the value in the RDB file, rewrite the AOF
* log, create the digest for "DEBUG DIGEST", and free the value when a key
* is deleted. */
typedef void *(*moduleTypeLoadFunc)(struct RedisModuleIO *io, int encver);
typedef void (*moduleTypeSaveFunc)(struct RedisModuleIO *io, void *value);
typedef void (*moduleTypeRewriteFunc)(struct RedisModuleIO *io, struct redisObject *key, void *value);
typedef void (*moduleTypeDigestFunc)(struct RedisModuleDigest *digest, void *value);
typedef void (*moduleTypeFreeFunc)(void *value);
/* The module type, which is referenced in each value of a given type, defines
* the methods and links to the module exporting the type. */
typedef struct RedisModuleType {
uint64_t id; /* Higher 54 bits of type ID + 10 lower bits of encoding ver. */
struct RedisModule *module;
moduleTypeLoadFunc rdb_load;
moduleTypeSaveFunc rdb_save;
moduleTypeRewriteFunc aof_rewrite;
moduleTypeDigestFunc digest;
moduleTypeFreeFunc free;
char name[10]; /* 9 bytes name + null term. Charset: A-Z a-z 0-9 _- */
} moduleType;
/* In Redis objects 'robj' structures of type OBJ_MODULE, the value pointer
* is set to the following structure, referencing the moduleType structure
* in order to work with the value, and at the same time providing a raw
* pointer to the value, as created by the module commands operating with
* the module type.
*
* So for example in order to free such a value, it is possible to use
* the following code:
*
* if (robj->type == OBJ_MODULE) {
* moduleValue *mt = robj->ptr;
* mt->type->free(mt->value);
* zfree(mt); // We need to release this in-the-middle struct as well.
* }
*/
typedef struct moduleValue {
moduleType *type;
void *value;
} moduleValue;
/* This is a wrapper for the 'rio' streams used inside rdb.c in Redis, so that
* the user does not have to take the total count of the written bytes nor
* to care about error conditions. */
typedef struct RedisModuleIO {
size_t bytes; /* Bytes read / written so far. */
rio *rio; /* Rio stream. */
moduleType *type; /* Module type doing the operation. */
int error; /* True if error condition happened. */
} RedisModuleIO;
#define moduleInitIOContext(iovar,mtype,rioptr) do { \
iovar.rio = rioptr; \
iovar.type = mtype; \
iovar.bytes = 0; \
iovar.error = 0; \
} while(0);
/* Objects encoding. Some kind of objects like Strings and Hashes can be
* internally represented in multiple ways. The 'encoding' field of the object
* is set to one of this fields for this object. */
......@@ -478,7 +536,7 @@ typedef struct redisObject {
/* Macro used to obtain the current LRU clock.
* If the current resolution is lower than the frequency we refresh the
* LRU clock (as it should be in production servers) we return the
* precomputed value, otherwise we need to resort to a function call. */
* precomputed value, otherwise we need to resort to a system call. */
#define LRU_CLOCK() ((1000/server.hz <= LRU_CLOCK_RESOLUTION) ? server.lruclock : getLRUClock())
/* Macro used to initialize a Redis object allocated on the stack.
......@@ -625,6 +683,12 @@ struct saveparam {
int changes;
};
struct moduleLoadQueueEntry {
sds path;
int argc;
robj **argv;
};
struct sharedObjectsStruct {
robj *crlf, *ok, *err, *emptybulk, *czero, *cone, *cnegone, *pong, *space,
*colon, *nullbulk, *nullmultibulk, *queued,
......@@ -984,8 +1048,8 @@ struct redisServer {
long long latency_monitor_threshold;
dict *latency_events;
/* Assert & bug reporting */
char *assert_failed;
char *assert_file;
const char *assert_failed;
const char *assert_file;
int assert_line;
int bug_report_start; /* True if bug report header was already logged. */
int watchdog_period; /* Software watchdog period in ms. 0 = off */
......@@ -1098,9 +1162,11 @@ extern dictType modulesDictType;
/* Modules */
void moduleInitModulesSystem(void);
int moduleLoad(const char *path);
int moduleLoad(const char *path, void **argv, int argc);
void moduleLoadFromQueue(void);
int *moduleGetCommandKeysViaAPI(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
moduleType *moduleTypeLookupModuleByID(uint64_t id);
void moduleTypeNameByID(char *name, uint64_t moduleid);
/* Utils */
long long ustime(void);
......@@ -1179,7 +1245,7 @@ void addReplyStatusFormat(client *c, const char *fmt, ...);
void listTypeTryConversion(robj *subject, robj *value);
void listTypePush(robj *subject, robj *value, int where);
robj *listTypePop(robj *subject, int where);
unsigned long listTypeLength(robj *subject);
unsigned long listTypeLength(const robj *subject);
listTypeIterator *listTypeInitIterator(robj *subject, long index, unsigned char direction);
void listTypeReleaseIterator(listTypeIterator *li);
int listTypeNext(listTypeIterator *li, listTypeEntry *entry);
......@@ -1219,7 +1285,7 @@ robj *createObject(int type, void *ptr);
robj *createStringObject(const char *ptr, size_t len);
robj *createRawStringObject(const char *ptr, size_t len);
robj *createEmbeddedStringObject(const char *ptr, size_t len);
robj *dupStringObject(robj *o);
robj *dupStringObject(const robj *o);
int isSdsRepresentableAsLongLong(sds s, long long *llval);
int isObjectRepresentableAsLongLong(robj *o, long long *llongval);
robj *tryObjectEncoding(robj *o);
......@@ -1234,11 +1300,12 @@ robj *createIntsetObject(void);
robj *createHashObject(void);
robj *createZsetObject(void);
robj *createZsetZiplistObject(void);
robj *createModuleObject(moduleType *mt, void *value);
int getLongFromObjectOrReply(client *c, robj *o, long *target, const char *msg);
int checkType(client *c, robj *o, int type);
int getLongLongFromObjectOrReply(client *c, robj *o, long long *target, const char *msg);
int getDoubleFromObjectOrReply(client *c, robj *o, double *target, const char *msg);
int getDoubleFromObject(robj *o, double *target);
int getDoubleFromObject(const robj *o, double *target);
int getLongLongFromObject(robj *o, long long *target);
int getLongDoubleFromObject(robj *o, long double *target);
int getLongDoubleFromObjectOrReply(client *c, robj *o, long double *target, const char *msg);
......@@ -1339,7 +1406,7 @@ void zzlNext(unsigned char *zl, unsigned char **eptr, unsigned char **sptr);
void zzlPrev(unsigned char *zl, unsigned char **eptr, unsigned char **sptr);
unsigned char *zzlFirstInRange(unsigned char *zl, zrangespec *range);
unsigned char *zzlLastInRange(unsigned char *zl, zrangespec *range);
unsigned int zsetLength(robj *zobj);
unsigned int zsetLength(const robj *zobj);
void zsetConvert(robj *zobj, int encoding);
void zsetConvertToZiplistIfNeeded(robj *zobj, size_t maxelelen);
int zsetScore(robj *zobj, sds member, double *score);
......@@ -1387,7 +1454,6 @@ void serverLogFromHandler(int level, const char *msg);
void usage(void);
void updateDictResizePolicy(void);
int htNeedsResize(dict *dict);
void oom(const char *msg);
void populateCommandTable(void);
void resetCommandTableStats(void);
void adjustOpenFilesLimit(void);
......@@ -1413,7 +1479,7 @@ int setTypeNext(setTypeIterator *si, sds *sdsele, int64_t *llele);
sds setTypeNextObject(setTypeIterator *si);
int setTypeRandomElement(robj *setobj, sds *sdsele, int64_t *llele);
unsigned long setTypeRandomElements(robj *set, unsigned long count, robj *aux_set);
unsigned long setTypeSize(robj *subject);
unsigned long setTypeSize(const robj *subject);
void setTypeConvert(robj *subject, int enc);
/* Hash data type */
......@@ -1426,7 +1492,7 @@ void hashTypeTryConversion(robj *subject, robj **argv, int start, int end);
void hashTypeTryObjectEncoding(robj *subject, robj **o1, robj **o2);
int hashTypeExists(robj *o, sds key);
int hashTypeDelete(robj *o, sds key);
unsigned long hashTypeLength(robj *o);
unsigned long hashTypeLength(const robj *o);
hashTypeIterator *hashTypeInitIterator(robj *subject);
void hashTypeReleaseIterator(hashTypeIterator *hi);
int hashTypeNext(hashTypeIterator *hi);
......@@ -1467,11 +1533,14 @@ void propagateExpire(redisDb *db, robj *key, int lazy);
int expireIfNeeded(redisDb *db, robj *key);
long long getExpire(redisDb *db, robj *key);
void setExpire(redisDb *db, robj *key, long long when);
robj *lookupKey(redisDb *db, robj *key);
robj *lookupKey(redisDb *db, robj *key, int flags);
robj *lookupKeyRead(redisDb *db, robj *key);
robj *lookupKeyWrite(redisDb *db, robj *key);
robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply);
robj *lookupKeyWriteOrReply(client *c, robj *key, robj *reply);
robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags);
#define LOOKUP_NONE 0
#define LOOKUP_NOTOUCH (1<<0)
void dbAdd(redisDb *db, robj *key, robj *val);
void dbOverwrite(redisDb *db, robj *key, robj *val);
void setKey(redisDb *db, robj *key, robj *val);
......@@ -1627,6 +1696,7 @@ void pexpireCommand(client *c);
void pexpireatCommand(client *c);
void getsetCommand(client *c);
void ttlCommand(client *c);
void touchCommand(client *c);
void pttlCommand(client *c);
void persistCommand(client *c);
void slaveofCommand(client *c);
......@@ -1729,11 +1799,11 @@ void *realloc(void *ptr, size_t size) __attribute__ ((deprecated));
#endif
/* Debugging stuff */
void _serverAssertWithInfo(client *c, robj *o, char *estr, char *file, int line);
void _serverAssert(char *estr, char *file, int line);
void _serverPanic(char *msg, char *file, int line);
void _serverAssertWithInfo(const client *c, const robj *o, const char *estr, const char *file, int line);
void _serverAssert(const char *estr, const char *file, int line);
void _serverPanic(const char *msg, const char *file, int line);
void bugReportStart(void);
void serverLogObjectDebugInfo(robj *o);
void serverLogObjectDebugInfo(const robj *o);
void sigsegvHandler(int sig, siginfo_t *info, void *secret);
sds genRedisInfoString(char *section);
void enableWatchdog(int period);
......
......@@ -308,13 +308,13 @@ int hashTypeDelete(robj *o, sds field) {
}
/* Return the number of elements in a hash. */
unsigned long hashTypeLength(robj *o) {
unsigned long hashTypeLength(const robj *o) {
unsigned long length = ULONG_MAX;
if (o->encoding == OBJ_ENCODING_ZIPLIST) {
length = ziplistLen(o->ptr) / 2;
} else if (o->encoding == OBJ_ENCODING_HT) {
length = dictSize((dict*)o->ptr);
length = dictSize((const dict*)o->ptr);
} else {
serverPanic("Unknown hash encoding");
}
......
......@@ -71,7 +71,7 @@ robj *listTypePop(robj *subject, int where) {
return value;
}
unsigned long listTypeLength(robj *subject) {
unsigned long listTypeLength(const robj *subject) {
if (subject->encoding == OBJ_ENCODING_QUICKLIST) {
return quicklistCount(subject->ptr);
} else {
......@@ -195,7 +195,7 @@ void listTypeConvert(robj *subject, int enc) {
*----------------------------------------------------------------------------*/
void pushGenericCommand(client *c, int where) {
int j, waiting = 0, pushed = 0;
int j, pushed = 0;
robj *lobj = lookupKeyWrite(c->db,c->argv[1]);
if (lobj && lobj->type != OBJ_LIST) {
......@@ -204,7 +204,6 @@ void pushGenericCommand(client *c, int where) {
}
for (j = 2; j < c->argc; j++) {
c->argv[j] = tryObjectEncoding(c->argv[j]);
if (!lobj) {
lobj = createQuicklistObject();
quicklistSetOptions(lobj->ptr, server.list_max_ziplist_size,
......@@ -214,7 +213,7 @@ void pushGenericCommand(client *c, int where) {
listTypePush(lobj,c->argv[j],where);
pushed++;
}
addReplyLongLong(c, waiting + (lobj ? listTypeLength(lobj) : 0));
addReplyLongLong(c, (lobj ? listTypeLength(lobj) : 0));
if (pushed) {
char *event = (where == LIST_HEAD) ? "lpush" : "rpush";
......@@ -232,68 +231,78 @@ void rpushCommand(client *c) {
pushGenericCommand(c,LIST_TAIL);
}
void pushxGenericCommand(client *c, robj *refval, robj *val, int where) {
void pushxGenericCommand(client *c, int where) {
int j, pushed = 0;
robj *subject;
listTypeIterator *iter;
listTypeEntry entry;
int inserted = 0;
if ((subject = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,subject,OBJ_LIST)) return;
if (refval != NULL) {
/* Seek refval from head to tail */
iter = listTypeInitIterator(subject,0,LIST_TAIL);
while (listTypeNext(iter,&entry)) {
if (listTypeEqual(&entry,refval)) {
listTypeInsert(&entry,val,where);
inserted = 1;
break;
}
}
listTypeReleaseIterator(iter);
for (j = 2; j < c->argc; j++) {
listTypePush(subject,c->argv[j],where);
pushed++;
}
if (inserted) {
signalModifiedKey(c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_LIST,"linsert",
c->argv[1],c->db->id);
server.dirty++;
} else {
/* Notify client of a failed insert */
addReply(c,shared.cnegone);
return;
}
} else {
char *event = (where == LIST_HEAD) ? "lpush" : "rpush";
addReplyLongLong(c,listTypeLength(subject));
listTypePush(subject,val,where);
if (pushed) {
char *event = (where == LIST_HEAD) ? "lpush" : "rpush";
signalModifiedKey(c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_LIST,event,c->argv[1],c->db->id);
server.dirty++;
}
addReplyLongLong(c,listTypeLength(subject));
server.dirty += pushed;
}
void lpushxCommand(client *c) {
c->argv[2] = tryObjectEncoding(c->argv[2]);
pushxGenericCommand(c,NULL,c->argv[2],LIST_HEAD);
pushxGenericCommand(c,LIST_HEAD);
}
void rpushxCommand(client *c) {
c->argv[2] = tryObjectEncoding(c->argv[2]);
pushxGenericCommand(c,NULL,c->argv[2],LIST_TAIL);
pushxGenericCommand(c,LIST_TAIL);
}
void linsertCommand(client *c) {
c->argv[4] = tryObjectEncoding(c->argv[4]);
int where;
robj *subject;
listTypeIterator *iter;
listTypeEntry entry;
int inserted = 0;
if (strcasecmp(c->argv[2]->ptr,"after") == 0) {
pushxGenericCommand(c,c->argv[3],c->argv[4],LIST_TAIL);
where = LIST_TAIL;
} else if (strcasecmp(c->argv[2]->ptr,"before") == 0) {
pushxGenericCommand(c,c->argv[3],c->argv[4],LIST_HEAD);
where = LIST_HEAD;
} else {
addReply(c,shared.syntaxerr);
return;
}
if ((subject = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,subject,OBJ_LIST)) return;
/* Seek pivot from head to tail */
iter = listTypeInitIterator(subject,0,LIST_TAIL);
while (listTypeNext(iter,&entry)) {
if (listTypeEqual(&entry,c->argv[3])) {
listTypeInsert(&entry,c->argv[4],where);
inserted = 1;
break;
}
}
listTypeReleaseIterator(iter);
if (inserted) {
signalModifiedKey(c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_LIST,"linsert",
c->argv[1],c->db->id);
server.dirty++;
} else {
/* Notify client of a failed insert */
addReply(c,shared.cnegone);
return;
}
addReplyLongLong(c,listTypeLength(subject));
}
void llenCommand(client *c) {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment