Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a code generator to to automatically create all redis commands scaffolding, arguments parsing and commands callbacks #153

Closed
danielealbano opened this issue Jul 10, 2022 · 0 comments · Fixed by #170
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@danielealbano
Copy link
Owner

danielealbano commented Jul 10, 2022

Implementing Redis commands currently require writing a lot of scaffolding code because the module doesn't have a knowledge of the type of data it can receive, only if they can be long and therefore be processed as stream of data or if they can be short enough to be read entirely in one go from the buffer.

The redis repository contains a number of json files, one per command, with the specs of the commands themselves (e.g. the command name, the command group, the ACLs required, the parameters it takes, the specifics for the key, etc.).

It's necessary to implement a command generator that will:

  • automatically generate the necessary data structs for the commands
  • automatically generate all the scaffolding code for these commands to handle keys, values, tokens and tokens parameters, filling as needed the data struct for the command
  • automatically take care of handling common errors (e.g. key too long, unsupported parameter, missing parameter for a specified token, etc.)

Link to the json files
https://github.com/redis/redis/tree/unstable/src/commands

It's necessary to analize the json structure before investigating how to build the generator

@danielealbano danielealbano added the enhancement New feature or request label Jul 10, 2022
@danielealbano danielealbano added this to the v0.2 milestone Jul 10, 2022
@danielealbano danielealbano self-assigned this Jul 10, 2022
@danielealbano danielealbano changed the title Implement a generator to implement a state machine to parse all the supported redis commands and to generate all the scaffolding for the redis commands Implement a code generator to to automatically create all redis commands scaffolding, arguments parsing and commands callbacks Jul 24, 2022
danielealbano added a commit that referenced this issue Aug 15, 2022
…ommands scaffolding arguments parsing and commands callbacks (#170)

This PR implements a new number of components with the goal to streamline the development of the Redis module and. at the same time, improve the compatibility and performances.
There are 3 new major components

A python script able to generate from a JSON file a number of headers and c files at runtime, the C files are compiled into a static library which gets injected onto the cachegrand-internal target, the headers are included as needed.
An automated parser able to read the data generated by the scaffolding generator and, in particular the list of arguments, their type and their tokens, and figure out if what is being received is legit and if yes figure out which fields have to be filled out of the command context (e.g., a sort of reflection but almost without overload as the most of the code is pre-generated)
A new hashtable single producer single consumer used to identify the commands requested by the user and to figure out the tokens of the arguments passed in the commands

This PR closes issue #153
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
1 participant