The utils
directory provides utilities to make using and developing projects
in HE simpler for everyone. The utility programs currently should be
considered at an alpha state and not completely stable in terms of the CLI or
all possible configurations.
Currently the utilities provided comprise:
- create-context
- encrypt
- decrypt
More utilities are expected to be released at a later date.
When developing HE programs the encoding is of utmost importance and usually are specific to the problem domain. We provide two simple example programs for encoding and decoding:
- example encoder (requires python >= 3.6)
- example decoder (requires python >= 3.6)
These example programs are used in our testing. We used Python, however, encoders and decoders programs can be implemented in whatever language the developer wishes.
Some example data can be generated by the example data generation script gen-data.py
which can be found in the tests
directory.
To compile the utilities, you must have HElib already installed on your system (see INSTALL.md in this distribution's root directory). The process is pure CMake. First, create a build directory and move into it. Second, run CMake,
cmake [-Dhelib_DIR=<directory-to-installed-HElib>/share/cmake/helib] <directory-to-utils>
then run make from the same directory with optional number of threads using the
-j
flag for example,
make [-j<number-of-threads>]
The create-context, encrypt and decrypt utility executables can be found in the
bin
directory. The example encoder and decoder are found in a separate
directory in <directory-to-utils>/coders
.
All tests for the utilities are written in bats (a test framework for bash) and require bats-core.
All utilities have a help method by passing the -h
flag, for example
./encrypt -h
Below is an example of a basic pipeline for the utilities. For simplicity we shall be using the BGV scheme although the utilities also provide CKKS functionality.
- Create a parameter file for the BGV scheme such as
p=2
m=3
r=1
c=2
Qbits=10
- Create the context using the previously generated parameter file
./bin/create-context <parameter-file> --info-file -o example
The flag -o
defines the prefix of the files generated by this script. If not
specified then the script will generate a prefix using the name of the
parameter file by default concatenated with a generated number. This file
prefix will be printed to the command line.
The flag --scheme <default=BGV|CKKS>
will generate the context for BGV or CKKS scheme.
The script will always generate two files (example.pk
and example.sk
. They
both hold the serialization of the context and the public key and secret key
respectively.
The flag --info-file
will redirect the information describing the algebra to
the file example.info
. By default this information is sent to the standard
output however generating the info file is useful for passing to the example
encode decode utilities.
The flag --bootstrap <default=NONE|THIN|THICK>
can be used enable the bootstrapping with THIN or THICK bootstrapping options, this will generate the required bootstrappable context with key-switching matrices, frobenius matrices and bootstrapping keys, which are added to the *.pk
and *.sk
files respectivey. If this flag is enabled while using the BGV scheme, it needs three additional parameters (mvec, gens, ords) which are formatted as space-separated list added to the parameter file.
- Encode some data
../coders/encode.py <data-file> example.info <scheme> > example.encoded
This script reads in data from <data-file>
where the first line represents
the number of lines to read in and each subsequent line contains the
coefficients of each slot of the plaintext.
The encoder reads the algebra from the info file example.info
to encode the
data in the correct format.
The encoder must also be told which scheme is being used, currently scheme can
be either BGV
or CKKS
.
The example encoder outputs the encoded data as plaintexts to the command line
by default which is then redirected to example.encoded
.
The output of the encoder is a collection of plaintexts and must be formatted as follows:
The first line contains the number of plaintexts outputted by the encoder. Each subsequent line contains a single plaintext.
Each plaintext has to be formatted as a comma-separated list surrounded by '[' and ']'. Each element of the list represents the slot of the plaintext formatted as a comma-separated list surrounded by '[' and ']'. Each element in the nested list represents the coefficient of the polynomial in the slot. Note, for the CKKS scheme, there are always a maximum of two elements in a slot of a plaintext which represent the real and imaginary values respectively.
An example output of encoding two plaintexts with three slots, each containing two coefficients is shown below
2
[[0, 1], [2, 3], [4, 5]]
[[6, 7], [8, 9], [10, 11]]
Note that the encoder reads in each line and encodes the values into a slot until the maximum number of slots is reached. When this happens the encoder begins encoding another plaintext until all data values are read in. If the maximum number of slots is not reached upon reaching the end of the data file, then the remaining slots of the plaintext are padded with slots containing 0.
- Encrypt the data
./bin/encrypt example.pk example.encoded -o example.ctxt
The encrypt script reads in a public key file example.pk
and some encoded
data example.encoded
.
Optionally, the use of -o
enables the user to specify the filename containing
the encrypted output. By default the script generates a file using the prefix
of the plaintext file and appending the extension .ctxt
.
- Decrypt the data
./bin/decrypt example.sk example.ctxt -o example.decrypted
The decrypt script reads in a secret key file example.sk
and some encrypted
data example.ctxt
.
Optionally, the use of -o
enables the user to specify the filename containing
the decrypted output. By default the script will print the decrypted plaintext
to the standard output.
- Decode the data
../coders/encode.py example.decrypted example.info <scheme>
The decoder reads some encoded data example.decrypted
and an info file
example.info
to ensure the data is decoded in the correct format.
The decoder must be told which scheme is being used, currently the scheme can
be either BGV
or CKKS
.
The input of decoder must be formatted in the same way as the output of the encoder as described above in step 3.
The example decoder outputs the decoded data to the standard output by default.
All tests for the utilities are written in bats (a test framework for bash) and require bats-core.
Note that the tests require that the utilities have been successfully compiled
in the build
directory and available in build/bin
. To run the tests, one
can simply execute the scripts from within the utils/tests
directory.
To run all tests type the command below. Optionally, the -j
flag can use
threads to parallelize the tests. Note, this requires GNU parallel, see the
bats documentation for more
information.
bats . [-j <number-of-threads>]
To run a specific test file.
bats <testfile> [-j <number-of-threads>]
or
./<testfile> [-j <number-of-threads>]
To run a specific test by name add the -f
flag e.g.
bats . -f <testname> [-j <number-of-threads>]
The <testname>
can be a substring of the test name and will match all valid
matches.
For debugging, it is sometimes useful to view the artifacts generated by the
tests. For this, set the environment variable DEBUG
to true
or 1
such as,
DEBUG=1 bats . [-j <number-of-threads>]