HEtest

  1. Introduction
  2. Running Lincoln’s pre-built binaries and circuits
  3. Making circuits
  4. Building the test harness and baseline

Download: HEtest.tgz

1. Introduction

In order to evaluate homomorphic encryption schemes for the IARPA SPAR project,
MIT Lincoln Laboratory has built the following:

  1. A test harness capable of executing the server and client executables for a homomorphic encryption scheme. The inputs to the test harness are the paths to: the server executable, the client executable, and the test script. The harness spawns the server and client, and then it iteratively feeds them key generation, circuit ingestion, encryption, homomorphic evaluation, and decryption requests. The key generation, encryption, and decryption requests are sent to the client, whereas the circuit ingestion and homomorphic evaluation requests are sent to the server.  The order and contents of these requests are spelled out by a test script.

  2. A circuit generator capable of generating circuits randomly from the families supported by IBM and Stealth, respectively.

  3. A baseline that parses and evaluates the circuits described in (2), but without homomorphic encryption. In fact, the baseline is able to play the role of the server in all five protocols described in (1).

  4. A dummy client that is able to respond to requests made to it by the test harness. Its responses are meaningless; for instance, it always responds to "key" messages with "0" rather than providing a legitimate public key. The purpose of the dummy client is to play the role of the client alongside the baseline described in (3), since the test harness requires both a client and a server to run.

Note that all of these tools can either be used directly or augmented in a reasonably straightforward way to support testing for new homomorphic encryption schemes that were not part of the SPAR project.

There are two ways to utilize Lincoln's software: either use our pre-built binaries, or compile the code yourself. The latter option will give you more flexibility, particularly if you wish to change some feature of our code to support your own testing.

2. Running Lincoln’s pre-built binaries and circuits

We have provided binaries for the test harness, baseline, and client. They are located in the /bin directory. Additionally, we have provided text files containing randomly-generated circuits from IBM and Stealth's supported circuit families.

To run the baseline evaluator of the IBM circuits, you can execute the following command:

./test-harness -s ibm-baseline-main -c dummy-client -t ../trunk/spar_python/circuit_generation/ibm/01_one_of_each_small/K=128/test.txt

The Stealth circuit families can be run similarly.

The first time you use the program, we strongly suggest that you run

./test-harness -h

and read the help menu that is presented to you.

This distribution includes text files that represent several circuits, their inputs, and their corresponding results. The circuit files for IBM and Stealth can be found in the subdirectories of these directories:

trunk/spar_python/circuit_generation/ibm/01_one_of_each_small/
trunk/spar_python/circuit_generation/ibm/02_one_of_each_medium/
trunk/spar_python/circuit_generation/ibm/04_single_gate_type/
trunk/spar_python/circuit_generation/stealth/01_one_of_each_small/
trunk/spar_python/circuit_generation/stealth/02_one_of_each_medium/
trunk/spar_python/circuit_generation/stealth/04_single_gate_type/
trunk/spar_python/circuit_generation/stealth/09_more_small/

Within these directories, simply navigate to a sub-directory until you see a test.txt file. This is the file that the test harness accepts with the -t flag.  The directory structure will tell you the type of circuit being tested.

3. Making circuits

As you may have noticed in the prior step, there are many more subdirectories of circuit_generation that do not currently have a test.txt file. One example of this is:

trunk/spar_python/circuit_generation/ibm/06_large

In order to keep this code distribution small in size, we have not included many of the larger circuit files. Instead, we've included the required code to build them yourself. In order to build new circuits, you can run our circuit_generator using the following commands:

cd trunk/spar_python/circuit_generation/ibm
./generate_all.sh

The Stealth circuits can be generated similarly in circuit_generation/stealth.  Please note though that the Stealth circuits can be much larger in size (up to tens of gigabytes) and take much longer to generate than the IBM ones.

If you instead want to generate only a few circuits, you can use the stealth_generate.py or ibm_generate.py commands in these directories, or just open the generate_all.sh file and delete the lines that you don't want.

Eventually, you may wish to augment our circuit generator tools to create circuits of your own. The circuit generation tools are written in Python, and they are located in /trunk/spar_python. The circuit generators have several tunable parameters.  Please read the following README files to get a detailed explanation of these parameters and how to alter them to your needs:

/trunk/spar_python/circuit_generation/ibm/README.txt
/trunk/spar_python/circuit_generation/stealth/README.txt

4. Building the test harness and baseline

If you are more adventurous and would like to compile our code from scratch or augment it to add new features, this section will explain how to build the test harness and baseline.

All of the test harness, baseline, and dummy client code is written in C++ and located in /trunk/cpp. You will likely need the following programs/libraries to compile the code:

  • g++
  • scons
  • lemon
  • flex
  • libevent-dev
  • libboost-all-dev

If you are using Ubuntu Server 12.04 (as Lincoln is), you can install any of these with the following command:

sudo apt-get install <package-name>

Once the packages are installed, compiling the code is a simple application of scons. (Just in case it doesn't work though, please don't delete the binaries
in /bin!) First, run:

scons -c --opt
scons -c --opt

from the directory /trunk to remove all build files. (Note that you should run this command twice!)

It is then a good idea to run our suite of unit tests to make sure your dependencies have been installed and everything is working properly:

scons -u --opt cpp_test
cd /trunk/spar_python/
nosetests

The first line runs all of our C++ unit tests, and the last line runs all of our Python unit tests. You should see no failures (although some tests will fail on very rare occasions due to random number generation).

Then, you can do the following:

scons -u --opt

to recompile. The first time you run scons, you may see an error message like this:

scons: *** [trunk/cpp/baseline/ta2/ibm/opt/unary-gate.o] Error 4
os.chdir('/trunk/cpp/baseline/ta2/stealth/opt')
scons: building terminated because of errors.

If you get an error, don't worry; just run "scons -u –opt" again and it will work the second time.
 
The binary executables are (from /trunk):
 
TEST HARNESS: cpp/test-harness/ta2/opt/test-harness
DUMMY CLIENT: cpp/test-harness/ta2/opt/dummy-client
IBM BASELINE: cpp/baseline/ta2/ibm/opt/ibm-baseline-main
STEALTH BASELINE: cpp/baseline/ta2/stealth/opt/stealth-baseline-main
 
To run the test-harness on the Stealth baseline and the dummy client, run (from trunk/cpp/test-harness/ta2/)

./opt/test-harness –s ../../baseline/ta2/stealth/opt/stealth-baseline-main –c ./opt/dummy-client –t <test-script>

The IBM baseline is run similarly.

The results of the test will be written to a file called "results" in the current directory.
 
Optionally, specify "–d <debug-file-name>" to have the test-harness write out 4 log files corresponding to the stdin/out of the client/server, or specify "–r" to have the test harness recover from a previous crash (and thus continue with the circuit under test at the time of the crash, rather than starting over).

We strongly suggest that you run the test-harness with the "-h" flag the first time you use it, as it will then print out a help screen.

 

top of page