Performance Analysis Script for C++ Program by Python and gnu-time

November 25, 2020

2505 views

3381 words

# Introduction

It is a python script for analyzing performance of a C++ program, including time and memory cost. It is useful for Data Structure and Algorithm course.

You need to provide:

```
data.cpp - the program to generate data for testing
std.cpp - the standard program that outputs the right answer
main.cpp - the program to test
```

# Implementation

It is gnu-time that provides the main functions, i.e. measuring the time and memory cost.

Follow the [document](https://www.gnu.org/software/time/) carefully to install it. The common issue is that you should use the real gnu-time, rather than "time" provided by the shell, which is mentioned in the doc. By the way, use `--help` option to learn how to use it.

For macOS users like me, follow this for installation: [https://formulae.brew.sh/formula/gnu-time](https://formulae.brew.sh/formula/gnu-time) .

Note that the `-f` (or`--format`) option is useful to get outputs we like.

Having gnu-time, we can write the python script below to test the program with increasing data. Use `os.popen` to get the outputs of gnu-time, and then print the results in whatever formats. For example, a table-like format is convenient for further processing the results with Excel, like plotting.

Read the codes to know how to use it. :)

```python
import sys
import os

def check_ans(output_file1, output_file2):
    with open(output_file1) as f:
        output1 = f.read()
    with open(output_file2) as f:
        output2 = f.read()
    return output1 == output2

if __name__ == '__main__':
    os.system('g++ data.cpp -O3 -std=c++14 -o data')
    os.system('g++ std.cpp -O2 -std=c++14 -o std')
    os.system('g++ main.cpp -O2 -std=c++14 -o main')

input_data  =   'input.txt'
    ans_txt     =   'ans.txt'
    output_txt  =   'output.txt'

scale = 10000
    step = 10000
    loop = 10
    if len(sys.argv) >= 2:
        scale = int(sys.argv[1])
        if len(sys.argv) >= 3:
            step = int(sys.argv[2])
            if len(sys.argv) >= 4:
                loop = int(sys.argv[3])

print('\tscale\t\tProg Time(s)\tProg Mem(KB)\tStd Time(s)\tStd Mem(KB)')
    for i in range(0, loop):

os.system('./data {} > {}'.format(scale, input_data))
        print('\t' + str(scale) + '\t\t', end='')

prog_info = os.popen(
            '(gtime -f"%e\t\t%M\t\t" ./main < {} > {}) 2>&1'.format(input_data, output_txt)
        ).readlines()
        print(prog_info[0][:-1], end='')

std_info = os.popen(
            '(gtime -f"%e\t\t%M\t\t" ./std < {} > {}) 2>&1'.format(input_data, ans_txt)
        ).readlines()
        print(std_info[0][:-1], end='\n')

not_same = os.system('diff -q {} {}'.format(ans_txt, output_txt))
        if not_same:
            print('WA!')
            break
        else:
            scale += step
```

Then we can get things like:

```shell
$ python tester.py 100000 100000 30

scale		Prog Time(s)	Prog Mem(KB)	Std Time(s)	Std Mem(KB)
	100000		0.04		1952		0.04		2412
	200000		0.09		2664		0.09		4144
	300000		0.14		3616		0.15		5040
	400000		0.22		5176		0.21		7304
	500000		0.27		5724		0.26		8260
	600000		0.29		7152		0.32		9976
	700000		0.35		9408		0.38		12104
	800000		0.42		10636		0.44		13028
	900000		0.46		9704		0.52		14436
	1000000		0.54		12020		0.60		17308
```

Last modification：November 26, 2020