Introduction
It is a python script for analyzing performance of a C++ program, including time and memory cost. It is useful for Data Structure and Algorithm course.
You need to provide:
data.cpp - the program to generate data for testing
std.cpp - the standard program that outputs the right answer
main.cpp - the program to test
Implementation
It is gnu-time that provides the main functions, i.e. measuring the time and memory cost.
Follow the document carefully to install it. The common issue is that you should use the real gnu-time, rather than "time" provided by the shell, which is mentioned in the doc. By the way, use --help
option to learn how to use it.
For macOS users like me, follow this for installation: https://formulae.brew.sh/formula/gnu-time .
Note that the -f
(or--format
) option is useful to get outputs we like.
Having gnu-time, we can write the python script below to test the program with increasing data. Use os.popen
to get the outputs of gnu-time, and then print the results in whatever formats. For example, a table-like format is convenient for further processing the results with Excel, like plotting.
Read the codes to know how to use it. :)
import sys
import os
def check_ans(output_file1, output_file2):
with open(output_file1) as f:
output1 = f.read()
with open(output_file2) as f:
output2 = f.read()
return output1 == output2
if __name__ == '__main__':
os.system('g++ data.cpp -O3 -std=c++14 -o data')
os.system('g++ std.cpp -O2 -std=c++14 -o std')
os.system('g++ main.cpp -O2 -std=c++14 -o main')
input_data = 'input.txt'
ans_txt = 'ans.txt'
output_txt = 'output.txt'
scale = 10000
step = 10000
loop = 10
if len(sys.argv) >= 2:
scale = int(sys.argv[1])
if len(sys.argv) >= 3:
step = int(sys.argv[2])
if len(sys.argv) >= 4:
loop = int(sys.argv[3])
print('\tscale\t\tProg Time(s)\tProg Mem(KB)\tStd Time(s)\tStd Mem(KB)')
for i in range(0, loop):
os.system('./data {} > {}'.format(scale, input_data))
print('\t' + str(scale) + '\t\t', end='')
prog_info = os.popen(
'(gtime -f"%e\t\t%M\t\t" ./main < {} > {}) 2>&1'.format(input_data, output_txt)
).readlines()
print(prog_info[0][:-1], end='')
std_info = os.popen(
'(gtime -f"%e\t\t%M\t\t" ./std < {} > {}) 2>&1'.format(input_data, ans_txt)
).readlines()
print(std_info[0][:-1], end='\n')
not_same = os.system('diff -q {} {}'.format(ans_txt, output_txt))
if not_same:
print('WA!')
break
else:
scale += step
Then we can get things like:
$ python tester.py 100000 100000 30
scale Prog Time(s) Prog Mem(KB) Std Time(s) Std Mem(KB)
100000 0.04 1952 0.04 2412
200000 0.09 2664 0.09 4144
300000 0.14 3616 0.15 5040
400000 0.22 5176 0.21 7304
500000 0.27 5724 0.26 8260
600000 0.29 7152 0.32 9976
700000 0.35 9408 0.38 12104
800000 0.42 10636 0.44 13028
900000 0.46 9704 0.52 14436
1000000 0.54 12020 0.60 17308