Add benchmark script

This commit is contained in:
Samuel Sloniker 2022-07-05 16:29:32 -07:00
parent b133facd70
commit c54c639b2f
3 changed files with 95 additions and 0 deletions

View File

@ -72,3 +72,13 @@ An example model, which is designed to distinguish between texts written by
Mark Twain and those written by William Shakespeare, is available in `models`. Mark Twain and those written by William Shakespeare, is available in `models`.
The raw model is in `models/raw.json`; the compiled model is in The raw model is in `models/raw.json`; the compiled model is in
`models/compiled.json`. `models/compiled.json`.
## Benchmark
A benchmark script is available for comparing performance of GPTC between
different Python versions. To use it, run `benchmark.py` with all of the Python
installations you want to test. It tests both compilation and classification.
It uses the default Twain/Shakespeare model for both, and for classification it
uses [Mark Antony's "Friends, Romans, countrymen"
speech](https://en.wikipedia.org/wiki/Friends,_Romans,_countrymen,_lend_me_your_ears)
from Shakespeare's *Julius Caesar*.

50
benchmark.py Normal file
View File

@ -0,0 +1,50 @@
import timeit
import gptc
import json
import sys
compile_iterations = 100
classify_iterations = 10000
with open("models/raw.json") as f:
raw_model = json.load(f)
with open("models/benchmark_text.txt") as f:
text = f.read()
classifier = gptc.Classifier(gptc.compile(raw_model))
print("Benchmarking GPTC on Python", sys.version)
print(
"Average compilation time over",
compile_iterations,
"iterations:",
round(
1000000
* timeit.timeit(
"gptc.compile(raw_model)",
number=compile_iterations,
globals=globals(),
)
/ compile_iterations
),
"microseconds",
)
print(
"Average classification time over",
classify_iterations,
"iterations:",
round(
1000000
* timeit.timeit(
"classifier.classify(text)",
number=classify_iterations,
globals=globals(),
)
/ classify_iterations
),
"microseconds",
)

35
models/benchmark_text.txt Normal file
View File

@ -0,0 +1,35 @@
Friends, Romans, countrymen, lend me your ears;
I come to bury Caesar, not to praise him.
The evil that men do lives after them;
The good is oft interred with their bones;
So let it be with Caesar. The noble Brutus
Hath told you Caesar was ambitious:
If it were so, it was a grievous fault,
And grievously hath Caesar answer'd it.
Here, under leave of Brutus and the rest-
For Brutus is an honourable man;
So are they all, all honourable men-
Come I to speak in Caesar's funeral.
He was my friend, faithful and just to me:
But Brutus says he was ambitious;
And Brutus is an honourable man.
He hath brought many captives home to Rome
Whose ransoms did the general coffers fill:
Did this in Caesar seem ambitious?
When that the poor have cried, Caesar hath wept:
Ambition should be made of sterner stuff:
Yet Brutus says he was ambitious;
And Brutus is an honourable man.
You all did see that on the Lupercal
I thrice presented him a kingly crown,
Which he did thrice refuse: was this ambition?
Yet Brutus says he was ambitious;
And, sure, he is an honourable man.
I speak not to disprove what Brutus spoke,
But here I am to speak what I do know.
You all did love him once, not without cause:
What cause withholds you then, to mourn for him?
O judgment! thou art fled to brutish beasts,
And men have lost their reason. Bear with me;
My heart is in the coffin there with Caesar,
And I must pause till it come back to me.