Add benchmark script
This commit is contained in:
parent
b133facd70
commit
c54c639b2f
10
README.md
10
README.md
|
@ -72,3 +72,13 @@ An example model, which is designed to distinguish between texts written by
|
||||||
Mark Twain and those written by William Shakespeare, is available in `models`.
|
Mark Twain and those written by William Shakespeare, is available in `models`.
|
||||||
The raw model is in `models/raw.json`; the compiled model is in
|
The raw model is in `models/raw.json`; the compiled model is in
|
||||||
`models/compiled.json`.
|
`models/compiled.json`.
|
||||||
|
|
||||||
|
## Benchmark
|
||||||
|
|
||||||
|
A benchmark script is available for comparing performance of GPTC between
|
||||||
|
different Python versions. To use it, run `benchmark.py` with all of the Python
|
||||||
|
installations you want to test. It tests both compilation and classification.
|
||||||
|
It uses the default Twain/Shakespeare model for both, and for classification it
|
||||||
|
uses [Mark Antony's "Friends, Romans, countrymen"
|
||||||
|
speech](https://en.wikipedia.org/wiki/Friends,_Romans,_countrymen,_lend_me_your_ears)
|
||||||
|
from Shakespeare's *Julius Caesar*.
|
||||||
|
|
50
benchmark.py
Normal file
50
benchmark.py
Normal file
|
@ -0,0 +1,50 @@
|
||||||
|
import timeit
|
||||||
|
import gptc
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
compile_iterations = 100
|
||||||
|
classify_iterations = 10000
|
||||||
|
|
||||||
|
with open("models/raw.json") as f:
|
||||||
|
raw_model = json.load(f)
|
||||||
|
|
||||||
|
with open("models/benchmark_text.txt") as f:
|
||||||
|
text = f.read()
|
||||||
|
|
||||||
|
classifier = gptc.Classifier(gptc.compile(raw_model))
|
||||||
|
|
||||||
|
print("Benchmarking GPTC on Python", sys.version)
|
||||||
|
|
||||||
|
print(
|
||||||
|
"Average compilation time over",
|
||||||
|
compile_iterations,
|
||||||
|
"iterations:",
|
||||||
|
round(
|
||||||
|
1000000
|
||||||
|
* timeit.timeit(
|
||||||
|
"gptc.compile(raw_model)",
|
||||||
|
number=compile_iterations,
|
||||||
|
globals=globals(),
|
||||||
|
)
|
||||||
|
/ compile_iterations
|
||||||
|
),
|
||||||
|
"microseconds",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
print(
|
||||||
|
"Average classification time over",
|
||||||
|
classify_iterations,
|
||||||
|
"iterations:",
|
||||||
|
round(
|
||||||
|
1000000
|
||||||
|
* timeit.timeit(
|
||||||
|
"classifier.classify(text)",
|
||||||
|
number=classify_iterations,
|
||||||
|
globals=globals(),
|
||||||
|
)
|
||||||
|
/ classify_iterations
|
||||||
|
),
|
||||||
|
"microseconds",
|
||||||
|
)
|
35
models/benchmark_text.txt
Normal file
35
models/benchmark_text.txt
Normal file
|
@ -0,0 +1,35 @@
|
||||||
|
Friends, Romans, countrymen, lend me your ears;
|
||||||
|
I come to bury Caesar, not to praise him.
|
||||||
|
The evil that men do lives after them;
|
||||||
|
The good is oft interred with their bones;
|
||||||
|
So let it be with Caesar. The noble Brutus
|
||||||
|
Hath told you Caesar was ambitious:
|
||||||
|
If it were so, it was a grievous fault,
|
||||||
|
And grievously hath Caesar answer'd it.
|
||||||
|
Here, under leave of Brutus and the rest-
|
||||||
|
For Brutus is an honourable man;
|
||||||
|
So are they all, all honourable men-
|
||||||
|
Come I to speak in Caesar's funeral.
|
||||||
|
He was my friend, faithful and just to me:
|
||||||
|
But Brutus says he was ambitious;
|
||||||
|
And Brutus is an honourable man.
|
||||||
|
He hath brought many captives home to Rome
|
||||||
|
Whose ransoms did the general coffers fill:
|
||||||
|
Did this in Caesar seem ambitious?
|
||||||
|
When that the poor have cried, Caesar hath wept:
|
||||||
|
Ambition should be made of sterner stuff:
|
||||||
|
Yet Brutus says he was ambitious;
|
||||||
|
And Brutus is an honourable man.
|
||||||
|
You all did see that on the Lupercal
|
||||||
|
I thrice presented him a kingly crown,
|
||||||
|
Which he did thrice refuse: was this ambition?
|
||||||
|
Yet Brutus says he was ambitious;
|
||||||
|
And, sure, he is an honourable man.
|
||||||
|
I speak not to disprove what Brutus spoke,
|
||||||
|
But here I am to speak what I do know.
|
||||||
|
You all did love him once, not without cause:
|
||||||
|
What cause withholds you then, to mourn for him?
|
||||||
|
O judgment! thou art fled to brutish beasts,
|
||||||
|
And men have lost their reason. Bear with me;
|
||||||
|
My heart is in the coffin there with Caesar,
|
||||||
|
And I must pause till it come back to me.
|
Loading…
Reference in New Issue
Block a user