Add benchmark script
This commit is contained in:
parent
b133facd70
commit
c54c639b2f
10
README.md
10
README.md
|
@ -72,3 +72,13 @@ An example model, which is designed to distinguish between texts written by
|
|||
Mark Twain and those written by William Shakespeare, is available in `models`.
|
||||
The raw model is in `models/raw.json`; the compiled model is in
|
||||
`models/compiled.json`.
|
||||
|
||||
## Benchmark
|
||||
|
||||
A benchmark script is available for comparing performance of GPTC between
|
||||
different Python versions. To use it, run `benchmark.py` with all of the Python
|
||||
installations you want to test. It tests both compilation and classification.
|
||||
It uses the default Twain/Shakespeare model for both, and for classification it
|
||||
uses [Mark Antony's "Friends, Romans, countrymen"
|
||||
speech](https://en.wikipedia.org/wiki/Friends,_Romans,_countrymen,_lend_me_your_ears)
|
||||
from Shakespeare's *Julius Caesar*.
|
||||
|
|
50
benchmark.py
Normal file
50
benchmark.py
Normal file
|
@ -0,0 +1,50 @@
|
|||
import timeit
|
||||
import gptc
|
||||
import json
|
||||
import sys
|
||||
|
||||
compile_iterations = 100
|
||||
classify_iterations = 10000
|
||||
|
||||
with open("models/raw.json") as f:
|
||||
raw_model = json.load(f)
|
||||
|
||||
with open("models/benchmark_text.txt") as f:
|
||||
text = f.read()
|
||||
|
||||
classifier = gptc.Classifier(gptc.compile(raw_model))
|
||||
|
||||
print("Benchmarking GPTC on Python", sys.version)
|
||||
|
||||
print(
|
||||
"Average compilation time over",
|
||||
compile_iterations,
|
||||
"iterations:",
|
||||
round(
|
||||
1000000
|
||||
* timeit.timeit(
|
||||
"gptc.compile(raw_model)",
|
||||
number=compile_iterations,
|
||||
globals=globals(),
|
||||
)
|
||||
/ compile_iterations
|
||||
),
|
||||
"microseconds",
|
||||
)
|
||||
|
||||
|
||||
print(
|
||||
"Average classification time over",
|
||||
classify_iterations,
|
||||
"iterations:",
|
||||
round(
|
||||
1000000
|
||||
* timeit.timeit(
|
||||
"classifier.classify(text)",
|
||||
number=classify_iterations,
|
||||
globals=globals(),
|
||||
)
|
||||
/ classify_iterations
|
||||
),
|
||||
"microseconds",
|
||||
)
|
35
models/benchmark_text.txt
Normal file
35
models/benchmark_text.txt
Normal file
|
@ -0,0 +1,35 @@
|
|||
Friends, Romans, countrymen, lend me your ears;
|
||||
I come to bury Caesar, not to praise him.
|
||||
The evil that men do lives after them;
|
||||
The good is oft interred with their bones;
|
||||
So let it be with Caesar. The noble Brutus
|
||||
Hath told you Caesar was ambitious:
|
||||
If it were so, it was a grievous fault,
|
||||
And grievously hath Caesar answer'd it.
|
||||
Here, under leave of Brutus and the rest-
|
||||
For Brutus is an honourable man;
|
||||
So are they all, all honourable men-
|
||||
Come I to speak in Caesar's funeral.
|
||||
He was my friend, faithful and just to me:
|
||||
But Brutus says he was ambitious;
|
||||
And Brutus is an honourable man.
|
||||
He hath brought many captives home to Rome
|
||||
Whose ransoms did the general coffers fill:
|
||||
Did this in Caesar seem ambitious?
|
||||
When that the poor have cried, Caesar hath wept:
|
||||
Ambition should be made of sterner stuff:
|
||||
Yet Brutus says he was ambitious;
|
||||
And Brutus is an honourable man.
|
||||
You all did see that on the Lupercal
|
||||
I thrice presented him a kingly crown,
|
||||
Which he did thrice refuse: was this ambition?
|
||||
Yet Brutus says he was ambitious;
|
||||
And, sure, he is an honourable man.
|
||||
I speak not to disprove what Brutus spoke,
|
||||
But here I am to speak what I do know.
|
||||
You all did love him once, not without cause:
|
||||
What cause withholds you then, to mourn for him?
|
||||
O judgment! thou art fled to brutish beasts,
|
||||
And men have lost their reason. Bear with me;
|
||||
My heart is in the coffin there with Caesar,
|
||||
And I must pause till it come back to me.
|
Loading…
Reference in New Issue
Block a user