History · Versus Spot and Oink
Wall-clock medians of the same operation on the same input in automata (through its Python bindings, like Spot) and in the reference, from main · c432f068. The ratio is ours / reference: above 1 we are slower. They depend on the machine and are never compared automatically.
Settings: {"automata":"0.1.0","budget_s":1.0,"oink":{"binary":"/nix/store/lpniqhrsws0fsk208p8djv3rdim4bqyl-oink-parity-unstable-2026-06-03/bin/oink","solvers":["zlk","ppp"],"workers":1},"python":"3.13.17","spot":"2.16.2"}
| input | reference | ours | reference | ours / reference | size (ours / reference) | same result |
|---|---|---|---|---|---|---|
| hard/l-dsa-3 | spot | 122.58 µs | 117.70 µs | 1.04× | – / – | yes |
| hard/l-nba-2 | spot | 56.20 µs | 37.44 µs | 1.50× | – / – | yes |
| hard/m-nba-3 | spot | 238.82 µs | 140.78 µs | 1.70× | – / – | yes |
| ltl/and-gf-3 | spot | 9.16 µs | 12.98 µs | 0.71× | – / – | yes |
| ltl/eh-patterns-0 | spot | 21.78 µs | 20.94 µs | 1.04× | – / – | yes |
| ltl/gf-equiv-3 | spot | 20.04 µs | 34.12 µs | 0.59× | – / – | yes |
| ltl/gf-implies-3 | spot | 10.68 µs | 19.90 µs | 0.54× | – / – | yes |
| ltl/gh-q-3 | spot | 50.76 µs | 30.14 µs | 1.68× | – / – | yes |
| ltl/gh-r-3 | spot | 20.76 µs | 68.50 µs | 0.30× | – / – | yes |
| ltl/go-theta-3 | spot | 11.20 µs | 22.00 µs | 0.51× | – / – | yes |
| ltl/lily-patterns-1 | spot | 12.34 µs | 16.58 µs | 0.74× | – / – | yes |
| ltl/ms-phi-r-1 | spot | 10.22 µs | 38.62 µs | 0.26× | – / – | yes |
| ltl/ms-phi-s-1 | spot | 10.48 µs | 41.02 µs | 0.26× | – / – | yes |
| ltl/rv-counter-3 | spot | 28.78 µs | 14.28 µs | 2.02× | – / – | yes |
| ltl/sb-patterns-0 | spot | 12.46 µs | 14.68 µs | 0.85× | – / – | yes |
| ltl/sejk-k-2 | spot | 13.48 µs | 58.24 µs | 0.23× | – / – | yes |
| ltl/tv-uu-3 | spot | 41.32 µs | 39.36 µs | 1.05× | – / – | yes |
| ltl/u-left-3 | spot | 10.56 µs | 9.36 µs | 1.13× | – / – | yes |
| many_aps/and-gf-12 | spot | 168.96 µs | 76.56 µs | 2.21× | – / – | yes |
| many_aps/gf-equiv-12 | spot | 131.12 µs | 91.96 µs | 1.43× | – / – | yes |
| many_aps/r-right-10 | spot | 66.88 µs | 26.88 µs | 2.49× | – / – | yes |
| many_aps/random-n16-a8-p4-s1 | spot | 269.88 µs | 136.08 µs | 1.98× | – / – | yes |
| many_aps/random-n24-a12-p4-s1 | spot | 1.18 ms | 448.70 µs | 2.62× | – / – | yes |
| many_aps/random-n32-a16-p4-s1 | spot | 3.86 ms | 1.23 ms | 3.13× | – / – | yes |
| random/n128-a2-p6-s1 | spot | 338.82 µs | 215.86 µs | 1.57× | – / – | yes |
| random/n16-a1-p3-s1 | spot | 22.88 µs | 16.76 µs | 1.37× | – / – | yes |
| random/n256-a2-p8-s1 | spot | 891.45 µs | 489.03 µs | 1.82× | – / – | yes |
| random/n64-a2-p4-s1 | spot | 141.46 µs | 101.86 µs | 1.39× | – / – | yes |
| input | reference | ours | reference | ours / reference | size (ours / reference) | same result |
|---|---|---|---|---|---|---|
| hard/l-dsa-3 | spot | 71.66 µs | 62.14 µs | 1.15× | – / – | yes |
| hard/l-nba-2 | spot | 35.82 µs | 24.40 µs | 1.47× | – / – | yes |
| hard/m-nba-3 | spot | 148.08 µs | 91.18 µs | 1.62× | – / – | yes |
| ltl/and-gf-3 | spot | 5.90 µs | 8.66 µs | 0.68× | – / – | yes |
| ltl/eh-patterns-0 | spot | 14.28 µs | 17.90 µs | 0.80× | – / – | yes |
| ltl/gf-equiv-3 | spot | 11.42 µs | 19.08 µs | 0.60× | – / – | yes |
| ltl/gf-implies-3 | spot | 6.32 µs | 11.58 µs | 0.55× | – / – | yes |
| ltl/gh-q-3 | spot | 33.12 µs | 29.06 µs | 1.14× | – / – | yes |
| ltl/gh-r-3 | spot | 12.32 µs | 34.86 µs | 0.35× | – / – | yes |
| ltl/go-theta-3 | spot | 6.86 µs | 16.04 µs | 0.43× | – / – | yes |
| ltl/lily-patterns-1 | spot | 8.04 µs | 13.98 µs | 0.58× | – / – | yes |
| ltl/ms-phi-r-1 | spot | 6.04 µs | 19.24 µs | 0.31× | – / – | yes |
| ltl/ms-phi-s-1 | spot | 6.12 µs | 25.32 µs | 0.24× | – / – | yes |
| ltl/rv-counter-3 | spot | 18.08 µs | 14.52 µs | 1.25× | – / – | yes |
| ltl/sb-patterns-0 | spot | 7.64 µs | 12.12 µs | 0.63× | – / – | yes |
| ltl/sejk-k-2 | spot | 7.84 µs | 28.00 µs | 0.28× | – / – | yes |
| ltl/tv-uu-3 | spot | 28.46 µs | 30.08 µs | 0.95× | – / – | yes |
| ltl/u-left-3 | spot | 6.72 µs | 8.68 µs | 0.77× | – / – | yes |
| many_aps/and-gf-12 | spot | 125.80 µs | 54.50 µs | 2.31× | – / – | yes |
| many_aps/gf-equiv-12 | spot | 74.80 µs | 53.08 µs | 1.41× | – / – | yes |
| many_aps/r-right-10 | spot | 49.00 µs | 25.78 µs | 1.90× | – / – | yes |
| many_aps/random-n16-a8-p4-s1 | spot | 223.70 µs | 100.84 µs | 2.22× | – / – | yes |
| many_aps/random-n24-a12-p4-s1 | spot | 1.12 ms | 396.64 µs | 2.83× | – / – | yes |
| many_aps/random-n32-a16-p4-s1 | spot | 3.80 ms | 1.12 ms | 3.39× | – / – | yes |
| random/n128-a2-p6-s1 | spot | 198.46 µs | 128.34 µs | 1.55× | – / – | yes |
| random/n16-a1-p3-s1 | spot | 13.46 µs | 10.80 µs | 1.25× | – / – | yes |
| random/n256-a2-p8-s1 | spot | 512.25 µs | 317.64 µs | 1.61× | – / – | yes |
| random/n64-a2-p4-s1 | spot | 87.22 µs | 65.72 µs | 1.33× | – / – | yes |
| input | reference | ours | reference | ours / reference | size (ours / reference) | same result |
|---|---|---|---|---|---|---|
| ladder_10000 | oink-ppp | 2.75 ms | 328.00 µs | 8.39× | 10000 / 10000 | yes |
| ladder_10000 | oink-zlk | 2.75 ms | 335.00 µs | 8.21× | 10000 / 10000 | yes |
| ladder_10000 | spot | 2.75 ms | 10.17 ms | 0.27× | 10000 / 10000 | yes |
| ladder_50000 | oink-ppp | 18.93 ms | 2.12 ms | 8.92× | 50000 / 50000 | yes |
| ladder_50000 | oink-zlk | 18.93 ms | 2.47 ms | 7.66× | 50000 / 50000 | yes |
| ladder_50000 | spot | 18.93 ms | 77.59 ms | 0.24× | 50000 / 50000 | yes |
| random_few_10000 | oink-ppp | 6.98 ms | 1.46 ms | 4.80× | 10000 / 10000 | yes |
| random_few_10000 | oink-zlk | 6.98 ms | 2.76 ms | 2.53× | 10000 / 10000 | yes |
| random_few_10000 | spot | 6.98 ms | 21.11 ms | 0.33× | 10000 / 10000 | yes |
| random_few_50000 | oink-ppp | 77.03 ms | 16.00 ms | 4.81× | 50000 / 50000 | yes |
| random_few_50000 | oink-zlk | 77.03 ms | 20.71 ms | 3.72× | 50000 / 50000 | yes |
| random_few_50000 | spot | 77.03 ms | 679.94 ms | 0.11× | 50000 / 50000 | yes |
| random_many_10000 | oink-ppp | 6.31 ms | 1.64 ms | 3.84× | 10000 / 10000 | yes |
| random_many_10000 | oink-zlk | 6.31 ms | 3.04 ms | 2.07× | 10000 / 10000 | yes |
| random_many_10000 | spot | 6.31 ms | 29.88 ms | 0.21× | 10000 / 10000 | yes |
| random_many_50000 | oink-ppp | 82.90 ms | 35.20 ms | 2.35× | 50000 / 50000 | yes |
| random_many_50000 | oink-zlk | 82.90 ms | 51.28 ms | 1.62× | 50000 / 50000 | yes |
| random_many_50000 | spot | 82.90 ms | 779.68 ms | 0.11× | 50000 / 50000 | yes |
Geometric mean of ours / reference over the inputs of an operation, one point per commit of main; the values in the tooltips are per mille (1.5 is 1500).