History · Versus Spot and Oink
CsrTs: real compressed sparse rows, nondeterministic, any alphabet
status done · queued 2026-11-11 17:40 · started 2026-11-11 17:40 · finished 2026-11-11 17:50
· · rustc 1.95.0 (59807616e 2026-04-14) (built from a source tarball) · valgrind-3.26.0 · pinned to cpu 1
| benchmark | base | head | change | heap bytes |
|---|---|---|---|---|
| storage::storage::minimize_dense::redundant_2000 | 4 676 910 | 4 834 920 | +3.38 % | +0.00 % |
| storage::storage::minimize_dense::redundant_8000 | 18 364 895 | 18 996 871 | +3.44 % | +0.00 % |
| storage::storage::predecessors_of_all_states_csr::k2_2000 | 102 046 | 638 345 | +525.55 % | |
| storage::storage::reachable_states_csr::k2_8000 | 1 101 115 | 1 148 908 | +4.34 % | +0.00 % |
| storage::storage::reachable_states_dense::k2_8000 | 1 133 699 | 1 365 712 | +20.47 % | +0.00 % |
| storage::storage::reachable_states_linked::k2_8000 | 924 279 | 1 148 290 | +24.24 % | +0.00 % |
| storage::storage::run_many_words_csr::count_1000 | 551 036 | 726 035 | +31.76 % | +0.00 % |
| storage::storage::run_one_word_csr::len_100000 | 2 600 054 | 3 500 048 | +34.61 % | +0.00 % |
| storage::storage::clone_csr::k2_10000 | 30 590 | 25 138 | -17.82 % | -8.70 % |
| storage::storage::complete_dense::partial_5000 | 155 628 | 150 672 | -3.18 % | -58.82 % |
| storage::storage::edges_from_all_states_csr::k2_2000 | 72 047 | 48 041 | -33.32 % | +0.00 % |
| storage::storage::edges_from_all_states_csr::k8_2000 | 108 047 | 48 041 | -55.54 % | +0.00 % |
| storage::storage::successor_all_pairs_csr::k2_2000 | 122 242 | 108 241 | -11.45 % | +0.00 % |
| storage::storage::successor_all_pairs_csr::k8_2000 | 702 302 | 552 301 | -21.36 % | +0.00 % |
| ts::ts::add_edges_in_order::k2_5000 | 1 402 671 | 1 362 671 | -2.85 % | +0.00 % |
| ts::ts::add_edges_in_order::k8_5000 | 6 669 285 | 6 509 285 | -2.40 % | +0.00 % |
| ts::ts::add_edges_in_reverse::k2_5000 | 1 402 671 | 1 362 671 | -2.85 % | +0.00 % |
| ts::ts::erase_state_colors_collect::scrambled_10000 | 2 377 172 | 2 317 172 | -2.52 % | +0.00 % |
| ts::ts::map_edge_colors_collect::scrambled_10000 | 2 368 784 | 2 308 784 | -2.53 % | +0.00 % |
| ts::ts::map_edge_colors_collect::scrambled_40000 | 9 415 409 | 9 175 409 | -2.55 % | +0.00 % |
| dfa::dfa::accepts_many_words::scrambled_1000 | 343 072 | 343 072 | +0.00 % | +0.00 % |
| dfa::dfa::complement::scrambled_1000 | 366 068 | 366 068 | +0.00 % | +0.00 % |
| dfa::dfa::complement::scrambled_4000 | 1 435 873 | 1 435 873 | +0.00 % | +0.00 % |
| dfa::dfa::equivalent::deep_difference_100 | 61 667 | 61 667 | +0.00 % | +0.00 % |
| dfa::dfa::equivalent::deep_difference_400 | 212 158 | 212 158 | +0.00 % | +0.00 % |
| dfa::dfa::equivalent::different_100 | 9 047 | 9 047 | +0.00 % | +0.00 % |
| dfa::dfa::equivalent::different_400 | 9 088 | 9 088 | +0.00 % | +0.00 % |
| dfa::dfa::equivalent::minimized_100 | 35 849 | 35 849 | +0.00 % | +0.00 % |
| dfa::dfa::equivalent::minimized_400 | 120 587 | 120 587 | +0.00 % | +0.00 % |
| dfa::dfa::give_word::empty_1000 | 1 872 613 | 1 872 613 | +0.00 % | +0.00 % |
| dfa::dfa::give_word::empty_4000 | 6 930 823 | 6 930 823 | +0.00 % | +0.00 % |
| dfa::dfa::give_word::far_1000 | 164 342 | 164 342 | +0.00 % | +0.00 % |
| dfa::dfa::give_word::far_4000 | 946 982 | 946 982 | +0.00 % | +0.00 % |
| dfa::dfa::intersection::scrambled_100 | 3 319 709 | 3 319 713 | +0.00 % | +0.00 % |
| dfa::dfa::intersection::scrambled_200 | 13 224 873 | 13 224 877 | +0.00 % | +0.00 % |
| dfa::dfa::intersection_is_empty::empty_100 | 59 266 | 59 266 | +0.00 % | +0.00 % |
| dfa::dfa::intersection_is_empty::empty_200 | 109 269 | 109 269 | +0.00 % | +0.00 % |
| dfa::dfa::is_empty_language::empty_1000 | 1 872 615 | 1 872 615 | +0.00 % | +0.00 % |
| dfa::dfa::is_empty_language::empty_4000 | 6 930 825 | 6 930 825 | +0.00 % | +0.00 % |
| dfa::dfa::minimize::minimal_1000 | 3 450 924 | 3 461 926 | +0.32 % | +0.00 % |
| dfa::dfa::minimize::minimal_4000 | 13 472 602 | 13 516 602 | +0.33 % | +0.00 % |
| dfa::dfa::minimize::redundant_1000 | 2 292 807 | 2 297 812 | +0.22 % | +0.00 % |
| dfa::dfa::minimize::redundant_4000 | 8 869 106 | 8 889 109 | +0.23 % | +0.00 % |
| dfa::dfa::minimize::scrambled_1000 | 3 670 256 | 3 674 125 | +0.11 % | +0.00 % |
| dfa::dfa::minimize::scrambled_4000 | 14 250 469 | 14 265 475 | +0.11 % | +0.00 % |
| dfa::dfa::minimize_moore_with_edge_colors::scrambled_1000 | 3 669 252 | 3 673 121 | +0.11 % | +0.00 % |
| dfa::dfa::minimize_moore_with_edge_colors::scrambled_4000 | 14 239 073 | 14 254 079 | +0.11 % | +0.00 % |
| dfa::dfa::union::scrambled_100 | 3 319 709 | 3 319 713 | +0.00 % | +0.00 % |
| dfa::dfa::union::scrambled_200 | 13 224 873 | 13 224 877 | +0.00 % | +0.00 % |
| dpa::dpa::accepted_word::chain_100 | 113 482 | 113 484 | +0.00 % | |
| dpa::dpa::accepted_word::chain_400 | 383 293 | 383 295 | +0.00 % | |
| dpa::dpa::accepted_word::scrambled_100 | 145 688 | 145 690 | +0.00 % | |
| dpa::dpa::accepted_word::scrambled_400 | 496 798 | 496 800 | +0.00 % | |
| dpa::dpa::accepted_word_fixtures::ltl | 546 210 | 546 252 | +0.01 % | |
| dpa::dpa::accepted_word_fixtures::synthetic | 1 402 296 | 1 402 310 | +0.00 % | |
| dpa::dpa::accepted_word_fixtures_by_atoms::ltl | 1 734 675 | 1 734 717 | +0.00 % | |
| dpa::dpa::accepted_word_fixtures_by_atoms::synthetic | 2 873 256 | 2 873 270 | +0.00 % | |
| dpa::dpa::collect_dpa::scrambled_1000 | 299 640 | 299 640 | +0.00 % | |
| dpa::dpa::collect_dpa::scrambled_4000 | 1 169 031 | 1 169 031 | +0.00 % | |
| dpa::dpa::complement::chain_1000 | 4 096 940 | 4 096 944 | +0.00 % | |
| dpa::dpa::complement::chain_4000 | 16 653 211 | 16 653 215 | +0.00 % | |
| dpa::dpa::complement::scrambled_1000 | 4 958 494 | 4 958 534 | +0.00 % | |
| dpa::dpa::complement::scrambled_4000 | 20 037 458 | 20 037 498 | +0.00 % | |
| dpa::dpa::complement_fixtures::ltl | 1 480 503 | 1 480 693 | +0.01 % | |
| dpa::dpa::complement_fixtures::many_aps | 13 919 764 | 13 919 870 | +0.00 % | |
| dpa::dpa::complement_fixtures::synthetic | 9 652 583 | 9 652 742 | +0.00 % | |
| dpa::dpa::complement_without_normalizing::scrambled_1000 | 574 441 | 574 429 | -0.00 % | |
| dpa::dpa::complement_without_normalizing::scrambled_4000 | 2 259 823 | 2 259 822 | -0.00 % | |
| dpa::dpa::equivalent_fixtures::many_aps | 21 649 390 | 21 649 756 | +0.00 % | |
| dpa::dpa::hoa_parse::ltl | 5 509 161 | 5 507 276 | -0.03 % | |
| dpa::dpa::hoa_parse::many_aps | 88 889 281 | 88 878 935 | -0.01 % | |
| dpa::dpa::hoa_parse::synthetic | 25 096 681 | 25 084 967 | -0.05 % | |
| dpa::dpa::hoa_print::ltl | 19 039 651 | 19 039 652 | +0.00 % | |
| dpa::dpa::hoa_print::many_aps | 19 675 345 | 19 675 345 | +0.00 % | |
| dpa::dpa::hoa_print::synthetic | 27 397 176 | 27 397 177 | +0.00 % | |
| dpa::dpa::included_fixtures::many_aps | 18 597 823 | 18 598 029 | +0.00 % | |
| dpa::dpa::included_in::complement_100 | 270 299 | 270 313 | +0.01 % | |
| dpa::dpa::included_in::complement_25 | 88 075 | 88 085 | +0.01 % | |
| dpa::dpa::included_in::complement_50 | 151 571 | 151 583 | +0.01 % | |
| dpa::dpa::included_in::normalized_100 | 543 003 | 543 031 | +0.01 % | |
| dpa::dpa::included_in::normalized_25 | 184 848 | 184 868 | +0.01 % | |
| dpa::dpa::included_in::normalized_50 | 313 519 | 313 539 | +0.01 % | |
| dpa::dpa::language_equivalent::complement_100 | 279 874 | 279 888 | +0.01 % | |
| dpa::dpa::language_equivalent::complement_25 | 90 600 | 90 610 | +0.01 % | |
| dpa::dpa::language_equivalent::complement_50 | 156 446 | 156 458 | +0.01 % | |
| dpa::dpa::language_equivalent::normalized_100 | 993 356 | 993 400 | +0.00 % | |
| dpa::dpa::language_equivalent::normalized_25 | 326 815 | 326 845 | +0.01 % | |
| dpa::dpa::language_equivalent::normalized_50 | 581 323 | 581 353 | +0.01 % | |
| dpa::dpa::map_edge_colors_collect::scrambled_1000 | 297 297 | 297 297 | +0.00 % | |
| dpa::dpa::map_edge_colors_collect::scrambled_4000 | 1 157 692 | 1 157 692 | +0.00 % | |
| dpa::dpa::normalized::chain_1000 | 3 824 386 | 3 824 390 | +0.00 % | |
| dpa::dpa::normalized::chain_4000 | 15 564 657 | 15 564 661 | +0.00 % | |
| dpa::dpa::normalized::scrambled_1000 | 4 685 742 | 4 685 782 | +0.00 % | |
| dpa::dpa::normalized::scrambled_4000 | 18 948 706 | 18 948 746 | +0.00 % | |
| dpa::dpa::normalized::scrambled_k4_1000 | 26 000 725 | 26 000 787 | +0.00 % | |
| dpa::dpa::normalized_fixtures::ltl | 1 415 889 | 1 416 101 | +0.01 % | |
| dpa::dpa::normalized_fixtures::many_aps | 13 659 715 | 13 659 799 | +0.00 % | |
| dpa::dpa::normalized_fixtures::synthetic | 9 337 591 | 9 337 761 | +0.00 % | |
| dpa::dpa::prefix_partition::chain_16 | 3 339 069 | 3 339 205 | +0.00 % | |
| dpa::dpa::prefix_partition::chain_8 | 861 644 | 861 716 | +0.01 % | |
| dpa::dpa::prefix_partition::scrambled_16 | 3 707 909 | 3 708 189 | +0.01 % | |
| dpa::dpa::prefix_partition::scrambled_8 | 1 022 858 | 1 023 002 | +0.01 % | |
| dpa::dpa::rejected_word::chain_100 | 938 248 | 938 255 | +0.00 % | |
| dpa::dpa::rejected_word::chain_400 | 3 643 943 | 3 643 961 | +0.00 % | |
| dpa::dpa::rejected_word::scrambled_100 | 247 893 | 247 895 | +0.00 % | |
| dpa::dpa::rejected_word::scrambled_400 | 935 362 | 935 364 | +0.00 % | |
| dpa::dpa::sccs::chain_1000 | 500 888 | 500 890 | +0.00 % | |
| dpa::dpa::sccs::chain_4000 | 1 928 615 | 1 928 617 | +0.00 % | |
| dpa::dpa::sccs::scrambled_1000 | 567 845 | 567 847 | +0.00 % | |
| dpa::dpa::sccs::scrambled_4000 | 2 257 485 | 2 257 487 | +0.00 % | |
| dpa::dpa::witness_colors::chain_100 | 4 741 540 | 4 741 555 | +0.00 % | |
| dpa::dpa::witness_colors::chain_400 | 46 520 653 | 46 520 668 | +0.00 % | |
| dpa::dpa::witness_colors::scrambled_100 | 4 126 372 | 4 126 387 | +0.00 % | |
| dpa::dpa::witness_colors::scrambled_400 | 31 846 701 | 31 846 738 | +0.00 % | |
| dpa::dpa::witness_inequivalence::complement_100 | 279 873 | 279 887 | +0.01 % | |
| dpa::dpa::witness_inequivalence::complement_25 | 90 599 | 90 609 | +0.01 % | |
| dpa::dpa::witness_inequivalence::complement_50 | 156 445 | 156 457 | +0.01 % | |
| dpa::dpa::witness_not_subset_of::complement_100 | 270 298 | 270 312 | +0.01 % | |
| dpa::dpa::witness_not_subset_of::complement_25 | 88 074 | 88 084 | +0.01 % | |
| dpa::dpa::witness_not_subset_of::complement_50 | 151 570 | 151 582 | +0.01 % | |
| dpa::dpa::witness_not_subset_of::normalized_100 | 543 002 | 543 030 | +0.01 % | |
| dpa::dpa::witness_not_subset_of::normalized_25 | 184 847 | 184 867 | +0.01 % | |
| dpa::dpa::witness_not_subset_of::normalized_50 | 313 518 | 313 538 | +0.01 % | |
| dpa::dpa::witnesses_fixtures::ltl | 2 229 215 | 2 229 287 | +0.00 % | |
| dpa::dpa::witnesses_fixtures::synthetic | 19 359 356 | 19 359 384 | +0.00 % | |
| game::game::tangle_learning::few_priorities_1000 | 2 697 048 | 2 687 985 | -0.34 % | +0.00 % |
| game::game::tangle_learning::few_priorities_4000 | 8 120 676 | 8 092 832 | -0.34 % | +0.00 % |
| game::game::tangle_learning::ladder_1000 | 1 179 314 | 1 179 088 | -0.02 % | +0.00 % |
| game::game::tangle_learning::ladder_4000 | 4 408 134 | 4 407 443 | -0.02 % | +0.00 % |
| game::game::tangle_learning::many_priorities_1000 | 2 018 028 | 2 013 656 | -0.22 % | +0.00 % |
| game::game::tangle_learning::many_priorities_4000 | 6 752 835 | 6 739 293 | -0.20 % | +0.00 % |
| game::game::tangle_learning_min_odd::few_priorities_1000 | 1 521 615 | 1 520 824 | -0.05 % | +0.00 % |
| game::game::tangle_learning_min_odd::few_priorities_4000 | 8 017 700 | 8 008 714 | -0.11 % | +0.00 % |
| game::game::zielonka::few_priorities_1000 | 1 126 199 | 1 126 202 | +0.00 % | +0.00 % |
| game::game::zielonka::few_priorities_4000 | 3 470 254 | 3 470 257 | +0.00 % | +0.00 % |
| game::game::zielonka::ladder_1000 | 5 134 281 | 5 134 284 | +0.00 % | +0.00 % |
| game::game::zielonka::ladder_4000 | 19 816 373 | 19 816 376 | +0.00 % | +0.00 % |
| game::game::zielonka::many_priorities_1000 | 4 103 518 | 4 103 521 | +0.00 % | +0.00 % |
| game::game::zielonka::many_priorities_4000 | 12 909 837 | 12 909 840 | +0.00 % | +0.00 % |
| game::game::zielonka_min_odd::few_priorities_1000 | 2 139 077 | 2 139 080 | +0.00 % | +0.00 % |
| game::game::zielonka_min_odd::few_priorities_4000 | 6 302 075 | 6 302 078 | +0.00 % | +0.00 % |
| learning::learning::lstar_mealy::counting_16 | 1 816 847 | 1 796 618 | -1.11 % | -0.21 % |
| learning::learning::lstar_mealy::counting_8 | 481 469 | 473 174 | -1.72 % | -0.47 % |
| learning::learning::sprout_parity::inf_ab_100 | 1 270 757 | 1 269 787 | -0.08 % | -0.05 % |
| learning::learning::sprout_parity::inf_ab_400 | 2 610 820 | 2 611 163 | +0.01 % | -0.02 % |
| nondeterministic::nondeterministic::fixture_accepts::ltl | 762 545 | 762 661 | +0.02 % | |
| nondeterministic::nondeterministic::nba_accepts::nba_400 | 125 269 894 | 125 269 894 | +0.00 % | |
| nondeterministic::nondeterministic::nba_witness::nba_4000 | 14 689 961 | 14 689 963 | +0.00 % | |
| nondeterministic::nondeterministic::nca_accepts::nca_400 | 266 229 440 | 266 229 440 | +0.00 % | |
| nondeterministic::nondeterministic::nca_witness::nca_4000 | 19 458 261 | 19 458 265 | +0.00 % | |
| nondeterministic::nondeterministic::nma_witness::nma_1000 | 25 744 469 | 25 744 501 | +0.00 % | |
| nondeterministic::nondeterministic::npa_accepts::empty_npa_400 | 493 167 601 | 493 167 513 | -0.00 % | |
| nondeterministic::nondeterministic::npa_accepts::npa_100 | 30 411 628 | 30 411 608 | -0.00 % | |
| nondeterministic::nondeterministic::npa_accepts::npa_400 | 125 279 542 | 125 279 522 | -0.00 % | |
| nondeterministic::nondeterministic::npa_witness::empty_npa_1000 | 9 121 952 | 9 121 954 | +0.00 % | |
| nondeterministic::nondeterministic::npa_witness::empty_npa_4000 | 37 023 303 | 37 023 305 | +0.00 % | |
| nondeterministic::nondeterministic::npa_witness::npa_1000 | 2 479 531 | 2 479 531 | +0.00 % | |
| nondeterministic::nondeterministic::npa_witness::npa_4000 | 20 354 255 | 20 354 255 | +0.00 % | |
| nondeterministic::nondeterministic::nra_witness::nra_4000 | 20 639 567 | 20 639 571 | +0.00 % | |
| runs::runs::dfa_accepts_many_words::count_1000 | 4 303 213 | 4 303 213 | +0.00 % | +0.00 % |
| runs::runs::dfa_accepts_one_word::len_100 | 3 492 | 3 492 | +0.00 % | +0.00 % |
| runs::runs::dfa_accepts_one_word::len_10000 | 266 070 | 266 070 | +0.00 % | +0.00 % |
| runs::runs::dfa_accepts_one_word::len_100000 | 2 650 731 | 2 650 731 | +0.00 % | +0.00 % |
| runs::runs::dfa_reached_state::count_1000 | 3 570 211 | 3 570 211 | +0.00 % | +0.00 % |
| runs::runs::dpa_accepts_lassos::long | 9 150 745 | 9 145 303 | -0.06 % | +0.00 % |
| runs::runs::dpa_accepts_lassos::short | 406 490 | 400 803 | -1.40 % | +0.00 % |
| runs::runs::dpa_recurrent_edge_colors::long | 15 116 266 | 15 110 824 | -0.04 % | +0.00 % |
| runs::runs::dpa_recurrent_edge_colors::short | 1 189 790 | 1 184 103 | -0.48 % | +0.00 % |
| runs::runs::dpa_recurrent_states::long | 10 502 988 | 10 497 546 | -0.05 % | +0.00 % |
| runs::runs::dpa_recurrent_states::short | 572 037 | 566 350 | -0.99 % | +0.00 % |
| runs::runs::dpa_visited_sequences::long | 8 738 263 | 8 738 263 | +0.00 % | +0.00 % |
| runs::runs::dpa_visited_sequences::short | 370 678 | 370 678 | +0.00 % | +0.00 % |
| runs::runs::fixture_accepts_lassos::ltl | 8 766 271 | 8 769 271 | +0.03 % | |
| runs::runs::fixture_accepts_lassos::synthetic | 4 340 189 | 4 341 589 | +0.03 % | |
| scc::scc::scc_queries::small_cycles_1000 | 906 676 | 906 678 | +0.00 % | +0.00 % |
| scc::scc::scc_queries::small_cycles_4000 | 3 548 919 | 3 548 921 | +0.00 % | +0.00 % |
| scc::scc::tarjan::back_edges_1000 | 512 099 | 512 101 | +0.00 % | +0.00 % |
| scc::scc::tarjan::back_edges_4000 | 1 987 627 | 1 987 629 | +0.00 % | +0.00 % |
| scc::scc::tarjan::binary_tree_1000 | 571 217 | 571 219 | +0.00 % | +0.00 % |
| scc::scc::tarjan::binary_tree_4000 | 2 290 459 | 2 290 461 | +0.00 % | +0.00 % |
| scc::scc::tarjan::fragmented_1000 | 658 522 | 658 524 | +0.00 % | +0.00 % |
| scc::scc::tarjan::fragmented_4000 | 2 534 108 | 2 534 110 | +0.00 % | +0.00 % |
| scc::scc::tarjan::grid_1024 | 596 080 | 596 082 | +0.00 % | +0.00 % |
| scc::scc::tarjan::grid_4096 | 2 378 484 | 2 378 486 | +0.00 % | +0.00 % |
| scc::scc::tarjan::path_1000 | 1 929 119 | 1 929 121 | +0.00 % | +0.00 % |
| scc::scc::tarjan::path_4000 | 7 632 908 | 7 632 910 | +0.00 % | +0.00 % |
| scc::scc::tarjan::ring_1000 | 512 099 | 512 101 | +0.00 % | +0.00 % |
| scc::scc::tarjan::ring_4000 | 1 987 627 | 1 987 629 | +0.00 % | +0.00 % |
| scc::scc::tarjan::scrambled_1000 | 579 056 | 579 058 | +0.00 % | +0.00 % |
| scc::scc::tarjan::scrambled_4000 | 2 316 497 | 2 316 499 | +0.00 % | +0.00 % |
| scc::scc::tarjan::small_cycles_1000 | 905 658 | 905 660 | +0.00 % | +0.00 % |
| scc::scc::tarjan::small_cycles_4000 | 3 544 901 | 3 544 903 | +0.00 % | +0.00 % |
| scc::scc::tarjan_from::fragmented_1000 | 242 530 | 242 532 | +0.00 % | +0.00 % |
| scc::scc::tarjan_from::fragmented_4000 | 907 762 | 907 764 | +0.00 % | +0.00 % |
| scc::scc::tarjan_from::path_1000 | 1 921 158 | 1 921 160 | +0.00 % | +0.00 % |
| scc::scc::tarjan_from::path_4000 | 7 545 584 | 7 545 586 | +0.00 % | +0.00 % |
| scc::scc::tarjan_from::ring_1000 | 503 645 | 503 647 | +0.00 % | +0.00 % |
| scc::scc::tarjan_from::scrambled_1000 | 570 602 | 570 604 | +0.00 % | +0.00 % |
| storage::storage::add_edges_dense::k2_5000 | 1 356 401 | 1 356 401 | +0.00 % | +0.00 % |
| storage::storage::add_edges_dense::k8_5000 | 4 247 059 | 4 247 059 | +0.00 % | +0.00 % |
| storage::storage::add_edges_linked::k2_5000 | 1 402 670 | 1 402 670 | +0.00 % | +0.00 % |
| storage::storage::add_edges_linked::k8_5000 | 6 669 284 | 6 669 284 | +0.00 % | +0.00 % |
| storage::storage::clone_dense::k2_10000 | 15 113 | 15 139 | +0.17 % | +0.00 % |
| storage::storage::clone_linked::k2_10000 | 370 824 | 370 822 | -0.00 % | +0.00 % |
| storage::storage::complete_linked::partial_5000 | 1 210 854 | 1 225 141 | +1.18 % | -0.12 % |
| storage::storage::edges_from_all_states_dense::k2_2000 | 102 043 | 102 043 | +0.00 % | +0.00 % |
| storage::storage::edges_from_all_states_dense::k8_2000 | 140 043 | 140 043 | +0.00 % | +0.00 % |
| storage::storage::edges_from_all_states_linked::k2_2000 | 68 033 | 68 033 | +0.00 % | +0.00 % |
| storage::storage::edges_from_all_states_linked::k8_2000 | 152 033 | 152 033 | +0.00 % | +0.00 % |
| storage::storage::minimize_csr::redundant_2000 | 4 633 756 | 4 584 603 | -1.06 % | +0.00 % |
| storage::storage::minimize_csr::redundant_8000 | 18 159 861 | 17 948 590 | -1.16 % | +0.00 % |
| storage::storage::minimize_linked::redundant_2000 | 4 505 038 | 4 585 040 | +1.78 % | +0.00 % |
| storage::storage::minimize_linked::redundant_8000 | 17 714 721 | 18 034 689 | +1.81 % | +0.00 % |
| storage::storage::overwrite_edges_dense::k2_5000 | 2 126 431 | 2 126 433 | +0.00 % | +0.00 % |
| storage::storage::overwrite_edges_linked::k2_5000 | 3 790 775 | 3 790 776 | +0.00 % | +0.00 % |
| storage::storage::predecessors_of_all_states_dense::k2_2000 | 290 369 | 294 366 | +1.38 % | +0.00 % |
| storage::storage::predecessors_of_all_states_linked::k2_2000 | 68 033 | 68 033 | +0.00 % | +0.00 % |
| storage::storage::product_successor_csr::n200 | 64 096 | 64 587 | +0.77 % | +0.00 % |
| storage::storage::product_successor_dense::n200 | 73 087 | 73 087 | +0.00 % | +0.00 % |
| storage::storage::product_successor_linked::n200 | 46 547 | 46 547 | +0.00 % | +0.00 % |
| storage::storage::remove_state_dense::k2_10000 | 1 072 212 | 1 092 207 | +1.86 % | +0.00 % |
| storage::storage::remove_state_linked::k2_10000 | 1 094 012 | 1 094 012 | +0.00 % | +0.00 % |
| storage::storage::run_many_words_dense::count_1000 | 558 044 | 558 044 | +0.00 % | +0.00 % |
| storage::storage::run_many_words_linked::count_1000 | 481 044 | 481 044 | +0.00 % | +0.00 % |
| storage::storage::run_one_word_dense::len_100000 | 2 800 050 | 2 800 050 | +0.00 % | +0.00 % |
| storage::storage::run_one_word_linked::len_100000 | 2 350 039 | 2 350 039 | +0.00 % | +0.00 % |
| storage::storage::successor_all_pairs_dense::k2_2000 | 162 237 | 162 237 | +0.00 % | +0.00 % |
| storage::storage::successor_all_pairs_dense::k8_2000 | 814 297 | 814 297 | +0.00 % | +0.00 % |
| storage::storage::successor_all_pairs_linked::k2_2000 | 108 236 | 108 236 | +0.00 % | +0.00 % |
| storage::storage::successor_all_pairs_linked::k8_2000 | 750 296 | 750 296 | +0.00 % | +0.00 % |
| ts::ts::builder_with_edges::k2_5000 | 81 365 967 | 81 441 560 | +0.09 % | +0.00 % |
| ts::ts::canonical_naming::scrambled_1500 | 3 144 090 | 3 144 066 | -0.00 % | +0.00 % |
| ts::ts::canonical_naming::scrambled_500 | 972 576 | 972 552 | -0.00 % | +0.00 % |
| ts::ts::clone::scrambled_10000 | 371 163 | 371 163 | +0.00 % | +0.00 % |
| ts::ts::clone::scrambled_40000 | 1 481 400 | 1 481 400 | +0.00 % | +0.00 % |
| ts::ts::collect_dts::scrambled_10000 | 3 455 128 | 3 455 128 | +0.00 % | +0.00 % |
| ts::ts::collect_dts::scrambled_40000 | 13 759 607 | 13 759 607 | +0.00 % | +0.00 % |
| ts::ts::determinize::nth_from_end_10 | 4 414 320 | 4 414 320 | +0.00 % | +0.00 % |
| ts::ts::determinize::nth_from_end_12 | 19 628 006 | 19 628 006 | +0.00 % | +0.00 % |
| ts::ts::determinize::nth_from_end_8 | 1 007 268 | 1 007 268 | +0.00 % | +0.00 % |
| ts::ts::determinize_propositional::aps_6_n_6 | 4 333 844 | 4 333 842 | -0.00 % | +0.00 % |
| ts::ts::determinize_propositional::aps_7_n_6 | 4 525 387 | 4 525 385 | -0.00 % | +0.00 % |
| ts::ts::determinize_propositional::aps_8_n_6 | 26 425 542 | 26 425 540 | -0.00 % | +0.00 % |
| ts::ts::edges_from_all_states::scrambled_2000 | 72 669 | 72 669 | +0.00 % | +0.00 % |
| ts::ts::edges_from_all_states::scrambled_k8_2000 | 156 601 | 156 601 | +0.00 % | +0.00 % |
| ts::ts::into_dfa::scrambled_10000 | 3 635 166 | 3 635 166 | +0.00 % | +0.00 % |
| ts::ts::matching_product_successor_index::n200 | 75 662 | 75 662 | +0.00 % | +0.00 % |
| ts::ts::minimal_representative_of_all_states::scrambled_100 | 2 102 811 | 2 102 811 | +0.00 % | +0.00 % |
| ts::ts::minimal_representative_of_all_states::scrambled_300 | 16 917 261 | 16 917 261 | +0.00 % | +0.00 % |
| ts::ts::minimal_representatives::scrambled_2000 | 3 386 040 | 3 386 040 | +0.00 % | +0.00 % |
| ts::ts::minimal_representatives::scrambled_8000 | 14 908 249 | 14 908 249 | +0.00 % | +0.00 % |
| ts::ts::predecessors_of_all_states::scrambled_2000 | 72 669 | 72 669 | +0.00 % | +0.00 % |
| ts::ts::predecessors_of_all_states::scrambled_8000 | 288 601 | 288 601 | +0.00 % | +0.00 % |
| ts::ts::reachable_states::scrambled_2000 | 290 091 | 290 091 | +0.00 % | +0.00 % |
| ts::ts::reachable_states::scrambled_8000 | 1 148 696 | 1 148 696 | +0.00 % | +0.00 % |
| ts::ts::remove_edges_from_matching::scrambled_10000 | 566 234 | 566 234 | +0.00 % | +0.00 % |
| ts::ts::remove_edges_from_matching::scrambled_40000 | 896 234 | 896 234 | +0.00 % | +0.00 % |
| ts::ts::remove_state::scrambled_10000 | 1 091 012 | 1 091 012 | +0.00 % | +0.00 % |
| ts::ts::remove_state::scrambled_40000 | 1 421 012 | 1 421 012 | +0.00 % | +0.00 % |
| ts::ts::scc_minimal_representatives::small_cycles_100 | 219 042 | 219 044 | +0.00 % | +0.00 % |
| ts::ts::scc_minimal_representatives::small_cycles_400 | 1 242 475 | 1 242 477 | +0.00 % | +0.00 % |
| ts::ts::subset_construction::nth_from_end_10 | 10 228 206 | 10 228 206 | +0.00 % | +0.00 % |
| ts::ts::subset_construction::nth_from_end_12 | 45 253 025 | 45 253 025 | +0.00 % | +0.00 % |
| ts::ts::subset_construction::nth_from_end_8 | 2 233 506 | 2 233 506 | +0.00 % | +0.00 % |
| ts::ts::successor_index_all_pairs::scrambled_2000 | 108 863 | 108 863 | +0.00 % | +0.00 % |
| ts::ts::successor_index_all_pairs::scrambled_k8_2000 | 750 855 | 750 855 | +0.00 % | +0.00 % |
Criterion means of a single run. They depend on the machine and are not compared; use the instruction counts above for that.
| benchmark | base | head |
|---|---|---|
| collect/clone/100000 | 3.93 ms (3.84 ms … 4.03 ms) | 4.32 ms (4.31 ms … 4.32 ms) |
| collect/clone/1000000 | 44.28 ms (43.85 ms … 44.72 ms) | 38.03 ms (37.59 ms … 38.47 ms) |
| collect/collect_owned/100000 | 7.23 ms (7.19 ms … 7.27 ms) | 8.32 ms (8.17 ms … 8.46 ms) |
| collect/collect_owned/1000000 | 81.41 ms (79.65 ms … 83.17 ms) | 88.83 ms (87.37 ms … 90.30 ms) |
| collect/collect_ref/100000 | 5.21 ms (5.19 ms … 5.22 ms) | 5.27 ms (5.25 ms … 5.29 ms) |
| collect/collect_ref/1000000 | 89.22 ms (88.40 ms … 90.04 ms) | 89.59 ms (87.77 ms … 91.41 ms) |
| collect/map_edge_colors_collect_owned/100000 | 8.02 ms (7.96 ms … 8.07 ms) | 8.79 ms (8.66 ms … 8.92 ms) |
| collect/map_edge_colors_collect_owned/1000000 | 89.84 ms (87.48 ms … 92.21 ms) | 76.33 ms (75.22 ms … 77.43 ms) |
| wallclock/dfa/accepts/word_100000 | 927.76 µs (909.78 µs … 945.75 µs) | 924.26 µs (907.21 µs … 941.30 µs) |
| wallclock/dfa/minimize/redundant_4000 | 1.40 ms (1.39 ms … 1.42 ms) | 1.44 ms (1.44 ms … 1.44 ms) |
| wallclock/dpa/complement/scrambled_4000 | 3.59 ms (3.59 ms … 3.60 ms) | 3.65 ms (3.64 ms … 3.65 ms) |
| wallclock/dpa/normalized/scrambled_4000 | 3.43 ms (3.36 ms … 3.50 ms) | 3.38 ms (3.37 ms … 3.39 ms) |
| wallclock/hoa/parse/ltl | 944.29 µs (940.39 µs … 948.18 µs) | 965.67 µs (948.50 µs … 982.84 µs) |
| wallclock/nfa/determinize/nth_from_end_12 | 3.45 ms (3.43 ms … 3.48 ms) | 3.43 ms (3.42 ms … 3.45 ms) |
| wallclock/nfa/subset_construction/nth_from_end_12 | 8.96 ms (8.78 ms … 9.15 ms) | 9.07 ms (9.03 ms … 9.12 ms) |
| wallclock/scc/tarjan/scrambled_10000 | 2.21 ms (2.19 ms … 2.24 ms) | 2.25 ms (2.21 ms … 2.29 ms) |
| input | reference | ours | reference | ours / reference | size (ours / reference) | same result |
|---|---|---|---|---|---|---|
| hard/l-dsa-3 | spot | 155.36 µs | 116.62 µs | 1.33× | – / – | yes |
| hard/l-nba-2 | spot | 67.62 µs | 35.88 µs | 1.88× | – / – | yes |
| hard/m-nba-3 | spot | 277.88 µs | 136.18 µs | 2.04× | – / – | yes |
| ltl/and-gf-3 | spot | 11.66 µs | 12.56 µs | 0.93× | – / – | yes |
| ltl/eh-patterns-0 | spot | 29.46 µs | 20.82 µs | 1.41× | – / – | yes |
| ltl/gf-equiv-3 | spot | 27.60 µs | 34.32 µs | 0.80× | – / – | yes |
| ltl/gf-implies-3 | spot | 14.44 µs | 19.84 µs | 0.73× | – / – | yes |
| ltl/gh-q-3 | spot | 64.16 µs | 29.86 µs | 2.15× | – / – | yes |
| ltl/gh-r-3 | spot | 30.80 µs | 74.16 µs | 0.42× | – / – | yes |
| ltl/go-theta-3 | spot | 14.46 µs | 21.56 µs | 0.67× | – / – | yes |
| ltl/lily-patterns-1 | spot | 14.92 µs | 15.92 µs | 0.94× | – / – | yes |
| ltl/ms-phi-r-1 | spot | 13.58 µs | 38.44 µs | 0.35× | – / – | yes |
| ltl/ms-phi-s-1 | spot | 13.68 µs | 39.36 µs | 0.35× | – / – | yes |
| ltl/rv-counter-3 | spot | 31.40 µs | 13.04 µs | 2.41× | – / – | yes |
| ltl/sb-patterns-0 | spot | 15.96 µs | 13.94 µs | 1.14× | – / – | yes |
| ltl/sejk-k-2 | spot | 17.78 µs | 57.18 µs | 0.31× | – / – | yes |
| ltl/tv-uu-3 | spot | 48.78 µs | 37.14 µs | 1.31× | – / – | yes |
| ltl/u-left-3 | spot | 13.08 µs | 8.32 µs | 1.57× | – / – | yes |
| many_aps/and-gf-12 | spot | 179.46 µs | 65.04 µs | 2.76× | – / – | yes |
| many_aps/gf-equiv-12 | spot | 163.42 µs | 90.14 µs | 1.81× | – / – | yes |
| many_aps/r-right-10 | spot | 77.32 µs | 26.12 µs | 2.96× | – / – | yes |
| many_aps/random-n16-a8-p4-s1 | spot | 305.98 µs | 127.82 µs | 2.39× | – / – | yes |
| many_aps/random-n24-a12-p4-s1 | spot | 1.29 ms | 431.12 µs | 2.99× | – / – | yes |
| many_aps/random-n32-a16-p4-s1 | spot | 3.97 ms | 1.16 ms | 3.42× | – / – | yes |
| random/n128-a2-p6-s1 | spot | 359.30 µs | 206.70 µs | 1.74× | – / – | yes |
| random/n16-a1-p3-s1 | spot | 24.96 µs | 15.46 µs | 1.61× | – / – | yes |
| random/n256-a2-p8-s1 | spot | 964.19 µs | 473.65 µs | 2.04× | – / – | yes |
| random/n64-a2-p4-s1 | spot | 155.50 µs | 96.04 µs | 1.62× | – / – | yes |
| input | reference | ours | reference | ours / reference | size (ours / reference) | same result |
|---|---|---|---|---|---|---|
| hard/l-dsa-3 | spot | 79.18 µs | 60.80 µs | 1.30× | – / – | yes |
| hard/l-nba-2 | spot | 40.28 µs | 22.80 µs | 1.77× | – / – | yes |
| hard/m-nba-3 | spot | 167.94 µs | 83.94 µs | 2.00× | – / – | yes |
| ltl/and-gf-3 | spot | 7.50 µs | 9.40 µs | 0.80× | – / – | yes |
| ltl/eh-patterns-0 | spot | 18.58 µs | 17.74 µs | 1.05× | – / – | yes |
| ltl/gf-equiv-3 | spot | 15.06 µs | 18.78 µs | 0.80× | – / – | yes |
| ltl/gf-implies-3 | spot | 8.20 µs | 11.52 µs | 0.71× | – / – | yes |
| ltl/gh-q-3 | spot | 39.72 µs | 27.20 µs | 1.46× | – / – | yes |
| ltl/gh-r-3 | spot | 16.00 µs | 34.54 µs | 0.46× | – / – | yes |
| ltl/go-theta-3 | spot | 9.06 µs | 16.18 µs | 0.56× | – / – | yes |
| ltl/lily-patterns-1 | spot | 10.20 µs | 13.84 µs | 0.74× | – / – | yes |
| ltl/ms-phi-r-1 | spot | 8.16 µs | 19.48 µs | 0.42× | – / – | yes |
| ltl/ms-phi-s-1 | spot | 8.06 µs | 24.98 µs | 0.32× | – / – | yes |
| ltl/rv-counter-3 | spot | 20.26 µs | 14.24 µs | 1.42× | – / – | yes |
| ltl/sb-patterns-0 | spot | 10.08 µs | 12.18 µs | 0.83× | – / – | yes |
| ltl/sejk-k-2 | spot | 10.32 µs | 29.12 µs | 0.35× | – / – | yes |
| ltl/tv-uu-3 | spot | 34.30 µs | 30.36 µs | 1.13× | – / – | yes |
| ltl/u-left-3 | spot | 8.78 µs | 9.44 µs | 0.93× | – / – | yes |
| many_aps/and-gf-12 | spot | 151.92 µs | 55.44 µs | 2.74× | – / – | yes |
| many_aps/gf-equiv-12 | spot | 92.20 µs | 51.28 µs | 1.80× | – / – | yes |
| many_aps/r-right-10 | spot | 55.22 µs | 23.36 µs | 2.36× | – / – | yes |
| many_aps/random-n16-a8-p4-s1 | spot | 242.46 µs | 96.88 µs | 2.50× | – / – | yes |
| many_aps/random-n24-a12-p4-s1 | spot | 1.23 ms | 382.54 µs | 3.22× | – / – | yes |
| many_aps/random-n32-a16-p4-s1 | spot | 3.85 ms | 1.09 ms | 3.53× | – / – | yes |
| random/n128-a2-p6-s1 | spot | 213.22 µs | 122.16 µs | 1.75× | – / – | yes |
| random/n16-a1-p3-s1 | spot | 15.32 µs | 10.20 µs | 1.50× | – / – | yes |
| random/n256-a2-p8-s1 | spot | 552.71 µs | 291.52 µs | 1.90× | – / – | yes |
| random/n64-a2-p4-s1 | spot | 92.36 µs | 60.14 µs | 1.54× | – / – | yes |
| input | reference | ours | reference | ours / reference | size (ours / reference) | same result |
|---|---|---|---|---|---|---|
| ladder_10000 | oink-ppp | 2.68 ms | 319.00 µs | 8.41× | 10000 / 10000 | yes |
| ladder_10000 | oink-zlk | 2.68 ms | 320.00 µs | 8.39× | 10000 / 10000 | yes |
| ladder_10000 | spot | 2.68 ms | 9.54 ms | 0.28× | 10000 / 10000 | yes |
| ladder_50000 | oink-ppp | 20.61 ms | 2.24 ms | 9.21× | 50000 / 50000 | yes |
| ladder_50000 | oink-zlk | 20.61 ms | 2.21 ms | 9.31× | 50000 / 50000 | yes |
| ladder_50000 | spot | 20.61 ms | 76.42 ms | 0.27× | 50000 / 50000 | yes |
| random_few_10000 | oink-ppp | 7.86 ms | 1.46 ms | 5.40× | 10000 / 10000 | yes |
| random_few_10000 | oink-zlk | 7.86 ms | 2.71 ms | 2.90× | 10000 / 10000 | yes |
| random_few_10000 | spot | 7.86 ms | 21.12 ms | 0.37× | 10000 / 10000 | yes |
| random_few_50000 | oink-ppp | 79.87 ms | 17.05 ms | 4.69× | 50000 / 50000 | yes |
| random_few_50000 | oink-zlk | 79.87 ms | 18.62 ms | 4.29× | 50000 / 50000 | yes |
| random_few_50000 | spot | 79.87 ms | 705.28 ms | 0.11× | 50000 / 50000 | yes |
| random_many_10000 | oink-ppp | 6.92 ms | 1.53 ms | 4.53× | 10000 / 10000 | yes |
| random_many_10000 | oink-zlk | 6.92 ms | 2.72 ms | 2.54× | 10000 / 10000 | yes |
| random_many_10000 | spot | 6.92 ms | 30.90 ms | 0.22× | 10000 / 10000 | yes |
| random_many_50000 | oink-ppp | 76.28 ms | 30.90 ms | 2.47× | 50000 / 50000 | yes |
| random_many_50000 | oink-zlk | 76.28 ms | 45.66 ms | 1.67× | 50000 / 50000 | yes |
| random_many_50000 | spot | 76.28 ms | 706.70 ms | 0.11× | 50000 / 50000 | yes |
== head f30631e1b439e865b0fb238377b9803d2cd59c27: building ["scc", "ts", "runs", "dfa", "dpa", "storage", "nondeterministic", "learning", "game"] ["wallclock"]
Compiling automata v0.2.1 (/var/lib/automata-bench/work/src-head)
Finished `bench` profile [optimized] target(s) in 2m 06s
Executable benches/dfa.rs (/var/lib/automata-bench/work/target-head/release/deps/dfa-a84ba41f760e70ed)
Executable benches/dpa.rs (/var/lib/automata-bench/work/target-head/release/deps/dpa-23bb829f074058ad)
Executable benches/game.rs (/var/lib/automata-bench/work/target-head/release/deps/game-14422f732ec85029)
Executable benches/learning.rs (/var/lib/automata-bench/work/target-head/release/deps/learning-ee53c00379271c59)
Executable benches/nondeterministic.rs (/var/lib/automata-bench/work/target-head/release/deps/nondeterministic-f164d05108dbce73)
Executable benches/runs.rs (/var/lib/automata-bench/work/target-head/release/deps/runs-85a152aa046a29b4)
Executable benches/scc.rs (/var/lib/automata-bench/work/target-head/release/deps/scc-1b22c641fd9d2d89)
Executable benches/storage.rs (/var/lib/automata-bench/work/target-head/release/deps/storage-f95e3d4a8c506941)
Executable benches/ts.rs (/var/lib/automata-bench/work/target-head/release/deps/ts-4948f6cc6a43d201)
Executable benches/wallclock.rs (/var/lib/automata-bench/work/target-head/release/deps/wallclock-93fb383f608d3bab)
== head: running scc
Finished `bench` profile [optimized] target(s) in 0.23s
Running benches/scc.rs (/var/lib/automata-bench/work/target-head/release/deps/scc-1b22c641fd9d2d89)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running ts
Finished `bench` profile [optimized] target(s) in 0.19s
Running benches/ts.rs (/var/lib/automata-bench/work/target-head/release/deps/ts-4948f6cc6a43d201)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running runs
Finished `bench` profile [optimized] target(s) in 0.20s
Running benches/runs.rs (/var/lib/automata-bench/work/target-head/release/deps/runs-85a152aa046a29b4)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running dfa
Finished `bench` profile [optimized] target(s) in 0.21s
Running benches/dfa.rs (/var/lib/automata-bench/work/target-head/release/deps/dfa-a84ba41f760e70ed)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running dpa
Finished `bench` profile [optimized] target(s) in 0.22s
Running benches/dpa.rs (/var/lib/automata-bench/work/target-head/release/deps/dpa-23bb829f074058ad)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running storage
Finished `bench` profile [optimized] target(s) in 0.22s
Running benches/storage.rs (/var/lib/automata-bench/work/target-head/release/deps/storage-f95e3d4a8c506941)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running nondeterministic
Finished `bench` profile [optimized] target(s) in 0.21s
Running benches/nondeterministic.rs (/var/lib/automata-bench/work/target-head/release/deps/nondeterministic-f164d05108dbce73)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running learning
Finished `bench` profile [optimized] target(s) in 0.20s
Running benches/learning.rs (/var/lib/automata-bench/work/target-head/release/deps/learning-ee53c00379271c59)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running game
Finished `bench` profile [optimized] target(s) in 0.19s
Running benches/game.rs (/var/lib/automata-bench/work/target-head/release/deps/game-14422f732ec85029)
gungraun_runner: Warn : The RUST_LOG environment variable to set the log level changed to GUNGRAUN_LOG
== head: running wallclock (wall clock)
Finished `bench` profile [optimized] target(s) in 0.21s
Running benches/wallclock.rs (/var/lib/automata-bench/work/target-head/release/deps/wallclock-93fb383f608d3bab)
Gnuplot not found, using plotters backend
Benchmarking wallclock/scc/tarjan/scrambled_10000
Benchmarking wallclock/scc/tarjan/scrambled_10000: Analyzing
Benchmarking wallclock/dpa/normalized/scrambled_4000
Benchmarking wallclock/dpa/normalized/scrambled_4000: Analyzing
Benchmarking wallclock/dpa/complement/scrambled_4000
Benchmarking wallclock/dpa/complement/scrambled_4000: Analyzing
Benchmarking wallclock/dfa/minimize/redundant_4000
Benchmarking wallclock/dfa/minimize/redundant_4000: Analyzing
Benchmarking wallclock/dfa/accepts/word_100000
Benchmarking wallclock/dfa/accepts/word_100000: Analyzing
Benchmarking wallclock/nfa/subset_construction/nth_from_end_12
Benchmarking wallclock/nfa/subset_construction/nth_from_end_12: Analyzing
Benchmarking wallclock/nfa/determinize/nth_from_end_12
Benchmarking wallclock/nfa/determinize/nth_from_end_12: Analyzing
Benchmarking wallclock/hoa/parse/ltl
Benchmarking wallclock/hoa/parse/ltl: Analyzing
Benchmarking collect/clone/100000
Benchmarking collect/clone/100000: Analyzing
Benchmarking collect/collect_owned/100000
Benchmarking collect/collect_owned/100000: Analyzing
Benchmarking collect/collect_ref/100000
Benchmarking collect/collect_ref/100000: Analyzing
Benchmarking collect/map_edge_colors_collect_owned/100000
Benchmarking collect/map_edge_colors_collect_owned/100000: Analyzing
Benchmarking collect/clone/1000000
Benchmarking collect/clone/1000000: Analyzing
Benchmarking collect/collect_owned/1000000
Benchmarking collect/collect_owned/1000000: Analyzing
Benchmarking collect/collect_ref/1000000
Benchmarking collect/collect_ref/1000000: Analyzing
Benchmarking collect/map_edge_colors_collect_owned/1000000
Benchmarking collect/map_edge_colors_collect_owned/1000000: Analyzing
== head: building the Python bindings
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings with abi3 support
🐍 Found CPython 3.13 at /nix/store/vilynzly1pyhawy04ljmrp7n9ffr24qy-python3-3.13.17/bin/python3
📡 Using build options features from pyproject.toml
Compiling automata v0.2.1 (/var/lib/automata-bench/work/src-head)
Compiling automata-py v0.1.0 (/var/lib/automata-bench/work/src-head/python)
Finished `release` profile [optimized] target(s) in 1m 50s
📦 Built wheel for abi3 Python ≥ 3.11 to /var/lib/automata-bench/work/versus-head/wheels/automatars-0.1.0-cp311-abi3-manylinux_2_34_aarch64.whl
== head: running benches/versus/compare.py (Spot, Oink)
dpa/equivalence/hard/l-dsa-3 vs spot: 0.155 ms / 0.117 ms
dpa/equivalence/hard/l-nba-2 vs spot: 0.068 ms / 0.036 ms
dpa/equivalence/hard/m-nba-3 vs spot: 0.278 ms / 0.136 ms
dpa/equivalence/ltl/and-gf-3 vs spot: 0.012 ms / 0.013 ms
dpa/equivalence/ltl/eh-patterns-0 vs spot: 0.029 ms / 0.021 ms
dpa/equivalence/ltl/gf-equiv-3 vs spot: 0.028 ms / 0.034 ms
dpa/equivalence/ltl/gf-implies-3 vs spot: 0.014 ms / 0.020 ms
dpa/equivalence/ltl/gh-q-3 vs spot: 0.064 ms / 0.030 ms
dpa/equivalence/ltl/gh-r-3 vs spot: 0.031 ms / 0.074 ms
dpa/equivalence/ltl/go-theta-3 vs spot: 0.014 ms / 0.022 ms
dpa/equivalence/ltl/lily-patterns-1 vs spot: 0.015 ms / 0.016 ms
dpa/equivalence/ltl/ms-phi-r-1 vs spot: 0.014 ms / 0.038 ms
dpa/equivalence/ltl/ms-phi-s-1 vs spot: 0.014 ms / 0.039 ms
dpa/equivalence/ltl/rv-counter-3 vs spot: 0.031 ms / 0.013 ms
dpa/equivalence/ltl/sb-patterns-0 vs spot: 0.016 ms / 0.014 ms
dpa/equivalence/ltl/sejk-k-2 vs spot: 0.018 ms / 0.057 ms
dpa/equivalence/ltl/tv-uu-3 vs spot: 0.049 ms / 0.037 ms
dpa/equivalence/ltl/u-left-3 vs spot: 0.013 ms / 0.008 ms
dpa/equivalence/many_aps/and-gf-12 vs spot: 0.179 ms / 0.065 ms
dpa/equivalence/many_aps/gf-equiv-12 vs spot: 0.163 ms / 0.090 ms
dpa/equivalence/many_aps/r-right-10 vs spot: 0.077 ms / 0.026 ms
dpa/equivalence/many_aps/random-n16-a8-p4-s1 vs spot: 0.306 ms / 0.128 ms
dpa/equivalence/many_aps/random-n24-a12-p4-s1 vs spot: 1.288 ms / 0.431 ms
dpa/equivalence/many_aps/random-n32-a16-p4-s1 vs spot: 3.965 ms / 1.158 ms
dpa/equivalence/random/n128-a2-p6-s1 vs spot: 0.359 ms / 0.207 ms
dpa/equivalence/random/n16-a1-p3-s1 vs spot: 0.025 ms / 0.015 ms
dpa/equivalence/random/n256-a2-p8-s1 vs spot: 0.964 ms / 0.474 ms
dpa/equivalence/random/n64-a2-p4-s1 vs spot: 0.156 ms / 0.096 ms
dpa/inclusion/hard/l-dsa-3 vs spot: 0.079 ms / 0.061 ms
dpa/inclusion/hard/l-nba-2 vs spot: 0.040 ms / 0.023 ms
dpa/inclusion/hard/m-nba-3 vs spot: 0.168 ms / 0.084 ms
dpa/inclusion/ltl/and-gf-3 vs spot: 0.007 ms / 0.009 ms
dpa/inclusion/ltl/eh-patterns-0 vs spot: 0.019 ms / 0.018 ms
dpa/inclusion/ltl/gf-equiv-3 vs spot: 0.015 ms / 0.019 ms
dpa/inclusion/ltl/gf-implies-3 vs spot: 0.008 ms / 0.012 ms
dpa/inclusion/ltl/gh-q-3 vs spot: 0.040 ms / 0.027 ms
dpa/inclusion/ltl/gh-r-3 vs spot: 0.016 ms / 0.035 ms
dpa/inclusion/ltl/go-theta-3 vs spot: 0.009 ms / 0.016 ms
dpa/inclusion/ltl/lily-patterns-1 vs spot: 0.010 ms / 0.014 ms
dpa/inclusion/ltl/ms-phi-r-1 vs spot: 0.008 ms / 0.019 ms
dpa/inclusion/ltl/ms-phi-s-1 vs spot: 0.008 ms / 0.025 ms
dpa/inclusion/ltl/rv-counter-3 vs spot: 0.020 ms / 0.014 ms
dpa/inclusion/ltl/sb-patterns-0 vs spot: 0.010 ms / 0.012 ms
dpa/inclusion/ltl/sejk-k-2 vs spot: 0.010 ms / 0.029 ms
dpa/inclusion/ltl/tv-uu-3 vs spot: 0.034 ms / 0.030 ms
dpa/inclusion/ltl/u-left-3 vs spot: 0.009 ms / 0.009 ms
dpa/inclusion/many_aps/and-gf-12 vs spot: 0.152 ms / 0.055 ms
dpa/inclusion/many_aps/gf-equiv-12 vs spot: 0.092 ms / 0.051 ms
dpa/inclusion/many_aps/r-right-10 vs spot: 0.055 ms / 0.023 ms
dpa/inclusion/many_aps/random-n16-a8-p4-s1 vs spot: 0.242 ms / 0.097 ms
dpa/inclusion/many_aps/random-n24-a12-p4-s1 vs spot: 1.233 ms / 0.383 ms
dpa/inclusion/many_aps/random-n32-a16-p4-s1 vs spot: 3.849 ms / 1.089 ms
dpa/inclusion/random/n128-a2-p6-s1 vs spot: 0.213 ms / 0.122 ms
dpa/inclusion/random/n16-a1-p3-s1 vs spot: 0.015 ms / 0.010 ms
dpa/inclusion/random/n256-a2-p8-s1 vs spot: 0.553 ms / 0.292 ms
dpa/inclusion/random/n64-a2-p4-s1 vs spot: 0.092 ms / 0.060 ms
game/solve/random_few_10000 vs spot: 7.855 ms / 21.118 ms
game/solve/random_few_10000 vs oink-zlk: 7.855 ms / 2.710 ms
game/solve/random_few_10000 vs oink-ppp: 7.855 ms / 1.456 ms
game/solve/random_few_50000 vs spot: 79.865 ms / 705.277 ms
game/solve/random_few_50000 vs oink-zlk: 79.865 ms / 18.625 ms
game/solve/random_few_50000 vs oink-ppp: 79.865 ms / 17.046 ms
game/solve/random_many_10000 vs spot: 6.918 ms / 30.903 ms
game/solve/random_many_10000 vs oink-zlk: 6.918 ms / 2.720 ms
game/solve/random_many_10000 vs oink-ppp: 6.918 ms / 1.526 ms
game/solve/random_many_50000 vs spot: 76.281 ms / 706.696 ms
game/solve/random_many_50000 vs oink-zlk: 76.281 ms / 45.658 ms
game/solve/random_many_50000 vs oink-ppp: 76.281 ms / 30.901 ms
game/solve/ladder_10000 vs spot: 2.684 ms / 9.544 ms
game/solve/ladder_10000 vs oink-zlk: 2.684 ms / 0.320 ms
game/solve/ladder_10000 vs oink-ppp: 2.684 ms / 0.319 ms
game/solve/ladder_50000 vs spot: 20.611 ms / 76.424 ms
game/solve/ladder_50000 vs oink-zlk: 20.611 ms / 2.213 ms
game/solve/ladder_50000 vs oink-ppp: 20.611 ms / 2.239 ms
== head: 266 benchmarks, 16 wall-clock measurements, 74 comparisons