This refactors things so that eval_phase_two can expose logs even when the script succeeds.
It also enriches traces to be either Logs or Labels, so that we can tell the difference between the two when inspecting the traces.
This is likely even better than what was done for property testing. We
shall revise that one perhaps one day.
Signed-off-by: KtorZ <5680256+KtorZ@users.noreply.github.com>
Going for a terminal plot, for now, as this was the original idea and it is immediately visual. All benchmark points can also be obtained as JSON when redirecting the output, like for tests. So all-in-all, we provide a flexible output which should be useful. Whether it is the best we can do, time (and people/users) will tell.
Signed-off-by: KtorZ <5680256+KtorZ@users.noreply.github.com>
The idea is to get a good sample of measures from running benchmarks
with various sizes, so one can get an idea of how well a function
performs at various sizes.
Given that size can be made arbitrarily large, and that we currently
report all benchmarks, I installed a fibonacci heuristic to gather
data points from 0 to the max size using an increasing stepping.
Defined as a trait as I already anticipate we might need different
sizing strategy, likely driven by the user via a command-line option;
but for now, this will do.
Signed-off-by: KtorZ <5680256+KtorZ@users.noreply.github.com>
This commit removes some duplication between bench and test runners,
as well as fixing the results coming out of running benchmarks.
Running benchmarks is expected to yield multiple measures, for each of
the iteration. For now, it'll suffice to show results for each size;
but eventually, we'll possibly try to interpolate results with
different curves and pick the best candidate.
Signed-off-by: KtorZ <5680256+KtorZ@users.noreply.github.com>
In particular, using a concrete enum instead of a string to avoid an
unnecessary incomplete pattern-match, and remove superfluous comments.
Signed-off-by: KtorZ <5680256+KtorZ@users.noreply.github.com>
This is to avoid pruning a definition which may end up needed later
on. The issue can be seen when definition to a Pair is used *before*
another Map definitions that uses this same Pair.
Before this commit, the Map definition would simply remove the
definition generated for the Pair, since it would be pointless (and it
is a lot easier to generate those pointless definition than trying to
remember we are currently generating definition for a Map).
So now, we defer the removal of the orphan definition to after all
defnitions have been generated by basically looking at a dependency
graph. I _could have_ used pet-graph on this to solve it similar to
how we do package dependencies; but given that we only really need to
that for pairs, the problem is relatively simple to solve (though
cumbersome since we need to traverse all defintions).
Fixes#1086.
This allows conditions like ```expect x == 1``` to match performance with ```x == 1 && ...```
Also change builtins forcing to accommodate the new case-constr apply optimization
* Fix: Deeply nested assignments would offset the new columns count calculation. Now we track relevant columns and their path to ensure each row has wildcards if they don't contain the relevant column
* Add test plus clippy fix
* Clippy fix
* New version clippy fix
1. A new option `show_json_schema` which, when enabled, will print the JSON schema of the command output if the target isn't an ANSI-capable terminal.
2. Some modifications to the help message and error handling for the new option `show_json_schema`.
This is now done to avoid flooding the help screen with an entire
JSON schema. Plus, it makes the schema more easily exportable as an
actual JSON schema.
* Refactor and structuring optimizations to be less computationally heavy
* Forgot to commit the new file containing the optimization do over
* Point to correct functions in shrinker2
* Split out inline_constr_ops since it adds in builtins that can then be swept up by the builtin force reduction
* Fix: issue where identity reducer was always returning true
* Forward inlining on lambdas produces better results. This is due to a forward pass being able to apply an argument that may have no_inline at the top where as vice-versa would reduce the arg first.
* Clippy and test fixes
* Clear no_inlines when inlining a function
* Convert shrinker2 to replace shrinker and update tests
KtorZ