Weird file CI issues with benchmark_effect_size test

[abrown]

On Windows, the benchmark_effect_size test has started intermittently failing since #170. It is unclear to me what could have modified the execution of the code that copies the built engine to a location provided by tempfile::NamedTempFile. I added an assert in hopes that it would trigger the error sooner but the OS always thinks the file exists:

sightglass/crates/cli/tests/tests.rs

Line 213 in cd8c11b

assert!(alt_engine_path.exists());

One example of this failure is here:

thread 'benchmark::benchmark_effect_size' panicked at 'Unexpected failure.
code--1073741819
...
command=`"D:\\a\\sightglass\\sightglass\\target\\debug\\sightglass-cli.exe" "benchmark" "--engine" "\\\\?\\D:\\a\\sightglass\\sightglass\\engines\\wasmtime\\engine.dll" "--engine" "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\.tmpaNVVum" "--processes" "1" "--iterations-per-process" "3" "../../benchmarks-next/noop/benchmark.wasm"`

If this random link is to be believed, the code referenced above is an access violation that might indicate referencing a null pointer, e.g. But why? The original engine library seems to run fine according to the logs; only the copied engine library has the problem.

[abrown]

@fitzgen, should we now consider this closed? If the CI issues with benchmark_effect_size are no longer a problem, maybe my "access violation" hypothesis was incorrect and it was the variance issue you saw?

[fitzgen]

Yeah sure, we can always reopen if we see it again.

[abrown]

I don't think this is resolved. @jlb6740 just triggered it again yesterday (logs). I'm going to disable the test on Windows until we figure out what is going on.