7.5. Benchmarking#
Tip
This notebook benchmarks JAX on a single CPU core. Compare with Julia results as reported in ComPWA/polarimetry#27. See also the Extended benchmark #68 discussion.
Note
This notebook uses only one run and one loop for %timeit
, because JAX seems to cache its return values.
Physical cores: 2
Total cores: 4
%%time
polarimetry_exprs = formulate_polarimetry(amplitude_builder, reference_subsystem)
unfolded_polarimetry_exprs = [
cached.unfold(expr, model.amplitudes) for expr in polarimetry_exprs
]
unfolded_intensity_expr = cached.unfold(model)
CPU times: user 6.35 s, sys: 13 ms, total: 6.36 s
Wall time: 6.36 s
7.5.1. DataTransformer
performance#
n_events = 100_000
phsp_sample = generate_phasespace_sample(model.decay, n_events, seed=0)
transformer = create_data_transformer(model)
%timeit -n1 -r1 transformer(phsp_sample) # first run, so no cache and JIT-compilation
%timeit -n1 -r1 transformer(phsp_sample) # second run with cache
%timeit -n1 -r1 transformer(phsp_sample) # third run with cache
phsp_sample = transformer(phsp_sample)
random_point = {k: v[0] if len(v.shape) > 0 else v for k, v in phsp_sample.items()}
241 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
894 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
1.11 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
res = 54
grid_sample = generate_meshgrid_sample(model.decay, res)
%timeit -n1 -r1 transformer(grid_sample) # first run, without cache, but already compiled
%timeit -n1 -r1 transformer(grid_sample) # second run with cache
%timeit -n1 -r1 transformer(grid_sample) # third run with cache
grid_sample = transformer(grid_sample)
282 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
339 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
149 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
7.5.2. Parametrized function#
Total number of mathematical operations:
\(\alpha_x\): 133,630
\(\alpha_y\): 133,634
\(\alpha_z\): 133,630
\(I_\mathrm{tot}\): 43,198
%%time
parametrized_polarimetry_funcs = [
cached.lambdify(expr, model.parameter_defaults)
for expr in unfolded_polarimetry_exprs
]
parametrized_intensity_func = cached.lambdify(
unfolded_intensity_expr, model.parameter_defaults
)
CPU times: user 6.92 s, sys: 15.9 ms, total: 6.94 s
Wall time: 6.94 s
rng = np.random.default_rng(seed=0)
original_parameters = dict(parametrized_intensity_func.parameters)
modified_parameters = {
k: rng.uniform(0.9, 1.1) * v
for k, v in parametrized_intensity_func.parameters.items()
}
7.5.2.1. One data point#
7.5.2.1.1. JIT-compilation#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(random_point)
<TimeitResult : 640 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](random_point)
array = parametrized_polarimetry_funcs[1](random_point)
array = parametrized_polarimetry_funcs[2](random_point)
<TimeitResult : 3.42 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.1.2. Compiled performance#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(random_point)
<TimeitResult : 626 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](random_point)
array = parametrized_polarimetry_funcs[1](random_point)
array = parametrized_polarimetry_funcs[2](random_point)
<TimeitResult : 1.43 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.2. 54x54 grid sample#
7.5.2.2.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(grid_sample)
<TimeitResult : 974 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](grid_sample)
array = parametrized_polarimetry_funcs[1](grid_sample)
array = parametrized_polarimetry_funcs[2](grid_sample)
<TimeitResult : 5.86 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.2.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(grid_sample)
<TimeitResult : 699 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](grid_sample)
array = parametrized_polarimetry_funcs[1](grid_sample)
array = parametrized_polarimetry_funcs[2](grid_sample)
<TimeitResult : 1.33 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.3. 100.000 event phase space sample#
7.5.2.3.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 968 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 5.82 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.3.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 646 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 4.42 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.4. Recompilation after parameter modification#
parametrized_intensity_func.update_parameters(modified_parameters)
for func in parametrized_polarimetry_funcs:
func.update_parameters(modified_parameters)
7.5.2.4.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 1.1 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 5.84 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.4.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 613 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 4.41 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
parametrized_intensity_func.update_parameters(original_parameters)
for func in parametrized_polarimetry_funcs:
func.update_parameters(original_parameters)
7.5.3. All parameters substituted#
subs_polarimetry_exprs = [
cached.xreplace(expr, model.parameter_defaults)
for expr in unfolded_polarimetry_exprs
]
subs_intensity_expr = cached.xreplace(unfolded_intensity_expr, model.parameter_defaults)
Number of mathematical operations after substituting all parameters:
\(\alpha_x\): 29,312
\(\alpha_y\): 29,316
\(\alpha_z\): 29,312
\(I_\mathrm{tot}\): 9,544
%%time
polarimetry_funcs = [cached.lambdify(expr) for expr in subs_polarimetry_exprs]
intensity_func = cached.lambdify(subs_intensity_expr)
CPU times: user 3.04 s, sys: 57 ms, total: 3.09 s
Wall time: 3.09 s
7.5.3.1. One data point#
7.5.3.1.1. JIT-compilation#
%%timeit -n1 -r1 -q -o
array = intensity_func(random_point)
<TimeitResult : 412 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](random_point)
array = polarimetry_funcs[1](random_point)
array = polarimetry_funcs[2](random_point)
<TimeitResult : 2.22 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.1.2. Compiled performance#
%%timeit -n1 -r1 -q -o
array = intensity_func(random_point)
<TimeitResult : 231 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](random_point)
array = polarimetry_funcs[1](random_point)
array = polarimetry_funcs[2](random_point)
<TimeitResult : 340 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.2. 54x54 grid sample#
7.5.3.2.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = intensity_func(grid_sample)
<TimeitResult : 659 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](grid_sample)
array = polarimetry_funcs[1](grid_sample)
array = polarimetry_funcs[2](grid_sample)
<TimeitResult : 3.52 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.2.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = intensity_func(grid_sample)
<TimeitResult : 259 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](grid_sample)
array = polarimetry_funcs[1](grid_sample)
array = polarimetry_funcs[2](grid_sample)
<TimeitResult : 399 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.3. 100.000 event phase space sample#
7.5.3.3.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = intensity_func(phsp_sample)
<TimeitResult : 634 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](phsp_sample)
array = polarimetry_funcs[1](phsp_sample)
array = polarimetry_funcs[2](phsp_sample)
<TimeitResult : 3.46 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.3.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = intensity_func(phsp_sample)
<TimeitResult : 113 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](phsp_sample)
array = polarimetry_funcs[1](phsp_sample)
array = polarimetry_funcs[2](phsp_sample)
<TimeitResult : 2.26 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.4. Summary#
parametrized | substituted | |||
---|---|---|---|---|
I | ɑ | I | ɑ | |
random point (compilation) | 640 ms | 3.42 s | 412 ms | 2.22 s |
random point (cached) | 626 μs | 1.43 ms | 231 μs | 340 μs |
54x54 grid | 974 ms | 5.86 s | 659 ms | 3.52 s |
54x54 grid (cached) | 699 μs | 1.33 ms | 259 μs | 399 μs |
100,000 phsp | 968 ms | 5.82 s | 634 ms | 3.46 s |
100,000 phsp (cached) | 646 μs | 4.42 ms | 113 μs | 2.26 ms |
modified 100,000 phsp | 1.1 s | 5.84 s | ||
modified 100,000 phsp (cached) | 613 μs | 4.41 ms |