7.5. Benchmarking#
Tip
This notebook benchmarks JAX on a single CPU core. Compare with Julia results as reported in ComPWA/polarimetry#27. See also the Extended benchmark #68 discussion.
Note
This notebook uses only one run and one loop for %timeit, because JAX seems to cache its return values.
Physical cores: 2
Total cores: 4
%%time
polarimetry_exprs = formulate_polarimetry(amplitude_builder, reference_subsystem)
unfolded_polarimetry_exprs = [
cached.unfold(expr, model.amplitudes) for expr in polarimetry_exprs
]
unfolded_intensity_expr = cached.unfold(model)
CPU times: user 2.15 s, sys: 2.94 ms, total: 2.15 s
Wall time: 2.15 s
7.5.1. DataTransformer performance#
n_events = 100_000
phsp_sample = generate_phasespace_sample(model.decay, n_events, seed=0)
transformer = create_data_transformer(model)
%timeit -n1 -r1 transformer(phsp_sample) # first run, so no cache and JIT-compilation
%timeit -n1 -r1 transformer(phsp_sample) # second run with cache
%timeit -n1 -r1 transformer(phsp_sample) # third run with cache
phsp_sample = transformer(phsp_sample)
random_point = {k: v[0] if len(v.shape) > 0 else v for k, v in phsp_sample.items()}
302 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
5.61 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
5.87 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
res = 54
grid_sample = generate_meshgrid_sample(model.decay, res)
%timeit -n1 -r1 transformer(grid_sample) # first run, without cache, but already compiled
%timeit -n1 -r1 transformer(grid_sample) # second run with cache
%timeit -n1 -r1 transformer(grid_sample) # third run with cache
grid_sample = transformer(grid_sample)
318 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
369 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
243 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)
7.5.2. Parametrized function#
Total number of mathematical operations:
\(\alpha_x\): 86,766
\(\alpha_y\): 86,770
\(\alpha_z\): 86,766
\(I_\mathrm{tot}\): 28,430
%%time
parametrized_polarimetry_funcs = [
cached.lambdify(expr, model.parameter_defaults)
for expr in unfolded_polarimetry_exprs
]
parametrized_intensity_func = cached.lambdify(
unfolded_intensity_expr, model.parameter_defaults
)
CPU times: user 106 ms, sys: 1.95 ms, total: 108 ms
Wall time: 108 ms
rng = np.random.default_rng(seed=0)
original_parameters = dict(parametrized_intensity_func.parameters)
modified_parameters = {
k: rng.uniform(0.9, 1.1) * v
for k, v in parametrized_intensity_func.parameters.items()
}
7.5.2.1. One data point#
7.5.2.1.1. JIT-compilation#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(random_point)
<TimeitResult : 166 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](random_point)
array = parametrized_polarimetry_funcs[1](random_point)
array = parametrized_polarimetry_funcs[2](random_point)
<TimeitResult : 643 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.1.2. Compiled performance#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(random_point)
<TimeitResult : 796 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](random_point)
array = parametrized_polarimetry_funcs[1](random_point)
array = parametrized_polarimetry_funcs[2](random_point)
<TimeitResult : 1.6 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.2. 54x54 grid sample#
7.5.2.2.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(grid_sample)
<TimeitResult : 175 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](grid_sample)
array = parametrized_polarimetry_funcs[1](grid_sample)
array = parametrized_polarimetry_funcs[2](grid_sample)
<TimeitResult : 683 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.2.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(grid_sample)
<TimeitResult : 985 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](grid_sample)
array = parametrized_polarimetry_funcs[1](grid_sample)
array = parametrized_polarimetry_funcs[2](grid_sample)
<TimeitResult : 1.94 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.3. 100.000 event phase space sample#
7.5.2.3.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 167 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 702 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.3.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 957 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 2.19 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.4. Recompilation after parameter modification#
parametrized_intensity_func.update_parameters(modified_parameters)
for func in parametrized_polarimetry_funcs:
func.update_parameters(modified_parameters)
7.5.2.4.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 199 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 667 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.2.4.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = parametrized_intensity_func(phsp_sample)
<TimeitResult : 732 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = parametrized_polarimetry_funcs[0](phsp_sample)
array = parametrized_polarimetry_funcs[1](phsp_sample)
array = parametrized_polarimetry_funcs[2](phsp_sample)
<TimeitResult : 2.88 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
parametrized_intensity_func.update_parameters(original_parameters)
for func in parametrized_polarimetry_funcs:
func.update_parameters(original_parameters)
7.5.3. All parameters substituted#
subs_polarimetry_exprs = [
cached.xreplace(expr, model.parameter_defaults)
for expr in unfolded_polarimetry_exprs
]
subs_intensity_expr = cached.xreplace(unfolded_intensity_expr, model.parameter_defaults)
Number of mathematical operations after substituting all parameters:
\(\alpha_x\): 31,488
\(\alpha_y\): 31,492
\(\alpha_z\): 31,488
\(I_\mathrm{tot}\): 10,360
%%time
polarimetry_funcs = [cached.lambdify(expr) for expr in subs_polarimetry_exprs]
intensity_func = cached.lambdify(subs_intensity_expr)
CPU times: user 74.8 ms, sys: 990 μs, total: 75.8 ms
Wall time: 75.5 ms
7.5.3.1. One data point#
7.5.3.1.1. JIT-compilation#
%%timeit -n1 -r1 -q -o
array = intensity_func(random_point)
<TimeitResult : 646 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](random_point)
array = polarimetry_funcs[1](random_point)
array = polarimetry_funcs[2](random_point)
<TimeitResult : 3.27 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.1.2. Compiled performance#
%%timeit -n1 -r1 -q -o
array = intensity_func(random_point)
<TimeitResult : 310 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](random_point)
array = polarimetry_funcs[1](random_point)
array = polarimetry_funcs[2](random_point)
<TimeitResult : 429 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.2. 54x54 grid sample#
7.5.3.2.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = intensity_func(grid_sample)
<TimeitResult : 114 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](grid_sample)
array = polarimetry_funcs[1](grid_sample)
array = polarimetry_funcs[2](grid_sample)
<TimeitResult : 446 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.2.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = intensity_func(grid_sample)
<TimeitResult : 129 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](grid_sample)
array = polarimetry_funcs[1](grid_sample)
array = polarimetry_funcs[2](grid_sample)
<TimeitResult : 226 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.3. 100.000 event phase space sample#
7.5.3.3.1. Compiled but uncached#
%%timeit -n1 -r1 -q -o
array = intensity_func(phsp_sample)
<TimeitResult : 98.6 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](phsp_sample)
array = polarimetry_funcs[1](phsp_sample)
array = polarimetry_funcs[2](phsp_sample)
<TimeitResult : 471 ms ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.3.3.2. Second run with cache#
%%timeit -n1 -r1 -q -o
array = intensity_func(phsp_sample)
<TimeitResult : 231 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
%%timeit -n1 -r1 -q -o
array = polarimetry_funcs[0](phsp_sample)
array = polarimetry_funcs[1](phsp_sample)
array = polarimetry_funcs[2](phsp_sample)
<TimeitResult : 251 μs ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)>
7.5.4. Summary#
| parametrized | substituted | |||
|---|---|---|---|---|
| I | ɑ | I | ɑ | |
| random point (compilation) | 166 ms | 643 ms | 646 ms | 3.27 s |
| random point (cached) | 796 μs | 1.6 ms | 310 μs | 429 μs |
| 54x54 grid | 175 ms | 683 ms | 114 ms | 446 ms |
| 54x54 grid (cached) | 985 μs | 1.94 ms | 129 μs | 226 μs |
| 100,000 phsp | 167 ms | 702 ms | 98.6 ms | 471 ms |
| 100,000 phsp (cached) | 957 μs | 2.19 ms | 231 μs | 251 μs |
| modified 100,000 phsp | 199 ms | 667 ms | ||
| modified 100,000 phsp (cached) | 732 μs | 2.88 ms | ||