Benchmarks

Here, we collect some simple benchmarks of BSeries.jl. Take them with a grain of salt since they run on virtual machines in the cloud to generate the documentation automatically. You can of course also copy the code and run the benchmarks locally yourself.

Comparing different symbolic packages

Symbolic computations of modified_equations and modifying_integrators in BSeries.jl support

• SymEngine.jl,
• SymPy.jl,
• SymPyPythonCall.jl, and
• Symbolics.jl

as symbolic backends. Here, we compare them in the context of the explicit midpoint method and the nonlinear oscillator ODE

\[u'(t) = \frac{1}{\| u(t) \|^2} \begin{pmatrix} -u_2(t) \\ u_1(t) \end{pmatrix}.\]

This particular combination of explicit Runge-Kutta method and ODE is special since the explicit midpoint method is unconditionally energy-conserving for this problem^{[RanochaKetcheson2020]}.

First, we set up some code to perform the benchmarks. Here, we use a very naive approach, run the code twice (to see the effect of compilation) and use @time to print the runtime. More sophisticated approaches should of course use something like @benchmark from BenchmarkTools.jl. However, this simple and cheap version suffices to compare the orders of magnitude.

using BSeries, StaticArrays

function benchmark(u, dt, subs, order)
  # explicit midpoint method
  A = @SArray [0 0; 1//2 0]
  b = @SArray [0, 1//1]
  c = @SArray [0, 1//2]

  # nonlinear oscillator
  f = [-u[2], u[1]] / (u[1]^2 + u[2]^2)

  println("\n Computing the series coefficients:")
  @time coefficients = modifying_integrator(A, b, c, order)
  @time coefficients = modifying_integrator(A, b, c, order)

  println("\n Computing the series including elementary differentials:")
  @time series = modifying_integrator(f, u, dt, A, b, c, order)
  @time series = modifying_integrator(f, u, dt, A, b, c, order)

  substitution_variables = Dict(u[1] => 1//1, u[2] => 0//1)

  println("\n Substituting the initial condition:")
  @time subs.(series, (substitution_variables, ))
  @time subs.(series, (substitution_variables, ))

  println("\n")
end

benchmark (generic function with 1 method)

Next, we load the symbolic packages and run the benchmarks.

using SymEngine: SymEngine
using SymPyPythonCall: SymPyPythonCall
using Symbolics: Symbolics

println("SymEngine")
dt   = SymEngine.symbols("dt")
u    = SymEngine.symbols("u1, u2")
subs = SymEngine.subs
benchmark(u, dt, subs, 8)

println("SymPy")
dt   = SymPyPythonCall.symbols("dt")
u    = SymPyPythonCall.symbols("u1, u2")
subs = SymPyPythonCall.subs
benchmark(u, dt, subs, 8)

println("Symbolics")
Symbolics.@variables dt
u = Symbolics.@variables u1 u2
subs = Symbolics.substitute
benchmark(u, dt, subs, 8)

SymEngine

 Computing the series coefficients:
  0.004942 seconds (466 allocations: 94.578 KiB)
  0.004966 seconds (466 allocations: 94.578 KiB)

 Computing the series including elementary differentials:
  1.278295 seconds (1.73 M allocations: 83.003 MiB, 4.62% gc time, 98.64% compilation time)
  0.014680 seconds (51.58 k allocations: 1.081 MiB)

 Substituting the initial condition:
  0.086422 seconds (174.48 k allocations: 8.276 MiB, 94.22% compilation time)
  0.004219 seconds (12.74 k allocations: 138.969 KiB)


SymPy

 Computing the series coefficients:
  0.004955 seconds (466 allocations: 94.578 KiB)
  0.004895 seconds (466 allocations: 94.578 KiB)

 Computing the series including elementary differentials:
  2.403410 seconds (2.12 M allocations: 104.485 MiB, 47.09% compilation time)
  0.803602 seconds (30.26 k allocations: 874.445 KiB)

 Substituting the initial condition:
  0.445738 seconds (229.62 k allocations: 11.324 MiB, 38.40% compilation time)
  0.196318 seconds (166 allocations: 3.641 KiB)


Symbolics

 Computing the series coefficients:
  0.004956 seconds (466 allocations: 94.578 KiB)
  0.004889 seconds (466 allocations: 94.578 KiB)

 Computing the series including elementary differentials:
  4.982337 seconds (7.23 M allocations: 385.834 MiB, 1.28% gc time, 90.45% compilation time: <1% of which was recompilation)
  0.475961 seconds (3.67 M allocations: 204.090 MiB, 11.99% gc time)

 Substituting the initial condition:
  6.655223 seconds (23.35 M allocations: 826.171 MiB, 1.59% gc time, 47.13% compilation time)
  3.447596 seconds (20.32 M allocations: 672.479 MiB, 1.78% gc time)

These results were obtained using the following versions.

using InteractiveUtils
versioninfo()

using Pkg
Pkg.status(["BSeries", "RootedTrees", "SymEngine", "SymPyPythonCall", "Symbolics"],
           mode=PKGMODE_MANIFEST)

Julia Version 1.11.3
Commit d63adeda50d (2025-01-21 19:42 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Environment:
  JULIA_PKG_SERVER_REGISTRY_PREFERENCE = eager
  LD_LIBRARY_PATH = /opt/hostedtoolcache/Python/3.9.21/x64/lib
  JULIA_PYTHONCALL_EXE = /home/runner/work/BSeries.jl/BSeries.jl/docs/.CondaPkg/env/bin/python
Status `~/work/BSeries.jl/BSeries.jl/docs/Manifest.toml`
  [ebb8d67c] BSeries v0.1.66 `~/work/BSeries.jl/BSeries.jl`
  [47965b36] RootedTrees v2.23.1
⌃ [123dc426] SymEngine v0.10.0
⌅ [bc8888f7] SymPyPythonCall v0.4.2
⌅ [0c5d862f] Symbolics v5.36.0
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`

Comparison with other packages

There are also other open source packages for B-series. Currently, we are aware of the Python packages

• BSeries
• pybs
• orderconditions

If you know about similar open source packages out there, please inform us, e.g., by creating an issue on GitHub.

The packages listed above and BSeries.jl all use different approaches and have different features. Thus, comparisons must be restricted to their common subset of features. Here, we present some simple performance comparisons. Again, we just use (the equivalent of) @time twice to get an idea of the performance after compilation, allowing us to compare orders of magnitude.

Python package BSeries

First, we start with the Python package BSeries and the following benchmark script.

import sys
from importlib.metadata import version
print("Python version", sys.version)
print("Package version", version('pybs'))

import time
import BSeries.bs as bs
import nodepy.runge_kutta_method as rk

midpoint_method = rk.loadRKM("Mid22")
up_to_order = 9


print("\nModified equation")

start_time = time.time()
series = bs.modified_equation(None, None,
                              midpoint_method.A, midpoint_method.b,
                              up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
series = bs.modified_equation(None, None,
                              midpoint_method.A, midpoint_method.b,
                              up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nModifying integrator")

start_time = time.time()
series = bs.modifying_integrator(None, None,
                                 midpoint_method.A, midpoint_method.b,
                                 up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
series = bs.modifying_integrator(None, None,
                                 midpoint_method.A, midpoint_method.b,
                                 up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

The results are as follows.

Python version 3.9.21 (main, Dec 12 2024, 19:08:08) 
[GCC 13.2.0]
Package version 0.3

Modified equation
19063/26880
 13.971107482910156 seconds
19063/26880
 14.33669376373291 seconds

Modifying integrator
5460293/241920
 12.720726251602173 seconds
5460293/241920
 12.7080397605896 seconds

Python package pybs

Next, we look at the Python package pybs and the following benchmark script. Note that this package does not provide functionality for modifying integrators.

import sys
from importlib.metadata import version
print("Python version", sys.version)
print("Package version", version('pybs'))

import time
import pybs
from pybs.rungekutta import methods as rk_methods

midpoint_method = rk_methods.RKmidpoint
up_to_order = 9
number_of_terms = pybs.unordered_tree.number_of_trees_up_to_order(up_to_order+1)

from itertools import islice
def first_values(f, n):
  return (f(tree) for tree in islice(pybs.unordered_tree.tree_generator(), 0, n))


print("\nModified equation")

start_time = time.time()
midpoint_series = midpoint_method.phi()
series = pybs.series.modified_equation(midpoint_series)
result = sum(first_values(series, number_of_terms))
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
midpoint_series = midpoint_method.phi()
series = pybs.series.modified_equation(midpoint_series)
result = sum(first_values(series, number_of_terms))
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nEnergy preservation")

start_time = time.time()
a = pybs.series.AVF
b = pybs.series.modified_equation(a)
result = pybs.series.energy_preserving_upto_order(b, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
a = pybs.series.AVF
b = pybs.series.modified_equation(a)
result = pybs.series.energy_preserving_upto_order(b, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nSymplecticity (conservation of quadratic invariants)")

start_time = time.time()
a = rk_methods.RKimplicitMidpoint.phi()
result = pybs.series.symplectic_up_to_order(a, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
a = rk_methods.RKimplicitMidpoint.phi()
result = pybs.series.symplectic_up_to_order(a, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

The results are as follows.

Python version 3.9.21 (main, Dec 12 2024, 19:08:08) 
[GCC 13.2.0]
Package version 0.3

Modified equation
19063/26880
 5.728819847106934 seconds
19063/26880
 5.639835834503174 seconds

Energy preservation
9
 5.721026420593262 seconds
9
 5.637234210968018 seconds

Symplecticity (conservation of quadratic invariants)
9
 0.03538799285888672 seconds
9
 0.020270824432373047 seconds

Python package orderconditions

Next, we look at the Python package orderconditions of Valentin Dallerit and the following benchmark script.

import sys
print("Python version", sys.version)

import time
from orderConditions import BSeries
import nodepy.runge_kutta_method as rk

midpoint_method = rk.loadRKM("Mid22")
up_to_order = 9


print("\nModified equation")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modified_equation(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modified_equation(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nModifying integrator")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modifying_integrator(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modifying_integrator(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

The results are as follows.

Python version 3.9.21 (main, Dec 12 2024, 19:08:08) 
[GCC 13.2.0]

Modified equation
19063/26880
 0.959726095199585 seconds
19063/26880
 0.9462010860443115 seconds

Modifying integrator
5460293/241920
 0.8227019309997559 seconds
5460293/241920
 0.828486442565918 seconds

This Julia package BSeries.jl

Finally, we perform the same task using BSeries.jl in Julia.

using BSeries, StaticArrays

A = @SArray [0 0; 1//2 0]
b = @SArray [0, 1//1]
c = @SArray [0, 1//2]
up_to_order = 9


println("Modified equation")
@time begin
  series = modified_equation(A, b, c, up_to_order)
  println(sum(values(series)))
end

@time begin
  series = modified_equation(A, b, c, up_to_order)
  println(sum(values(series)))
end


println("\nModifying integrator")
@time begin
  series = modifying_integrator(A, b, c, up_to_order)
  println(sum(values(series)))
end

@time begin
  series = modifying_integrator(A, b, c, up_to_order)
  println(sum(values(series)))
end


println("\nEnergy preservation")
@time begin
  series = bseries(AverageVectorFieldMethod(), up_to_order)
  println(is_energy_preserving(series))
end

@time begin
  series = bseries(AverageVectorFieldMethod(), up_to_order)
  println(is_energy_preserving(series))
end


println("\nSymplecticity (conservation of quadratic invariants)")
@time begin
  # implicit midpoint method = first Gauss method
  A = @SArray [1//2;;]
  b = @SArray [1//1]
  rk = RungeKuttaMethod(A, b)
  series = bseries(rk, up_to_order)
  println(is_symplectic(series))
end

@time begin
  # implicit midpoint method = first Gauss method
  A = @SArray [1//2;;]
  b = @SArray [1//1]
  rk = RungeKuttaMethod(A, b)
  series = bseries(rk, up_to_order)
  println(is_symplectic(series))
end

Modified equation
19063//26880
  0.139321 seconds (71.38 k allocations: 4.070 MiB, 60.43% compilation time)
19063//26880
  0.054968 seconds (1.12 k allocations: 423.375 KiB)

Modifying integrator
5460293//241920
  0.030743 seconds (1.15 k allocations: 390.266 KiB, 16.53% compilation time)
5460293//241920
  0.025510 seconds (1.09 k allocations: 387.875 KiB)

Energy preservation
true
  2.239186 seconds (4.74 M allocations: 245.130 MiB, 1.07% gc time, 97.23% compilation time)
true
  0.061020 seconds (64.79 k allocations: 4.908 MiB)

Symplecticity (conservation of quadratic invariants)
true
  0.394424 seconds (391.57 k allocations: 19.958 MiB, 99.80% compilation time)
true
  0.000981 seconds (1.83 k allocations: 253.125 KiB)

References

Hendrik Ranocha and David Ketcheson (2020) Energy Stability of Explicit Runge-Kutta Methods for Nonautonomous or Nonlinear Problems. SIAM Journal on Numerical Analysis DOI: 10.1137/19M1290346