Benchmarks

Here, we collect some simple benchmarks of BSeries.jl. Take them with a grain of salt since they run on virtual machines in the cloud to generate the documentation automatically. You can of course also copy the code and run the benchmarks locally yourself.

Comparing different symbolic packages

Symbolic computations of modified_equations and modifying_integrators in BSeries.jl support

• SymEngine.jl,
• SymPy.jl,
• SymPyPythonCall.jl, and
• Symbolics.jl

as symbolic backends. Here, we compare them in the context of the explicit midpoint method and the nonlinear oscillator ODE

\[u'(t) = \frac{1}{\| u(t) \|^2} \begin{pmatrix} -u_2(t) \\ u_1(t) \end{pmatrix}.\]

This particular combination of explicit Runge-Kutta method and ODE is special since the explicit midpoint method is unconditionally energy-conserving for this problem^{[RanochaKetcheson2020]}.

First, we set up some code to perform the benchmarks. Here, we use a very naive approach, run the code twice (to see the effect of compilation) and use @time to print the runtime. More sophisticated approaches should of course use something like @benchmark from BenchmarkTools.jl. However, this simple and cheap version suffices to compare the orders of magnitude.

using BSeries, StaticArrays

function benchmark(u, dt, subs, order)
  # explicit midpoint method
  A = @SArray [0 0; 1//2 0]
  b = @SArray [0, 1//1]
  c = @SArray [0, 1//2]

  # nonlinear oscillator
  f = [-u[2], u[1]] / (u[1]^2 + u[2]^2)

  println("\n Computing the series coefficients:")
  @time coefficients = modifying_integrator(A, b, c, order)
  @time coefficients = modifying_integrator(A, b, c, order)

  println("\n Computing the series including elementary differentials:")
  @time series = modifying_integrator(f, u, dt, A, b, c, order)
  @time series = modifying_integrator(f, u, dt, A, b, c, order)

  substitution_variables = Dict(u[1] => 1//1, u[2] => 0//1)

  println("\n Substituting the initial condition:")
  @time subs.(series, (substitution_variables, ))
  @time subs.(series, (substitution_variables, ))

  println("\n")
end

benchmark (generic function with 1 method)

Next, we load the symbolic packages and run the benchmarks.

using SymEngine: SymEngine
using SymPyPythonCall: SymPyPythonCall
using Symbolics: Symbolics

println("SymEngine")
dt   = SymEngine.symbols("dt")
u    = SymEngine.symbols("u1, u2")
subs = SymEngine.subs
benchmark(u, dt, subs, 8)

println("SymPy")
dt   = SymPyPythonCall.symbols("dt")
u    = SymPyPythonCall.symbols("u1, u2")
subs = SymPyPythonCall.subs
benchmark(u, dt, subs, 8)

println("Symbolics")
Symbolics.@variables dt
u = Symbolics.@variables u1 u2
subs = Symbolics.substitute
benchmark(u, dt, subs, 8)

SymEngine

 Computing the series coefficients:
  0.004911 seconds (466 allocations: 94.656 KiB)
  0.004898 seconds (466 allocations: 94.656 KiB)

 Computing the series including elementary differentials:
  1.248972 seconds (1.64 M allocations: 79.623 MiB, 2.56% gc time, 98.55% compilation time)
  0.013263 seconds (51.44 k allocations: 1.354 MiB)

 Substituting the initial condition:
  0.090196 seconds (172.93 k allocations: 8.437 MiB, 94.58% compilation time)
  0.003940 seconds (10.24 k allocations: 233.906 KiB)


SymPy

 Computing the series coefficients:
  0.004932 seconds (466 allocations: 94.656 KiB)
  0.004850 seconds (466 allocations: 94.656 KiB)

 Computing the series including elementary differentials:
  2.411383 seconds (2.11 M allocations: 104.490 MiB, 1.04% gc time, 49.99% compilation time)
  0.782070 seconds (30.26 k allocations: 874.562 KiB)

 Substituting the initial condition:
  0.409585 seconds (227.21 k allocations: 11.253 MiB, 42.52% compilation time)
  0.165799 seconds (166 allocations: 3.641 KiB)


Symbolics

 Computing the series coefficients:
  0.004975 seconds (466 allocations: 94.656 KiB)
  0.004901 seconds (466 allocations: 94.656 KiB)

 Computing the series including elementary differentials:
  4.028159 seconds (8.27 M allocations: 347.955 MiB, 2.69% gc time, 89.20% compilation time)
  0.326922 seconds (3.52 M allocations: 105.000 MiB, 1.74% compilation time)

 Substituting the initial condition:
  4.956648 seconds (20.52 M allocations: 703.849 MiB, 6.51% gc time, 73.72% compilation time: <1% of which was recompilation)
  1.263813 seconds (17.02 M allocations: 527.158 MiB, 5.34% gc time, 0.05% compilation time)

These results were obtained using the following versions.

using InteractiveUtils
versioninfo()

using Pkg
Pkg.status(["BSeries", "RootedTrees", "SymEngine", "SymPyPythonCall", "Symbolics"],
           mode=PKGMODE_MANIFEST)

Julia Version 1.11.6
Commit 9615af0f269 (2025-07-09 12:58 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Environment:
  JULIA_PKG_SERVER_REGISTRY_PREFERENCE = eager
  LD_LIBRARY_PATH = /opt/hostedtoolcache/Python/3.9.23/x64/lib
  JULIA_PYTHONCALL_EXE = /home/runner/work/BSeries.jl/BSeries.jl/docs/.CondaPkg/.pixi/envs/default/bin/python
Status `~/work/BSeries.jl/BSeries.jl/docs/Manifest.toml`
  [ebb8d67c] BSeries v0.1.67 `~/work/BSeries.jl/BSeries.jl`
  [47965b36] RootedTrees v2.23.1
  [123dc426] SymEngine v0.13.0
  [bc8888f7] SymPyPythonCall v0.5.1
  [0c5d862f] Symbolics v6.52.0

Comparison with other packages

There are also other open source packages for B-series. Currently, we are aware of the Python packages

• BSeries
• pybs
• orderconditions

If you know about similar open source packages out there, please inform us, e.g., by creating an issue on GitHub.

The packages listed above and BSeries.jl all use different approaches and have different features. Thus, comparisons must be restricted to their common subset of features. Here, we present some simple performance comparisons. Again, we just use (the equivalent of) @time twice to get an idea of the performance after compilation, allowing us to compare orders of magnitude.

Python package BSeries

First, we start with the Python package BSeries and the following benchmark script.

import sys
from importlib.metadata import version
print("Python version", sys.version)
print("Package version", version('pybs'))

import time
import BSeries.bs as bs
import nodepy.runge_kutta_method as rk

midpoint_method = rk.loadRKM("Mid22")
up_to_order = 9


print("\nModified equation")

start_time = time.time()
series = bs.modified_equation(None, None,
                              midpoint_method.A, midpoint_method.b,
                              up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
series = bs.modified_equation(None, None,
                              midpoint_method.A, midpoint_method.b,
                              up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nModifying integrator")

start_time = time.time()
series = bs.modifying_integrator(None, None,
                                 midpoint_method.A, midpoint_method.b,
                                 up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
series = bs.modifying_integrator(None, None,
                                 midpoint_method.A, midpoint_method.b,
                                 up_to_order, True)
result = sum(series.values())
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

The results are as follows.

Python version 3.9.23 (main, Jun  4 2025, 04:11:23) 
[GCC 13.3.0]
Package version 0.3

Modified equation
19063/26880
 16.175946474075317 seconds
19063/26880
 16.577430486679077 seconds

Modifying integrator
5460293/241920
 12.862091302871704 seconds
5460293/241920
 13.1637282371521 seconds

Python package pybs

Next, we look at the Python package pybs and the following benchmark script. Note that this package does not provide functionality for modifying integrators.

import sys
from importlib.metadata import version
print("Python version", sys.version)
print("Package version", version('pybs'))

import time
import pybs
from pybs.rungekutta import methods as rk_methods

midpoint_method = rk_methods.RKmidpoint
up_to_order = 9
number_of_terms = pybs.unordered_tree.number_of_trees_up_to_order(up_to_order+1)

from itertools import islice
def first_values(f, n):
  return (f(tree) for tree in islice(pybs.unordered_tree.tree_generator(), 0, n))


print("\nModified equation")

start_time = time.time()
midpoint_series = midpoint_method.phi()
series = pybs.series.modified_equation(midpoint_series)
result = sum(first_values(series, number_of_terms))
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
midpoint_series = midpoint_method.phi()
series = pybs.series.modified_equation(midpoint_series)
result = sum(first_values(series, number_of_terms))
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nEnergy preservation")

start_time = time.time()
a = pybs.series.AVF
b = pybs.series.modified_equation(a)
result = pybs.series.energy_preserving_upto_order(b, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
a = pybs.series.AVF
b = pybs.series.modified_equation(a)
result = pybs.series.energy_preserving_upto_order(b, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nSymplecticity (conservation of quadratic invariants)")

start_time = time.time()
a = rk_methods.RKimplicitMidpoint.phi()
result = pybs.series.symplectic_up_to_order(a, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
a = rk_methods.RKimplicitMidpoint.phi()
result = pybs.series.symplectic_up_to_order(a, up_to_order)
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

The results are as follows.

Python version 3.9.23 (main, Jun  4 2025, 04:11:23) 
[GCC 13.3.0]
Package version 0.3

Modified equation
19063/26880
 5.7237019538879395 seconds
19063/26880
 5.691795349121094 seconds

Energy preservation
9
 5.767525911331177 seconds
9
 5.646322727203369 seconds

Symplecticity (conservation of quadratic invariants)
9
 0.03461956977844238 seconds
9
 0.01937699317932129 seconds

Python package orderconditions

Next, we look at the Python package orderconditions of Valentin Dallerit and the following benchmark script.

import sys
print("Python version", sys.version)

import time
from orderConditions import BSeries
import nodepy.runge_kutta_method as rk

midpoint_method = rk.loadRKM("Mid22")
up_to_order = 9


print("\nModified equation")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modified_equation(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modified_equation(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")


print("\nModifying integrator")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modifying_integrator(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

start_time = time.time()
BSeries.set_order(up_to_order)
Y1 = BSeries.y() + midpoint_method.A[1,0] * BSeries.hf()
rk2 = BSeries.y() + midpoint_method.b[0] * BSeries.hf() + midpoint_method.b[1] * BSeries.compo_hf(Y1)
series = BSeries.modifying_integrator(rk2)
result = series.sum()
end_time = time.time()
print(result)
print("", end_time - start_time, "seconds")

The results are as follows.

Python version 3.9.23 (main, Jun  4 2025, 04:11:23) 
[GCC 13.3.0]

Modified equation
19063/26880
 1.0970251560211182 seconds
19063/26880
 1.0744047164916992 seconds

Modifying integrator
5460293/241920
 0.9320647716522217 seconds
5460293/241920
 0.9344227313995361 seconds

This Julia package BSeries.jl

Finally, we perform the same task using BSeries.jl in Julia.

using BSeries, StaticArrays

A = @SArray [0 0; 1//2 0]
b = @SArray [0, 1//1]
c = @SArray [0, 1//2]
up_to_order = 9


println("Modified equation")
@time begin
  series = modified_equation(A, b, c, up_to_order)
  println(sum(values(series)))
end

@time begin
  series = modified_equation(A, b, c, up_to_order)
  println(sum(values(series)))
end


println("\nModifying integrator")
@time begin
  series = modifying_integrator(A, b, c, up_to_order)
  println(sum(values(series)))
end

@time begin
  series = modifying_integrator(A, b, c, up_to_order)
  println(sum(values(series)))
end


println("\nEnergy preservation")
@time begin
  series = bseries(AverageVectorFieldMethod(), up_to_order)
  println(is_energy_preserving(series))
end

@time begin
  series = bseries(AverageVectorFieldMethod(), up_to_order)
  println(is_energy_preserving(series))
end


println("\nSymplecticity (conservation of quadratic invariants)")
@time begin
  # implicit midpoint method = first Gauss method
  A = @SArray [1//2;;]
  b = @SArray [1//1]
  rk = RungeKuttaMethod(A, b)
  series = bseries(rk, up_to_order)
  println(is_symplectic(series))
end

@time begin
  # implicit midpoint method = first Gauss method
  A = @SArray [1//2;;]
  b = @SArray [1//1]
  rk = RungeKuttaMethod(A, b)
  series = bseries(rk, up_to_order)
  println(is_symplectic(series))
end

Modified equation
19063//26880
  0.136900 seconds (71.57 k allocations: 4.097 MiB, 61.66% compilation time)
19063//26880
  0.052298 seconds (1.12 k allocations: 423.492 KiB)

Modifying integrator
5460293//241920
  0.030059 seconds (1.15 k allocations: 390.344 KiB, 17.44% compilation time)
5460293//241920
  0.025000 seconds (1.09 k allocations: 387.953 KiB)

Energy preservation
true
  2.252625 seconds (4.61 M allocations: 239.036 MiB, 2.71% gc time, 97.43% compilation time)
true
  0.056894 seconds (64.79 k allocations: 4.904 MiB)

Symplecticity (conservation of quadratic invariants)
true
  0.398460 seconds (388.64 k allocations: 19.832 MiB, 99.79% compilation time)
true
  0.000754 seconds (1.83 k allocations: 253.164 KiB)

References

Hendrik Ranocha and David Ketcheson (2020) Energy Stability of Explicit Runge-Kutta Methods for Nonautonomous or Nonlinear Problems. SIAM Journal on Numerical Analysis DOI: 10.1137/19M1290346