Depend on the data available the graph could contain
a different structure. This refactor will use one structure
and use `x/yAxisKey` to select the data that will be shown.
This will increase the flexibility when adding features as data can be
calculated/stored along side the original data.
Pull Request: https://projects.blender.org/blender/blender/pulls/157120
Adds an option to run multiple benchmark runs and graph error bars.
`./benchmark.py run --count 3`
`./benchmark.py update --count 3`
Error bars will only be drawn when data is available containing multiple
runs.
Pull Request: https://projects.blender.org/blender/blender/pulls/156638
Adding the actual GPU device name and host CPU to the test results.
For Cycles the CPU isn't that important when using GPUs as it would
mostly perform the scheduling, however when running EEVEE tests the
CPU is more involved.
This PR will extract the correct GPU device names and adds the CPU name
to a separate identifier. The used CPU is then added to the report, when
found in the results and different than the test device.
Pull Request: https://projects.blender.org/blender/blender/pulls/155419
Log scale helps visualize results with very different runtime or memory
usage in a single chart. However big performance changes do not stand
out as much and may be missed.
Linear with outliner clipping keeps the linear display but clips the Y
axis by detecting outliers. This solves the chart being compressed too
much with extreme outliers, but may also hide some lines entirely.
Pull Request: https://projects.blender.org/blender/blender/pulls/152794
Choice is between 1 month, 4 months, 12 months and all time. The default
is 4 months like the release cycle.
The long time range makes the charts harder to read and slower to
display, this helps with both. It also means that results from older
machines are now hidden by default instead of having multiple charts.
Pull Request: https://projects.blender.org/blender/blender/pulls/152794
This solves performance issues with big charts on browsers like Firefox,
has some features that we want to use (log scale), and generally seems
more modern and well maintained.
Pull Request: https://projects.blender.org/blender/blender/pulls/152794
Stored undo step data for position changes in sculpt mode are now
automatically compressed. Compression is run in background threads,
reducing memory consumption during sculpting sessions while adding
little performance overhead.
For testing and benchmarks, memory usage is now available through
`bpy.app.undo_memory_info()`. Undo memory usage is now tracked by the
existing automated benchmark tests. Some changes to the web benchmark
visualization present the data a bit better.
ZSTD compression is run asynchronously in a backround task pool.
Compression is only blocking if the data is requested immediately for
undo/redo.
Co-authored-by: Hans Goudey <hans@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/141310
The general graphing mechanism will create one graph for each output
variable. So it's not limited to time and memory, but that is what the
Cycles tests now output.
These are scripts for benchmarking Blender features on real-world .blend
files. They were originally written for benchmarking Cycles performance, and
were made generic so they can be used for more Blender features.
The benchmarks can be run locally by developers. But the plan is to also run
these as part of continuous integration to track performance over time.
Currently there are tests for Cycles rendering and .blend file loading.
Documentation:
https://wiki.blender.org/wiki/Tools/Tests/Performance
Main features:
* User created configurations to quickly run, re-run and analyze a selected
subset of tests.
* Supports both benchmarking with existing builds, and automatic building of
specified git commits, tags and branches.
* Generate HTML page with bar and line graphs from test results.
* Controlled using simple command line tool.
* For writing tests, convenient abstraction to run a Python function in Blender
with arguments and return value.
Ref T74730
Differential Revision: https://developer.blender.org/D11662
