While testing driver stack changes while moving our AMD GPU workers
from Rocky Linux 8 to Ubuntu 24.04 these tests came up as errors. Some
of these errors could be attributed to hardware differences (W7600 vs.
W7800).
Pull Request: https://projects.blender.org/blender/blender/pulls/163975
A new command line option `--debug-gpu-backend-no-fallback` is added
that skips the GPU backend support checks. This is primarily useful for
the GPU tests that should not fall back to another backend. This has the
useful side effect of skipping `vk_instance_create_for_platform_checks`,
which would otherwise create its own Vulkan instance. `vkCreateInstance`
can be an expensive function call depending on the used GPU driver (see
https://projects.blender.org/infrastructure/meta/issues/231).
Pull Request: https://projects.blender.org/blender/blender/pulls/163203
Implement optional hardware raytraced shadows for the Workbench engine.
They can be enabled in the `Preferences > Viewport > Quality` settings.
The option will be shown as inactive if the current backend/GPU doesn't
support hardware ray queries (only Vulkan is supported ATM).
A new `gpu.capabilities.ray_query_support_get()` has been added to the
Python API to support this UI functionality.
The visual results are the same as the stencil shadows implementation,
but it provides a significant performance boost on modern hardware.
The raytracing functionality has been implemented in a fragment shader
that mimics the output of stencil-based implementation, so the render
pipeline stays mostly the same.
Co-authored by: Jeroen Bakker
Pull Request: https://projects.blender.org/blender/blender/pulls/161047
As the vulkan backend is now default, `get_gpu_device_vendor()` returns
the selected GPU for Vulkan. In the rare case that OpenGL uses a
different GPU, this causes render tests to use incorrect tolerances and
possibly fail. The PR adds the GPU backend argument to
`get_gpu_device_vendor()` to avoid this\*.
> \* this = confusing me
Pull Request: https://projects.blender.org/blender/blender/pulls/161255
Platform can get unstable. The artifacts that were detected seemed
like invalid synchronization of index buffers, or incorrect execution
of vertex buffers. This could be related to the combination of AMD
official drivers and Rocky8. More investigation is needed, but for
now we just disable these tests.
Pull Request: https://projects.blender.org/blender/blender/pulls/160204
Initially noted by devops as a problem: UI tests when crashing
spawn our crash dialog, that will just sit there for 1200 seconds
until the CI environment decides the test has failed and kills the
process, clicking away the dialog also works, but neither option is
ideal here.
The crash handler knows when we are in background mode, (`-B`) and
suppresses this dialog so this is why it has not been an issue for the
normal tests. However when we do crash we get an unhelpful message
saying `Writing: blender.crash.txt` which is not collected by buildbot
so unless a developer can get a devops person to go retrieve this file
its contents will be left to ones imagination.
This PR adds a `--console-crash-handler` argument that does two things:
1 - Suppress the crash dialog even when we are not in background mode
2 - Rather than writing the crash data to blender.crash.txt write this
information to stderr so it shows up in the CI logs.
It also updates all invocations of blender I could find in our test
scripts to pass this new flag. The benchmark scripts have not been
updated as they regularly run against older blender versions that may
not support the new flag.
Pull Request: https://projects.blender.org/blender/blender/pulls/159983
This will be replaced by the texture cache. Advanced OpenImageIO features
will no longer be available, and texture filtering results will be different.
But performance will be better and there will be consistency with SVM.
OSL specifc image tests were removed as these now match SVM exactly and are
tested by WITH_CYCLES_TEST_OSL.
Pull Request: https://projects.blender.org/blender/blender/pulls/154913
Part of #148449.
Add a Workbench-specific folder for testing Workbench-specific features.
Generate a different list of tests in CMake, removing tests without a
meaningful translation to Workbench.
Removes attributes, bsdf, displacement, integrator, light,
light_linking, node_inlining, principled_bsdf, raycast, shader, shadow,
and sss.
Some features (shadows, outlines, curvature) don't have specific tests
and are instead enabled/disabled across the added tests.
This has the advantage of testing that they work correctly in
combination with other features without increasing the number of tests.
This also adds tests for viewport-specific volume features
(interpolation, density, slices).
These tests are added to the openvdb folder since they require
`WITH_OPENVDB` to work.
They could be disabled on the Python side for other engines, or moved
to a workbench_openvdb folder if necessary.
However, since they use very low-resolution volumes, it may be good to
keep them to test interpolation in EEVEE and Cycles, even if they're a
bit redundant.
Pull Request: https://projects.blender.org/blender/blender/pulls/153558
The raycast node disables self intersection when the ray start is the
surface position.
However, this fails when evaluating bump.
This commit adds an extra offset to the ray start (tmin) when
evaluating bump.
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/154222
This PR enables strip curve drawing when performing the workbench
rendertests. On Intel/vulkan the lines are to far off. Using strip will
reduce platform differences. Downside is that (basic) line rendering is not
covered anymore by a render test.
Pull Request: https://projects.blender.org/blender/blender/pulls/146820
The image log render tests uses an EXR that contains INF. Workbench
with specular enabled will mix these INFs, but there is a difference
in behavior between OpenGL and Vulkan.
In OpenGL mix(0.05, INF, 0.0) will result in INF
In Vulkan this results in NaN.
This should eventually be solved in the engine to ensure consistency.
For now we disable the render test and document the limitation.
Pull Request: https://projects.blender.org/blender/blender/pulls/146648
Due to an incorrect assumption float buffers were converted to sRGB
values when uploading to an sRGBA8 texture. This is done when rendering
flames in workbench and resulted in to bright renders.
This PR removes sRGB encoding when uploading float values to sRGBA8 textures.
Fixes:
- render/openvdb/fire
- render/openvdb/principled_blackbody
- render/openvdb/smoke_fire
Pull Request: https://projects.blender.org/blender/blender/pulls/146636
This changes the engine identifier back to `BLENDER_EEVEE`.
We keep the `BLENDER_EEVEE_NEXT` identifier around for
versioning reasons (have to detect when it is the active
engine of a older file).
This also rename a bunch of pannels that were using `next`
in their name.
This is a breaking change for Addons compatibility.
Pull Request: https://projects.blender.org/blender/blender/pulls/140282
This was introduced during EEVEE-next developement
cycle to not make the buildbot fail because of EEVEE
render tests.
These have stabilized now and we can remove this option.
Pull Request: https://projects.blender.org/blender/blender/pulls/137545
This commit reworks the RenderReport base class to avoid adding
`--cycles-device` device arguments to non Cycles tests.
This reduces some warnings that can show up with EEVEE and
Workbench tests that accidentally used these arguments.
Pull Request: https://projects.blender.org/blender/blender/pulls/133724
Some changes to how argparse is used in render tests:
1. Use the common approach of one dash for single-letter options (`-b`)
and two dashes for longer options (`--blender`). In this commit that
just means changing single-dashed (`-testdir`) to double-dashed
(`--testdir`).
2. Remove unnecessary `nargs` arguments. The code was telling `argparse`
to put CLI arguments into a list of one item, and then had code to
turn that one-item list into the item itself. I've just removed the
`nargs` argument altogether, as that just produces the desired
value without requiring more code.
I've also removed `nargs="+"` from the handling of the `--blender`
parameter, as that allowed for multiple occurrences of `--blender
{path}` but was silently ignoring all of those except the first.
To ensure that required arguments are present, the code now uses
`required=True` instead of `nargs`.
3. Add a `description` parameter so that `--help` shows what the
test script actually does. Also it helps people (like me) who want
to figure out which blend file is actually being opened by the
test, without making the test itself more verbose.
No functional changes, except that you now cannot add multiple
`--blender` arguments any more (the CLI invocation will fail). This wasn't
used anywhere I could find, though.
Pull Request: https://projects.blender.org/blender/blender/pulls/131666
This PR enabled backend specific rendertest for EEVEE and Workbench.
Some changes that have been made are:
- Add suffix to the test identifying the backend (_opengl, _vulkan, _metal)
- Vulkan render tests are compared with the opengl results.
Most EEVEE tests run as expected there are some issues in the Vulkan
backend that needs to be addressed:
- Fully smooth reflective materials miss lighting.
- Tangent normals are off
None of the workbench tests pass. It has to do with downloading the depth
buffer. In Workbench they are stored as GPU_DEPTH32F_STENCIL8 and downloaded
as FLOAT. We didn't implement it in the vulkan backend yet and currently asserts.
The Vulkan render test run faster compared to OpenGL. On my system around
25-50% faster.
Pull Request: https://projects.blender.org/blender/blender/pulls/126784
Add silently fail option to GPU based render tests. This is a pre-requisite to enable
render tests on the buildbot. By default these render tests will pass silently.
* Test will pass when using the `--pass-silently` arguments.
* Only crashes will be reported as failed tests.
* To find out failing test, review the test reports.
`WITH_GPU_RENDER_TESTS_SILENT` compile option can be used to let tests pass (default)
or fail (default for developers).
Although some tests fail, they still passed. In the generated render report,
the silently passed failures are correctly reported to be failures.
Pull Request: https://projects.blender.org/blender/blender/pulls/117629
This change fixes confusion situation when the render output
is an RGBA image: the difference in color was not visible in
the report because alpha channel was all zeros. This is due
to idiff performing per-channel difference.
The solution to this problem is to have separate images for
color and alpha difference, which makes it clear where the
difference actually is coming from.
Some tests like cycles, sequencer and compositor batch together multiple
tests in a single Blender invocation. This makes them run faster, but
makes debugging harder. This is an option to disable that batching.
Pull Request: https://projects.blender.org/blender/blender/pulls/114603
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
When running the render test cases on MacOS/Intel the hair render
test fail. Most likely due to the dense geometry and the low
resolution of the test image.
This patch increases the fail threshold so these tests will pass.
Note that I haven't been able to test whether this is also the case
for Linux/Windows. If that is the case we should remove the platform
specific test.
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
CYCLES_TEST_DEVICES is a list of devices (CPU, CUDA, OPTIX, OPENCL). It is set
to CPU only by default.
Test output is now writen to build/tests/cycles/<device>, and the HTML report
has separate report pages for the different devices, with option to compare
between CPU and GPU renders.
Various GPU tests are still failing due to CPU/GPU differences, these are to be
fixed or blacklisted still.
Ref T82193
This adds a new `--debug-exit-on-error` flag. When it is set, Blender
will abort with a non-zero exit code when there are internal errors.
Currently, "internal errors" includes memory leaks detected by
guardedalloc and error/fatal log entries in clog.
The new flag is passed to Blender in various places where automated
tests are run. Furthermore, the `--debug-memory` flag is used in tests,
because that makes the verbose output more useful, when dealing
with memory leaks.
Reviewers: brecht, sergey
Differential Revision: https://developer.blender.org/D8665
CentOS on the buildbot still runs Python 3.6, which is also used for the
unit tests. This means that the tests can't use language features that
are available to Blender itself. And testing with a different version of
Python than will be used by the actual code seems like a bad idea to me.
This commit adds `TEST_PYTHON_EXECUTABLE` as advanced CMake option. This
will allow us to set a specific Python executable when we need it. When
not set, a platform-specific default will be used:
- On Windows, the `python….exe` from the installation directory. This is
just like before this patch, except that this patch adds the
overridability.
- On macOS/Linux, the `${PYTHON_EXECUTABLE}` as found by CMake.
Every platform should now have a value (configured by the user or
detected by CMake) for `TEST_PYTHON_EXE`, so there is no need to allow
running without. This also removes the need to have some Python files
marked as executable.
If `TEST_PYTHON_EXE` is not user-configured, and thus the above default
is used, a status message is logged by CMake. I've seen this a lot in
other projects, and I like that it shows which values are auto-detected.
However, it's not common in Blender, so if we want we can either remove
it now, or remove it after the buildbot has been set up correctly.
Differential Revision: https://developer.blender.org/D7395
Reviewed by: campbellbarton, mont29, sergey
Blender startup time and shader compilation is a big factor when running
hundreds of tests, so now all renders in the same ctest run in the same
process.
This was previously reverted due to skipping other tests when one test
crashed. Now if a test crashes, Blender is re-run with the remaining
tests so we get results from them still.
Blender startup time and shader compilation is a big factor when running
hundreds of tests, so now all renders in the same ctest run in the same
process. If a test crashes, the remaining tests in the same category will
be marked as skipped.
Benchmarked on a quad core with ctest -j8.
cycles: 118.1s -> 94.3s
eevee: 66.2s -> 29.2s
workbench: 31.7s -> 8.6s
Being able to compare Eevee reference images is useful for refactoring I'm
working on so might as well add them now, even if we can still improve them.
Workbench tests are just rendering the same files as Cycles and Eevee. This
doesn't really tests many workbench settings until we add tests specifically
for them, but does cover how it it handles the different object types.
