Mainly just boring renaming.
For the cases where there were both C and C++ headers with the same
name, the C one was either:
- Renamed to `_c.hh`.
- Merged into the existing `.hh` header.
Pull Request: https://projects.blender.org/blender/blender/pulls/160019
Replace C macros with C++ attributes where possible.
Note that these don't work in all situations
(such as ternary operators `do {...} while (..)`),
so the LIKELY/UNLIKELY have to be kept.
Ref !159578
Adds instrumentation for our memory allocation library, note that this
includes the alignment as well as the extra padding of `MemHead` and
`MemTail`, not only the requested size from calling code.
Additionally, actual calls to the "public" `MEM_` guarded and lockfree
implementation functions are also timed with `PRF_scope` to provide
insight on time spent allocating / copying / freeing memory. This was
done instead of adding these calls to the main `MEM_guardedalloc.h`
interface functions to avoid dragging the header into any consumer of
the API.
Pull Request: https://projects.blender.org/blender/blender/pulls/159260
This change removes the need to to carefully pair MEM_mallocN/MEM_new_for_free
with MEM_freeN and MEM_new with MEM_delete. Instead all function are now
variations of MEM_new and MEM_delete.
* MEM_new_for_free -> MEM_new
* MEM_mallocN -> MEM_new_uninitialized
* MEM_callocN -> MEM_new_zeroed
* MEM_freeN -> MEM_delete (for typed pointers)
* MEM_freeN -> MEM_delete_void (for void pointers)
* MEM_reallocN -> MEM_realloc_uninitialized
* MEM_recallocN -> MEM_realloc_zeroed
* MEM_dupallocN -> MEM_new (when possible)
* MEM_dupallocN -> MEM_dupalloc (for typed pointers)
* MEM_dupallocN -> MEM_dupalloc_void (for void pointers)
The MEM_malloc and MEM_calloc functions were renamed to make it clear that
they should be paired with MEM_delete and to clarify what they do.
The compiler will emit an error if MEM_delete or MEM_delete_void is used on
the wrong pointer type. However a remaining risk is casting a MEM_new allocation
with a non-trivial destructor to a void pointer (which is the same as before). This is
why MEM_delete_void and MEM_dupalloc_void exist as a separate functions,
to identify legacy code that has this risk and should be eliminated over time.
Note that MEM_new_array only supports trivially destructible types. This
avoids the need for an equivalent of the delete [] operator and the associated
mistakes that can be made. Instead MEM_delete can be used for everything. For
non-trivial types, data structures like Vector should be used instead.
MEM_dupalloc is now also more type safe. As before it is only supported on
trivially copyable types, and this is now enforced through static asserts to
prevent mistakes. It is also templated to remove the need for casts.
Pull Request: https://projects.blender.org/blender/blender/pulls/151387
Replacing various include directories, defines and libraries specified
directly.
Additionally, this makes it so bf::dependencies are automatically added
last to the target. This ensures include directories from bf::extern libraries
have priority over them, so that e.g. our own fmtlib will be used rather than
one that might be installed on the system.
Pull Request: https://projects.blender.org/blender/blender/pulls/151920
JEMALLOC is no longer maintained, use TBB_MALLOC_PROXY instead,
which was already done for WIN32.
Performance testing showed mixed results with neither allocator having a
clear advantage across systems. Variance depends on system-specific
factors (cache size, memory layout), and given JEMALLOC's uncertain
maintenance status - we deem this acceptable.
Co-authored-by: Bastien Montagne <bastien@blender.org>
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Ref !151524
Like MEM_new_for_free, this is only supported for trivially destructible
types, as we'd need to store the array length to support freeing.
This is mostly for DNA and converting legacy code, in new code Vector<>
should be used when possible.
Also relax condition to only destructible on MSVC, for same reason as
MEM_freeN.
Ref #139113
Pull Request: https://projects.blender.org/blender/blender/pulls/134531
The clamping was a band aid fix for high power HDRI issues
(see #62892).
Since we nowadays support 32bit float textures, we can just
tell users to use that instead of clamping the output.
Use CPU side float to half conversion for OpenGL and Metal to
avoid platform difference related to the handling of overflow.
Also make the value finite inside pixels to avoid infinities and
the potential platform dependent (fast_math on/off) behavior
that comes with them..
This moves `MEM_freeN_smart_ptr_deleter` to `MEM_guardedalloc.h`
to be able to reuse it.
The image_log and other image_colorspace tests exhibit a bit more
noise compared to the reference after this commit. That is expected.
Pull Request: https://projects.blender.org/blender/blender/pulls/148288
Currently `MEM_delete` frees pointers expecting that they match to the
pointers allocated with `MEM_new`, otherwise it can cause undefined
behavior when freeing memory(using `--debug-memory` flag breaks in
place, if not it can corrupts other data, generating a incorrect back-traces).
However polymorphic objects lifetime can be managed by pointer of their
most derived type or by any pointer in their ancestor tree that defines
a virtual destructor, which sometimes can differ in offset when pointing to
the same object.
This changes ensures the correct pointer is being freed, by using the pointer
to the most derived type (returned by`dynamic_cast<void *>(...);`[0]).
----------
[0] = [dynamic_cast](https://en.cppreference.com/w/cpp/language/dynamic_cast.html): `a) If expression is a pointer to (possibly cv-qualified) void, the result is a pointer to the most derived object pointed to by expression.`
-----------
As an example, given the followings structs:
```c++
struct A {
int a;
virtual ~A() = default;
};
struct B {
int b;
virtual ~B() = default;
};
struct Derived : public A , public B {
int c;
};
std::unique_ptr<A> a_ptr = std::make_unique<Derived>();
std::unique_ptr<B> b_ptr = std::make_unique<Derived>();
```
Using std smart pointers to manage `Derived` objects can be done with `A`
or `B` pointers.
However if a `Derived` object memory is managed with `MEM_delete`,
using a `B` pointer for freeing the memory currently may silently break Blender,
since it don't accounts for the full object memory layout, the `dynamic_cast<void *>(ptr);`
cast gives a more safe pointer for freeing the memory.
Note that object destruction is successfully handled through the virtual destructor.
----------
This instead could be an assert to ensure polymorphic objects to be deleted
as the most derived object type.
Pull Request: https://projects.blender.org/blender/blender/pulls/146269
The data constructed by this call remains in the 'C-alloc' realm, i.e.
it can be `MEM_dupallocN`'ed, and `MEM_freeN`'ed.
This is intended as a temporary API only, to facilitate transition to
full C++ handling of data in Blender. It's primary target is to allow
pseudo-POD types to use default values for their members. See e.g.
!134531.
Unlike !143827 and !138829, it does not change the current rule (`new`
must be paired with `delete`, and `alloc` must be paired with `free`).
Instead, it defines an explicit and temporary API to allow a very
limited form of construction to happen on C-allocated data, provided
that the type is default-constructible, and remains trivial after
construction.
### Notes
* The new API is purposely as restrictive as possible, trying to
only allow the current known needs (init with default member values).
This can easily be extended if needed.
* To try to stay as close as malloc/calloc behavior as possible, and
avoid the 'zero-initialization' gotcha, it does not use
value-initialization, but instead default-initialization on zero-
initialized memory.
_Ideally it would even not allow any user-defined default constructor,
but this does not seem simple to detect._
Pull Request: https://projects.blender.org/blender/blender/pulls/144141
This commit implements #125759.
It removes:
* Blender does not build on big endian systems anymore.
* Support for opening blendfiles written from a big endian system is
removed.
It keeps:
* Support to generate thumbnails from big endian blendfiles.
* BE support in `extern` or `intern` libraries, including Cycles.
* Support to open big endian versions of third party file formats:
- PLY files.
- Some image files (cineon, ...).
Pull Request: https://projects.blender.org/blender/blender/pulls/140138
Instead of allowing leaks when parsing arguments, always cleanup before
calling exit(). This impacts -a (animation player), --help & --version
arguments, as well as scripts executed via --python which meant tests
that ran scripts could leak memory without raising an error as intended.
Avoid having suppress warnings & rationalize in code-comments when
leaking memory is/isn't acceptable, any leaks from the animation-player
are now reported as well.
This change exposed leaks: !140182, !140116.
Ref !140098
calloc is generally faster than zeroing separately after a regular
allocation. Our allocator API exposed an allocation call with "calloc"
in the name that didn't actually use "calloc" because it had an
alignment argument (there is no standardized calloc-with-alignment
provided by the OS). However, we can still use calloc internally if
the alignment fits within the default. That just aligns the function
better with performance expectations.
Pull Request: https://projects.blender.org/blender/blender/pulls/139749
Some Linux multi lib setups have a helper include file for Jemalloc that
in turn includes the actual header file. This makes our version regex fail.
As the Jemalloc version we are checking for is no longer in any of
the currently supported LTS linux distros, we can safely drop it.
Pull Request: https://projects.blender.org/blender/blender/pulls/139225
This reflects better the more detailed info in comments of each sections
of the API in that file, and the info in the on-going work for a related
handbook page at blender/blender-developer-docs!139.
This fixes most "One Definition Rule" violations inside blender proper
resulting from duplicate structures of the same name. The fixes were
made similar to that of !135491. See also #120444 for how this has come
up in the past.
These were found by using the following compile options:
-flto=4 -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing
Note: There are still various ODR issues remaining that require
more / different fixes than what was done here.
Pull Request: https://projects.blender.org/blender/blender/pulls/136371
The general idea is to keep the 'old', C-style MEM_callocN signature, and slowly
replace most of its usages with the new, C++-style type-safer template version.
* `MEM_cnew<T>` allocation version is renamed to `MEM_callocN<T>`.
* `MEM_cnew_array<T>` allocation version is renamed to `MEM_calloc_arrayN<T>`.
* `MEM_cnew<T>` duplicate version is renamed to `MEM_dupallocN<T>`.
Similar templates type-safe version of `MEM_mallocN` will be added soon
as well.
Following discussions in !134452.
NOTE: For now static type checking in `MEM_callocN` and related are slightly
different for Windows MSVC. This compiler seems to consider structs using the
`DNA_DEFINE_CXX_METHODS` macro as non-trivial (likely because their default
copy constructors are deleted). So using checks on trivially
constructible/destructible instead on this compiler/system.
Pull Request: https://projects.blender.org/blender/blender/pulls/134771
Followup to 48e26c3afe, and discussions in !134771 about keeping
'C-style' and 'C++ template type-safe style' implementations of our
guardedalloc separated. And it makes `MEM_freeN<T>` code simpler.
Also skip type-checking in `MEM_freeN<T>` only with MSVC, as clang-cl on
windows-arm64 does work fine with DNA structs using
`DNA_DEFINE_CXX_METHODS`.
Pull Request: https://projects.blender.org/blender/blender/pulls/134861
The main goal of these changes are to improve static (i.e. build-time)
checks on whether a given data can be allocated and freed with `malloc`
and `free` (C-style), or requires proper C++-style construction and
destruction (`new` and `delete`).
* Add new `MEM_malloc_arrayN_aligned` API.
* Make `MEM_freeN` a template function in C++, which does static assert on
type triviality.
* Add `MEM_SAFE_DELETE`, similar to `MEM_SAFE_FREE` but calling
`MEM_delete`.
The changes to `MEM_freeN` was painful and useful, as it allowed to fix a bunch
of invalid calls in existing codebase already.
It also highlighted a fair amount of places where it is called to free incomplete
type pointers, which is likely a sign of badly designed code (there should
rather be an API to destroy and free these data then, if the data type is not fully
publicly exposed). For now, these are 'worked around' by explicitly casting the
freed pointers to `void *` in these cases - which also makes them easy to search for.
Some of these will be addressed separately (see blender/blender!134765).
Finally, MSVC seems to consider structs defining new/delete operators (e.g. by
using the `MEM_CXX_CLASS_ALLOC_FUNCS` macro) as non-trivial. This does not
seem to follow the definition of type triviality, so for now static type checking in
`MEM_freeN` has been disabled for Windows. We'll likely have to do the same
with type-safe `MEM_[cm]allocN` API being worked on in blender/blender!134771
Based on ideas from Brecht in blender/blender!134452
Pull Request: https://projects.blender.org/blender/blender/pulls/134463
194e233d86 caused a discussion in the chat about the initialization
behavior of `MEM_new()`, and agreement was to not rely on
zero-initialization ever. Noted this in the API comment now.
Some people found the existing comment useful but it still left some
questions. Tried to clarify that now.
This is a crucial memory management function, it's important to have
behavior documented well, even if a full explanation is out-of-scope.
Also added another link in case people want to check more details.
Pull Request: https://projects.blender.org/blender/blender/pulls/134577
This implements the proposal from #124512. For that it contains the following
changes:
* Remove the global override of `new`/`delete` when `WITH_CXX_GUARDEDALLOC` was
enabled.
* Always use `MEM_CXX_CLASS_ALLOC_FUNCS` where it is currently used. This used
to be guarded by `WITH_CXX_GUARDEDALLOC` in some but not all cases. This means
that a few classes which didn't use our guarded allocator by default before,
are now using it.
Pull Request: https://projects.blender.org/blender/blender/pulls/130181
Add a new API to store data that is guaranteed to not be freed
before the memleak detector has run.
This will be used in next commit by the readfile code to improve
reporting on leaks from blendfile readingi process.
This is done by a two-layer approach:
A new templated `MEM_construct_leak_detection_data` allows to
create any type of data. Its ownership and lifetime are handled
internally, and guaranteed to not be destroyed before the memleak
detector has run.
Add a new template-based 'allocation string storage' system to
`intern/memutil`. This uses the new `Guardedalloc Persistent Storage`
system to store all 'complex' allocation messages, that cannot be
defined as literals.
Internally, the storage is done through an owning reference (a
`shared_ptr`) of the created data into a mutex-protected static
vector.
`MEM_init_memleak_detection` code ensures that this static storage
is created before the memleak detection data, so that it is destructed
after the memleak detector has ran.
The main container (`AllocStringStorageContainer`) is wrapping a
map of `{string -> AllocStringStorage<key_type, hash_type>}`.
The key is a storage identifier.
Each storage is also a map wrapped into a simple templated API
class (`AllocStringStorage`), where the values are the alloc strings,
and the keys type is defined by the user code.
Pull Request: https://projects.blender.org/blender/blender/pulls/125320
