Set a hard maximum on the size of JSON files, before trying to parse
them. This prevents out-of-memory errors of Blender when a malicious
server tries to send a huge JSON file.
Pull Request: https://projects.blender.org/blender/blender/pulls/154464
Add an option to the HTTP downloader that sets a size limit (in bytes).
Any download that's larger than this will be rejected with a
`ContentLengthTooBigError` exception.
This is useful to prevent Blender from choking on the parsing of huge
JSON files, which can be served relatively cheaply via compression
techniques.
```py
from _bpy_internal.http import downloader as http_dl
downloader = http_dl.ConditionalDownloader(
metadata_provider=metadata_provider)
downloader.max_size_bytes = 100
downloader.download_to_file("https://example.com/huge.json",
Path("/tmp/huge.json"))
```
The `DownloaderOptions` dataclass has also been extended with a
`max_size_bytes` parameter, so that this can be passed to the background
downloader process as well.
Note: the HTTP downloader is not used in Blender 5.1 (it's intended
for the Remote Asset Libraries in 5.2). However, since this can be
considered a fix for a security issue (potential DOS of Blender), I
feel it's still warranted to include this in 5.1. See !154464 for this
change in context.
Pull Request: https://projects.blender.org/blender/blender/pulls/155259
This adds a new `UString` ("unique string") type in `BLI_ustring.hh`. It is a
thin wrapper around `OpenImageIO::ustring`. OpenImageIO is already a required
dependency, so it's okay to rely on it being available here.
`UString` is just 8 byte large and can still convert to `StringRefNull` or
`const std::string &` in constant time.
See the in-depth code comment for more details of what a `UString` is. The tldr
is that it makes it very cheap to store and compare strings when the same
strings are reused often.
Additionally, this also adds a `_ustr` string literal operator which allows
creating ustrings very conveniently by writing `"my string"_ustr`. Other than
the similar `_us` operator in OpenImageIO, this one templated on the string and
can therefore efficiently cache the `UString` represenation of a string literal
instead of having to do a string lookup to make it unique every time. This
implementation also requires `FixedString` utility type which has been added in
`BLI_fixed_string.hh`.
Translation code has been updated to be able to detect string literals with a
`_ustr` suffix too.
----
For testing, I changed the panel name stored in `PanelDeclaration` from
`std::string` to `UString`. I intend to use this in more places after this
initial commit though. Having to add the `_ustr` suffix in many places is
slightly annoying but seems like the most efficient approach. The compiler
enforces this since there is no implicit conversion from string literals to
`UString` (it's not free).
Using `UString` for panel names (or pretty much all strings in node declarations
for that matter) is nice because these are fairly stable over an entire Blender
session. Currently, there are many `std::string`s stored in node declarations
which are large and are recreated every time a node declaration is build. Using
`UString`, especially with the `_ustr` prefix makes building the declaration
much cheaper because all the string operations are basically eliminated. Note
that many node declarations are not just build once but whenever properties
change because nodes can be dynamic.
The impact for just panel names is small (that's why I used it here), but I
expect even more benefits from using `UString` for descriptions (which are so
long that they require allocations) and especially socket identifiers which can
also make Geometry Nodes evaluation faster.
Pull Request: https://projects.blender.org/blender/blender/pulls/155211
Hide select core add-ons from the Preferences Add-ons tab and
Workspace Filter list, treating them as implementation details
rather than user-facing add-ons.
- Core add-ons in `_addons_hidden_core` are unconditionally
enabled on startup, independent of `preferences.addons`.
- Bypass workspace owner-ID filtering for these add-ons by registering
with an empty owner, avoiding `bl_owner_use_filter` workarounds.
- Centralize the hidden add-on list in `addon_utils`.
Cycles is excluded because hidden add-ons don't support saved
preferences (see #71486).
Ref !155233
Create submenus for common template types in the text editor as the menu
was getting too long.
Common categories such as Gizmo, UI & Operators are now in their own
directories, displayed as submenus.
Menu.menu_path now supports expanding sub-menus recursively.
Ref !154643
Change the way the language is called in Belarusian from "беларуску"
to "Беларуская": title case, ending.
I asked a Belarusian native speaker about it and here was their reply:
> Belarusian language would translate as Беларуская (Мова), or just
> Беларуская (the 'ая' ending is for feminine gender - which
> мова (language) is, in Belarusian).
Reported by Alexandr Fatih in #105461.
Pull Request: https://projects.blender.org/blender/blender/pulls/154582
Sanitize file paths in the downloaded asset listing of remote asset
libraries.
File paths are considered malicious when they reference a directory
outside the asset library. This covers both absolute paths and paths
that use `../` walk 'too far up'.
On the Python side, right after the listing was downloaded, any
malicious file paths are rewritten to relative file paths. On the C++
side, malicious file paths are rejected (because the Python code
should have already removed them).
Pull Request: https://projects.blender.org/blender/blender/pulls/154453
Part of the remote asset libraries project, see:
https://projects.blender.org/blender/blender/issues/134495
When Blender synchronises an asset library from a remote URL, it locks
the library to prevent other running Blenders to do the same. These
locks are now explicitly released when Blender quits.
This isn't necessary for the locking mechanism itself, as the OS
automatically releases the locks when the process ends. However, Python
will complain with a `ResourceWarning` if the files are left open, and
this is now prevented.
Pull Request: https://projects.blender.org/blender/blender/pulls/153284
Part of the remote asset libraries project, see:
https://projects.blender.org/blender/blender/issues/134495
New functionality:
- Automatic daily updating of the asset library listing
- Loading and displaying remote asset libraries in the asset browser and
asset shelf
- Filter flag for online assets in the asset browser and asset shelf
- Operator to download assets, available in context menu
- Prefetch handler to download assets when dragging is invoked
- "Downloading Asset..." hint when dragging assets being downloaded
Further notes:
- Assets that are still being downloaded cannot be dropped currently
(releasing the mouse button does nothing). We might change this but
the planned proper solution is some kind of "placeholder" data-block
that gets replaced with the downloaded data-block once available.
- Downloading is done in Python (already committed), which then "pings"
the asset system for status updates. This happens via the window
manager currently, we might move this to a more dedicated asset system
entry point.
- Asset/file browser previews for online assets use the UI preview
loading system, not the asset/file browser one. Supporting online
previews in the latter turned out tricky, plus it's redundant anyway.
Asset/file browsers should switch to the UI system at some point.
A bunch of more isolated code for remote asset library support was
already committed, see PR for more context. This keeps this commit
smaller and more manageable. And a bunch of more general code was
already committed to `main` throughout the project.
Co-authored-by: Sybren A. Stüvel <sybren@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/153284
Part of the remote asset libraries project, see:
https://projects.blender.org/blender/blender/issues/134495
No user visible changes expected.
This commit contains all the Python code necessary integration with
Blender's asset system (so for downloading asset listings, previews, and
the assets themselves), and for the `blender -c asset_listing` CLI
command.
The OpenAPI datamodel generator no longer includes the OpenAPI schema in
the generated code. That was intended to be used by a validation
library, but as it turns out, that couldn't handle the recursive
definitions we need. Now the code just relies on the validation from the
cattrs library, which is good enough for our purpose.
Available CLI commands:
blender -c asset_listing generate:
Generates the asset listing for all assets in the current directory &
subdirectories.
blender -c asset_listing download <URL>:
This is more of a test command, to download the asset listing. The URL
should point to the root of the asset library, so basically a
HTTP-exposed version of the same directory in which the `generate`
command above was given.
The files are downloaded to `./_asset_download_location`. That way the
current directory isn't spammed with multiple files.
Downloading includes a time-based 'stamp' for cache-busting, when
requesting the top-level JSON file of a remote asset library's listing.
This stamp is the number of seconds since some point in time, divided by
60, to make it change once per minute. This means that a caching service
like CloudFlare can cache the response, and only forward the query to
our HTTP server once every minute.
Co-authored-by: Julian Eisel <julian@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/153284
Macro operator doc-strings didn't properly document the parameters
for the other operators that can be passed in.
Expose them as a dictionary since internally dictionaries are
coerced into OperatorProperties.
Array properties that don't map to a mathutils type (Vector, Matrix,
etc.) incorrectly used their element type (e.g. `float` / `int`).
- For function arguments use `Sequence[...]` since it will coerce
accepts any sequence type.
- Struct properties use `bpy_prop_array`, because they're a
Blender specific type which is mutable.
sphinx_doc_gen.py:
- Add _PRIVATE_ATTR_INCLUDE so underscore-prefixed types can be
selectively included in API docs (needed to reference return values).
- Add :return:/:rtype: for operators.
- Fix context type entries (AnnotationLayer, FluidModifier, Curves).
bpy.props:
- Add :return:/:rtype: for property declaration functions.
gpu:
- Expose MatrixStackContext and OffScreenStackContext as documented
gpu.types so :rtype: references resolve.
bmesh.ops:
- Use Sequence[float] for vector input params.
- Add default_value for single-element BMO_OP_SLOT_ELEMENT_BUF slots.
bpy.types:
- Convert `tuple of X` to `tuple[X, ...]` and `list of X` to `list[X]`.
- Add :return:/:rtype: to Context.path_resolve and Operator.as_keywords.
- Fix Menu example link reference.
bpy_extras:
- Convert Python type annotations to :param:/:type:/:rtype:,
add missing doc-strings.
bpy.app.handlers:
- Add :type: with callable signatures.
- Fix depsgraph_update handler description
(the second argument is optional).
- Fix missing argument descriptions for render and undo/redo handlers.
bpy.context
- Use type hints for for property listing.
mathutils.kdtree:
- Fix doc-string syntax.
- Clarify that the size is an upper limit.
Various minor corrections to other modules.
When reading/writing from/to the communication pipe between the
background Python process and the main Blender process, catch more
exceptions to detect that the Blender process disappeared.
Python's documentation is unclear about which possible exceptions can be
raised by those functions. It now catches the documented `EOFError` as
well as a general `OSError`.
The implementation of the send/recv functions actually raises `OSError`
directly, and in practice I've also seen `ConnectionResetError` (when
receiving) and `BrokenPipeError` (when sending), both of which also
inherit from `OSError`. So this should, in theory, be enough.
Pull Request: https://projects.blender.org/blender/blender/pulls/153845
These messages from node UIs were not extracted because they are not
stored in the input socket's default value, but in a special
"menu_items" property.
Reported by Ye Gui in #43295
Pull Request: https://projects.blender.org/blender/blender/pulls/149409
In the HTTP downloader (`_bpy_internal.http`), make sure a custom
User-Agent HTTP header is set. This includes the version of Blender.
This makes the HTTP downloader use the same User-Agent header as the
code for interfacing with the Extensions platform.
Care is taken to ensure the HTTP downloader code can still be used
outside of Blender; it gracefully handles errors when importing `bpy`.
Pull Request: https://projects.blender.org/blender/blender/pulls/153604
To connect to SQLite, the path to the SQLite file needs to be absolute,
otherwise it cannot be converted to a URI. This is now explicitly
checked, to make this requirement explicit, and to produce a nicer error
message.
I've thought of just making the path absolute in the Disk File Hash
Service itself (instead of raising an error). However, that introduces
the assumption that the path is relative to the current working
directory, which might be incorrect. So IMO it's better to make the
caller responsible for this.
Pull Request: https://projects.blender.org/blender/blender/pulls/153592
