Finally got an idea how to provide various options store these
efficiently, so it's implemented now. Five different storage variants
times four different type sizes.
Because storing arbitrary data as a string was not good:
- It *never* followed alignment requirements due to the last byte being
used for size. Instead the size is now stored before the data, and
thus the data is always on the 64 byte boundary.
- As it could contain arbitrary binary data, it could cause
magnum-sceneconverter --material-info to print garbage, corrupt the
terminal or, worst case, crash. Not good.
- It stored an implicit \0, which was unnecessary.
For file opening there's no longer an unatomic pair of exists() +
read(), but since Path::read() now returns an Optional, it means we can
reliably distinguish between empty files and failures.
While at it, also added TODOs for removal of the StringStl.h header
that's needed in various places for compatibility with APIs still using
STL strings.
Also not something the classic GPU vertex pipeline can handle, but
useful for other scenarios. Subsequently a support for array indices
will be added, allowing to directly represent for example OBJ files,
where each attribute has its own index buffer.
This is not something the classic GPU vertex pipeline can handle
(except maybe Vulkan, which can handle zero strides for instanced
attributes?), but useful for other scenarios. This means existing code
needs to be aware of and handle the new corner case.
Same as with MeshData2D/3D, the original ObjectData API and plugin
interfaces are preserved to keep existing code as well as existing
importer implementations working. As Magnum's own importers will get
updated to the new SceneData workflow, a backward compatibility layer
provided that translates it to the subset that the legacy ObjectData
understands.
With this commit, both existing plugin code can build (and test against)
the new workflow, and any ports to the new workflow can test against the
legacy interfaces. Except that for now the compatibility layer doesn't
deal with objects that have more than one mesh or for example a light
and a camera attached, this will be done in a separate step.
Looking at the snippets, these seem to have been written back when there
was no builtin shaders yet, it seems, not to mention
MeshTools::compile(), Trade::MeshData or any of the other high-level
APIs. Rather overwhelming to just throw huge code snippets at the user,
explaining a workflow with a custom-made mesh that's going to be drawn
with a custom-made shader, which is like level 999 of using the GL
library.
It was rather discouraging to start "Basic usage" with a boring-ass long
snippet. On the other hand showing just compile() first would lead
people to think it's all some opaque magic, so trying to balance that a
bit.
Also why the hell was the compile() snippet showing the horrendous GL
way of specifying attribute formats? This is not great either but at
least not redundant.
Using openMemory() instead of openData() allows the implementation to
assume the data will stay in scope for as long as needed, which can
prevent unnecessary copies in some plugin implementations.
It warranted a new flag, DataFlag::ExternallyOwned, to describe this
kind of memory. I couldn't reuse Owned as that's used for allocations
owned by the instance, which is too little for certain future use cases.
For example returning *Data instances referencing an Owned memory would
mean the user has to assume the memory is gone when the importer
instance is gone, and that's generally not true for memory passed to
openMemory().
Originally I thought I would do this later, but then realized the
existing plugin implementations would need to get all updated again to
be aware of the new flag, with some being forgotten, and it's just
easier to do the whole thing in a single step.
This allows to better describe memory ownership and transfer it instead
of forcing the plugins to allocate their own local copy if the import
happens in-place on the imported data. Right now that's mainly for the
openFile() use case, which implicitly allocated an Array with file
contents only to pass it to openData() which then made a copy because it
could not make any assumption about data scope.
In other words, certain plugins (TgaImporter, KtxImporter, DdsImporter,
CgltfImporter and possibly others) will now have their peak memory usage
*halved*.
Hah, so many overloads. Not providing mutable access to keys or layer
offsets as that would break the invariant of the internal array always
being sorted.
Interestingly enough, there were no docs whatsoever for image and scene
conversion, and neither it was mentioned what all scene data can be
imported. Sigh.
There's actually a lot of code involved in checking if all textures use
the same transform or coordinate set, especially when considering all
fallback variants and potential future expansion with separate texture
offset/scale/rotation attributes.
A lot of the complexity was thus hidden in plugin implementations, which
were each trying to find a common value for all textures to save the
user from doing the same. All that code can now be removed and left up
to the material APIs themselves -- now it's just about checking
hasCommonTextureTransformation() and then retrieving that one common
transformation, independently on how the material actually defines it.
Turns out glTF doesn't actually put metalness into R and roughness into
G, even though the naming suggests that. This was done originally, but
then they changed that in order to be compatible with UE4 and allow for
a more efficient storage of an occlusion map.
Because this feels extremely arbitrary, the docs have added rationale
for each of the packing variants, and I'm also renaming the packed
attribute and checks to imply the red channel isn't used.
The plugin interface version got bumped to avoid ABI issues when loading
plugins that weren't updated for the change, but apart from that this
shouldn't be a breaking change, as the API returns a type that can be
both an Optional and a Pointer.
AbstractMaterialData is now just a typedef to MaterialData, with all
existing public APIs moved to (and marked as deprecated, if they don't
make sense anymore). The new class doesn't have a virtual destructor as
that's not the desired use anymore -- and AbstractImporter::material()
APIs will be returning an Optional instead of a Pointer, which means any
potential subclasses will be sliced away.
PhongMaterialData is reimplemented using the new key/value store,
with no own members anymore -- thus having the same size as
MaterialData, and safe to be casted from it to access the helper APIs.
And the Vector3 version 5% slower in Release, on GCC at least. FFS,
what was I thinking with the gather() things. Nice in user code,
extremely bad in library code.
Originally wanted to offload this to someone else, but then realized I
need those for generic vertex attribute definitions, which I need for
instancing, which I need now. So here it is, at the bottom of the
dependency chain.
With API analogous to the (relatively) new AnimationData -- with one
buffer containing all index data and one buffer containing all vertex
data, both meant to be uploaded as-is to the GPU.
This will eventually replace MeshData2D and MeshData3D, backwards
compatibility and wiring up to other APIs will be done in follow-up
commits.
It'll get used outside of the root namespace and since the callbacks tend
to be quite complex, it would be silly to require users to implement one
callback for Trade, one for Text and one for Audio, for example.
Vladimír Vondruš