This is what commit ca722eac6e should
have been.
Sorry for breaking a non-deprecated MSVC build once again. The reason it
"appeared to work" for me was that on deprecated builds, Manager.h
includes Manager.hpp for backwards compatibility. On non-deprecated it
doesn't and MSVC ends up dying on a linker error. But the problem was
not related to extern templates, the problem was that there was a
compiler-specific ifdef in the cpp file which exported the class for all
compilers but MSVC, somehow. And so when I removed extern template from
the header, it still worked everywhere, except MSVC and except a
non-deprecated build.
Except for file callbacks, for these I have another change planned for
zero-copy import and it would be unwise to break stuff twice, providing
two sets of backwards compatibility wrappers. The image / scene
converter plugins went through a similar change earlier already and the
shader converters were made sane since the very beginning. OTOH audio
importers and text stuff are scheduled for merging with Trade or a
larger rework anyway, so I didn't see any point in updating those.
It's mostly a trivial change, except that returned String instances are
now also checked for non-default deleters same as Arrays because yes,
wow such flexibility compared to STL strings. Same was done for
ShaderTools::AbstractConverter already anyway, so nothing unheard-of
either.
The importer plugin interface version is bumped as this likely breaks
ABI in a nasty way that would lead to crashes.
For file opening there's no longer an unatomic pair of exists() +
read(), but since Path::read() now returns an Optional, it means we can
reliably distinguish between empty files and failures.
While at it, also added TODOs for removal of the StringStl.h header
that's needed in various places for compatibility with APIs still using
STL strings.
It was a clever harmless trick. Well, it was way more harmless than it
was clever, but even then it caused UBSan to complain. And that's Not A
Good Thing for various reasons, so let's just comply.
The main bad effect of this change is a *slightly* larger list of
exported symbols but until we actually get rid of the major bloats like
<iostream>, <string> and the like, this is not going to have any
measurable impact.
No functional change, just splitting them to two separate headers and
two separate tests. These will eventually become public SceneTools
APIs... once I figure out better naming.
With the previous commits, existing plugin implementations built and ran
against the new code, however it introduced several ABI breaks meaning
that existing plugin binaries would crash. This forces them to be
recompiled to match the new version string.
Mostly just to avoid the return types changing to incompatible types in
the future, breaking existing code. The internals are currently not
fully ready to operate with 64-bit object IDs, especially the AsArray()
APIs -- those I will have to solve in the future somehow. Returning
64-bit values in the pairs would add four byte padding after basically
each value, which is way too wasteful for the common case.
The Into() APIs could eventually get 64-bit overloads though.
Now it's a field and its corresponding object mapping, instead of
field and "objects":
- Goes better with the concept that there's not really any materialized
"object" anywhere, just fields mapped to them.
- No more weird singular/plural difference between field() and
objects(), it's field() and mapping() now.
- The objectCount() that actually wasn't really object count is now a
mappingBound(), an upper bound for object IDs contained in the object
mapping views. Which is quite self-explanatory without having to
mention every time that the range may be sparse.
Honestly I don't care much, this is just that the original
PrimitiveImporter tests started to fail because they expected
object3DForName() to return -1 for a 2D name, but that's no longer the
case. It would be possible to fix this, but I doubt anyone ever relied
on such behavior for 2D scenes, so just add a test that acknowledges
this new behavior.
As the comment says -- before the user code expected that if the scene
hierarchy is broken, particular objects will fail to import. Now the
whole scene fails to import, so we don't even get to know the actual
(expanded) deprecated 2D/3D object count. To reduce the suffering,
return at least the dimension-less object count there. It won't include
the duplicates from the single-function-object conversion but better
than reporting 0.
Same as with MeshData2D/3D, the original ObjectData API and plugin
interfaces are preserved to keep existing code as well as existing
importer implementations working. As Magnum's own importers will get
updated to the new SceneData workflow, a backward compatibility layer
provided that translates it to the subset that the legacy ObjectData
understands.
With this commit, both existing plugin code can build (and test against)
the new workflow, and any ports to the new workflow can test against the
legacy interfaces. Except that for now the compatibility layer doesn't
deal with objects that have more than one mesh or for example a light
and a camera attached, this will be done in a separate step.
For example if we're looking for a 2D image named "cubemap", the
importer will tell us there's 0 2D images, making a potential mistake
(2D vs 3D) more obvious.
Using openMemory() instead of openData() allows the implementation to
assume the data will stay in scope for as long as needed, which can
prevent unnecessary copies in some plugin implementations.
It warranted a new flag, DataFlag::ExternallyOwned, to describe this
kind of memory. I couldn't reuse Owned as that's used for allocations
owned by the instance, which is too little for certain future use cases.
For example returning *Data instances referencing an Owned memory would
mean the user has to assume the memory is gone when the importer
instance is gone, and that's generally not true for memory passed to
openMemory().
Originally I thought I would do this later, but then realized the
existing plugin implementations would need to get all updated again to
be aware of the new flag, with some being forgotten, and it's just
easier to do the whole thing in a single step.
This makes it much less annoying to pass arbitrarily typed data, such as
std::uint8_t or char8_t and what not. It was already done like this for
the new shader converter plugins, where the input is often 32-bit ints
for SPIR-V.
OTOH the internal virtual API is kept with ArrayView<const char>, as
that makes it easier to operate on by the implementations.
This allows to better describe memory ownership and transfer it instead
of forcing the plugins to allocate their own local copy if the import
happens in-place on the imported data. Right now that's mainly for the
openFile() use case, which implicitly allocated an Array with file
contents only to pass it to openData() which then made a copy because it
could not make any assumption about data scope.
In other words, certain plugins (TgaImporter, KtxImporter, DdsImporter,
CgltfImporter and possibly others) will now have their peak memory usage
*halved*.
There will be Flag::FlipY for images at some point, enabled by default
for compatibility with existing GL code, and so it makes sense to start
discouraging setFlags() as early as possible to avoid people resetting
the default by accident.
Also update the imageconverter, sceneconverter and shaderconverter utils
to use these instead of setFlags().
Makes more sense as the function isn't expected to fail (and thus any
kind of lazy population is not possible as it would be too late for
error checks anyway).
*Not* updating interface strings even though this is an ABI break
because we're doing that right after the skin import interface bump.
The plugin interface version got bumped to avoid ABI issues when loading
plugins that weren't updated for the change, but apart from that this
shouldn't be a breaking change, as the API returns a type that can be
both an Optional and a Pointer.
Similar to image mip level import, but this is largely left to be
importer-specific. For example PLY defines per-face data and sometimes
one might want to import them as-is, without them being turned into a
per-vertex property.
For backwards compatibility these will delegate to the new MeshData
interfaces for 3D (and nothing for 2D, because so far there were no 2D
scene importers).
Vladimír Vondruš