Texture pipeline

How a texture travels from a project's assets/ folder to a shipped export: the import-settings model, the export-time cook, and GPU block compression.

The one-line contract: the dev loop always renders the raw source PNG; only the exported payload is conditioned (resized, premultiplied, block-compressed). Iteration never waits on a cook.

Import settings (the .orkmeta sidecar)

Every id-tracked texture carries its import settings in its sidecar's <texture> block (core_project/AssetDatabase.h, sidecar v3). The block holds the DEFAULT settings plus optional per-platform override sub-blocks — <android>, <ios> and <web> — each stored as a delta over the default:

<orkmeta id="5f2c...">
    <texture filter="bilinear" wrap="clamp" maxSize="0" premultiply="false"
             generateMips="false" format="auto" quality="normal">
        <android maxSize="1024"/>
        <ios format="astc-4x4"/>
        <web maxSize="512"/>
    </texture>
</orkmeta>

A texture whose sidecar is id-only cooks with the defaults (format auto) — every id-tracked texture ships compressed unless it opts out. A texture with no sidecar at all carries no import intent and ships untouched.

filter/wrap are the only fields a RUNNING game reads; every other field is already baked into the shipped pixels by the time it sees them. A shipped build carries no sidecars at all, so the export resolves the sampler once — for the platform it packages for, per-platform overrides included — into the payload manifest's <TextureSamplers> block:

<TextureSamplers>
    <Sampler texture="tileset" filter="point" wrap="wrap"/>
</TextureSamplers>

An authoring project has no such block and derives the same answers from its sidecars when it loads. Either way a component asks the ONE TextureSamplerTable (core_project/TextureSamplerTable.h), keyed by a texture's bare stem so the cook's rename (tileset.png → tileset.dds) cannot break the lookup; a texture nobody authored a sampler for is absent from the table and samples bilinear/clamp.

Edit the settings in the editor (Inspector › Texture Import Settings, with Base/Android/iOS/Web override sections and a resolved preview), or write the sidecar directly — agents use the MCP write_project_file verb on the .orkmeta; the Inspector re-reads a sidecar the moment the file changes, so an external write is picked up live. No dedicated MCP verb exists or is needed.

The format model

One shared vocabulary on every platform slot:

auto | none | astc-4x4 | astc-6x6 | astc-8x8 | etc2 | bc1 | bc3 | bc7

• etc2 is a family, not one codec: the cook picks ETC2 RGB8 for an

  opaque texture and ETC2 RGBA8 (EAC alpha) when the source carries any alpha below 255. The BCn auto pick is alpha-aware the same way.

• Every explicit format is selectable on every slot; the cook validates

  the (format, platform, flavor) pair at export and refuses impossible combinations honestly (e.g. bc7 on a mobile platform) — never a half-cooked export.

• quality (low/normal/high) maps to encoder effort everywhere, and

  under format="auto" on iOS it also picks the ASTC block size (high → 4x4, normal → 6x6, low → 8x8). An explicit block-size format wins over the quality-implied one — quality then only tunes encoder effort.

The auto table

auto resolves per export platform and packaged render flavor:

Rationale for the corners:

• Desktop ships the BC family only: ASTC/ETC2 refuse on the desktop

  slot for BOTH flavors — classic desktop GL exposes neither, and the next flavor's desktop renderer compiles its ASTC/ETC2 pixel-format mappings only into its mobile builds (the hardware may decode them; the loader does not). Classic desktop additionally never picks bc7: its default GL renderer runs on desktops whose GL has no BC7 support, so auto stays at the universally supported bc1/bc3 and an explicit bc7 on the classic desktop slot is refused.

• **Classic GLES2 mobile ships PNG — the transcode road is refused, with

  reasons: the -classic mobile presets ARE the GLES2 compatibility flavor**, and on the universal-transcoder road (an ETC2 KTX1 out of the same cooked pipeline) the machinery half works while the device half does not:

  • The cook CAN emit it — the CPU-side path is proven: an explicit

    format="etc2" on a classic mobile slot already encodes an ETC2 KTX1 (.ktx) that the classic runtime's compressed-texture codec parses (ExportTextureCookTests builds and validates that container). So "classic's image codec can take the container" is true.

  • The GPU floor cannot — ETC2/EAC is core only in OpenGL ES 3.0; a

    GLES2 context has no guaranteed ETC2 (the widely-present GLES2 extension is ETC1 — RGB only, no alpha). .oitd is not an option either: it is Ogre-Next's native container, which the classic runtime does not read. So flipping auto to ETC2 would emit textures that the very devices this flavor exists to serve — old GLES2 hardware — may refuse to upload, trading a payload win for black textures on the target audience.

    So auto stays PNG on classic mobile (universally uploadable raw texels), the decision being about the GLES2 GPU floor, not the codec. An explicit etc2/ASTC override is still permitted — it ships the KTX1 the codec parses — but the cook warns loudly that a GLES2 context may not accept it: a per-device experiment, never a shipping default. (A device-capability probe that ships BOTH a PNG and a compressed variant and picks at load is the only way this flips, and a fat dual-payload is not worth it for a compatibility flavor.)

• Web ships PNG — the KTX2 transcoder road is refused, with reasons:

  compressed-texture support in a browser is a property of the visitor's GPU — desktop visitors expose the BC family, phones expose ETC2/ASTC, all as optional extensions, none guaranteed. An explicit format on the <web> slot is permitted but warns loudly: it only loads on clients exposing the matching extension. The one universal answer — KTX2 with a supercompressed UASTC/ETC1S intermediate transcoded at page load to whatever the visitor's context supports — is deliberately not built, because it cannot be done here without breaking the engine's closure discipline:

  • The browser player is the classic GLES2 → WebGL build, and **WebGL 1

    is the floor (tools/player/CMakeLists.txt allows a WebGL2 context only where the driver asks for GLES3 features). WebGL 1 guarantees no** compressed format, so a single shipped artifact cannot pick a GPU-native compressed format up front — it MUST transcode at load, or fall back to uncompressed.

  • Runtime transcoding needs the Basis Universal transcoder (libktx) compiled

    into the wasm player. But ktx is scoped !ios & !android & !emscripten in vcpkg.json — a deliberate exclusion: it is a host-only encoder dep for texcook, never shipped in a player closure. Pulling it into the emscripten build would reverse that decision, add the transcoder's code + tables to every page's wasm download, and require a KTX2/Basis loader in the classic backend (its texture codecs read KTX1, not KTX2 — there is no transcode path today).

  • The alternative — ship a WebGL-consumable compressed format directly,

    no transcoder — fails on the WebGL 1 floor above: there is no format all visitors accept, and the export cannot know a visitor's GPU.

    So auto stays PNG on web (byte-identical raw texels, universally decodable through the browser's own PNG path), and the <web> slot's explicit formats stay the warned per-visitor lottery. The refusal is a closure-discipline decision, not a capability gap; it would revisit only alongside a KTX2 codec + a sanctioned wasm transcoder dependency.

• Android auto is ASTC (matching iOS): the next flavor on Android boots

  Ogre-Next's Vulkan renderer, and every Vulkan-capable device at the shipping floor — arm64, API 28+ — decodes ASTC LDR (it is near-universal on that class of hardware, and the same encoder road iOS already ships). So auto resolves to ASTC there, block size by quality exactly like iOS (high 4x4 / normal 6x6 / low 8x8), in the native .oitd. ETC2 stays a first-class explicit override (format="etc2" → ETC2 RGB8/RGBA8 .oitd) for a project targeting older or ASTC-less Android GPUs; the cook validates and encodes it unchanged. The on-device load proof rides the Android Play/export device tests; the host asserts the cook output + the .oitd PixelFormatGpu (an ASTC value for auto, an ETC2 value for the explicit override).

• Cubemaps cook through the same matrix: a skybox cubemap

  (AtmosphereDesc::skyboxTexture) is a six-face .dds — one uncompressed BGRA8 container with a baked mip chain, baked by Util/make_sky_assets.py. A sidecar-carrying cubemap resolves its format through the SAME auto table and encoder as a 2D texture, so auto block-compresses it per platform (desktop BCn .dds; iOS/Android ASTC .oitd on next), and none (the sky baker's default stamp) ships it verbatim. Two properties of a cubemap are preserved exactly: the face order (+X,-X,+Y,-Y,+Z,-Z) and the baked mip chain — a sky cubemap's chain IS the prefiltered roughness chain the IBL samplers index (make_sky_assets.py), so the cook re-encodes each level as-is rather than regenerating it (maxSize/ premultiply/generateMips therefore do not apply to a cubemap). The BC container reuses the source .dds name in place; the mobile ASTC/ETC2 containers rename .dds → .oitd/.ktx, and the skybox loaders on both flavors resolve a missing .dds to its cooked sibling (the same cooked-extension fallback the 2D paths use). The stock skies still stamp format="none" (a 128px-face sky is ~0.5 MB and its prefiltered chain is quality-sensitive), so shipping bytes are unchanged until a cubemap opts in — the capability is there for any project that wants it.

Containers and what the runtime loads

The cook replaces foo.png in the payload with the compressed container, and every reference reaches the shipped name through the backends' cooked-extension fallback: a missing .png resolves to its .dds/.oitd/.ktx sibling. That covers both reference shapes — a scene's texture field and the bare, id-less ones (.omat texture names, gui atlas files, script-assigned sprite names). A cook run over a PROJECT directory (orkige_export cook-textures) renames the sidecar along with the file (foo.dds.orkmeta) so the project's asset ids keep resolving; a packaged payload sheds its sidecars entirely.

The same three containers carry a cubemap (six faces, the full mip chain): the .dds sets the cubemap caps + a face-major payload, the .oitd a TypeCube header with the faces mip-major (the Ogre-Next Image2 slice layout), the KTX1 numberOfFaces = 6. texcook --faces 6 encodes all six faces in one pass.

All shipped formats are the non-sRGB (UNORM) variants: the engine's render pipeline is deliberately gamma-space passthrough on both flavors (textures sampled raw, non-sRGB swapchain — the cross-flavor colour-parity convention), so no colorspace field exists in the settings; nothing would consume it.

Mip chains

generateMips bakes an offline mip chain: the cook downsamples the (resized) source area-averaged per level and the encoder compresses every level into the container — .dds, .ktx and .oitd all carry mip chains natively. The runtime never generates mips for compressed textures (there is nothing to generate them with on the GPU), so a compressed texture without a baked chain simply samples its base level. Default off; flip it for 3D content viewed at oblique angles.

Normal maps

A normal map encodes directions, not colours — BC1's 5:6:5 endpoint quantisation visibly distorts lighting. The generators that bake normal maps (make_terrain_mesh.py, make_material_demo.py) stamp format="none" so they ship as exact texels. Manually choosing ASTC (fine quality for normals) or bc7 on the next desktop slot is reasonable; this pipeline deliberately has no texture-type system — the per-texture format field IS the knob.

Atlas defaults

Glyph/UI atlases (make_gui_atlas.py) and sprite region atlases (make_sprite_atlas.py) are pixel-exact, point-sampled artwork — block compression smears glyph and frame edges. The generators stamp format="none" into the generated sidecar (preserving an existing asset id, never overriding a format a user already chose — Util/orkige_sidecar.py), so atlases ship as PNG while any decorative atlas can still opt in per asset.

The export cook

The exporter (tools/exporter) stages the project payload and runs the cook over it (ExportTextureCook.h), resolved for the target platform token (""/ios/android/web) and the packaged render flavor. The encoding itself is orkige_texcook (tools/texcook), a host library the exporter links plus its own texcook CLI. Its one dependency, libktx (vcpkg ktx), carries the ASTC encoder for every block size plus the universal encoder/transcoder pair that yields ETC2 and BC1/BC3/BC7 blocks; the .dds/.ktx/.oitd containers are written by that code.

Web/mobile-classic exports whose textures all resolve to PNG encode nothing.

Out of scope by construction: textures baked at runtime (TrueType font atlas pages, rasterised SVG sprites, anything through createTexture2D) are uploaded raw on device and never touch this pipeline; textures embedded inside .glb meshes ship as authored.

Verification

• texcook --selftest — every format encodes, container layouts verify,

  plus the cubemap round-trip (six faces to DDS/OITD/KTX, cube caps + a face-complete payload).

• ExportTextureCookTests — the auto table, override precedence (incl. the

  web slot), impossible-pair refusals, real encode legs (rename + sidecar + mip chains) and the cubemap legs (decode an uncompressed six-face .dds, cook it in place to a BC1 cube DDS with the mip chain + cube caps preserved, rename to a .oitd cube on Android, ship a non-cubemap .dds verbatim).

• render_cooked_cubemap (ctest, both flavors) — block-compresses the stock

  debug cubemap through the real cook and boots the render-facade selfcheck against it: the skybox leg proves the compressed cube LOADS with its +X face and baked mip chain intact (desktop BC; the mobile ASTC/ETC2 cube containers are structural + device-tested, like the 2D .oitd).

• player_cooked_textures (ctest, both flavors) — the real player renders a

  cooked payload from .dds, through both the asset-id rename and the bare-name fallback; the iOS and Android .oitd cooks (both ASTC by default) are asserted structurally (their on-device load proof rides the iOS-simulator/Android Play and export device tests).

• export_* ctests — every exported payload's compressed-texture set

  matches what its source project's settings resolve to per platform, the payload carries no .orkmeta, and the manifest bakes exactly the samplers the source project authors for that platform.

• TextureSamplerTableTests + ExportPayloadTests — the sampler table's key

  normalisation and both fill sources; a staged payload that sheds its sidecars and bakes the authored samplers instead.

• player_pak_sampler_selfcheck (ctest, both flavors) — a sidecar-free

  package with its texture MOUNTED inside a pak still samples the way the texture was authored (point/wrap), which only the baked block can supply.