Developer
Log in

Extensions

Plugin ABI

A plugin is a native library with one exported C function. This page is the ABI it speaks, version 1.3, taken from velum_plugin.h, the header every plugin builds against.

Source plugins, ABI 1.3

Studio's plugin host runs ABI 1.3, and plugins of kind source. Other kinds are planned. The header and the Rust SDK are MIT licensed.

The entry point

One symbol per plugin binary:

C
VELUM_PLUGIN_EXPORT const VelumPluginDesc *velum_plugin_entry(const VelumHostApi *host);

Plain C: no C++ or Rust types cross the boundary. The host passes its function table; the plugin returns its descriptor (or NULL to refuse to load), and keeps host for later calls. VELUM_PLUGIN_EXPORT is __declspec(dllexport) on Windows and default visibility elsewhere. The symbol's name is also VELUM_PLUGIN_ENTRY_SYMBOL.

C
static const VelumHostApi *host;
static VelumPluginDesc desc;

VELUM_PLUGIN_EXPORT const VelumPluginDesc *velum_plugin_entry(const VelumHostApi *h) {
    if (!h || h->abi_major != VELUM_PLUGIN_ABI_MAJOR) return NULL;
    host = h;
    desc.struct_size = sizeof desc;
    desc.abi_major = VELUM_PLUGIN_ABI_MAJOR;
    desc.abi_minor = 0;                       /* the lowest host minor this plugin needs */
    desc.id = VELUM_STR("com.example.glow");  /* equals plugin.json's id */
    desc.name = VELUM_STR("Glow");
    desc.version = VELUM_STR("1.0.0");        /* equals plugin.json's version */
    desc.sources = sources;                   /* your VelumSourceType pointers */
    desc.source_count = 1;
    return &desc;
}

In C++, velum_str("…") makes a VelumStr from a literal; in C, the VELUM_STR(lit) macro.

Versioning

VelumHostApi and every table a plugin hands back start with the same header:

C
uint32_t struct_size; uint16_t abi_major; uint16_t abi_minor;
  • A different major is incompatible. The host refuses to load the plugin. The current major is VELUM_PLUGIN_ABI_MAJOR (1).
  • Minors are append-only. New fields and function pointers only ever go at the end of a table, so a newer Studio keeps loading older plugins.
  • VelumPluginDesc.abi_minor is the lowest host minor the plugin needs. An older host refuses it. The current minor is VELUM_PLUGIN_ABI_MINOR (3). Declare it in plugin.json too, as "abi": {"major": 1, "minMinor": …}.
  • Check before you call. A plugin that can do without a newer minor declares the lower one and checks each newer function first: VELUM_HAS_FIELD(host, VelumHostApi, fn) is true when struct_size is long enough to hold fn.
Minor Added
1.0 Sources: properties, CPU frames (texture_write), audio (audio_pull), buttons
1.1 GPU frames (Vulkan, Direct3D 12, Metal), node and device pickers, lists, secrets, audio feeds (audio_input), rename and removal notices
1.2 Icon buttons, previews, plugin windows (window_open, window_update, render_preview), pointer events
1.3 props_commit (undoable self-edits), props_get_json, preset enums, schema_refresh, readouts, look properties, up to eight windows

Conventions

Handles VelumInstanceHost, VelumProps, VelumSchema, VelumTexture, VelumAudioBuf and VelumRenderCtx are opaque and owned by the host. Each is valid only for the call it was passed to, except VelumInstanceHost (from create() until destroy() returns) and a VelumProps kept with props_retain (until props_release).
Strings UTF-8 (ptr, len) pairs, VelumStr. Nothing relies on NUL termination, in either direction. ptr may be NULL when len is 0. A VelumStr the host returns is valid for the call that returned it.
Memory Freed by the side that allocated it. A blob a plugin returns to the host is allocated with host->alloc; the host frees it.
Errors Functions return VelumResult: 0 is OK, negative numbers are the control API's error codes. Say why with host->set_last_error(); Studio shows it in the Inspector.
Colors VelumColor is sRGB with straight (not premultiplied) alpha, each channel 0 to 1.

Results

Constant Value
VELUM_OK 0
VELUM_E_FORBIDDEN −32002
VELUM_E_NOT_FOUND −32011
VELUM_E_ALREADY_EXISTS −32012
VELUM_E_INVALID_STATE −32020
VELUM_E_BUSY −32021
VELUM_E_TIMEOUT −32022
VELUM_E_UNSUPPORTED −32030
VELUM_E_VERSION_MISMATCH −32050
VELUM_E_INVALID_ARGUMENT −32602
VELUM_E_INTERNAL −32603

Threads and deadlines

Every function names one thread class:

Class What runs there Rules
control The plugin's own serial queue: load, schema, create, update, destroy, activation, interaction, windows Ordinary code. Return promptly: a call running 2 seconds counts as a hang.
render The GPU worker: tick, render, render_preview Deadline-bound. A frame not ready by the core's compose deadline is shown late: the last ready frame is used, and a late frame is counted against the plugin.
audio Real time: audio_pull, audio_input No locks, no allocation, no system calls.
any Any thread
  • The host never calls one instance concurrently on the same thread class.
  • The host never runs a control call and a render call of the same instance at the same time: update() never races render().
  • audio_pull can run alongside either, so state shared with it must be lock-free.
  • VelumRenderInfo gives each render its frame number, time_ns and deadline_ns on the core's monotonic clock, and the program's canvas_fps.
  • Studio never waits on a plugin's GPU work: it polls, and shows the newest finished frame.

Crashes and hangs

Plugins run inside velum-plugin-host, never inside Studio, one host process per plugin. Studio watches each host:

  • A host that exits, misses its heartbeats (one every 500 ms) for 2 seconds, or sits in one plugin call for 2 seconds, is stopped and started again after 0.5, 1, 2, 4, then 8 seconds.
  • While it restarts, only that plugin's layers show a placeholder frame. The stream never waits.
  • A restarted host gets every instance back with the properties last committed.
  • The fifth crash within 10 minutes turns the plugin off for the session, and the Inspector says why.
  • The handshake and the plugin's load must finish within 10 seconds.
  • Before loading, the host checks the binary's SHA-256 against the package's signed manifest.

Lifecycle

  1. Load. The host calls velum_plugin_entry(host) and reads the descriptor: its id and version must equal plugin.json's.
  2. Schema. get_schema(schema) runs once per source type per load, before any create(), and declares the properties in Inspector order.
  3. Create. create(host_instance, props, &out_instance) makes an instance with its first values. Whatever you store in *out_instance is passed back to every other call.
  4. Size. size(instance, &w, &h) gives the picture's natural size in source pixels, asked after create() and after every update(). A change resizes the textures.
  5. Run. tick(dt) and render() once per frame on the render thread; audio_pull() per block on the audio thread for audio sources. on_activate() and on_deactivate() when the source goes on and off air. update(props) whenever values change, with all of them. interact(event) when someone presses one of its buttons.
  6. Destroy. destroy(instance), never while another call on that instance runs.
  7. Unload. shutdown(), once, before the binary is unloaded.

The plugin descriptor

VelumPluginDesc

Field Type Meaning
struct_size, abi_major, abi_minor header sizeof(VelumPluginDesc), the major built against, the lowest host minor needed
id VelumStr Reverse-DNS; must equal plugin.json's id
name VelumStr Shown to people
version VelumStr Semver; must equal plugin.json's version
sources, source_count const VelumSourceType *const *, size_t The source types it offers
shutdown void (*)(void) control: called once before unload. May be NULL.

VelumSourceType

Field Meaning
struct_size, abi_major, abi_minor The header
id Stable within the plugin, [a-z0-9_-]{1,64}. Never changes once shipped: scene collections store it as plugin.<plugin id>/<source id>.
name Shown in Add source, such as "Gradient"
flags VELUM_SOURCE_VIDEO (0x1), VELUM_SOURCE_AUDIO (0x2), or both. An audio source gets its own mixer channel.
vtable Its VelumSourceVtable

Source callbacks: VelumSourceVtable

Callback Thread Since What it does
get_schema(schema) control 1.0 Declare this source type's properties
create(host_instance, props, &out_instance) control 1.0 A new instance with its first values
destroy(instance) control 1.0 The instance goes
update(instance, props) control 1.0 Values changed; all of them are passed
size(instance, &width, &height) control 1.0 The natural size in source pixels
on_activate(instance), on_deactivate(instance) control 1.0 On air, off air. May be NULL.
interact(instance, event) control 1.0 A button was pressed (VELUM_EVENT_BUTTON, key = the button's key), or, from 1.2, a window event. May be NULL.
tick(instance, dt_seconds) render 1.0 Time moved on since the last tick
render(instance, ctx, out) render 1.0 Draw this frame into out
audio_pull(instance, buf, frames) audio 1.0 Fill one block (audio sources only; else NULL)
audio_input(instance, input) audio 1.1 A block of sound an audio-feed property chose, every 10 ms or sooner. May be NULL.
on_node_renamed(instance, uuid, old_name, new_name) control 1.1 A source or scene a picker points at was renamed. Return within 5 ms. May be NULL.
on_node_removed(instance, uuid) control 1.1 …or removed. May be NULL.
window_update(instance, window, props) control 1.2 Values edited in one of its windows, all passed. May be NULL.
render_preview(instance, window, key, ctx, out) render 1.2 Draw a visible preview at texture_info(out)'s size, the same way as render(). May be NULL.

A 1.0 vtable is shorter; struct_size says which one the host has. Fill it with designated initializers so the same source builds against any 1.x header.

Host functions: VelumHostApi

Check VELUM_HAS_FIELD(host, VelumHostApi, fn) before calling anything added after 1.0.

Basics

Function Thread Since
alloc(size, align), free(ptr) any 1.0 Memory the host frees (blobs returned to it)
log(instance, level, message) any 1.0 A line in Studio's session log, attributed to the plugin and the instance (when not NULL). Levels: VELUM_LOG_ERROR 1, WARN 2, INFO 3, DEBUG 4.
set_last_error(instance, message) any 1.0 Why the last call failed; shown in the Inspector

Reading property values

Function Thread Since
props_get_bool, props_get_int, props_get_float, props_get_string, props_get_color control, render 1.0 A value by key. VELUM_E_NOT_FOUND when unset or set as another type; the schema's default applies when unset.
props_retain(props), props_release(props) control 1.0 Keep a VelumProps past its call
props_get_noderef(props, key, &ref) control, render 1.1 A picker's source or scene: kind, uuid (36 characters) and its name now. Studio stores the UUID and resolves the name each time; a deleted node reads with its UUID and an empty name.
props_get_list_len, props_get_list_item control, render 1.1 A list's items
secret_read(instance, props, key, buf, cap, &len) control 1.1 A secret's value, read from the OS keychain at use time. VELUM_E_NOT_FOUND when none is saved; VELUM_E_INVALID_ARGUMENT (with len set) when buf is too small. Never log it, and don't keep it longer than you need it.
props_get_json(props, &out) control, render 1.3 Every value as Studio saved it, as JSON (key to stored value, such as {"t": "float", "v": 0.5}); properties never set are absent

Declaring properties (inside get_schema)

Function Since
schema_add_bool(schema, common, default) 1.0 A toggle
schema_add_int(schema, common, default, min, max, step, unit) 1.0 An integer, with a unit
schema_add_float(schema, common, default, min, max, step, unit) 1.0 A number, with a unit
schema_add_string(schema, common, default, kind) 1.0 Text: VELUM_STRING_PLAIN, MULTILINE, URL or FONT (a font family)
schema_add_enum(schema, common, default, options, count) 1.0 A choice; each VelumEnumOption has a stored value and a shown label
schema_add_color(schema, common, default) 1.0 A color
schema_add_path(schema, common, default, kind, filter) 1.0 A file or folder: VELUM_PATH_OPEN_FILE, SAVE_FILE or DIRECTORY; filter such as "Pictures (*.png *.jpg)"
schema_add_button(schema, common) 1.0 A button; pressing it calls interact()
schema_group_begin(schema, common), schema_group_end(schema) 1.0 Properties between them show under one heading. No nesting.
schema_add_noderef(schema, common, node_kinds) 1.1 A picker for VELUM_NODE_SOURCE, SCENE or AUDIO_SOURCE (a mixer channel), OR'd
schema_add_deviceref(schema, common, kind) 1.1 A device picker: VELUM_DEVICE_AUDIO_INPUT (empty = the system default)
schema_add_list(schema, common, item_kind, max_items) 1.1 A list of VELUM_LIST_TEXT, PATH or URL
schema_add_secret(schema, common) 1.1 Stored in the OS keychain; the collection holds only a handle
schema_add_button_icon(schema, common, icon) 1.2 A button with an icon, by name (such as "rotate-ccw")
schema_add_preview(schema, common) 1.2 A live preview, drawn by render_preview()
schema_enum_presets(schema, enum_key, values_json) 1.3 After the enum: options that set other properties, as a preset choice does. Editing one of those by hand picks the option left without values ("Custom").
schema_add_readout(schema, common) 1.3 A read-only line of text the plugin keeps current; no value, not saved

Every property starts with a VelumPropCommon:

Field
struct_size sizeof(VelumPropCommon)
flags See below
key [a-z0-9_]{1,64}, unique per source type
label Shown in the Inspector, sentence case
description An optional tooltip; length 0 for none
Flag Since
VELUM_PROP_ANIMATABLE 0x1 1.0 A Move or tween target
VELUM_PROP_AUTOMATABLE 0x2 1.0 Offered to hotkeys, automations ("Set plugin setting", "Trigger plugin action"), Stream Deck, Touch Portal and the control API (source.set, source.trigger, the source_settings topic). Only these can be changed from outside the Inspector. On a value it's a setting, changed like an Inspector edit; on a button it's a trigger, and nothing is saved. A property <prefix>_<rest> beside a plain string <prefix>_name is named after that string ("happy: Toggle expression") and left out of pickers while it's empty.
VELUM_PROP_AUDIO_FEED 0x4 1.1 On an audio-source picker or an audio-input device picker: Studio feeds the chosen sound to audio_input()
VELUM_PROP_LOOK 0x8 1.3 Part of the source's look. A window with such properties offers "Copy look from…": all of them, copied from another source of the same type in one undoable step. Older hosts ignore it.

Units

Every numeric property declares a unit.

Constant Value
VELUM_UNIT_UNITLESS 0
VELUM_UNIT_LAYOUT_LENGTH 1 Resolution-independent: a fraction of the parent's height
VELUM_UNIT_SOURCE_PIXELS 2 Pixels of the source's own picture
VELUM_UNIT_FRACTION 3 0 to 1
VELUM_UNIT_PERCENT 4 0 to 100
VELUM_UNIT_DEGREES 5
VELUM_UNIT_SECONDS 6
VELUM_UNIT_MILLISECONDS 7
VELUM_UNIT_DB 8
VELUM_UNIT_HZ 9
VELUM_UNIT_ABSOLUTE_PIXELS 10 Allowed, and shown as such; Studio offers to rescale it when the canvas changes

Changing its own settings and schema

Function Thread Since
props_commit(instance, label, values_json) control 1.3 Set this instance's own properties as one undoable change named label (Edit › Undo "label"), saved like an Inspector edit; update() then has them. values_json is an object of key to stored value, or null to unset. Returns at once; Studio commits shortly after. At most 1 MiB.
schema_refresh() any 1.3 The plugin's properties changed (an enum's options, say): Studio asks every get_schema again once the current call returns. Saved values are kept.
readout_set(instance, key, text) any 1.3 A readout's text now, at most 4000 bytes. Send it when it changes, not every frame.

Windows

Function Thread Since
window_open(instance, desc, get_schema, values) control 1.2 Create or replace a window, drawn by Studio with the Inspector's widgets from the schema your callback declares (with the same schema_add_* functions). values may be NULL to keep the current ones. Returns at once; Studio opens it asynchronously. At most four windows per plugin, eight from 1.3.
window_close(instance, key) control 1.2 Close it

VelumWindowDesc has flags (VELUM_WINDOW_MODAL, or 0 for modeless), a key ([a-z0-9_]{1,64}, unique within the instance), a title, and an initial width and height in logical pixels, 160 to 2048. Its events reach interact(): VELUM_EVENT_WINDOW_CLOSED, and VELUM_EVENT_POINTER_DOWN, MOVE and UP with x and y from 0 to 1 and buttons (left 1, right 2, middle 4). VelumEvent.window is empty for Inspector actions.

Drawing frames

From memory

Function Thread Since
texture_info(texture, &info) render 1.0 width, height and format
texture_write(texture, pixels, stride) render 1.0 The whole frame, rows of stride bytes. The host moves it onto the GPU.
render_info(ctx, &info) render 1.0 This render's frame, time_ns, deadline_ns and canvas_fps

The format is VELUM_FORMAT_RGBA8_SRGB_PREMULTIPLIED: 8-bit RGBA, sRGB-encoded, premultiplied alpha, rows top to bottom.

On the GPU (1.1)

A source can draw its frame on its own GPU pipeline, straight into the image Studio composes: no copy and no CPU pixels. In render():

  1. texture_gpu_device(texture, &device): which GPU (the Vulkan device UUID, the D3D12 adapter LUID, or the Metal registry id) and the host's own device, if you want to use it. VELUM_E_UNSUPPORTED means frames go through memory here: use texture_write.
  2. texture_gpu_target(texture, &target): this frame's image, a dmabuf (Linux), an NT handle (Windows) or an IOSurface (macOS), to import on your device, or the host's own import. Import each slot once per generation and keep it; a new generation means import again.
  3. Draw. Replace every pixel: the contents are undefined on entry. Values are 8-bit RGBA, sRGB-encoded, premultiplied. The image's own format is UNORM, so encode sRGB in your shader.
  4. texture_gpu_done(texture, &done): how the host knows your work is finished.
API The image Done
Vulkan (Linux) VK_FORMAT_R8G8B8A8_UNORM, DRM format modifier tiling with the explicit modifier, plane offset and row pitch given. Import a dup() of dmabuf_fd as a dedicated allocation. Leave it in TRANSFER_SRC_OPTIMAL. Signal a binary semaphore exported as a sync_file and hand the fd over (VELUM_GPU_DONE_SYNC_FILE), or signal the host's vk_semaphore (VELUM_GPU_DONE_HOST_SEMAPHORE)
Direct3D 12 (Windows) DXGI_FORMAT_R8G8B8A8_UNORM, render target and simultaneous access; any state. Open d3d12_resource_handle and d3d12_fence_handle with OpenSharedHandle. Queue Signal(fence, d3d12_fence_value) after your work (VELUM_GPU_DONE_D3D12_FENCE)
Direct3D 11 (Windows) OpenSharedResource1 and OpenSharedFence on the same handles, or wrap the host's D3D12 device with D3D11On12 Signal(fence, d3d12_fence_value) and Flush (VELUM_GPU_DONE_D3D12_FENCE)
Metal (macOS) An RGBA IOSurface shared with Studio. Make a texture on it on the MTLDevice whose registryID is mtl_registry_id (MTLPixelFormatRGBA8Unorm), or use the host's mtl_texture. Encode a signal of mtl_shared_event to mtl_event_value (VELUM_GPU_DONE_METAL_EVENT)
Any You already waited for your work (VELUM_GPU_DONE_IDLE)
wgpu The Rust SDK's wgpu feature does all of this

Studio never waits on your GPU work: it polls. A frame not done by the compose deadline is late, and the last done frame is shown. The host's device pointers are valid until the plugin is unloaded; use the host's queue only inside render(). Target handles are borrowed while generation stays the same: duplicate or open them to keep your own.

Audio

Function Thread Since
audio_info(buf, &info) audio 1.0 The block audio_pull fills: channels, sample_rate, frames, the planes and pts_ns, when its first frame plays on the core's clock

Audio is float32 planar at 48 kHz. An audio source's sound goes to its own mixer channel.

Audio feeds (1.1). A property flagged VELUM_PROP_AUDIO_FEED (an audio-source picker or an audio-input device picker) brings that sound to audio_input() as Studio's mixer has it: after the source's filters and sync offset, before its fader (so a muted source is silence), 48 kHz stereo planar. VelumAudioInput carries the key of the property that chose it, pts_ns when its first frame was mixed, and VELUM_AUDIO_INPUT_GAP in flags when frames were lost just before this block. Like audio_pull: no locks, no allocation.

Packaging

The binary ships in a .velumplugin with a plugin.json whose id, version and abi match the descriptor. velum-bundle pack-plugin --plugin plugin.json --binary linux-x64=glow.so --binary windows-x64=glow.dll --out glow.velumplugin packs it, and velum-bundle sign --key your-key.pem glow.velumplugin signs it.

to open · ↑ ↓ to move · Esc to close