Extensions
Plugin ABI
A plugin is a native library with one exported C function. This page is the ABI it speaks, version 1.3, taken from velum_plugin.h, the header every plugin builds against.
Source plugins, ABI 1.3
Studio's plugin host runs ABI 1.3, and plugins of kind source. Other kinds are planned. The header and the Rust SDK are MIT licensed.
The entry point
One symbol per plugin binary:
VELUM_PLUGIN_EXPORT const VelumPluginDesc *velum_plugin_entry(const VelumHostApi *host);Plain C: no C++ or Rust types cross the boundary. The host passes its function table; the plugin returns its descriptor (or NULL to refuse to load), and keeps host for later calls. VELUM_PLUGIN_EXPORT is __declspec(dllexport) on Windows and default visibility elsewhere. The symbol's name is also VELUM_PLUGIN_ENTRY_SYMBOL.
static const VelumHostApi *host;
static VelumPluginDesc desc;
VELUM_PLUGIN_EXPORT const VelumPluginDesc *velum_plugin_entry(const VelumHostApi *h) {
if (!h || h->abi_major != VELUM_PLUGIN_ABI_MAJOR) return NULL;
host = h;
desc.struct_size = sizeof desc;
desc.abi_major = VELUM_PLUGIN_ABI_MAJOR;
desc.abi_minor = 0; /* the lowest host minor this plugin needs */
desc.id = VELUM_STR("com.example.glow"); /* equals plugin.json's id */
desc.name = VELUM_STR("Glow");
desc.version = VELUM_STR("1.0.0"); /* equals plugin.json's version */
desc.sources = sources; /* your VelumSourceType pointers */
desc.source_count = 1;
return &desc;
}In C++, velum_str("…") makes a VelumStr from a literal; in C, the VELUM_STR(lit) macro.
Versioning
VelumHostApi and every table a plugin hands back start with the same header:
uint32_t struct_size; uint16_t abi_major; uint16_t abi_minor;- A different major is incompatible. The host refuses to load the plugin. The current major is
VELUM_PLUGIN_ABI_MAJOR(1). - Minors are append-only. New fields and function pointers only ever go at the end of a table, so a newer Studio keeps loading older plugins.
VelumPluginDesc.abi_minoris the lowest host minor the plugin needs. An older host refuses it. The current minor isVELUM_PLUGIN_ABI_MINOR(3). Declare it inplugin.jsontoo, as"abi": {"major": 1, "minMinor": …}.- Check before you call. A plugin that can do without a newer minor declares the lower one and checks each newer function first:
VELUM_HAS_FIELD(host, VelumHostApi, fn)is true whenstruct_sizeis long enough to holdfn.
| Minor | Added |
|---|---|
| 1.0 | Sources: properties, CPU frames (texture_write), audio (audio_pull), buttons |
| 1.1 | GPU frames (Vulkan, Direct3D 12, Metal), node and device pickers, lists, secrets, audio feeds (audio_input), rename and removal notices |
| 1.2 | Icon buttons, previews, plugin windows (window_open, window_update, render_preview), pointer events |
| 1.3 | props_commit (undoable self-edits), props_get_json, preset enums, schema_refresh, readouts, look properties, up to eight windows |
Conventions
| Handles | VelumInstanceHost, VelumProps, VelumSchema, VelumTexture, VelumAudioBuf and VelumRenderCtx are opaque and owned by the host. Each is valid only for the call it was passed to, except VelumInstanceHost (from create() until destroy() returns) and a VelumProps kept with props_retain (until props_release). |
| Strings | UTF-8 (ptr, len) pairs, VelumStr. Nothing relies on NUL termination, in either direction. ptr may be NULL when len is 0. A VelumStr the host returns is valid for the call that returned it. |
| Memory | Freed by the side that allocated it. A blob a plugin returns to the host is allocated with host->alloc; the host frees it. |
| Errors | Functions return VelumResult: 0 is OK, negative numbers are the control API's error codes. Say why with host->set_last_error(); Studio shows it in the Inspector. |
| Colors | VelumColor is sRGB with straight (not premultiplied) alpha, each channel 0 to 1. |
Results
| Constant | Value |
|---|---|
VELUM_OK |
0 |
VELUM_E_FORBIDDEN |
−32002 |
VELUM_E_NOT_FOUND |
−32011 |
VELUM_E_ALREADY_EXISTS |
−32012 |
VELUM_E_INVALID_STATE |
−32020 |
VELUM_E_BUSY |
−32021 |
VELUM_E_TIMEOUT |
−32022 |
VELUM_E_UNSUPPORTED |
−32030 |
VELUM_E_VERSION_MISMATCH |
−32050 |
VELUM_E_INVALID_ARGUMENT |
−32602 |
VELUM_E_INTERNAL |
−32603 |
Threads and deadlines
Every function names one thread class:
| Class | What runs there | Rules |
|---|---|---|
control |
The plugin's own serial queue: load, schema, create, update, destroy, activation, interaction, windows | Ordinary code. Return promptly: a call running 2 seconds counts as a hang. |
render |
The GPU worker: tick, render, render_preview |
Deadline-bound. A frame not ready by the core's compose deadline is shown late: the last ready frame is used, and a late frame is counted against the plugin. |
audio |
Real time: audio_pull, audio_input |
No locks, no allocation, no system calls. |
any |
Any thread |
- The host never calls one instance concurrently on the same thread class.
- The host never runs a
controlcall and arendercall of the same instance at the same time:update()never racesrender(). audio_pullcan run alongside either, so state shared with it must be lock-free.VelumRenderInfogives each render itsframenumber,time_nsanddeadline_nson the core's monotonic clock, and the program'scanvas_fps.- Studio never waits on a plugin's GPU work: it polls, and shows the newest finished frame.
Crashes and hangs
Plugins run inside velum-plugin-host, never inside Studio, one host process per plugin. Studio watches each host:
- A host that exits, misses its heartbeats (one every 500 ms) for 2 seconds, or sits in one plugin call for 2 seconds, is stopped and started again after 0.5, 1, 2, 4, then 8 seconds.
- While it restarts, only that plugin's layers show a placeholder frame. The stream never waits.
- A restarted host gets every instance back with the properties last committed.
- The fifth crash within 10 minutes turns the plugin off for the session, and the Inspector says why.
- The handshake and the plugin's load must finish within 10 seconds.
- Before loading, the host checks the binary's SHA-256 against the package's signed manifest.
Lifecycle
- Load. The host calls
velum_plugin_entry(host)and reads the descriptor: its id and version must equalplugin.json's. - Schema.
get_schema(schema)runs once per source type per load, before anycreate(), and declares the properties in Inspector order. - Create.
create(host_instance, props, &out_instance)makes an instance with its first values. Whatever you store in*out_instanceis passed back to every other call. - Size.
size(instance, &w, &h)gives the picture's natural size in source pixels, asked aftercreate()and after everyupdate(). A change resizes the textures. - Run.
tick(dt)andrender()once per frame on the render thread;audio_pull()per block on the audio thread for audio sources.on_activate()andon_deactivate()when the source goes on and off air.update(props)whenever values change, with all of them.interact(event)when someone presses one of its buttons. - Destroy.
destroy(instance), never while another call on that instance runs. - Unload.
shutdown(), once, before the binary is unloaded.
The plugin descriptor
VelumPluginDesc
| Field | Type | Meaning |
|---|---|---|
struct_size, abi_major, abi_minor |
header | sizeof(VelumPluginDesc), the major built against, the lowest host minor needed |
id |
VelumStr |
Reverse-DNS; must equal plugin.json's id |
name |
VelumStr |
Shown to people |
version |
VelumStr |
Semver; must equal plugin.json's version |
sources, source_count |
const VelumSourceType *const *, size_t |
The source types it offers |
shutdown |
void (*)(void) |
control: called once before unload. May be NULL. |
VelumSourceType
| Field | Meaning |
|---|---|
struct_size, abi_major, abi_minor |
The header |
id |
Stable within the plugin, [a-z0-9_-]{1,64}. Never changes once shipped: scene collections store it as plugin.<plugin id>/<source id>. |
name |
Shown in Add source, such as "Gradient" |
flags |
VELUM_SOURCE_VIDEO (0x1), VELUM_SOURCE_AUDIO (0x2), or both. An audio source gets its own mixer channel. |
vtable |
Its VelumSourceVtable |
Source callbacks: VelumSourceVtable
| Callback | Thread | Since | What it does |
|---|---|---|---|
get_schema(schema) |
control | 1.0 | Declare this source type's properties |
create(host_instance, props, &out_instance) |
control | 1.0 | A new instance with its first values |
destroy(instance) |
control | 1.0 | The instance goes |
update(instance, props) |
control | 1.0 | Values changed; all of them are passed |
size(instance, &width, &height) |
control | 1.0 | The natural size in source pixels |
on_activate(instance), on_deactivate(instance) |
control | 1.0 | On air, off air. May be NULL. |
interact(instance, event) |
control | 1.0 | A button was pressed (VELUM_EVENT_BUTTON, key = the button's key), or, from 1.2, a window event. May be NULL. |
tick(instance, dt_seconds) |
render | 1.0 | Time moved on since the last tick |
render(instance, ctx, out) |
render | 1.0 | Draw this frame into out |
audio_pull(instance, buf, frames) |
audio | 1.0 | Fill one block (audio sources only; else NULL) |
audio_input(instance, input) |
audio | 1.1 | A block of sound an audio-feed property chose, every 10 ms or sooner. May be NULL. |
on_node_renamed(instance, uuid, old_name, new_name) |
control | 1.1 | A source or scene a picker points at was renamed. Return within 5 ms. May be NULL. |
on_node_removed(instance, uuid) |
control | 1.1 | …or removed. May be NULL. |
window_update(instance, window, props) |
control | 1.2 | Values edited in one of its windows, all passed. May be NULL. |
render_preview(instance, window, key, ctx, out) |
render | 1.2 | Draw a visible preview at texture_info(out)'s size, the same way as render(). May be NULL. |
A 1.0 vtable is shorter; struct_size says which one the host has. Fill it with designated initializers so the same source builds against any 1.x header.
Host functions: VelumHostApi
Check VELUM_HAS_FIELD(host, VelumHostApi, fn) before calling anything added after 1.0.
Basics
| Function | Thread | Since | |
|---|---|---|---|
alloc(size, align), free(ptr) |
any | 1.0 | Memory the host frees (blobs returned to it) |
log(instance, level, message) |
any | 1.0 | A line in Studio's session log, attributed to the plugin and the instance (when not NULL). Levels: VELUM_LOG_ERROR 1, WARN 2, INFO 3, DEBUG 4. |
set_last_error(instance, message) |
any | 1.0 | Why the last call failed; shown in the Inspector |
Reading property values
| Function | Thread | Since | |
|---|---|---|---|
props_get_bool, props_get_int, props_get_float, props_get_string, props_get_color |
control, render | 1.0 | A value by key. VELUM_E_NOT_FOUND when unset or set as another type; the schema's default applies when unset. |
props_retain(props), props_release(props) |
control | 1.0 | Keep a VelumProps past its call |
props_get_noderef(props, key, &ref) |
control, render | 1.1 | A picker's source or scene: kind, uuid (36 characters) and its name now. Studio stores the UUID and resolves the name each time; a deleted node reads with its UUID and an empty name. |
props_get_list_len, props_get_list_item |
control, render | 1.1 | A list's items |
secret_read(instance, props, key, buf, cap, &len) |
control | 1.1 | A secret's value, read from the OS keychain at use time. VELUM_E_NOT_FOUND when none is saved; VELUM_E_INVALID_ARGUMENT (with len set) when buf is too small. Never log it, and don't keep it longer than you need it. |
props_get_json(props, &out) |
control, render | 1.3 | Every value as Studio saved it, as JSON (key to stored value, such as {"t": "float", "v": 0.5}); properties never set are absent |
Declaring properties (inside get_schema)
| Function | Since | |
|---|---|---|
schema_add_bool(schema, common, default) |
1.0 | A toggle |
schema_add_int(schema, common, default, min, max, step, unit) |
1.0 | An integer, with a unit |
schema_add_float(schema, common, default, min, max, step, unit) |
1.0 | A number, with a unit |
schema_add_string(schema, common, default, kind) |
1.0 | Text: VELUM_STRING_PLAIN, MULTILINE, URL or FONT (a font family) |
schema_add_enum(schema, common, default, options, count) |
1.0 | A choice; each VelumEnumOption has a stored value and a shown label |
schema_add_color(schema, common, default) |
1.0 | A color |
schema_add_path(schema, common, default, kind, filter) |
1.0 | A file or folder: VELUM_PATH_OPEN_FILE, SAVE_FILE or DIRECTORY; filter such as "Pictures (*.png *.jpg)" |
schema_add_button(schema, common) |
1.0 | A button; pressing it calls interact() |
schema_group_begin(schema, common), schema_group_end(schema) |
1.0 | Properties between them show under one heading. No nesting. |
schema_add_noderef(schema, common, node_kinds) |
1.1 | A picker for VELUM_NODE_SOURCE, SCENE or AUDIO_SOURCE (a mixer channel), OR'd |
schema_add_deviceref(schema, common, kind) |
1.1 | A device picker: VELUM_DEVICE_AUDIO_INPUT (empty = the system default) |
schema_add_list(schema, common, item_kind, max_items) |
1.1 | A list of VELUM_LIST_TEXT, PATH or URL |
schema_add_secret(schema, common) |
1.1 | Stored in the OS keychain; the collection holds only a handle |
schema_add_button_icon(schema, common, icon) |
1.2 | A button with an icon, by name (such as "rotate-ccw") |
schema_add_preview(schema, common) |
1.2 | A live preview, drawn by render_preview() |
schema_enum_presets(schema, enum_key, values_json) |
1.3 | After the enum: options that set other properties, as a preset choice does. Editing one of those by hand picks the option left without values ("Custom"). |
schema_add_readout(schema, common) |
1.3 | A read-only line of text the plugin keeps current; no value, not saved |
Every property starts with a VelumPropCommon:
| Field | |
|---|---|
struct_size |
sizeof(VelumPropCommon) |
flags |
See below |
key |
[a-z0-9_]{1,64}, unique per source type |
label |
Shown in the Inspector, sentence case |
description |
An optional tooltip; length 0 for none |
| Flag | Since | |
|---|---|---|
VELUM_PROP_ANIMATABLE 0x1 |
1.0 | A Move or tween target |
VELUM_PROP_AUTOMATABLE 0x2 |
1.0 | Offered to hotkeys, automations ("Set plugin setting", "Trigger plugin action"), Stream Deck, Touch Portal and the control API (source.set, source.trigger, the source_settings topic). Only these can be changed from outside the Inspector. On a value it's a setting, changed like an Inspector edit; on a button it's a trigger, and nothing is saved. A property <prefix>_<rest> beside a plain string <prefix>_name is named after that string ("happy: Toggle expression") and left out of pickers while it's empty. |
VELUM_PROP_AUDIO_FEED 0x4 |
1.1 | On an audio-source picker or an audio-input device picker: Studio feeds the chosen sound to audio_input() |
VELUM_PROP_LOOK 0x8 |
1.3 | Part of the source's look. A window with such properties offers "Copy look from…": all of them, copied from another source of the same type in one undoable step. Older hosts ignore it. |
Units
Every numeric property declares a unit.
| Constant | Value | |
|---|---|---|
VELUM_UNIT_UNITLESS |
0 | |
VELUM_UNIT_LAYOUT_LENGTH |
1 | Resolution-independent: a fraction of the parent's height |
VELUM_UNIT_SOURCE_PIXELS |
2 | Pixels of the source's own picture |
VELUM_UNIT_FRACTION |
3 | 0 to 1 |
VELUM_UNIT_PERCENT |
4 | 0 to 100 |
VELUM_UNIT_DEGREES |
5 | |
VELUM_UNIT_SECONDS |
6 | |
VELUM_UNIT_MILLISECONDS |
7 | |
VELUM_UNIT_DB |
8 | |
VELUM_UNIT_HZ |
9 | |
VELUM_UNIT_ABSOLUTE_PIXELS |
10 | Allowed, and shown as such; Studio offers to rescale it when the canvas changes |
Changing its own settings and schema
| Function | Thread | Since | |
|---|---|---|---|
props_commit(instance, label, values_json) |
control | 1.3 | Set this instance's own properties as one undoable change named label (Edit › Undo "label"), saved like an Inspector edit; update() then has them. values_json is an object of key to stored value, or null to unset. Returns at once; Studio commits shortly after. At most 1 MiB. |
schema_refresh() |
any | 1.3 | The plugin's properties changed (an enum's options, say): Studio asks every get_schema again once the current call returns. Saved values are kept. |
readout_set(instance, key, text) |
any | 1.3 | A readout's text now, at most 4000 bytes. Send it when it changes, not every frame. |
Windows
| Function | Thread | Since | |
|---|---|---|---|
window_open(instance, desc, get_schema, values) |
control | 1.2 | Create or replace a window, drawn by Studio with the Inspector's widgets from the schema your callback declares (with the same schema_add_* functions). values may be NULL to keep the current ones. Returns at once; Studio opens it asynchronously. At most four windows per plugin, eight from 1.3. |
window_close(instance, key) |
control | 1.2 | Close it |
VelumWindowDesc has flags (VELUM_WINDOW_MODAL, or 0 for modeless), a key ([a-z0-9_]{1,64}, unique within the instance), a title, and an initial width and height in logical pixels, 160 to 2048. Its events reach interact(): VELUM_EVENT_WINDOW_CLOSED, and VELUM_EVENT_POINTER_DOWN, MOVE and UP with x and y from 0 to 1 and buttons (left 1, right 2, middle 4). VelumEvent.window is empty for Inspector actions.
Drawing frames
From memory
| Function | Thread | Since | |
|---|---|---|---|
texture_info(texture, &info) |
render | 1.0 | width, height and format |
texture_write(texture, pixels, stride) |
render | 1.0 | The whole frame, rows of stride bytes. The host moves it onto the GPU. |
render_info(ctx, &info) |
render | 1.0 | This render's frame, time_ns, deadline_ns and canvas_fps |
The format is VELUM_FORMAT_RGBA8_SRGB_PREMULTIPLIED: 8-bit RGBA, sRGB-encoded, premultiplied alpha, rows top to bottom.
On the GPU (1.1)
A source can draw its frame on its own GPU pipeline, straight into the image Studio composes: no copy and no CPU pixels. In render():
texture_gpu_device(texture, &device): which GPU (the Vulkan device UUID, the D3D12 adapter LUID, or the Metal registry id) and the host's own device, if you want to use it.VELUM_E_UNSUPPORTEDmeans frames go through memory here: usetexture_write.texture_gpu_target(texture, &target): this frame's image, a dmabuf (Linux), an NT handle (Windows) or an IOSurface (macOS), to import on your device, or the host's own import. Import eachslotonce pergenerationand keep it; a new generation means import again.- Draw. Replace every pixel: the contents are undefined on entry. Values are 8-bit RGBA, sRGB-encoded, premultiplied. The image's own format is UNORM, so encode sRGB in your shader.
texture_gpu_done(texture, &done): how the host knows your work is finished.
| API | The image | Done |
|---|---|---|
| Vulkan (Linux) | VK_FORMAT_R8G8B8A8_UNORM, DRM format modifier tiling with the explicit modifier, plane offset and row pitch given. Import a dup() of dmabuf_fd as a dedicated allocation. Leave it in TRANSFER_SRC_OPTIMAL. |
Signal a binary semaphore exported as a sync_file and hand the fd over (VELUM_GPU_DONE_SYNC_FILE), or signal the host's vk_semaphore (VELUM_GPU_DONE_HOST_SEMAPHORE) |
| Direct3D 12 (Windows) | DXGI_FORMAT_R8G8B8A8_UNORM, render target and simultaneous access; any state. Open d3d12_resource_handle and d3d12_fence_handle with OpenSharedHandle. |
Queue Signal(fence, d3d12_fence_value) after your work (VELUM_GPU_DONE_D3D12_FENCE) |
| Direct3D 11 (Windows) | OpenSharedResource1 and OpenSharedFence on the same handles, or wrap the host's D3D12 device with D3D11On12 |
Signal(fence, d3d12_fence_value) and Flush (VELUM_GPU_DONE_D3D12_FENCE) |
| Metal (macOS) | An RGBA IOSurface shared with Studio. Make a texture on it on the MTLDevice whose registryID is mtl_registry_id (MTLPixelFormatRGBA8Unorm), or use the host's mtl_texture. |
Encode a signal of mtl_shared_event to mtl_event_value (VELUM_GPU_DONE_METAL_EVENT) |
| Any | You already waited for your work (VELUM_GPU_DONE_IDLE) |
|
| wgpu | The Rust SDK's wgpu feature does all of this |
Studio never waits on your GPU work: it polls. A frame not done by the compose deadline is late, and the last done frame is shown. The host's device pointers are valid until the plugin is unloaded; use the host's queue only inside render(). Target handles are borrowed while generation stays the same: duplicate or open them to keep your own.
Audio
| Function | Thread | Since | |
|---|---|---|---|
audio_info(buf, &info) |
audio | 1.0 | The block audio_pull fills: channels, sample_rate, frames, the planes and pts_ns, when its first frame plays on the core's clock |
Audio is float32 planar at 48 kHz. An audio source's sound goes to its own mixer channel.
Audio feeds (1.1). A property flagged VELUM_PROP_AUDIO_FEED (an audio-source picker or an audio-input device picker) brings that sound to audio_input() as Studio's mixer has it: after the source's filters and sync offset, before its fader (so a muted source is silence), 48 kHz stereo planar. VelumAudioInput carries the key of the property that chose it, pts_ns when its first frame was mixed, and VELUM_AUDIO_INPUT_GAP in flags when frames were lost just before this block. Like audio_pull: no locks, no allocation.
Packaging
The binary ships in a .velumplugin with a plugin.json whose id, version and abi match the descriptor. velum-bundle pack-plugin --plugin plugin.json --binary linux-x64=glow.so --binary windows-x64=glow.dll --out glow.velumplugin packs it, and velum-bundle sign --key your-key.pem glow.velumplugin signs it.

