External IME Linking (AIDL)

BiBi Keyboard (asr-keyboard) provides a standard AIDL service that lets other IMEs use its speech recognition. Four IMEs are currently supported:

IME	Protocol	Notes
Modified Fcitx5 IME	BiBi linking protocol	Long-press Space for voice input
Modified Trime IME	BiBi linking protocol	Voice key / toolbar microphone
fxliang's Fcitx5 IME	fxliang voice plugin protocol	IME-side recording; BiBi Keyboard recognizes
Foxy IME	fxliang voice plugin protocol	Pick BiBi Keyboard in its voice input plugin

The server side uses a hand-written Binder protocol, but it is fully compatible with AIDL-generated stubs/proxies. Clients can use .aidl generated code, or call it via raw Binder transact (as Fcitx/Trime do).

User Guide

The ASR provider follows BiBi Keyboard's current settings. The calling IME cannot override it.

All linking methods depend on the same prerequisite switch (the only prerequisite; no floating ball or accessibility service needed):

1. Install the latest BiBi Keyboard (OSS or Pro; Pro is preferred if installed)
2. Enable external linking in BiBi Keyboard: Settings → Input → Input Settings → Audio & External Link → Allow external IME linking (AIDL)

Modified Fcitx5

1. Install the modified Fcitx IME: https://github.com/BryceWG/fcitx5-android-bibi-keyboard/releases (you need to uninstall the original Fcitx first and back up your data)
2. In Fcitx, enable: Settings → Virtual Keyboard → Long-press Space Bar Behavior → Voice Input (AIDL)
3. While using Fcitx, long-press Space to start voice input and release to finish

fxliang's Fcitx5

1. Install fxliang's Fcitx5 IME (supported since BiBi Keyboard v4.0.1)
2. In its voice input settings, choose "BiBi Keyboard Link" as the recognition provider
3. Start recording from the IME's own voice key: recording happens on the IME side, while recognition and AI polish run in BiBi Keyboard following its current settings

Trime

1. Install the modified Trime IME: https://github.com/BryceWG/trime-bibi-keyboard/releases
2. In Trime, enable: Settings → General Settings → BiBi AIDL Voice Input
3. Usage:
   • Long-press a key that has VOICE_ASSIST to start recording, then release to finish
   • If your current theme has no VOICE_ASSIST long-press entry, enable the "Toolbar microphone button" in Trime settings; tap once to start, tap again to stop and commit

Foxy IME

1. Enable the voice input plugin in Foxy IME and choose BiBi Keyboard as the recognition provider
2. You can then invoke BiBi Keyboard recognition from Foxy's voice input, following BiBi Keyboard's current ASR and AI post-processing settings

More external linking capabilities

• Clipboard sync: the latest modified Fcitx5 / Trime can enable “BiBi Keyboard clipboard sync” in their clipboard settings. Clipboard sync must also be configured and enabled in BiBi Keyboard; see Clipboard Sync.
• Pro input-field context: when Pro's input-field context option is enabled, the modified IME supplies limited cursor-adjacent text when requested for AI post-processing.
• Pro learn hotwords from corrections: when enabled in Pro, the modified IME briefly observes corrections after a voice result is committed and reports the settled edit from the same input field. Password, email, URL, phone fields are excluded.

AIDL vs floating-ball IME bridge

AIDL linking is for modified IMEs such as Fcitx/Trime to actively call BiBi Keyboard recognition. IME Bridge instead uses LSPosed/LSPatch so the floating ball can send results through the current third-party keyboard. They serve different setups; most users only need the one that matches their IME workflow. See IME Bridge Module.

Developer Guide

Package priority

1. com.brycewg.asrkb.pro
2. com.brycewg.asrkb

Clients should try binding in this order and prefer the installed Pro package (same interface and behavior).

Service interface (IExternalSpeechService)

Descriptor: com.brycewg.asrkb.aidl.IExternalSpeechService

Transaction codes (match AIDL stub):

Method	Transaction code	Description
startSession	FIRST_CALL_TRANSACTION + 0	server-recording session
stopSession	FIRST_CALL_TRANSACTION + 1	stop current session
cancelSession	FIRST_CALL_TRANSACTION + 2	cancel current session
isRecording	FIRST_CALL_TRANSACTION + 3	whether session is recording
isAnyRecording	FIRST_CALL_TRANSACTION + 4	whether any session is recording
getVersion	FIRST_CALL_TRANSACTION + 5	app version name
startPcmSession	FIRST_CALL_TRANSACTION + 6	client-pushed PCM session
writePcm	FIRST_CALL_TRANSACTION + 7	push one PCM frame
finishPcm	FIRST_CALL_TRANSACTION + 8	finish PCM input and process
getInputRequirements	FIRST_CALL_TRANSACTION + 9	query optional input data (Pro extension)
setInputContext	FIRST_CALL_TRANSACTION + 10	attach limited cursor context (Pro extension)
reportEdit	FIRST_CALL_TRANSACTION + 11	report a correction after dictation (Pro extension)

AIDL signatures:

fun startSession(config: SpeechConfig?, callback: ISpeechCallback): Int
fun stopSession(sessionId: Int)
fun cancelSession(sessionId: Int)
fun isRecording(sessionId: Int): Boolean
fun isAnyRecording(): Boolean
fun getVersion(): String
fun startPcmSession(config: SpeechConfig?, callback: ISpeechCallback): Int
fun writePcm(sessionId: Int, pcm: ByteArray, sampleRate: Int, channels: Int)
fun finishPcm(sessionId: Int)
fun getInputRequirements(sessionId: Int): Int
fun setInputContext(sessionId: Int, generation: Long, inputType: Int, imeOptions: Int, beforeCursor: String, afterCursor: String): Boolean
fun reportEdit(sessionId: Int, generation: Long, beforeCursor: String, afterCursor: String, reason: String): Boolean

Config object (SpeechConfig)

SpeechConfig is a nullable Parcelable with fields:

• vendorId: String?
• streamingPreferred: Boolean
• punctuationEnabled: Boolean?
• autoStopOnSilence: Boolean?
• sessionTag: String?

Current behavior:

• startSession / startPcmSession ignore all config fields except vendorId == "mock". The actual vendor/streaming mode always follows BiBi Keyboard's current in-app settings.
• Only startSession supports vendorId == "mock" connectivity test mode (see below).

Callback interface (ISpeechCallback)

Descriptor: com.brycewg.asrkb.aidl.ISpeechCallback

fun onState(sessionId: Int, state: Int, message: String)
fun onPartial(sessionId: Int, text: String)
fun onFinal(sessionId: Int, text: String)
fun onError(sessionId: Int, code: Int, message: String)
fun onAmplitude(sessionId: Int, amplitude: Float)

Key Methods

startSession (server-recording)

BiBi Keyboard handles recording and audio upload.

Return values:

• >0: success; returns server-generated sessionId
• -2: busy (an existing session is recording)
• -3: feature disabled or engine not ready
• -4: BiBi Keyboard lacks RECORD_AUDIO permission

Callbacks:

• For feature disabled (-3), it will call onError(-1, 403, "feature disabled") first.
• For permission (-4), it will call onError(-1, 401, "record permission denied") first.
• -2 / -3 (engine not ready) are indicated only via return value (no extra callback).

Connectivity test (mock)

When SpeechConfig.vendorId == "mock", it skips real recording: the server directly calls onPartial("【testing】...") and onFinal("External AIDL integration OK (mock)") without needing record permission.

startPcmSession / writePcm / finishPcm (client-pushed PCM)

The client records audio and pushes PCM frames to BiBi Keyboard (Fcitx uses this mode).

startPcmSession return values:

• >0: success; returns sessionId
• -2: busy
• -3: feature disabled
• -5: current vendor does not support pushed PCM (unsupported)

Notes:

• Pushed PCM mode does not check BiBi Keyboard's microphone permission; the client handles recording permission itself.
• Recommended format: PCM16LE / 16000Hz / mono, around 200ms per frame. The server currently does not strictly validate sample rate/channels, but mismatches may hurt results for some engines.
• finishPcm(sessionId) is equivalent to stopSession(sessionId) and indicates end of audio input, waiting for final result.

Optional input context and correction reporting (Pro 4.3.0+)

After starting a session, call getInputRequirements(sessionId) and inspect its bit mask:

Bit	Value	Meaning
bit 0	1	AI post-processing requests input-field context
bit 1	2	learning hotwords from corrections requests post-dictation observation

When the result is non-zero, the client may call setInputContext(...) with the input target's generation, inputType, imeOptions, and text before/after the cursor. The service rejects sensitive editors and limits each side to 1,500 characters.

When bit 1 is active, the client may briefly observe the same input target after committing the final result and send the settled snapshot through reportEdit(...). The service accepts only a matching caller UID, session, and editor generation; learning tickets expire after about 90 seconds.

Backward compatibility

These three transactions are Pro extensions. Treat an unknown transaction or a 0 result as “no optional input requested” and continue the original ASR flow. An older service must not make recording fail.

stopSession / cancelSession

Both are void; no success status is returned.

• stopSession: end input and enter processing; later you will receive onFinal or onError.
• cancelSession: cancel and cleanup; it is not guaranteed that onFinal will never be called (AIDL notes "final result not guaranteed").

isRecording / isAnyRecording

• isRecording(sessionId): whether the given session is recording/accepting input.
• isAnyRecording(): whether any active session exists.

getVersion

Returns semantic version name (BuildConfig.VERSION_NAME), e.g. "4.4.5".

Callback States & Errors

onState: state values

state (Int)	Meaning	Common message
0	idle / finished	final / canceled
1	recording	recording
2	processing	processing
3	error	error text

onError: code values

code	Meaning
401	missing record permission (only possible from startSession)
403	external linking is disabled
500	server internal error (engine/network/etc.)

Vendor & Streaming Mode Decision

External calls always follow BiBi Keyboard's current settings (ignore SpeechConfig).

Cloud vendors:

• Volc: prefs.volcStreamingEnabled selects VolcStreamAsrEngine
• DashScope: prefs.dashStreamingEnabled
• Soniox: prefs.sonioxStreamingEnabled
• ElevenLabs: prefs.elevenStreamingEnabled
• OpenAI: follows the in-app OpenAI streaming (Realtime) switch, for all invocation sources
• Gemini / SiliconFlow / Zhipu GLM / OpenRouter / Xiaomi MiMo / StepAudio / Cohere: fixed non-streaming file engines

Local vendors:

• X-ASR: streaming only
• SenseVoice / FunASR Nano / Qwen3-ASR / Parakeet / FireRedASR V2: non-streaming file engines (pseudo-streaming is UI-only and not exposed externally)

Result Filters

Final results (onFinal) go through unified post-filters:

1. If trimFinalTrailingPunct is enabled, trim trailing punctuation/emoji.
2. Speech preset replacement: if matched, replace with preset content.
3. If postProcessEnabled is enabled and LLM keys are valid, run AI post-processing; on failure/empty output, fall back to simple processing.

Entry points: AsrFinalFilters.applySimple / AsrFinalFilters.applyWithAi.

Session Cleanup

Server removes the session and releases resources:

• after onFinal
• after onError
• immediately on cancelSession

Clients should proactively call cancelSession on window/focus changes to avoid dangling sessions.

Fcitx (bibi/lexi) Integration Example

The modified Fcitx IME (bibi) integrates BiBi Keyboard linking. The example directory in this repo still uses the old name fcitx5-android-lexi-keyboard.

Client implementation file:

fcitx5-android-lexi-keyboard/app/src/main/java/org/fcitx/fcitx5/android/link/AsrkbSpeechClient.kt

Key behaviors:

• bind package order: com.brycewg.asrkb.pro → com.brycewg.asrkb
• component: com.brycewg.asrkb.api.ExternalSpeechService
• raw Binder transact calls (no AIDL generated classes)
• uses pushed PCM mode: startPcmSession → loop writePcm → finishPcm/cancelSession
• long-press Space to start; on release, call finishPcm if any PCM was sent, else cancelSession
• onPartial for preview (setComposingText)
• onFinal commits (commitText) and unbinds service
• onAmplitude drives waveform overlay animation

Error handling:

• -2: show "busy"
• -3 / 403: show "enable external IME linking in BiBi Keyboard"
• -5: show "current vendor doesn't support pushed PCM"
• record permission is handled by Fcitx; pushed PCM mode won't return 401/-4 from server

fxliang voice plugin protocol (IVoiceInputProvider)

fxliang's Fcitx5 and Foxy IME use a separate plugin protocol, different from IExternalSpeechService above: recording happens on the IME side, and PCM frames are pushed to BiBi Keyboard for recognition.

• Service component: com.brycewg.asrkb.api.FxliangFcitxVoiceInputProviderService
• Intent actions: org.fcitx.fcitx5.android.plugin.VOICE_INPUT (Fcitx5) and com.fxliang.foxy.plugin.VOICE_INPUT (Foxy), plus debug variants
• Interfaces: org.fcitx.fcitx5.android.common.ipc.IVoiceInputProvider / IVoiceInputCallback (isAvailable / startSession / feedAudio / endStream, etc.)
• Audio format: PCM16LE / 16000 Hz / mono
• Gated by the same "Allow external IME linking (AIDL)" switch: when disabled, isAvailable() returns false and startSession calls back onError(403, "feature disabled")
• Recognition and post-processing reuse ExternalSpeechSession; vendor/streaming decisions and result filters match the sections above

Best Practices

Client guidelines

1. Bind with Context.BIND_AUTO_CREATE and try Pro → OSS in order.
2. Store returned sessionId after startSession/startPcmSession. Do not generate/reuse ids.
3. Call cancelSession(sessionId) on focus/window changes.
4. Use onPartial for live preview; after onFinal/onError you should cleanup/unbind.
5. If you want the client to control recording (recommended for IME use), prefer pushed PCM mode.