This library provides a way to work with audio worklets and streams using modern web technologies. It allows for the manual writing of audio frames to a buffer and supports various buffer writing strategies.
This library was created for use in my project fbdplay_wasm. In this project, we utilize only a very limited set of WebAudio functionalities. It might lack features for general use.
This library uses modern Web APIs and is designed for contemporary browsers only. Tested and confirmed working on:
Feature | Chrome | Chrome (Android) | Firefox | Safari (macOS) | Safari (iOS) | Edge | Opera |
---|---|---|---|---|---|---|---|
Basic Support | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | ❓ |
Manual Buffer Writing | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | ❓ |
Timer-Based Buffer Writing | ✅ | ✅ | ✅ | 🔺 | ❓ | ✅ | ❓ |
Worker-Based Stability | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | ❓ |
Legend:
Note: Compatibility with Safari on iOS has not been confirmed due to lack of test devices.
To check compatibility in your environment, please run the demo in the example directory.
To install the library, run:
npm install @ain1084/audio-worklet-stream
You need to add @ain1084/audio-worklet-stream
to the optimizeDeps.exclude section in vite.config.ts
. Furthermore, include the necessary CORS settings to enable the use of SharedArrayBuffer
.
vite.config.ts
import { defineConfig } from 'vite'
export default defineConfig({
optimizeDeps: {
exclude: ['@ain1084/audio-worklet-stream']
},
plugins: [
{
name: 'configure-response-headers',
configureServer: (server) => {
server.middlewares.use((_req, res, next) => {
res.setHeader('Cross-Origin-Embedder-Policy', 'require-corp')
res.setHeader('Cross-Origin-Opener-Policy', 'same-origin')
next()
})
},
},
],
})
If you are using Nuxt3, add it under vite in nuxt.config.ts
.
nuxt.config.ts
export default defineNuxtConfig({
vite: {
optimizeDeps: {
exclude: ['@ain1084/audio-worklet-stream']
},
plugins: [
{
name: 'configure-response-headers',
configureServer: (server) => {
server.middlewares.use((_req, res, next) => {
res.setHeader('Cross-Origin-Embedder-Policy', 'require-corp')
res.setHeader('Cross-Origin-Opener-Policy', 'same-origin')
next()
})
},
},
],
},
nitro: {
rollupConfig: {
external: '@ain1084/audio-worklet-stream',
},
routeRules: {
'/**': {
cors: true,
headers: {
'Cross-Origin-Embedder-Policy': 'require-corp',
'Cross-Origin-Opener-Policy': 'same-origin',
},
},
},
},
})
This library continuously plays audio sample frames using AudioWorkletNode. The audio sample frames need to be supplied externally via a ring buffer. The library provides functionality to retrieve the number of written and read (played) frames and allows stopping playback at a specified frame.
The output-only AudioNode is implemented by the OutputStreamNode class, which inherits from AudioWorkletNode. This class adds functionalities such as stream playback, stopping, and retrieving playback position to the AudioWorkletNode.
Instances of OutputStreamNode cannot be constructed directly. First, an instance of StreamNodeFactory needs to be created. The StreamNodeFactory is instantiated by calling its static create method with a BaseAudioContext as an argument. This method internally loads the necessary modules. Then, through the returned instance, the construction of OutputStreamNode becomes possible.
The library does not handle the construction or destruction of AudioContext. When constructing AudioContext, be sure to do so in response to a user interaction, such as a UI event (e.g., button press).
Example:
let audioContext: AudioContext | null = null
let factory: StreamNodeFactory | null = null
const clicked = async () => {
if (!audioContext) {
audioContext = new AudioContext()
factory = await StreamNodeFactory.create(audioContext)
}
const node = await factory.createManualBufferNode( { channelCount: 2, frameBufferSize: 4096 })
}
As outlined in the overview, OutputStreamNode requires external audio samples. These samples must be written to a ring buffer, and there are several methods to achieve this.
OutputStreamFactory.createManualBufferNode
method, specifying the number of channels and frames.stop()
method of OutputStreamNode. You can specify the frame at which to stop playback. If you want to play all the written frames, you can specify the total number of written frames, which can be obtained via the FrameBufferWriter.OutputStreamFactory.createTimedBufferNode()
method, specifying the number of channels, timer interval, and the FrameBufferFiller that supplies samples to the buffer.fill
method of the FrameBufferFiller, which supplies audio frames via the FrameBufferWriter.fill
method of the FrameBufferFiller returns false, it indicates the end of the audio frame supply. Once OutputStreamNode outputs all the written frames, the stream automatically stops and disconnects.stop()
method.OutputStreamFactory.createWorkerBufferNode()
method.Understanding these parameters is crucial for optimal audio performance:
Example calculation: For 48kHz sample rate and 20ms fillInterval:
The actual values may slightly differ from the above because they are rounded up to 128 sample units.
The example can be found at https://github.com/ain1084/audio-worklet-stream/tree/main/example.
The provided example demonstrates how to use the library to manually write audio frames to a buffer. It includes:
example/src/main.ts
): Sets up and starts the audio stream using different buffer writing strategies.example/src/sine-wave-frame-buffer-filler.ts
): Implements a frame buffer filler that generates a sine wave.example/src/sine-wave-generator.ts
): Generates sine wave values for the buffer filler.example/src/worker.ts
): Sets up a worker to handle buffer filling tasks.example/index.html
): Provides the HTML structure and buttons to control the audio stream.For more details, refer to the example/README.md.
This guide provides tips and best practices for optimizing the performance of your audio application using the Audio Worklet Stream Library.
Our library is optimized for processing audio frames in large batches. To maximize performance:
For the most stable playback, especially in scenarios where the main thread might be busy:
By following these guidelines, you can ensure that your audio application runs efficiently and provides a smooth user experience.
This guide covers advanced usage scenarios and techniques for the Audio Worklet Stream Library.
While the library provides manual, timed, and worker-based strategies, you can create custom strategies:
BufferWriteStrategy
interface.onInit
, onStart
, and onStopped
methods to define your custom behavior.Example:
class CustomStrategy implements BufferWriteStrategy {
async onInit(node: OutputStreamNode): Promise<boolean> {
// Custom initialization logic
}
onStart(node: OutputStreamNode): boolean {
// Custom start logic
}
onStopped(): void {
// Custom stop logic
}
}
For complex audio processing tasks:
BufferFillWorker
.SharedArrayBuffer
for efficient data sharing between the main thread and the Worker.When working with multiple audio streams:
OutputStreamNode
instances for each stream.You can combine this library with other Web Audio API features:
OutputStreamNode
to other AudioNodes for additional processing.For robust applications:
These advanced techniques will help you leverage the full power of the Audio Worklet Stream Library in complex audio applications.
This guide helps you troubleshoot common issues when using the Audio Worklet Stream Library.
If you're experiencing frequent UnderrunEvents:
@ain1084/audio-worklet-stream
is properly installed.optimizeDeps.exclude
setting in Vite.If you're experiencing poor performance:
If you're still facing issues after trying these solutions, please open an issue on our GitHub repository with a detailed description of the problem and steps to reproduce it.
When using @ain1084/audio-worklet-stream
in a Nuxt 3 project, you may encounter issues during SSR (Server-Side Rendering) or when importing the package as an ESM module. This can result in errors like:
[nuxt] [request error] [unhandled] [500] Cannot find module '/path/to/node_modules/@ain1084/audio-worklet-stream/dist/esm/events' imported from '/path/to/node_modules/@ain1084/audio-worklet-stream/dist/esm/index.js'
Disable SSR for the Component
You can disable SSR for the component that uses the package. This can be done by using <client-only>
:
<client-only>
<MyComponent />
</client-only>
Use ssr: false in nuxt.config.ts
You can disable SSR for the entire project in nuxt.config.ts
:
export default defineNuxtConfig({
ssr: false,
// other configurations
})
Use import.meta.server and import.meta.client
For a more granular control, you can use import.meta.server
and import.meta.client
to conditionally import the module only on the client-side. Note that this method is more complex compared to 1 and 2:
if (import.meta.client) {
const { StreamNodeFactory } = await import('@ain1084/audio-worklet-stream');
// Use StreamNodeFactory
}
To ensure proper operation, it is essential to use ssr: false
or <client-only>
for components and to exclude @ain1084/audio-worklet-stream
from Vite's optimization in your nuxt.config.ts
:
export default defineNuxtConfig({
ssr: false, // or use <client-only> for specific components
vite: {
optimizeDeps: {
exclude: ['@ain1084/audio-worklet-stream']
},
plugins: [
{
name: 'configure-response-headers',
configureServer: (server) => {
server.middlewares.use((_req, res, next) => {
res.setHeader('Cross-Origin-Embedder-Policy', 'require-corp')
res.setHeader('Cross-Origin-Opener-Policy', 'same-origin')
next()
})
},
},
],
},
nitro: {
rollupConfig: {
external: '@ain1084/audio-worklet-stream',
},
// Ensure CORS settings for SharedArrayBuffer
routeRules: {
'/**': {
cors: true,
headers: {
'Cross-Origin-Embedder-Policy': 'require-corp',
'Cross-Origin-Opener-Policy': 'same-origin',
},
},
},
},
})
We are considering potential enhancements for future releases, including:
Please note that these are just considerations and may or may not be implemented in future versions. We always aim to balance new features with maintaining the library's stability and simplicity.
Vite as a Bundler: This library utilizes Vite to enable the loading and placement of workers without complex configurations. It may not work out-of-the-box with WebPack due to differences in how bundlers handle workers. While similar methods may exist for WebPack, this library currently only supports Vite. Initially, a bundler-independent approach was considered, but a suitable method could not be found.
Security Requirements: Since this library uses SharedArrayBuffer
, ensuring browser compatibility requires meeting specific security requirements. For more details, refer to the MDN Web Docs on SharedArrayBuffer Security Requirements.
Contributions are welcome! Please open an issue or submit a pull request on GitHub.
This project is licensed under multiple licenses:
You can choose either license depending on your project needs.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.