CasCap.Api.Voice 0.3.0

CasCap.Api.Voice

Speech-to-text and text-to-speech pipelines for .NET 10/11 applications. Use provider-neutral transcription and synthesis contracts while selecting Whisper, Azure or Piper behind dependency injection.

Installation

dotnet package add CasCap.Api.Voice

Usage

Register either pipeline in a .NET Generic Host:

using CasCap.Abstractions;
using CasCap.Models;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddSpeechToText();
builder.Services.AddTextToSpeech();

using var host = builder.Build();
var transcription = host.Services.GetRequiredService<IVoiceTranscriptionService>();
var result = await transcription.Transcribe(
    await File.ReadAllBytesAsync("input.wav"), "audio/wav");

if (result.TranscriptAvailable)
{
    var synthesis = host.Services.GetRequiredService<IVoiceSynthesisService>();
    var reply = await synthesis.TrySynthesizeAsync(
        "The request is complete.", inboundWasVoice: true);

    if (reply.Outcome == VoiceSynthesisOutcome.Success)
        await File.WriteAllBytesAsync("reply.ogg", reply.Audio.ToArray());
}

Provide a reachable transcription backend. Synthesis returns Skipped until its mode is enabled; with the Piper configuration below it produces Ogg Opus audio. The example's transcription input is a local WAV file, not an audio fixture shipped in the package.

IVoiceTranscriptionService and IVoiceSynthesisService are in CasCap.Abstractions; configuration, enums and result records are in CasCap.Models. Check TranscriptAvailable or AudioAvailable before consuming content. Validation, limits and provider failures produce typed outcomes; caller cancellation propagates rather than becoming a successful result.

Providers

AzureSpeechContainer, Kokoro and CoquiXtts are enum values, not supported adapters; selecting one for enabled synthesis throws NotImplementedException when the provider client is resolved. Audio or text is transmitted to the selected backend, including a cloud service when an Azure provider is selected. The library does not deploy that backend.

Configuration

The host supplies configuration through normal .NET providers:

Transcription accepts up to 5 MiB compressed, 19,200,000 decoded bytes and 300 seconds of audio. Configure MaxCompressedBytes, MaxDecodedBytes and MaxDurationSeconds independently; all must pass. Supported containers include WAV, AAC, MP3, MP4/M4A, Ogg/Opus and FLAC, with matching file signatures. Non-normalized input requires ffmpeg and is converted to 16 kHz mono signed 16-bit PCM WAV. Conversion uses pipes, not temporary files. One transcription runs per service instance; its timeout includes admission, conversion and the backend request.

SpeechToTextConfig.Mode communicates Disabled, Shadow or Enabled to a consuming host's orchestration. A direct Transcribe call does not inspect that mode: the host must gate its own receive, processing and reply paths. The library itself contains no agent or messaging workflow.

TextToSpeechConfig.Mode is enforced by synthesis: Disabled skips, MatchInbound speaks only when inboundWasVoice is true, and Always permits any reply. Empty or non-speakable text is skipped; normalized text over MaxCharacters returns TooLong.

For Azure transcription set AzureEndpoint; for Azure Speech synthesis set AzureSpeechEndpoint. For Azure OpenAI set AzureOpenAiEndpoint, AzureOpenAiDeployment and a service-supported AzureOpenAiApiVersion (the code defaults to 2025-03-01-preview). The host supplies IOptions<AzureAuthConfig> with a TokenCredential; these registration methods do not acquire one. Provider-specific voice settings are AzureSpeechVoice, AzureOpenAiVoice or PiperVoice.

Set either pipeline's FfmpegPath when ffmpeg is not on PATH. See the runtime requirements before choosing an application image.

Configuration Examples

Default transcription, no spoken replies

{
  "CasCap": {
    "SpeechToTextConfig": {
      "Provider": "WhisperAsr",
      "WhisperAsrEndpoint": "http://localhost:9000"
    },
    "TextToSpeechConfig": {
      "Mode": "Disabled"
    }
  }
}

whisper.cpp transcription and Piper replies

{
  "CasCap": {
    "SpeechToTextConfig": {
      "Provider": "WhisperCpp",
      "WhisperCppEndpoint": "http://localhost:8080",
      "Mode": "Enabled",
      "Language": "en",
      "MaxCompressedBytes": 5242880,
      "MaxDecodedBytes": 19200000,
      "MaxDurationSeconds": 300,
      "TimeoutMs": 120000,
      "FfmpegPath": "ffmpeg"
    },
    "TextToSpeechConfig": {
      "Mode": "MatchInbound",
      "Provider": "Piper",
      "PiperEndpoint": "localhost:10200",
      "MaxCharacters": 1000,
      "TimeoutMs": 60000,
      "FfmpegPath": "ffmpeg"
    },
    "VoiceMetricsConfig": {
      "MetricNamePrefix": "voice"
    }
  }
}

Piper uses host:port, not an HTTP URL. Supply real remote endpoints and credentials through secret-backed configuration.

Data Flow

The two pipelines are independent; the host decides whether a transcript should lead to a reply. The package dependency graph shows which shared libraries support these providers.

flowchart LR
  audio([Encoded audio]) --> validate[Validate type, signature and size]
  validate --> normalize[Normalize WAV if needed]
  normalize --> limits[Check decoded size and duration]
  limits --> stt[Selected speech-to-text provider]
  stt --> transcript([Typed transcription result])
  text([Reply text]) --> policy[Reply mode and length policy]
  policy --> tts[Selected text-to-speech provider]
  tts --> result([Typed synthesis result])
  classDef owned fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
  classDef terminal fill:#dcfce7,stroke:#15803d,color:#14532d
  class validate,normalize,limits,stt,policy,tts owned
  class audio,transcript,text,result terminal

Diagnostics and Testing

The transcription meter is named by MetricNamePrefix. Instruments use <prefix>.voice.transcriptions, <prefix>.voice.audio.duration, and duration/speed measurements for transcoding and transcription. Dimensions are bounded provider/outcome values, not media names, recordings or transcripts. Exporter registration belongs to the consuming host.

Logs carry sizes, durations, counts, status codes and outcomes, never audio, transcripts, reply text or local paths. Each transcription or synthesis logs once at the pipeline boundary: Information on success, Warning for a rejected, timed-out or empty result (an over-long reply is skipped at Debug) and Error for a backend failure; a missing ffmpeg binary adds one Error. Provider adapters log their own rejections, empty results and encoding failures only at Debug, because the pipeline or the caller already reports the thrown exception or empty response.

Provider adapters implement Microsoft.Extensions.AI ISpeechToTextClient and ITextToSpeechClient. Registrations use TryAdd, so applications can replace the clients or whole pipelines. Use the Testing companion for canned clients without a speech backend.

License

Released under the MIT License.