Skip to content

Batch API

Jump to bottom
Gunpal Jain edited this page Nov 15, 2026 · 1 revision

Batch API

Batch processing runs many generateContent (or embedding) requests asynchronously at a lower price than interactive calls. You submit a job, poll it (or get a webhook), and read the results when it finishes.

BatchClient (namespace GenerativeAI.Clients) works on both platforms, with different inputs:

Gemini API Vertex AI (batch prediction jobs)
Input Inline requests (InlinedRequests) or an uploaded JSONL file (FileName) Cloud Storage (GcsUri) or BigQuery (BigqueryUri)
Output Inline responses, or a results file Cloud Storage prefix or BigQuery table (BatchJobDestination)
Embedding batches (CreateEmbeddingsAsync) Yes No (NotSupportedException)
Per-job WebhookConfig Yes No (NotSupportedException)
Cancel Synchronous request Accepted immediately, applied asynchronously

Prerequisites

  • NuGet package: Google_GenerativeAI
  • Gemini API: an API key. Vertex AI: a project, a region and credentials with access to the GCS bucket or BigQuery dataset. See Authentication.
using GenerativeAI;
using GenerativeAI.Clients;
using GenerativeAI.Types;

// Gemini API
var googleAi = new GoogleAi(apiKey);
BatchClient batch = googleAi.BatchClient;

// Vertex AI
var vertexAi = new VertexAI(projectId, "us-central1");
BatchClient vertexBatch = vertexAi.BatchClient;
// or: new BatchClient(platformAdapter)

Gemini API: inline requests

var source = new BatchJobSource
{
    InlinedRequests = new List<InlinedRequest>
    {
        new InlinedRequest
        {
            Contents = new List<Content> { new Content("Write a haiku about rain.", Roles.User) },
            Metadata = new Dictionary<string, string> { ["key"] = "haiku-1" }
        },
        new InlinedRequest
        {
            Contents = new List<Content> { new Content("Write a haiku about snow.", Roles.User) },
            GenerationConfig = new GenerationConfig { Temperature = 0.2 }
        }
    }
};

BatchJob job = await batch.CreateAsync(GoogleAIModels.Gemini38Flash, source);
Console.WriteLine($"{job.Name} {job.State}");   // e.g. "batches/..." JOB_STATE_PENDING

Each InlinedRequest can set Contents, SystemInstruction, GenerationConfig (including a response schema), SafetySettings, Tools, ToolConfig, CachedContent and Metadata.

Gemini API: file input

Upload a JSONL file of requests with the Files API (see Files Client) and pass its name:

var source = new BatchJobSource { FileName = "files/abc123" };
var job = await batch.CreateAsync(GoogleAIModels.Gemini38Flash, source);

Vertex AI: batch prediction from Cloud Storage or BigQuery

// Cloud Storage JSONL in, Cloud Storage out
var job = await vertexBatch.CreateAsync(
    VertexAIModels.Gemini.Gemini38Flash,
    new BatchJobSource { GcsUri = new List<string> { "gs://my-bucket/input/requests.jsonl" } },
    new BatchJobDestination { GcsUri = "gs://my-bucket/output/" });

// BigQuery in, BigQuery out
var bqJob = await vertexBatch.CreateAsync(
    VertexAIModels.Gemini.Gemini38Flash,
    new BatchJobSource { BigqueryUri = "bq://my-project.my_dataset.requests" },
    new BatchJobDestination { BigqueryUri = "bq://my-project.my_dataset" });

The input format defaults to jsonl (GCS) or bigquery; set BatchJobSource.Format / BatchJobDestination.Format to override it. FileName and InlinedRequests are not supported on Vertex AI and throw NotSupportedException.

Get a job and wait for it

GetAsync returns a BatchJobResponse; the job itself is in Metadata on both platforms.

BatchJob? current;
do
{
    await Task.Delay(TimeSpan.FromSeconds(30));
    var response = await batch.GetAsync(job.Name!);
    current = response.Metadata;
    Console.WriteLine(current?.State);
}
while (current?.State is JobState.JOB_STATE_QUEUED or JobState.JOB_STATE_PENDING
       or JobState.JOB_STATE_RUNNING or JobState.JOB_STATE_CANCELLING);

Gemini API BATCH_STATE_* values are converted to the same JobState enum (JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, JOB_STATE_CANCELLED, JOB_STATE_EXPIRED, ...). BatchJob.BatchStats has request counts, and BatchJob.Error explains a failure.

To avoid polling on the Gemini API, pass a WebhookConfig (see below) or register a webhook for WebhookEvents.BatchSucceeded / BatchFailed / BatchExpired. See Webhooks.

Read the results

Gemini API, inline requests: the responses come back in ContentOutput (or EmbeddingOutput for embedding batches):

var finished = await batch.GetAsync(job.Name!);
var results = finished.ContentOutput?.InlinedResponses?.InlinedResponses;
if (results != null)
{
    foreach (var item in results)
    {
        if (item.Response != null)
            Console.WriteLine(item.Response.Text());
        else
            Console.WriteLine($"Request failed: {item.Error?.Message}");
    }
}

For file input, BatchJob.Destination holds the result file (FileName) or inline responses; download a results file with the Files API.

Vertex AI: results are written to your destination. BatchJob.OutputInfo tells you where (GcsOutputDirectory, or BigqueryOutputDataset and BigqueryOutputTable):

var done = (await vertexBatch.GetAsync(job.Name!)).Metadata;
Console.WriteLine(done?.OutputInfo?.GcsOutputDirectory);

List jobs

ListBatchJobsResponse page = await batch.ListAsync(pageSize: 20);
foreach (var j in page.BatchJobs ?? new List<BatchJob>())
    Console.WriteLine($"{j.Name} {j.State}");
// Pass page.NextPageToken as pageToken for the next page.

Cancel and delete

await batch.CancelAsync(job.Name!);
await batch.DeleteBatchJobAsync(job.Name!);

On Vertex AI, cancellation is asynchronous: CancelAsync returns once the request is accepted, and the job then moves through JOB_STATE_CANCELLING to JOB_STATE_CANCELLED. Poll GetAsync if you need to confirm it. A cancelled job cannot be resumed. Deleting a job removes its metadata but not output files that were already written.

Embedding batches (Gemini API only)

CreateEmbeddingsAsync(model, source, destination) creates an asynchronous embedding batch, for example with GoogleAIModels.GeminiEmbedding. Results come back in BatchJobResponse.EmbeddingOutput. On Vertex AI it throws NotSupportedException.

Per-job webhook (Gemini API only)

var job = await batch.CreateAsync(
    GoogleAIModels.Gemini38Flash,
    source,
    destination: null,
    webhookConfig: new WebhookConfig { Uris = new List<string> { "https://example.com/batch-done" } });

Error handling

  • API errors are thrown as GenerativeAI.Exceptions.ApiException (ErrorCode, ErrorStatus, ErrorMessage).
  • ArgumentException: empty model or job name, or a source without a valid input for the platform.
  • NotSupportedException: an input or option the platform doesn't support (see the table above).
  • Individual inline requests can fail while the job succeeds: check InlinedResponse.Error.
try
{
    var job = await batch.CreateAsync(GoogleAIModels.Gemini38Flash, source);
}
catch (ApiException ex) when (ex.ErrorCode == 429)
{
    Console.WriteLine("Batch quota exhausted; try again later.");
}

Related pages

Home

Getting Started

Core Features

Gemini APIs

Vertex AI

Integrations

Guides

Legacy

Clone this wiki locally