Skip to content

Multimodal Features

Jump to bottom
Gunpal Jain edited this page Nov 15, 2026 · 4 revisions

Gemini models accept images, video, audio, PDFs and text files together with a text prompt. This page shows how to attach that media to a request: by local file path, inline data, an uploaded file or a remote URI.

Package: Google_GenerativeAI. Setup: Initialization.

Ways to Attach Media

Method Where Notes
GeminiModel.GenerateContentAsync(prompt, filePath) / StreamContentAsync(prompt, filePath) Gemini API Small files of a supported type are sent inline. Larger files (up to 2 GB) are uploaded with the Files API, and the call waits for them to become ACTIVE.
request.AddInlineFile(filePath) Both Reads the file and sends it inline as base64 (keep inline data under about 20 MB).
request.AddInlineData(base64, mimeType) Both You provide the base64 data yourself.
request.AddRemoteFile(RemoteFile) Gemini API A file uploaded with FileClient. See Files Client.
request.AddRemoteFile(uri, mimeType) Both A Files API URI, a gs:// URI (Vertex AI) or a public URL.

The file-path overloads are on GeminiModel. Create one with GoogleAi.CreateGeminiModel(...).

using GenerativeAI;
using GenerativeAI.Types;

var googleAI = new GoogleAi("YOUR_API_KEY");
GeminiModel model = googleAI.CreateGeminiModel(GoogleAIModels.Gemini38Flash);

Identifying Objects in an Image (request builder)

var request = new GenerateContentRequest();
request.AddInlineFile("image.png");
request.AddText("Identify objects in the image?");

var result = await model.GenerateContentAsync(request);
Console.WriteLine(result.Text());

Image by File Path

var result = await model.GenerateContentAsync("Identify objects in the image?", "image.png");
Console.WriteLine(result.Text());

Video by File Path

var result = await model.GenerateContentAsync("Describe this video?", "TestData/testvideo.mp4");
Console.WriteLine(result.Text());

Large videos are uploaded with the Files API first. GeminiModel.TimeoutForFileStateCheck (seconds, default 300) sets how long the call waits for the file to finish processing.

Audio by File Path

var result = await model.GenerateContentAsync("Describe this audio?", "TestData/testaudio.mp3");
Console.WriteLine(result.Text());

Streaming with Multimodal Input

await foreach (var chunk in model.StreamContentAsync("Identify objects in the image?", "image.png"))
{
    Console.WriteLine($"Chunk: {chunk.Text()}");
}

Chat with Multimodal Input and Streaming

ChatSession has no file-path overload. Add the file to a request instead:

var chat = model.StartChat();

var request = new GenerateContentRequest();
request.AddInlineFile("image.png");
request.AddText("Identify objects in the image?");

await foreach (var chunk in chat.StreamContentAsync(request))
{
    Console.Write(chunk.Text());
}

var response2 = await chat.GenerateContentAsync("Can you estimate the number of blueberries?");
Console.WriteLine(response2.Text());

Using an Uploaded File

RemoteFile file = await model.Files!.UploadFileAsync("report.pdf");
await model.Files.AwaitForFileStateActiveAsync(file, 120, CancellationToken.None);

var request = new GenerateContentRequest();
request.AddRemoteFile(file);
request.AddText("Summarize this report.");

var response = await model.GenerateContentAsync(request);

Vertex AI

On Vertex AI, models created with VertexAI.CreateGenerativeModel are plain GenerativeModel instances without the file-path overloads. Use inline data, or reference files in Cloud Storage:

var vertex = new VertexAI("your-project-id", "global");
var vModel = vertex.CreateGenerativeModel(VertexAIModels.Gemini.Gemini38Flash);

var request = new GenerateContentRequest();
request.AddRemoteFile("gs://cloud-samples-data/generative-ai/audio/pixel.mp3", "audio/mp3");
request.AddText("What is this audio about?");

var response = await vModel.GenerateContentAsync(request);

Media Resolution

GenerationConfig.MediaResolution (MEDIA_RESOLUTION_LOW, MEDIA_RESOLUTION_MEDIUM or MEDIA_RESOLUTION_HIGH) trades image and video detail against token cost:

request.GenerationConfig = new GenerationConfig { MediaResolution = MediaResolution.MEDIA_RESOLUTION_LOW };

Important Considerations

  • File paths: Make sure the paths exist and your application can read them. Missing files throw FileNotFoundException.
  • Supported types: The SDK checks MIME types against InlineMimeTypes.AllowedMimeTypes (inline) and FilesConstants.SupportedMimeTypes (upload). Unsupported types throw NotSupportedException.
  • Model capabilities: Check that your model supports the input modality you send.
  • Streaming: Chunks are partial. Join them if you need the complete output.

Related Pages

API Reference

Home

Getting Started

Core Features

Gemini APIs

Vertex AI

Integrations

Guides

Legacy

Clone this wiki locally