Orcus

Image input

Send images to Gemini from data URLs, https URLs or raw bytes. The package uploads them and attaches the returned file references to the request.

Image input is a content part on a user message. Text and images can be mixed in the same turn, in any order.

src/images.ts
await model.invoke([  {    role: 'user',    content: [      { type: 'text', text: 'What is in this picture?' },      { type: 'image', data: pngBytes, mime_type: 'image/png' },      { type: 'image_url', image_url: { url: 'https://example.com/a.png' } },      { type: 'image', data: 'iVBORw0KGgoAAAANSUhEUg...', mime_type: 'image/png' },    ],  },])

Accepted sources

  • Raw bytes: a Uint8Array in data, with mime_type.
  • Base64: a base64 string in data, with or without a data: URL prefix.
  • Remote URL: an https URL in image_url.url or url.
  • OpenAI shape: image_url accepts either a plain string or { url, mime_type }.

Part type aliases

Text parts accept `text`, `input_text` and `output_text`. Image parts accept `image`, `input_image` and `image_url`, which makes it easy to reuse content you already built for another SDK.

Uploads

Remote images and raw bytes are uploaded through the Google upload endpoint and the returned file references are attached to the payload. Upload page tokens are cached per client and refreshed on demand, so keep one long lived client per configuration rather than creating one per request.

Helpers

`detectImageMime` and `resolveImageSource` are exported if you need to resolve or validate an image source yourself before building the message.

Not supported

  • Audio, video and document parts.
  • Plain http URLs. Use https.
  • Streaming a generated image back as a URL. The package reads text, tool calls and metadata, not generated media.

Hosted URLs are fetched from your server

A remote image URL is fetched from wherever the SDK runs. Do not accept arbitrary URLs from untrusted input without validating them first.