Batch Processing

Use model.batch() to process multiple prompts with a single method call. You can configure concurrency, share invocation options across requests, handle individual failures, and cancel batch work with an AbortSignal. Successful results preserve the original input order, regardless of completion order.

Basic usage

import 'dotenv/config'import { DeepSeekWeb } from '@orcushq/deepseek' const model = new DeepSeekWeb({  model: 'deepseek-v3',  dsSessionId: process.env.DEEPSEEK_SESSION_ID ?? '',  authorization: process.env.DEEPSEEK_AUTHORIZATION ?? '',}) const responses = await model.batch([  'Explain HTTP in one sentence.',  'Explain TCP in one sentence.',  'Explain DNS in one sentence.',]) for (const response of responses) {  console.log(response.content)}

Set DEEPSEEK_SESSION_ID and DEEPSEEK_AUTHORIZATION in your .env file before running the example. The SDK does not load environment variables automatically; dotenv/config loads them for this example.

Each successful result is a DeepSeekResponse. Use response.content to access the generated text and other response fields when needed. An empty input array returns an empty array.

Concurrency

Use concurrency to set the maximum number of requests that can run simultaneously. The default client-level value is 4; you can set it in the DeepSeekWeb constructor or override it per batch() call. Effective concurrency never exceeds the number of inputs.

const responses = await model.batch(  [    'Summarize document A.',    'Summarize document B.',    'Summarize document C.',    'Summarize document D.',  ],  {    concurrency: 2,  },)

Concurrency must be a positive integer. Values such as 0, negative numbers, decimals, NaN, and Infinity are rejected with a configuration error.

Result ordering

Results preserve input order even when requests finish in a different order. You can safely associate responses[i] with the input at index i.

const prompts = [  'Explain HTTP.',  'Explain TCP.',  'Explain DNS.',] const responses = await model.batch(prompts) console.log(responses[0]?.content) // Result for 'Explain HTTP.'console.log(responses[1]?.content) // Result for 'Explain TCP.'console.log(responses[2]?.content) // Result for 'Explain DNS.'

Options

• concurrency: Maximum number of simultaneous requests. Defaults to 4 unless overridden by client configuration.
• continueOnError: Defaults to false. When true, processing continues after individual failures, then throws BatchAggregateError if any request failed.
• signal: An optional AbortSignal for cancelling batch work and propagating cancellation to requests.
• Invocation options: Supported DeepSeekInvokeOptions, including options such as model and timeoutMs, apply to each input in the batch.

Invocation options are shared across all inputs in a single batch call; they are not configured separately for each input.

Error handling

By default, batch() uses fail-fast behavior. If a request fails, the batch rejects with the original error and stops scheduling new work. Requests already in progress may still finish; rejection does not guarantee that every in-flight request has been forcibly cancelled.

try {  const responses = await model.batch(prompts)   for (const response of responses) {    console.log(response.content)  }} catch (error) {  console.error('Batch failed:', error)}

Continue after individual failures

Set continueOnError: true to process the remaining inputs after individual failures. If any request fails, the method throws BatchAggregateError after processing finishes. The aggregate error exposes errors and failures, but does not provide a results collection containing successful responses. Inspect the failure details to identify unsuccessful inputs.

try {  const responses = await model.batch(prompts, {    concurrency: 3,    continueOnError: true,  })   for (const response of responses) {    console.log(response.content)  }} catch (error) {  if (error instanceof Error && error.name === 'BatchAggregateError') {    const aggregate = error as {      errors?: readonly unknown[]      failures?: readonly unknown[]    }     console.error('Failed requests:', aggregate.failures)    console.error('Errors:', aggregate.errors)  } else {    throw error  }}

The failures entries contain the failed input index and its error. Failure entries follow failure encounter order, which may differ from input order. BatchAggregateError extends the standard Error class and is exported as a type only, so detect it using error.name === "BatchAggregateError" rather than importing it as a runtime class.

Cancellation

Pass an AbortSignal to request cancellation. An already-aborted signal prevents work from starting. When cancellation occurs, the scheduler stops assigning new inputs and propagates the signal to active invocations.

const controller = new AbortController() const batchPromise = model.batch(prompts, {  concurrency: 3,  signal: controller.signal,}) // Cancel when your application no longer needs the results.controller.abort() try {  const responses = await batchPromise  console.log(responses)} catch (error) {  console.error('Batch stopped:', error)}

Structured-output batch

You can combine batch processing with withStructuredOutput() to validate each response against a Zod schema. Each successful structured result exposes value, which contains the parsed and schema-validated object. Structured batch failures follow the batch error-handling behavior described above.

import { z } from 'zod' const Person = z.object({  name: z.string(),  age: z.number(),}) const structured = model.withStructuredOutput(Person) try {  const results = await structured.batch(    [      'Return a person named Alice aged 30.',      'Return a person named Bob aged 25.',    ],    {      concurrency: 2,    },  )   for (const result of results) {    console.log(result.value)  }} catch (error) {  if (error instanceof Error && error.name === 'BatchAggregateError') {    const aggregate = error as {      errors?: readonly unknown[]      failures?: readonly unknown[]    }     console.error('Batch failures:', aggregate.failures)    console.error('Errors:', aggregate.errors)  } else {    throw error  }}

Each successful structured result contains value, raw, response, and metadata. Use value for the validated object, not content or value from a normal DeepSeekResponse. If a batch fails with continueOnError: true, inspect the aggregate error for failure details; successful partial results are not exposed through BatchAggregateError.results.

Timeouts and retries

Invocation-level timeout and retry behavior is delegated to the existing invocation and transport layers. A request timeout applies to the individual invocation; batch does not provide a separate overall timeout for the entire batch. Configure concurrency appropriately for your workload and handle upstream or rate-limit errors.

The constructor-level retry policy is not forwarded to the turn-level retry logic used by batch() in the current implementation. Internal transport calls also disable transport-level retries. Do not assume a client-level retry setting changes batch retry behavior; see Reliability for the method-specific retry scope.

Important notes

• Batching is a client-side convenience API, not a server-side batch job or a provider-native bulk endpoint.
• Each input is processed as an independent invocation.
• Higher concurrency does not guarantee higher throughput; provider limits and session capacity still apply.
• Results are collected in memory and returned after the operation completes.
• Already-running requests may finish after a fail-fast rejection.
• When continueOnError: true and any input fails, inspect BatchAggregateError for failure details; successful partial results are not exposed by that error.