BatchAPI - BrowserTest

Overview

The BatchAPI provides methods for efficiently processing multiple screenshots and tests with built-in concurrency control and error handling.

Methods

screenshots()

Processes multiple screenshots concurrently.

screenshots(requests: ScreenshotRequest[], options?: BatchOptions): Promise<BatchScreenshotResults>

Parameters

Parameter	Type	Required	Description
`requests`	`ScreenshotRequest[]`	Yes	Array of screenshot requests
`options`	`BatchOptions`	No	Batch processing options

ScreenshotRequest

interface ScreenshotRequest {
  /** URL to screenshot */
  url: string;
  /** Full page capture (default: false) */
  fullPage?: boolean;
  /** Viewport width (default: 1280) */
  width?: number;
  /** Viewport height (default: 720) */
  height?: number;
  /** Image format: 'png' | 'jpeg' (default: 'png') */
  format?: 'png' | 'jpeg';
  /** Image quality for JPEG (default: 80) */
  quality?: number;
  /** Wait time before screenshot (default: 0) */
  waitFor?: number;
  /** CSS selector for element screenshot */
  selector?: string | null;
  /** Request timeout in milliseconds (default: 10000) */
  timeout?: number;
  /** Block ads and trackers (default: true) */
  blockAds?: boolean;
}

BatchOptions

interface BatchOptions {
  /** Maximum concurrent operations (default: 3) */
  concurrency?: number;
  /** Continue processing on individual failures (default: true) */
  continueOnError?: boolean;
  /** Overall batch timeout in milliseconds */
  timeout?: number;
}

Returns

Promise<BatchScreenshotResults> - Batch screenshot results

BatchScreenshotResults

interface BatchScreenshotResults {
  successful: number;
  total: number;
  results: Array<{
    url: string;
    success: boolean;
    data?: ScreenshotResult['data'];
    error?: string;
    timeMs?: number;
  }>;
  meta: {
    totalTimeMs: number;
    averageTimeMs: number;
    concurrencyUsed: number;
  };
}

Examples

// Basic batch screenshots
const results = await bt.batch.screenshots([
  { url: 'https://site1.com', fullPage: true },
  { url: 'https://site2.com', width: 1920, height: 1080 },
  { url: 'https://site3.com', format: 'jpeg', quality: 90 }
]);

console.log(`${results.successful}/${results.total} screenshots completed`);

// With batch options
const controlledBatch = await bt.batch.screenshots(
  urls.map(url => ({ url, fullPage: true })),
  {
    concurrency: 2, // Process 2 at a time
    timeout: 300000 // 5 minute overall timeout
  }
);

tests()

Processes multiple tests concurrently.

tests(requests: TestRequest[], options?: BatchOptions): Promise<BatchTestResults>

Parameters

Parameter	Type	Required	Description
`requests`	`TestRequest[]`	Yes	Array of test requests
`options`	`BatchOptions`	No	Batch processing options

TestRequest

interface TestRequest {
  /** Test instructions */
  instructions: string;
  /** URL to test */
  url: string;
  /** JSON schema for structured output */
  outputSchema?: object;
  /** Test configuration */
  config?: TestConfig;
}

Returns

Promise<BatchTestResults> - Batch test results

BatchTestResults

interface BatchTestResults {
  successful: number;
  total: number;
  results: Array<{
    instructions: string;
    url: string;
    success: boolean;
    data?: TestResult['results'];
    error?: string;
    timeMs?: number;
  }>;
  meta: {
    totalTimeMs: number;
    averageTimeMs: number;
    concurrencyUsed: number;
  };
}

Examples

// Basic batch tests
const testResults = await bt.batch.tests([
  {
    instructions: 'Check homepage loads correctly',
    url: 'https://example.com'
  },
  {
    instructions: 'Verify login form validation',
    url: 'https://example.com/login'
  },
  {
    instructions: 'Test search functionality',
    url: 'https://example.com/search'
  }
]);

console.log(`${testResults.successful}/${testResults.total} tests passed`);

// With structured output
const advancedTests = await bt.batch.tests([
  {
    instructions: 'Test user registration flow',
    url: 'https://app.com/register',
    outputSchema: {
      type: 'object',
      properties: {
        registrationSuccessful: { type: 'boolean' },
        userCreated: { type: 'boolean' },
        emailSent: { type: 'boolean' }
      }
    },
    config: { timeout: 60000 }
  }
]);

createTests()

Creates multiple async test jobs.

createTests(requests: TestRequest[]): Promise<BatchJobResults>

Parameters

Parameter	Type	Required	Description
`requests`	`TestRequest[]`	Yes	Array of test requests

Returns

Promise<BatchJobResults> - Batch job creation results

BatchJobResults

interface BatchJobResults {
  jobs: Array<{
    jobId: string;
    status: 'pending' | 'running' | 'completed' | 'failed';
    url: string;
    instructions: string;
  }>;
  meta: {
    total: number;
    created: number;
    failed: number;
  };
}

Example

const jobBatch = await bt.batch.createTests([
  {
    instructions: 'Comprehensive site audit',
    url: 'https://example.com',
    config: { timeout: 300000 }
  },
  {
    instructions: 'Performance testing',
    url: 'https://example.com',
    config: { timeout: 180000 }
  }
]);

console.log(`Created ${jobBatch.meta.created} async jobs`);

// Monitor all jobs
for (const job of jobBatch.jobs) {
  const status = await bt.testing.getStatus(job.jobId);
  console.log(`Job ${job.jobId}: ${status.job.status}`);
}

Advanced Usage

Custom Batch Processing

class CustomBatchProcessor {
  constructor(private bt: BrowserTest) {}

  async processWithRetry<TRequest, TResult>(
    requests: TRequest[],
    processor: (request: TRequest) => Promise<TResult>,
    options: {
      concurrency?: number;
      maxRetries?: number;
      retryDelay?: number;
    } = {}
  ): Promise<{
    successful: TResult[];
    failed: Array<{ request: TRequest; error: string; retries: number }>;
  }> {
    const {
      concurrency = 3,
      maxRetries = 2,
      retryDelay = 1000
    } = options;

    const successful: TResult[] = [];
    const failed: Array<{ request: TRequest; error: string; retries: number }> = [];

    // Process in chunks based on concurrency
    for (let i = 0; i < requests.length; i += concurrency) {
      const chunk = requests.slice(i, i + concurrency);
      const chunkPromises = chunk.map(async (request) => {
        let lastError: string = '';
        let retries = 0;

        for (let attempt = 0; attempt <= maxRetries; attempt++) {
          try {
            const result = await processor(request);
            successful.push(result);
            return;
          } catch (error) {
            lastError = error instanceof Error ? error.message : 'Unknown error';
            retries = attempt;

            if (attempt < maxRetries) {
              await new Promise(resolve => setTimeout(resolve, retryDelay * (attempt + 1)));
            }
          }
        }

        failed.push({ request, error: lastError, retries });
      });

      await Promise.all(chunkPromises);
    }

    return { successful, failed };
  }

  async batchScreenshotsWithRetry(
    urls: string[],
    options: Partial<ScreenshotRequest> = {}
  ) {
    return this.processWithRetry(
      urls,
      async (url) => {
        const result = await this.bt.screenshot.take({ url, ...options });
        return result;
      },
      { concurrency: 2, maxRetries: 2 }
    );
  }
}

// Usage
const processor = new CustomBatchProcessor(bt);
const results = await processor.batchScreenshotsWithRetry(
  ['https://site1.com', 'https://site2.com', 'https://site3.com'],
  { fullPage: true }
);

console.log(`Processed: ${results.successful.length} successful, ${results.failed.length} failed`);

Progress Tracking

class BatchProgressTracker {
  constructor(private total: number) {
    this.startTime = Date.now();
  }

  private completed = 0;
  private startTime: number;

  update(count: number = 1) {
    this.completed += count;
    const progress = ((this.completed / this.total) * 100).toFixed(1);
    const elapsed = Date.now() - this.startTime;
    const rate = this.completed / (elapsed / 1000); // items per second
    const eta = ((this.total - this.completed) / rate) * 1000; // milliseconds

    console.log(`Progress: ${this.completed}/${this.total} (${progress}%) - ETA: ${Math.round(eta / 1000)}s`);
  }

  getStats() {
    const elapsed = Date.now() - this.startTime;
    return {
      completed: this.completed,
      total: this.total,
      progress: (this.completed / this.total) * 100,
      elapsedMs: elapsed,
      rate: this.completed / (elapsed / 1000)
    };
  }
}

// Usage
async function trackedBatchScreenshots(urls: string[]) {
  const tracker = new BatchProgressTracker(urls.length);

  const results = await bt.batch.screenshots(
    urls.map(url => ({ url, fullPage: true })),
    {
      concurrency: 2
    }
  );

  // Update progress for each completed item
  results.results.forEach(() => tracker.update());

  const stats = tracker.getStats();
  console.log(`Batch completed in ${stats.elapsedMs}ms at ${stats.rate.toFixed(2)} items/sec`);

  return results;
}

Memory-Efficient Processing

class MemoryEfficientBatchProcessor {
  constructor(private bt: BrowserTest) {}

  async processLargeBatch<T>(
    items: T[],
    processor: (item: T) => Promise<any>,
    options: {
      batchSize?: number;
      delayBetweenBatches?: number;
      onBatchComplete?: (batchIndex: number, results: any[]) => void;
    } = {}
  ) {
    const {
      batchSize = 10,
      delayBetweenBatches = 1000,
      onBatchComplete
    } = options;

    const allResults: any[] = [];

    for (let i = 0; i < items.length; i += batchSize) {
      const batch = items.slice(i, i + batchSize);
      const batchIndex = Math.floor(i / batchSize) + 1;
      const totalBatches = Math.ceil(items.length / batchSize);

      console.log(`Processing batch ${batchIndex}/${totalBatches} (${batch.length} items)`);

      try {
        const batchPromises = batch.map(item => processor(item));
        const batchResults = await Promise.all(batchPromises);

        allResults.push(...batchResults);

        if (onBatchComplete) {
          onBatchComplete(batchIndex, batchResults);
        }

        // Force garbage collection if available
        if (global.gc) {
          global.gc();
        }

      } catch (error) {
        console.error(`Batch ${batchIndex} failed:`, error);
        // Continue with next batch or handle error
      }

      // Delay between batches to prevent overwhelming the system
      if (i + batchSize < items.length) {
        await new Promise(resolve => setTimeout(resolve, delayBetweenBatches));
      }
    }

    return allResults;
  }

  async batchScreenshotLargeSet(urls: string[]) {
    return this.processLargeBatch(
      urls,
      async (url) => {
        const result = await this.bt.screenshot.take({ url, fullPage: true });
        return { url, success: true, data: result.data };
      },
      {
        batchSize: 5,
        delayBetweenBatches: 2000,
        onBatchComplete: (batchIndex, results) => {
          console.log(`Batch ${batchIndex} completed: ${results.length} screenshots`);
        }
      }
    );
  }
}

// Usage
const largeProcessor = new MemoryEfficientBatchProcessor(bt);
const results = await largeProcessor.batchScreenshotLargeSet(largeUrlArray);

Error Handling

Batch-Level Error Handling

async function robustBatchOperation(requests, operation, options = {}) {
  const {
    continueOnError = true,
    maxErrors = 10,
    errorCallback = null
  } = options;

  const results = [];
  let errorCount = 0;

  for (const request of requests) {
    if (!continueOnError && errorCount >= maxErrors) {
      console.log('Too many errors, stopping batch operation');
      break;
    }

    try {
      const result = await operation(request);
      results.push({ success: true, data: result, request });
    } catch (error) {
      errorCount++;
      const errorInfo = {
        success: false,
        error: error.message,
        request,
        attempt: errorCount
      };

      results.push(errorInfo);

      if (errorCallback) {
        errorCallback(errorInfo);
      }

      console.warn(`Operation failed for request:`, request, 'Error:', error.message);
    }
  }

  return {
    results,
    summary: {
      total: requests.length,
      successful: results.filter(r => r.success).length,
      failed: errorCount,
      successRate: ((results.filter(r => r.success).length / requests.length) * 100).toFixed(1) + '%'
    }
  };
}

// Usage
const batchResults = await robustBatchOperation(
  urls,
  async (url) => await bt.screenshot.take({ url }),
  {
    continueOnError: true,
    maxErrors: 5,
    errorCallback: (error) => console.error('Screenshot failed:', error.request, error.error)
  }
);

Circuit Breaker Pattern for Batches

class BatchCircuitBreaker {
  constructor(
    private bt: BrowserTest,
    private failureThreshold = 3,
    private recoveryTimeout = 30000
  ) {
    this.failureCount = 0;
    this.lastFailureTime = null;
    this.state = 'closed';
  }

  private failureCount: number;
  private lastFailureTime: number | null;
  private state: 'closed' | 'open' | 'half-open';

  async executeBatch(requests, operation, options = {}) {
    if (this.state === 'open') {
      if (this.lastFailureTime && Date.now() - this.lastFailureTime > this.recoveryTimeout) {
        this.state = 'half-open';
        console.log('Circuit breaker transitioning to half-open');
      } else {
        throw new Error('Circuit breaker is open - batch operation blocked');
      }
    }

    try {
      const results = await operation(requests, options);
      this.onSuccess();
      return results;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  private onSuccess() {
    this.failureCount = 0;
    this.state = 'closed';
  }

  private onFailure() {
    this.failureCount++;
    this.lastFailureTime = Date.now();

    if (this.failureCount >= this.failureThreshold) {
      this.state = 'open';
      console.log('Circuit breaker opened due to repeated failures');
    }
  }

  getState() {
    return {
      state: this.state,
      failureCount: this.failureCount,
      lastFailureTime: this.lastFailureTime
    };
  }
}

// Usage
const circuitBreaker = new BatchCircuitBreaker(bt, 3, 60000);

try {
  const results = await circuitBreaker.executeBatch(
    urls,
    (requests) => bt.batch.screenshots(requests.map(url => ({ url })))
  );
  console.log('Batch completed successfully');
} catch (error) {
  console.error('Batch failed:', error.message);
  console.log('Circuit breaker state:', circuitBreaker.getState());
}

Performance Optimization

Optimal Concurrency

async function findOptimalConcurrency(urls: string[], maxConcurrency = 5) {
  const results = [];

  for (let concurrency = 1; concurrency <= maxConcurrency; concurrency++) {
    const startTime = Date.now();

    try {
      const batch = await bt.batch.screenshots(
        urls.slice(0, 10).map(url => ({ url, fullPage: true })), // Test with subset
        { concurrency }
      );

      const duration = Date.now() - startTime;
      const throughput = batch.successful / (duration / 1000); // screenshots per second

      results.push({
        concurrency,
        duration,
        throughput,
        successful: batch.successful,
        total: batch.total
      });

      console.log(`Concurrency ${concurrency}: ${throughput.toFixed(2)} screenshots/sec`);

    } catch (error) {
      console.error(`Failed at concurrency ${concurrency}:`, error.message);
      break;
    }
  }

  // Return the best performing configuration
  const best = results.sort((a, b) => b.throughput - a.throughput)[0];
  console.log(`Optimal concurrency: ${best.concurrency} (${best.throughput.toFixed(2)} screenshots/sec)`);

  return best;
}

// Usage
const optimalConfig = await findOptimalConcurrency(testUrls);
// Use optimalConfig.concurrency for future batches

Resource Pooling

class ResourcePool {
  constructor(private maxConcurrent: number) {
    this.available = maxConcurrent;
    this.queue = [];
  }

  private available: number;
  private queue: Array<{ resolve: Function; reject: Function; operation: Function }>;

  async execute<T>(operation: () => Promise<T>): Promise<T> {
    return new Promise((resolve, reject) => {
      this.queue.push({ resolve, reject, operation });
      this.processQueue();
    });
  }

  private async processQueue() {
    if (this.available > 0 && this.queue.length > 0) {
      this.available--;
      const { resolve, reject, operation } = this.queue.shift()!;

      try {
        const result = await operation();
        resolve(result);
      } catch (error) {
        reject(error);
      } finally {
        this.available++;
        this.processQueue();
      }
    }
  }
}

// Usage
const pool = new ResourcePool(3); // Max 3 concurrent operations

const results = await Promise.all(
  urls.map(url =>
    pool.execute(() => bt.screenshot.take({ url, fullPage: true }))
  )
);

Best Practices

Batch Size Optimization

function calculateOptimalBatchSize(estimatedItemTime: number, targetBatchTime: number = 300000) {
  // Target batch completion time (5 minutes default)
  // Assuming some overhead per item
  const overheadPerItem = 500; // 500ms overhead
  const effectiveItemTime = estimatedItemTime + overheadPerItem;

  const optimalSize = Math.floor(targetBatchTime / effectiveItemTime);

  // Clamp to reasonable bounds
  return Math.max(1, Math.min(optimalSize, 50));
}

// Usage
const optimalBatchSize = calculateOptimalBatchSize(5000); // 5 seconds per screenshot
console.log(`Optimal batch size: ${optimalBatchSize}`);

// Process in optimal chunks
for (let i = 0; i < urls.length; i += optimalBatchSize) {
  const batch = urls.slice(i, i + optimalBatchSize);
  const results = await bt.batch.screenshots(batch.map(url => ({ url })));
  // Process results...
}

Monitoring and Alerting

class BatchMonitor {
  constructor(private bt: BrowserTest) {
    this.activeBatches = new Map();
  }

  private activeBatches: Map<string, {
    id: string;
    startTime: number;
    totalItems: number;
    completedItems: number;
  }>;

  startBatch(id: string, totalItems: number) {
    this.activeBatches.set(id, {
      id,
      startTime: Date.now(),
      totalItems,
      completedItems: 0
    });
  }

  updateProgress(batchId: string, completed: number) {
    const batch = this.activeBatches.get(batchId);
    if (batch) {
      batch.completedItems = completed;
      this.checkAlerts(batch);
    }
  }

  private checkAlerts(batch) {
    const progress = batch.completedItems / batch.totalItems;
    const elapsed = Date.now() - batch.startTime;
    const estimatedTotal = elapsed / progress;
    const remaining = estimatedTotal - elapsed;

    if (remaining > 300000) { // More than 5 minutes remaining
      console.warn(`Batch ${batch.id} will take ${Math.round(remaining / 60000)} more minutes`);
    }

    if (elapsed > 600000 && progress < 0.5) { // 10 minutes with less than 50% done
      console.error(`Batch ${batch.id} is running slowly (${(progress * 100).toFixed(1)}% after ${Math.round(elapsed / 60000)} minutes)`);
    }
  }

  endBatch(batchId: string) {
    const batch = this.activeBatches.get(batchId);
    if (batch) {
      const duration = Date.now() - batch.startTime;
      console.log(`Batch ${batchId} completed in ${Math.round(duration / 1000)}s`);
      this.activeBatches.delete(batchId);
    }
  }

  getActiveBatches() {
    return Array.from(this.activeBatches.values());
  }
}

// Usage
const monitor = new BatchMonitor(bt);

async function monitoredBatch(urls: string[]) {
  const batchId = `batch-${Date.now()}`;
  monitor.startBatch(batchId, urls.length);

  const results = await bt.batch.screenshots(
    urls.map(url => ({ url })),
    {
      concurrency: 3
    }
  );

  monitor.updateProgress(batchId, results.successful);
  monitor.endBatch(batchId);

  return results;
}

​Overview

​Methods

​screenshots()

​Parameters

​ScreenshotRequest

​BatchOptions

​Returns

​BatchScreenshotResults

​Examples

​tests()

​Parameters

​TestRequest

​Returns

​BatchTestResults

​Examples

​createTests()

​Parameters

​Returns

​BatchJobResults

​Example

​Advanced Usage

​Custom Batch Processing

​Progress Tracking

​Memory-Efficient Processing

​Error Handling

​Batch-Level Error Handling

​Circuit Breaker Pattern for Batches

​Performance Optimization

​Optimal Concurrency

​Resource Pooling

​Best Practices

​Batch Size Optimization

​Monitoring and Alerting

Overview

Methods

screenshots()

Parameters

ScreenshotRequest

BatchOptions

Returns

BatchScreenshotResults

Examples

tests()

Parameters

TestRequest

Returns

BatchTestResults

Examples

createTests()

Parameters

Returns

BatchJobResults

Example

Advanced Usage

Custom Batch Processing

Progress Tracking

Memory-Efficient Processing

Error Handling

Batch-Level Error Handling

Circuit Breaker Pattern for Batches

Performance Optimization

Optimal Concurrency

Resource Pooling

Best Practices

Batch Size Optimization

Monitoring and Alerting