Batch Operations

Overview

Batch operations allow you to process multiple screenshots and tests simultaneously, with built-in concurrency control and error handling. This is perfect for large-scale testing and monitoring scenarios.

Batch Screenshots

Basic Batch Screenshot

const batchResults = await bt.screenshot.takeBatch({
  urls: [
    'https://site1.com',
    'https://site2.com',
    'https://site3.com',
    'https://site4.com',
    'https://site5.com'
  ],
  fullPage: true,
  format: 'png'
});

console.log(`Processed ${batchResults.meta.successful} out of ${batchResults.meta.total} screenshots`);
console.log(`Total time: ${batchResults.meta.timeMs}ms`);

Individual Options per Screenshot

const mixedBatch = await bt.screenshot.takeBatch({
  requests: [
    {
      url: 'https://mobile-site.com',
      width: 375,
      height: 667,
      fullPage: true
    },
    {
      url: 'https://desktop-site.com',
      width: 1920,
      height: 1080,
      fullPage: true,
      format: 'jpeg',
      quality: 90
    },
    {
      url: 'https://tablet-site.com',
      width: 768,
      height: 1024,
      selector: '.main-content'
    }
  ]
});

Concurrency Control

Control how many operations run simultaneously:

const controlledBatch = await bt.batch.screenshots([
  { url: 'https://site1.com', fullPage: true },
  { url: 'https://site2.com', fullPage: true },
  { url: 'https://site3.com', fullPage: true },
  { url: 'https://site4.com', fullPage: true },
  { url: 'https://site5.com', fullPage: true }
], {
  concurrency: 2 // Process 2 screenshots at a time
});

console.log('Batch completed with controlled concurrency');

Batch Testing

Basic Batch Test

const testResults = await bt.batch.tests([
  {
    instructions: 'Check if homepage loads correctly',
    url: 'https://example.com'
  },
  {
    instructions: 'Verify login form validation',
    url: 'https://example.com/login'
  },
  {
    instructions: 'Test search functionality',
    url: 'https://example.com/search'
  }
]);

console.log(`${testResults.successful} out of ${testResults.total} tests passed`);

Advanced Batch Testing

const advancedBatch = await bt.batch.tests([
  {
    instructions: 'Navigate to products page and verify items load',
    url: 'https://shop.com/products',
    outputSchema: {
      type: 'object',
      properties: {
        productsLoaded: { type: 'boolean' },
        itemCount: { type: 'number' }
      }
    }
  },
  {
    instructions: 'Test shopping cart functionality',
    url: 'https://shop.com/cart',
    config: {
      viewport: { width: 1280, height: 720 },
      timeout: 30000
    }
  }
], {
  concurrency: 1, // Sequential execution for complex tests
  timeout: 60000
});

Create Tests in Batch

For async jobs, create multiple tests that run in the background:

const batchJobs = await bt.batch.createTests([
  {
    instructions: 'Complete e-commerce checkout flow',
    url: 'https://shop.com/checkout'
  },
  {
    instructions: 'Test user registration process',
    url: 'https://app.com/register'
  },
  {
    instructions: 'Verify API documentation loads',
    url: 'https://api-docs.com'
  }
]);

console.log('Created jobs:', batchJobs.jobs.map(job => job.jobId));

// Monitor all jobs
for (const job of batchJobs.jobs) {
  const status = await bt.testing.getStatus(job.jobId);
  console.log(`Job ${job.jobId}: ${status.job.status}`);
}

Mixed Batch Operations

Screenshots and Tests Together

async function comprehensiveSiteCheck(url) {
  // Take screenshots first
  const screenshots = await bt.screenshot.takeBatch({
    requests: [
      { url, fullPage: true, format: 'png' },
      { url: `${url}/about`, fullPage: true, format: 'png' },
      { url: `${url}/contact`, fullPage: true, format: 'png' }
    ]
  });

  // Run tests in parallel
  const tests = await bt.batch.tests([
    {
      instructions: 'Check homepage for broken links',
      url
    },
    {
      instructions: 'Verify contact form works',
      url: `${url}/contact`
    },
    {
      instructions: 'Test navigation menu',
      url
    }
  ]);

  return {
    screenshots: screenshots.results,
    tests: tests.results,
    summary: {
      screenshotsTaken: screenshots.meta.successful,
      testsPassed: tests.successful,
      totalTime: Math.max(screenshots.meta.timeMs, tests.meta?.timeMs || 0)
    }
  };
}

Error Handling

Handling Partial Failures

const batchResults = await bt.screenshot.takeBatch({
  urls: [
    'https://reliable-site.com',
    'https://unreliable-site.com', // Might fail
    'https://another-site.com',
    'https://slow-site.com' // Might timeout
  ],
  fullPage: true
});

// Process results, handling failures gracefully
const successful = batchResults.results.filter(result => result.success);
const failed = batchResults.results.filter(result => !result.success);

console.log(`${successful.length} screenshots succeeded`);
console.log(`${failed.length} screenshots failed`);

failed.forEach(failure => {
  console.log(`Failed: ${failure.url} - ${failure.error}`);
});

// Continue with successful results
await processSuccessfulScreenshots(successful);

Retry Logic for Batch Operations

async function batchWithRetry(requests, options = {}) {
  const maxRetries = options.maxRetries || 3;
  let results = [];
  let remainingRequests = [...requests];

  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    if (remainingRequests.length === 0) break;

    console.log(`Attempt ${attempt} for ${remainingRequests.length} requests`);

    const batchResults = await bt.screenshot.takeBatch({
      requests: remainingRequests,
      ...options
    });

    const successful = batchResults.results.filter(r => r.success);
    const failed = batchResults.results.filter(r => !r.success);

    results.push(...successful);

    if (failed.length === 0) break;

    remainingRequests = failed.map(f => ({
      url: f.url,
      ...options // Retry with same options
    }));

    if (attempt < maxRetries) {
      console.log(`Waiting before retry...`);
      await new Promise(resolve => setTimeout(resolve, 2000));
    }
  }

  return {
    successful: results,
    failed: remainingRequests,
    totalAttempts: maxRetries
  };
}

Performance Optimization

Optimal Concurrency

// Test different concurrency levels
async function findOptimalConcurrency(urls, maxConcurrency = 5) {
  const results = [];

  for (let concurrency = 1; concurrency <= maxConcurrency; concurrency++) {
    const startTime = Date.now();

    const batch = await bt.batch.screenshots(
      urls.map(url => ({ url, fullPage: true })),
      { concurrency }
    );

    const duration = Date.now() - startTime;

    results.push({
      concurrency,
      duration,
      throughput: urls.length / (duration / 1000), // screenshots per second
      successful: batch.successful
    });
  }

  // Return the best performing configuration
  return results.sort((a, b) => b.throughput - a.throughput)[0];
}

Memory Management

// Process large batches in chunks
async function processLargeBatch(urls, chunkSize = 10) {
  const results = [];

  for (let i = 0; i < urls.length; i += chunkSize) {
    const chunk = urls.slice(i, i + chunkSize);
    console.log(`Processing chunk ${Math.floor(i / chunkSize) + 1} of ${Math.ceil(urls.length / chunkSize)}`);

    const chunkResults = await bt.screenshot.takeBatch({
      urls: chunk,
      fullPage: true
    });

    results.push(...chunkResults.results);

    // Optional: save intermediate results to disk
    await saveIntermediateResults(chunkResults);

    // Prevent memory issues with large datasets
    if (global.gc) global.gc();
  }

  return results;
}

Monitoring & Reporting

Batch Progress Tracking

class BatchMonitor {
  constructor(totalOperations) {
    this.total = totalOperations;
    this.completed = 0;
    this.startTime = Date.now();
  }

  update(progress) {
    this.completed = progress;
    const percentage = ((this.completed / this.total) * 100).toFixed(1);
    const elapsed = Date.now() - this.startTime;
    const estimated = elapsed * (this.total / this.completed);
    const remaining = estimated - elapsed;

    console.log(`Progress: ${percentage}% (${this.completed}/${this.total})`);
    console.log(`Estimated remaining: ${(remaining / 1000).toFixed(0)}s`);
  }

  complete() {
    const totalTime = Date.now() - this.startTime;
    console.log(`Batch completed in ${(totalTime / 1000).toFixed(1)}s`);
    console.log(`Average time per operation: ${(totalTime / this.total).toFixed(0)}ms`);
  }
}

// Usage
const monitor = new BatchMonitor(urls.length);
const results = await bt.screenshot.takeBatch({
  urls,
  progressCallback: (completed) => monitor.update(completed)
});
monitor.complete();

Batch Analytics

async function analyzeBatchResults(results) {
  const analytics = {
    total: results.length,
    successful: results.filter(r => r.success).length,
    failed: results.filter(r => !r.success).length,
    averageTime: results.reduce((sum, r) => sum + (r.meta?.timeMs || 0), 0) / results.length,
    errorTypes: {},
    sizeStats: {
      min: Infinity,
      max: 0,
      avg: 0
    }
  };

  // Analyze errors
  results.filter(r => !r.success).forEach(result => {
    const errorType = categorizeError(result.error);
    analytics.errorTypes[errorType] = (analytics.errorTypes[errorType] || 0) + 1;
  });

  // Analyze sizes for successful screenshots
  const sizes = results.filter(r => r.success && r.data?.screenshot)
    .map(r => r.data.screenshot.length);

  if (sizes.length > 0) {
    analytics.sizeStats.min = Math.min(...sizes);
    analytics.sizeStats.max = Math.max(...sizes);
    analytics.sizeStats.avg = sizes.reduce((a, b) => a + b) / sizes.length;
  }

  return analytics;
}

Real-World Examples

Website Monitoring

async function monitorWebsites() {
  const sites = [
    'https://primary-site.com',
    'https://backup-site.com',
    'https://api-site.com',
    'https://docs-site.com'
  ];

  const results = await bt.screenshot.takeBatch({
    requests: sites.map(url => ({
      url,
      fullPage: true,
      timeout: 30000
    }))
  }, {
    concurrency: 2
  });

  // Send alerts for failed sites
  const failedSites = results.results.filter(r => !r.success);
  if (failedSites.length > 0) {
    await sendAlert('Website monitoring alert', {
      failedSites: failedSites.map(s => s.url),
      totalChecked: sites.length,
      failureRate: (failedSites.length / sites.length * 100).toFixed(1) + '%'
    });
  }

  return results;
}

Visual Regression Testing

async function visualRegressionTest(pages, baselineDir) {
  // Capture current screenshots
  const currentResults = await bt.screenshot.takeBatch({
    requests: pages.map(page => ({
      url: page.url,
      fullPage: true,
      viewport: page.viewport || { width: 1280, height: 720 }
    }))
  });

  const regressions = [];

  for (const result of currentResults.results) {
    if (!result.success) continue;

    const baselinePath = path.join(baselineDir, `${result.url.replace(/[^a-zA-Z0-9]/g, '_')}.png`);
    const baselineExists = fs.existsSync(baselinePath);

    if (!baselineExists) {
      // Save as new baseline
      fs.writeFileSync(baselinePath, Buffer.from(result.data.screenshot, 'base64'));
      console.log(`New baseline saved for ${result.url}`);
    } else {
      // Compare with baseline
      const baseline = fs.readFileSync(baselinePath);
      const diff = await compareImages(result.data.screenshot, baseline);

      if (diff.percentage > 0.1) { // More than 0.1% difference
        regressions.push({
          url: result.url,
          difference: diff.percentage,
          current: result.data.screenshot,
          baseline: baseline.toString('base64')
        });
      }
    }
  }

  return {
    regressions,
    totalPages: pages.length,
    checkedPages: currentResults.meta.successful
  };
}

Content Validation

async function validateContent(pages) {
  const tests = pages.map(page => ({
    instructions: `
      Navigate to the page
      Check for required elements:
      - Page title should not be empty
      - Main content should be visible
      - No console errors
      - All images should load
      - Links should not be broken
    `,
    url: page.url,
    outputSchema: {
      type: 'object',
      properties: {
        titlePresent: { type: 'boolean' },
        contentVisible: { type: 'boolean' },
        noConsoleErrors: { type: 'boolean' },
        imagesLoad: { type: 'boolean' },
        linksValid: { type: 'boolean' },
        loadTime: { type: 'number' }
      }
    }
  }));

  const results = await bt.batch.tests(tests, {
    concurrency: 3,
    timeout: 45000
  });

  // Generate report
  const report = {
    timestamp: new Date().toISOString(),
    totalPages: pages.length,
    passed: results.successful,
    failed: results.total - results.successful,
    details: results.results.map((result, index) => ({
      url: pages[index].url,
      success: result.success,
      data: result.data,
      error: result.error
    }))
  };

  return report;
}

Best Practices

Batch Size Optimization

Start small: Test with small batches to find optimal sizes
Monitor resources: Watch memory and CPU usage
Use concurrency: Balance speed with resource usage
Handle failures: Implement retry logic for reliability
Monitor progress: Track batch completion and errors

Error Recovery

// Implement circuit breaker pattern
class BatchCircuitBreaker {
  constructor(failureThreshold = 5, recoveryTimeout = 60000) {
    this.failureCount = 0;
    this.failureThreshold = failureThreshold;
    this.recoveryTimeout = recoveryTimeout;
    this.lastFailureTime = null;
    this.state = 'closed'; // closed, open, half-open
  }

  async execute(operation) {
    if (this.state === 'open') {
      if (Date.now() - this.lastFailureTime > this.recoveryTimeout) {
        this.state = 'half-open';
      } else {
        throw new Error('Circuit breaker is open');
      }
    }

    try {
      const result = await operation();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  onSuccess() {
    this.failureCount = 0;
    this.state = 'closed';
  }

  onFailure() {
    this.failureCount++;
    this.lastFailureTime = Date.now();

    if (this.failureCount >= this.failureThreshold) {
      this.state = 'open';
    }
  }
}

Resource Management

// Implement resource pools
class ResourcePool {
  constructor(maxConcurrent = 3) {
    this.maxConcurrent = maxConcurrent;
    this.running = 0;
    this.queue = [];
  }

  async execute(operation) {
    return new Promise((resolve, reject) => {
      this.queue.push({ operation, resolve, reject });
      this.processQueue();
    });
  }

  async processQueue() {
    if (this.running >= this.maxConcurrent || this.queue.length === 0) {
      return;
    }

    this.running++;
    const { operation, resolve, reject } = this.queue.shift();

    try {
      const result = await operation();
      resolve(result);
    } catch (error) {
      reject(error);
    } finally {
      this.running--;
      this.processQueue();
    }
  }
}

Getting Started

Core Features

Advanced Usage

Overview

Batch Screenshots

Basic Batch Screenshot

Individual Options per Screenshot

Concurrency Control

Batch Testing

Basic Batch Test

Advanced Batch Testing

Create Tests in Batch

Mixed Batch Operations

Screenshots and Tests Together

Error Handling

Handling Partial Failures

Retry Logic for Batch Operations

Performance Optimization

Optimal Concurrency

Memory Management

Monitoring & Reporting

Batch Progress Tracking

Batch Analytics

Real-World Examples

Website Monitoring

Visual Regression Testing

Content Validation

Best Practices

Batch Size Optimization

Error Recovery

Resource Management

​Overview

​Batch Screenshots

​Basic Batch Screenshot

​Individual Options per Screenshot

​Concurrency Control

​Batch Testing

​Basic Batch Test

​Advanced Batch Testing

​Create Tests in Batch

​Mixed Batch Operations

​Screenshots and Tests Together

​Error Handling

​Handling Partial Failures

​Retry Logic for Batch Operations

​Performance Optimization

​Optimal Concurrency

​Memory Management

​Monitoring & Reporting

​Batch Progress Tracking

​Batch Analytics

​Real-World Examples

​Website Monitoring

​Visual Regression Testing

​Content Validation

​Best Practices

​Batch Size Optimization

​Error Recovery

​Resource Management

Overview

Batch Screenshots

Basic Batch Screenshot

Individual Options per Screenshot

Concurrency Control

Batch Testing

Basic Batch Test

Advanced Batch Testing

Create Tests in Batch

Mixed Batch Operations

Screenshots and Tests Together

Error Handling

Handling Partial Failures

Retry Logic for Batch Operations

Performance Optimization

Optimal Concurrency

Memory Management

Monitoring & Reporting

Batch Progress Tracking

Batch Analytics

Real-World Examples

Website Monitoring

Visual Regression Testing

Content Validation

Best Practices

Batch Size Optimization

Error Recovery

Resource Management