Async Test Jobs

Overview

Async jobs allow you to run complex, long-running tests in the background. This is perfect for tests that take longer than typical API timeouts or need to run multiple complex operations.

Creating Async Jobs

From Templates

The most common way to create async jobs is through templates:

// Create a template first
const template = await bt.template.create({
  name: 'Complex E-commerce Test',
  instructions: `
    Navigate to the product catalog
    Add multiple items to cart
    Proceed through checkout
    Test payment processing
    Verify order confirmation
    Check email notifications
  `,
  outputSchema: {
    type: 'object',
    properties: {
      cartOperations: { type: 'boolean' },
      checkoutFlow: { type: 'boolean' },
      paymentProcessing: { type: 'boolean' },
      orderConfirmation: { type: 'boolean' },
      emailNotifications: { type: 'boolean' },
      totalTime: { type: 'number' }
    }
  }
});

// Invoke template to create async job
const job = await bt.template.invoke(template.template.id, {
  url: 'https://example-shop.com'
});

console.log('Job created with ID:', job.jobId);
console.log('Initial status:', job.status);

Direct Job Creation

Create jobs directly without templates:

const job = await bt.testing.create({
  instructions: `
    Perform comprehensive site audit:
    1. Check all pages for broken links
    2. Validate forms on each page
    3. Test responsive design
    4. Verify accessibility compliance
    5. Generate performance report
  `,
  url: 'https://example.com',
  config: {
    timeout: 300000, // 5 minutes
    viewport: { width: 1920, height: 1080 }
  },
  outputSchema: {
    type: 'object',
    properties: {
      brokenLinks: { type: 'array', items: { type: 'string' } },
      formValidation: { type: 'object' },
      responsiveTests: { type: 'object' },
      accessibilityScore: { type: 'number' },
      performanceMetrics: { type: 'object' }
    }
  }
});

Job Status Monitoring

Basic Status Check

const status = await bt.testing.getStatus(job.jobId);

console.log('Job status:', status.job.status);
console.log('Progress:', status.job.progress || 'N/A');
console.log('Created:', status.job.createdAt);
console.log('Updated:', status.job.updatedAt);

if (status.job.status === 'completed') {
  console.log('Results:', status.job.results);
} else if (status.job.status === 'failed') {
  console.log('Error:', status.job.error);
}

Polling for Completion

async function waitForJobCompletion(jobId, options = {}) {
  const {
    maxWaitTime = 300000, // 5 minutes default
    pollInterval = 5000,   // Check every 5 seconds
    onProgress = null
  } = options;

  const startTime = Date.now();

  while (Date.now() - startTime < maxWaitTime) {
    const status = await bt.testing.getStatus(jobId);

    if (onProgress) {
      onProgress(status);
    }

    switch (status.job.status) {
      case 'completed':
        return {
          success: true,
          results: status.job.results,
          totalTime: Date.now() - startTime
        };

      case 'failed':
        return {
          success: false,
          error: status.job.error,
          totalTime: Date.now() - startTime
        };

      case 'running':
        console.log(`Job running... ${status.job.progress || ''}`);
        break;

      case 'pending':
        console.log('Job queued...');
        break;
    }

    await new Promise(resolve => setTimeout(resolve, pollInterval));
  }

  throw new Error(`Job ${jobId} timed out after ${maxWaitTime / 1000} seconds`);
}

// Usage
const result = await waitForJobCompletion(job.jobId, {
  onProgress: (status) => {
    console.log(`Progress: ${status.job.status}`);
  }
});

if (result.success) {
  console.log('Job completed successfully!', result.results);
} else {
  console.error('Job failed:', result.error);
}

Job Management

List Jobs

// Get all jobs
const allJobs = await bt.testing.list();

// Filter by status
const runningJobs = await bt.testing.list({
  status: 'running'
});

const completedJobs = await bt.testing.list({
  status: 'completed',
  limit: 20
});

// Filter by template
const templateJobs = await bt.testing.list({
  templateId: template.template.id
});

// Recent jobs across all templates
const recentJobs = await bt.testing.list({
  limit: 50,
  sortBy: 'createdAt',
  sortOrder: 'desc'
});

Job Details

// Get detailed job information
const jobDetails = await bt.testing.getStatus(jobId);

// Access comprehensive job data
console.log('Job Details:');
console.log('- ID:', jobDetails.job.id);
console.log('- Status:', jobDetails.job.status);
console.log('- Template ID:', jobDetails.job.templateId);
console.log('- URL:', jobDetails.job.url);
console.log('- Created:', new Date(jobDetails.job.createdAt));
console.log('- Started:', jobDetails.job.startedAt ? new Date(jobDetails.job.startedAt) : 'Not started');
console.log('- Completed:', jobDetails.job.completedAt ? new Date(jobDetails.job.completedAt) : 'Not completed');

// Configuration used
console.log('Configuration:', jobDetails.job.config);

// Results (if completed)
if (jobDetails.job.results) {
  console.log('Results:', jobDetails.job.results);
}

Advanced Job Patterns

Job Chains

Run jobs sequentially where each depends on the previous:

class JobChain {
  constructor(bt) {
    this.bt = bt;
    this.jobs = [];
  }

  async addJob(templateId, config) {
    this.jobs.push({ templateId, config });
    return this;
  }

  async execute(startUrl) {
    const results = [];
    let currentUrl = startUrl;

    for (const jobConfig of this.jobs) {
      console.log(`Executing job ${results.length + 1}/${this.jobs.length}`);

      const job = await this.bt.template.invoke(jobConfig.templateId, {
        url: currentUrl,
        ...jobConfig.config
      });

      const result = await waitForJobCompletion(job.jobId);

      if (!result.success) {
        throw new Error(`Job chain failed at step ${results.length + 1}: ${result.error}`);
      }

      results.push(result);

      // Use result URL for next job if available
      if (result.results?.finalUrl) {
        currentUrl = result.results.finalUrl;
      }
    }

    return results;
  }
}

// Usage
const chain = new JobChain(bt)
  .addJob(loginTemplateId, { /* login config */ })
  .addJob(navigateTemplateId, { /* navigation config */ })
  .addJob(testTemplateId, { /* test config */ });

const chainResults = await chain.execute('https://app.com');

Parallel Job Execution

Run multiple jobs simultaneously:

async function runParallelJobs(jobConfigs) {
  const jobPromises = jobConfigs.map(config => {
    return bt.template.invoke(config.templateId, config.params);
  });

  // Create all jobs
  const jobs = await Promise.all(jobPromises);
  console.log(`Created ${jobs.length} parallel jobs`);

  // Monitor all jobs
  const monitoringPromises = jobs.map(async (job, index) => {
    console.log(`Monitoring job ${index + 1}: ${job.jobId}`);

    const result = await waitForJobCompletion(job.jobId, {
      onProgress: (status) => {
        console.log(`Job ${index + 1} status: ${status.job.status}`);
      }
    });

    return {
      jobIndex: index,
      jobId: job.jobId,
      ...result
    };
  });

  const results = await Promise.all(monitoringPromises);

  // Analyze results
  const successful = results.filter(r => r.success);
  const failed = results.filter(r => !r.success);

  console.log(`Completed: ${successful.length} successful, ${failed.length} failed`);

  return {
    all: results,
    successful,
    failed
  };
}

// Usage
const parallelResults = await runParallelJobs([
  { templateId: smokeTestTemplate, params: { url: 'https://site1.com' } },
  { templateId: smokeTestTemplate, params: { url: 'https://site2.com' } },
  { templateId: smokeTestTemplate, params: { url: 'https://site3.com' } }
]);

Job Retry Logic

async function executeJobWithRetry(templateId, params, options = {}) {
  const {
    maxRetries = 3,
    retryDelay = 10000,
    backoffMultiplier = 2
  } = options;

  let lastError;

  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      console.log(`Job attempt ${attempt}/${maxRetries}`);

      const job = await bt.template.invoke(templateId, params);
      const result = await waitForJobCompletion(job.jobId, {
        maxWaitTime: 600000 // 10 minutes per attempt
      });

      if (result.success) {
        console.log(`Job succeeded on attempt ${attempt}`);
        return result;
      } else {
        throw new Error(result.error);
      }

    } catch (error) {
      lastError = error;
      console.error(`Attempt ${attempt} failed:`, error.message);

      if (attempt < maxRetries) {
        const delay = retryDelay * Math.pow(backoffMultiplier, attempt - 1);
        console.log(`Waiting ${delay / 1000}s before retry...`);
        await new Promise(resolve => setTimeout(resolve, delay));
      }
    }
  }

  throw new Error(`Job failed after ${maxRetries} attempts. Last error: ${lastError.message}`);
}

Job Queues and Scheduling

Job Queue Management

class JobQueue {
  constructor(bt, options = {}) {
    this.bt = bt;
    this.maxConcurrent = options.maxConcurrent || 3;
    this.running = new Set();
    this.queue = [];
    this.completed = [];
  }

  async add(templateId, params, priority = 0) {
    const job = { templateId, params, priority, id: Date.now() + Math.random() };
    this.queue.push(job);
    this.queue.sort((a, b) => b.priority - a.priority); // Higher priority first
    return job.id;
  }

  async process() {
    while (this.queue.length > 0 && this.running.size < this.maxConcurrent) {
      const job = this.queue.shift();
      this.running.add(job.id);

      // Process job asynchronously
      this.processJob(job).finally(() => {
        this.running.delete(job.id);
        this.process(); // Process next job
      });
    }
  }

  async processJob(job) {
    try {
      console.log(`Processing job ${job.id}`);

      const btJob = await this.bt.template.invoke(job.templateId, job.params);
      const result = await waitForJobCompletion(btJob.jobId);

      this.completed.push({
        id: job.id,
        success: result.success,
        results: result.results,
        error: result.error
      });

      console.log(`Job ${job.id} completed: ${result.success ? 'SUCCESS' : 'FAILED'}`);

    } catch (error) {
      console.error(`Job ${job.id} failed:`, error);
      this.completed.push({
        id: job.id,
        success: false,
        error: error.message
      });
    }
  }

  getStats() {
    return {
      queued: this.queue.length,
      running: this.running.size,
      completed: this.completed.length,
      successful: this.completed.filter(j => j.success).length,
      failed: this.completed.filter(j => !j.success).length
    };
  }
}

// Usage
const queue = new JobQueue(bt, { maxConcurrent: 2 });

// Add multiple jobs
await queue.add(templateId, { url: 'https://site1.com' }, 1);
await queue.add(templateId, { url: 'https://site2.com' }, 2); // Higher priority
await queue.add(templateId, { url: 'https://site3.com' }, 1);

// Start processing
await queue.process();

// Monitor progress
setInterval(() => {
  console.log('Queue stats:', queue.getStats());
}, 5000);

Scheduled Jobs

class JobScheduler {
  constructor(bt) {
    this.bt = bt;
    this.scheduledJobs = new Map();
  }

  schedule(templateId, params, cronExpression, jobName) {
    // This is a simplified scheduler - in production, use a proper cron library
    const job = {
      templateId,
      params,
      cronExpression,
      name: jobName,
      nextRun: this.calculateNextRun(cronExpression)
    };

    this.scheduledJobs.set(jobName, job);
    console.log(`Scheduled job "${jobName}" for ${job.nextRun}`);
  }

  calculateNextRun(cronExpression) {
    // Simplified - parse cron and calculate next run time
    // In production, use a library like 'node-cron' or 'cron'
    return new Date(Date.now() + 60000); // Run in 1 minute for demo
  }

  async checkAndRunScheduledJobs() {
    const now = new Date();

    for (const [name, job] of this.scheduledJobs) {
      if (now >= job.nextRun) {
        console.log(`Running scheduled job: ${name}`);

        try {
          const btJob = await this.bt.template.invoke(job.templateId, job.params);
          console.log(`Scheduled job "${name}" started with ID: ${btJob.jobId}`);

          // Update next run time
          job.nextRun = this.calculateNextRun(job.cronExpression);

        } catch (error) {
          console.error(`Failed to start scheduled job "${name}":`, error);
        }
      }
    }
  }

  start() {
    // Check every minute for due jobs
    setInterval(() => {
      this.checkAndRunScheduledJobs();
    }, 60000);
  }
}

// Usage
const scheduler = new JobScheduler(bt);

// Schedule daily health check
scheduler.schedule(
  healthCheckTemplate,
  { url: 'https://my-app.com' },
  '0 9 * * *', // 9 AM daily
  'daily-health-check'
);

scheduler.start();

Monitoring and Analytics

Job Performance Analytics

async function analyzeJobPerformance(jobIds) {
  const jobStatuses = await Promise.all(
    jobIds.map(id => bt.testing.getStatus(id))
  );

  const completedJobs = jobStatuses.filter(s => s.job.status === 'completed');

  const analytics = {
    totalJobs: jobIds.length,
    completedJobs: completedJobs.length,
    successRate: (completedJobs.length / jobIds.length) * 100,
    averageDuration: 0,
    minDuration: Infinity,
    maxDuration: 0,
    errorTypes: {}
  };

  completedJobs.forEach(job => {
    const duration = new Date(job.job.completedAt) - new Date(job.job.createdAt);

    analytics.averageDuration += duration;
    analytics.minDuration = Math.min(analytics.minDuration, duration);
    analytics.maxDuration = Math.max(analytics.maxDuration, duration);
  });

  analytics.averageDuration /= completedJobs.length;

  // Analyze errors for failed jobs
  const failedJobs = jobStatuses.filter(s => s.job.status === 'failed');
  failedJobs.forEach(job => {
    const errorType = categorizeError(job.job.error);
    analytics.errorTypes[errorType] = (analytics.errorTypes[errorType] || 0) + 1;
  });

  return analytics;
}

Real-time Dashboard

class JobDashboard {
  constructor(bt) {
    this.bt = bt;
    this.jobs = new Map();
  }

  async refresh() {
    const allJobs = await this.bt.testing.list({ limit: 100 });
    this.jobs.clear();

    allJobs.jobs.forEach(job => {
      this.jobs.set(job.id, job);
    });
  }

  getSummary() {
    const jobs = Array.from(this.jobs.values());
    const byStatus = jobs.reduce((acc, job) => {
      acc[job.status] = (acc[job.status] || 0) + 1;
      return acc;
    }, {});

    return {
      total: jobs.length,
      ...byStatus,
      successRate: jobs.filter(j => j.status === 'completed').length / jobs.length * 100
    };
  }

  getRecentFailures(limit = 5) {
    return Array.from(this.jobs.values())
      .filter(job => job.status === 'failed')
      .sort((a, b) => new Date(b.updatedAt) - new Date(a.updatedAt))
      .slice(0, limit);
  }

  getLongRunning(thresholdMinutes = 10) {
    const threshold = thresholdMinutes * 60 * 1000;
    const now = Date.now();

    return Array.from(this.jobs.values())
      .filter(job => {
        const started = job.startedAt ? new Date(job.startedAt).getTime() : now;
        return job.status === 'running' && (now - started) > threshold;
      });
  }
}

// Usage
const dashboard = new JobDashboard(bt);

// Refresh data periodically
setInterval(() => dashboard.refresh(), 30000);

// Display dashboard
function displayDashboard() {
  const summary = dashboard.getSummary();
  const failures = dashboard.getRecentFailures();
  const longRunning = dashboard.getLongRunning();

  console.clear();
  console.log('=== Job Dashboard ===');
  console.log(`Total Jobs: ${summary.total}`);
  console.log(`Running: ${summary.running || 0}`);
  console.log(`Completed: ${summary.completed || 0}`);
  console.log(`Failed: ${summary.failed || 0}`);
  console.log(`Success Rate: ${summary.successRate?.toFixed(1) || 0}%`);

  if (failures.length > 0) {
    console.log('\nRecent Failures:');
    failures.forEach(job => {
      console.log(`- ${job.id}: ${job.error}`);
    });
  }

  if (longRunning.length > 0) {
    console.log('\nLong Running Jobs:');
    longRunning.forEach(job => {
      const duration = Math.floor((Date.now() - new Date(job.startedAt).getTime()) / 60000);
      console.log(`- ${job.id}: ${duration} minutes`);
    });
  }
}

setInterval(displayDashboard, 5000);

Best Practices

Job Lifecycle Management

Set appropriate timeouts: Don’t let jobs run indefinitely
Implement monitoring: Track job status and performance
Handle failures gracefully: Implement retry logic and error recovery
Clean up old jobs: Archive or delete completed jobs periodically
Resource limits: Monitor and limit concurrent jobs

Performance Optimization

// Optimize job configuration
const optimizedJob = await bt.testing.create({
  instructions: 'Optimized test execution',
  url: 'https://example.com',
  config: {
    // Performance settings
    timeout: 120000, // Reasonable timeout
    waitFor: 1000,   // Minimal wait time
    viewport: { width: 1280, height: 720 }, // Standard viewport

    // Resource optimization
    blockAds: true,     // Reduce network requests
    disableImages: false, // Keep images for visual tests
    reducedMotion: true  // Speed up animations
  }
});

Error Handling and Recovery

// Comprehensive error handling
async function executeReliableJob(templateId, params) {
  try {
    const job = await bt.template.invoke(templateId, params);
    const result = await Promise.race([
      waitForJobCompletion(job.jobId, { maxWaitTime: 300000 }),
      timeoutPromise(300000) // Additional safety timeout
    ]);

    return result;

  } catch (error) {
    // Log error details
    console.error('Job execution failed:', {
      templateId,
      params,
      error: error.message,
      timestamp: new Date().toISOString()
    });

    // Attempt cleanup if job was created
    if (error.jobId) {
      try {
        // Cancel job if possible
        await bt.testing.cancel(error.jobId);
      } catch (cancelError) {
        console.error('Failed to cancel job:', cancelError);
      }
    }

    throw error;
  }
}

function timeoutPromise(ms) {
  return new Promise((_, reject) => {
    setTimeout(() => reject(new Error('Operation timed out')), ms);
  });
}

Getting Started

Core Features

Advanced Usage

Overview

Creating Async Jobs

From Templates

Direct Job Creation

Job Status Monitoring

Basic Status Check

Polling for Completion

Job Management

List Jobs

Job Details

Advanced Job Patterns

Job Chains

Parallel Job Execution

Job Retry Logic

Job Queues and Scheduling

Job Queue Management

Scheduled Jobs

Monitoring and Analytics

Job Performance Analytics

Real-time Dashboard

Best Practices

Job Lifecycle Management

Performance Optimization

Error Handling and Recovery

​Overview

​Creating Async Jobs

​From Templates

​Direct Job Creation

​Job Status Monitoring

​Basic Status Check

​Polling for Completion

​Job Management

​List Jobs

​Job Details

​Advanced Job Patterns

​Job Chains

​Parallel Job Execution

​Job Retry Logic

​Job Queues and Scheduling

​Job Queue Management

​Scheduled Jobs

​Monitoring and Analytics

​Job Performance Analytics

​Real-time Dashboard

​Best Practices

​Job Lifecycle Management

​Performance Optimization

​Error Handling and Recovery

Overview

Creating Async Jobs

From Templates

Direct Job Creation

Job Status Monitoring

Basic Status Check

Polling for Completion

Job Management

List Jobs

Job Details

Advanced Job Patterns

Job Chains

Parallel Job Execution

Job Retry Logic

Job Queues and Scheduling

Job Queue Management

Scheduled Jobs

Monitoring and Analytics

Job Performance Analytics

Real-time Dashboard

Best Practices

Job Lifecycle Management

Performance Optimization

Error Handling and Recovery