TL; DR;

Unified base class to execute batch record processing asynchronously without having to worry (too much) about limits and how it actually executes. Intended for simpler jobs in order to ease off the pressure on trigger handlers rather than true big-batch tasks. Just keep calling executeAsync and let the class worry about how (ideally).

🔗 Shortcut to source code and longer version of the article

Async Apex 101

Most Salesforce orgs I have seen over the years tend to rely heavily (if not exclusively) on record operations to execute any business logic and automations. The pattern is fairly repetitive:

• Go through the trigger records evaluating some criteria and building a list of relevant records or Ids

• Do some thing(s) with the identified records

• Save the records

The second step in particular can be quite complex. Good idea to defer that logic (so long as it's not strictly required to complete instantaneously) to async context. This helps to keep transactions quick and also can make debugging easier by keeping logs for different business logic separated.

Async Apex Options

We have a number of options in Apex: Future, Queueable, Batch and Schedulable.

Future is great at handling simple tasks, but it only allows primitive parameters and is also not good when we talk about processing batches of things. The processing may be quite complex sometimes and we may not be able to handle all the records at the same time before "the Governor" shuts us down. Schedule has the second issue as well. That is why we would usually use Queueable or Batch.

Both of the above options are fairly simple to implement. But we have to choose first. Or implement both and decide which option to go with at runtime. Either one has very little boiler-plate code around it, but if we support both it can add up. We should check limits before launching too.

I wanted to create something where I could just provide “the way” to handle a batch of records and nothing else at all. I came up with an ApexJob class that attempts to do just that.

Implementing Async Jobs

The one method required of a “participating” class is handleBatch(). This is what the job is supposed to be achieving. Separate interface is probably a good idea so this is it:

public interface IAsyncHandler {

    void handleBatch(List<SObject> records);

    /**
       * Must override standard Object toString to return the full
       * name of the class including any outer class. e.g. AsyncJobTest.TestHandler
       *
       * @return return the full name of the Class implementing the interface
       */
    String toString();
}

Ok so not strictly only one. I’m forcing the implementing class to override toString() as well to return the fully qualified name of the class. It’s one of the things I ran into as I went. I need to be able to construct an instance of the class from the type name and the known approach* to get it from the instance doesn’t play nice with inner classes.

String.valueOf(myInstanceVariable).split(‘:’)[0]

Inheritance over Composition

Yes, this does not sound right, I know. But here it actually makes a lot of sense. I did start the other way round with AsyncJob doing the work using a provided “handler instance” implementing the above Interface. It took exactly 1 day in Production to notice that the Apex Jobs menu in Setup then becomes completely unreadable. Just an endless list of “AsyncJob” entries.

In this case it is actually much better to have the implementing classes extend AsyncJob instead. They themselves then become the Job Class listed in setup for admins to make sense of a lot easier. So no interface, but rather an abstract method after all.

Running Async Jobs

This is what needs to be called in order to “schedule” an async job. I need the records to act on and the type name of the handler. There is an overloaded version of this method which gives us a little bit more flexibility and hints at one of the core features of the AsyncJob utility. We can control the Batch Size and influence the type of Async Job.

public static Id executeAsync(List<SObject> records, String className)
public static Id executeAsync(List<SObject> records, String className, Integer batchSize, Strategy asyncStrategy)

I started with an option of passing in an instantiated handler. This allowed for it to be as complex to initialise as I needed it to be. But in the end the “handler” needs to be possible to initialise from the type name (I will clarify why, I promise).

Queue or Batch

Queueable jobs are fast and simple. Best choice for processing a relatively small number of records. Not normally an issue, but if there are many large batch jobs to be run, we want to keep the limits of concurrent batch jobs and the Flex Queue open. Each batch job also comes with 2 extra async executions (start and finish) that can add up in an extremely busy org. Wasteful if we only have 1 or 2 batches to do. Then again with a very large number of records it can be a lot faster and definitely more readable in your Apex Jobs list to run a Batch instead.

So the default setting I went with is what I call “Prefer Queueable”. This means that as long as Transaction Limits permit the jobs runs as Queueable. But if the number of batches is higher than say 5 it's a Batch instead. I can choose to “Prefer Batch” and essentially reverse the selection, or I can force Batch or Queueable and nothing else. If that can’t be done because of the transaction context we get an Exception.

private static Id executeAsync(AsyncJob job, Strategy asyncStrategy) {
    if (job.records == null || job.records.isEmpty()) {
        System.debug(LoggingLevel.DEBUG, 'No records provided. Not running async job!');
        return null;     
    }

    Long noOfBatches = Decimal.valueOf(job.records.size() / job.batchSize).round(System.RoundingMode.UP);
    if (
        asyncStrategy == Strategy.QUEUEABLE_ONLY ||
        asyncStrategy == Strategy.NO_BATCH ||
        (noOfBatches < BATCHABLE_PREFERRED_SIZE &&
        asyncStrategy != Strategy.BATCH_ONLY &&
        asyncStrategy != Strategy.NO_QUEUEABLE)
    ) {
        return runPreferQueueable(job, asyncStrategy);
    }     
    return runPreferBatch(job, asyncStrategy);
}

Queueable Batches

When the Queueable job is supposed to handle more records than are allowed in a single batch it has to re-requeue itself. AsyncJob handles this using the same instance of the handler. So as long as that is initialised with all the records in mind it does not have a problem handling the next batch in a completely new job and transaction.

private void handleBatchQueueable() {
    List<SObject> currentBatch = new List<SObject>();
    while (currentBatch.size() < this.batchSize && !this.recordsToProcess.isEmpty()) {
        currentBatch.add(this.recordsToProcess.remove(0));
    }
    this.handleBatch(currentBatch);
    if (!this.recordsToProcess.isEmpty()) {
        reQueueNotYetProcessedRecords(this.recordsToProcess);     
    }
}

private void reQueueNotYetProcessedRecords(List<SObject> recordsToStillProcess) {
    if (Test.isRunningTest()) {
        System.debug(LoggingLevel.DEBUG, 'Re-queuing unprocessed records/batches blocked in Unit Tests because of limits!');
        return;
    }
    System.enqueueJob(this.setRecords(recordsToStillProcess));
}

Governor Limits

In order for this to be truly useful, I have to not worry about Governor Limits. That’s why the AsyncJob contains methods like isQueueableAvailable() and isBatchAvailable().

For the former it’s just using the Limits class to see how many Queueable jobs are still available.

private static Boolean isQueueableAvailable() {
    Integer queueableJobsLimit = Limits.getLimitQueueableJobs();
    Integer queueableJobsUsed = Limits.getQueueableJobs();
    return queueableJobsLimit > queueableJobsUsed;
}

The latter is not so easy and for now the class is just being a bit silly and checking it’s not running a batch already. I didn’t find an efficient way to keep checking the Flex Queue or any way to find out I’m inside a Batch.finish() method (which is the only place in isBatch() where it’s possible to execute another one). Lots to improve here.

private static Boolean isBatchAvailable() {
    Boolean isNotExecutingBatchOrFuture = !System.isBatch() && !System.isFuture();     
    Boolean isNotQueueableInTest = !System.isQueueable() || !Test.isRunningTest();
    return isNotExecutingBatchOrFuture && isNotQueueableInTest;
}

The isNotQueueableInTest is another hurdle I fell over during testing. For some reason running Database.executeBatch() in Queueable inside a Unit Test is no-go.

Ultimate Backup – Event

If we can’t run Queueable or a Batch the AsyncJob falls back to a Platform Event. Those should always be available. The Event is designed to serialise the records (so yea, there will be a limit to this) and a Trigger on its insert will re-try to run the job again via the Queueable-first route.

In order for this to work the implementing class must provide a public empty constructor (hinted at that earlier) so that it can be initialised from the Type name stored in the Event payload.

EventBus.publish(
    new AsyncJob__e(
        Payload__c = JSON.serialize(job.records),
        HandlerTypeName__c = job.handlerClassName,
        BatchSize__c = job.batchSize
    )
);

trigger AsyncJob on AsyncJob__e(after insert) {
   for (AsyncJob__e event : Trigger.new) {
       String handlerClassName = event.HandlerTypeName__c;
       List<sObject> records = (List<SObject>) JSON.deserialize(event.Payload__c, List<SObject>.class);
       if (records == null || records.isEmpty()) {
           return;
       }
       AsyncJob.executeAsync(records, handlerClassName, batchSize, AsyncJob.Strategy.PREFER_QUEUEABLE);
   }
}

Limitations and Lessons

A big limitation is the inability to monitor state. Queueables do this fairly easily, but one has to be careful when using the Platform Event route as the state can be lost when serialising. Batchable provides the Database.Stateful interface, but the major issue I found is that a single class cannot implement it at the same time as also implementing Queueable. My AsyncJob class now has a NoBatch strategy for this purpose, but that’s far from ideal.

AsyncJob can now handle only lists of SObjects and those better have all the queried fields that you need. You can of course re-query to be safe, but that seems unnecessary as long as you have control over launching the job and/or use the Domain Layer consistently. Another option is to work on a List or Set of Ids to define a batch. That way you can force all the data collection to be done in the job itself, making it safer. This makes it more difficult when using the Batch route though.

If you go a little overboard with this approach there could potentially be many async jobs competing to update the same records. So care needs to be taken here to not run into record locking issues.

Thanks for reading! 🔗 Source code and full article