Handling API failure with Code Example in Javascript

Handling Rate limiting gracefully

In one of the projects we were making a lot of API requests to third party provider(for example: Google APIs). These APIs had certain rate limits implemented. We were being throttled, most of our API calls failed with 429 requests

You can read more about different rate limiting techniques applied here but in this blog post I want to discuss about one of the approaches that we used to handle them

Part 1: Retry Mechanism with Exponential Backoff

Unlike other HTTP code(like 401, 500), the 429 is not because of an issue in calling the API or from the server side. 429 occurs because there are certain limits that are applied to the resource to prevent it from being overused due to sudden attack. Think of it as max load limit kept on a particular API resource. But what should the application that is consuming these APIs do to overcome it? Letting it simply fail would not be the right approach because if you call the same API after cool off period it will work. That shows the application is showing inconsistent behvaiour. In order to avoid this, we implemented a axios interceptor to retry the failed request. You can either use some of the open source npm packages but we decided to write our own version. Some factors that need to be pre-defined are MAX_RETRIES - number of time you would like to retry the request before considering it failed.

Exponential Backoff is a common error handling strategy that retries the request after an exponential time period for failed API call.

import axios, { AxiosError, AxiosRequestConfig } from "axios";


let MAX_RETRIES = 5;
let ONE_MINUTE = 60 * 1000;

interface AxiosConfig extends AxiosRequestConfig {
  retryCount?: number;
}

interface AxiosConfigCustom extends AxiosConfig {
  retryCount: number;
}
interface AxiosErrorCustom extends AxiosError {
  config: AxiosConfigCustom;
}

axios.interceptors.response.use(
  (response) => {
    return response;
  },
  (error: AxiosErrorCustom) => {
    /**
    * Retry 429 requests with backoff
    * 429 => TOO MANY REQUESTS
    * 503 => SERVICE UNAVAILABLE
    * https://cloud.google.com/apis/design/errors
    */
    if (error.response?.status == 429 || error.response?.status === 503 || error.response?.status === 500) {
      // set a retry count parameter
      const retryCount = (error.config.retryCount || 0) + 1;
      error.config.retryCount = retryCount;
      if (retryCount <= MAX_RETRIES) {
        return new Promise((resolve, _) => {
          setTimeout(() => {
            resolve(axios(error.config));
          }, 2 * retryCount * ONE_MINUTE);
        });
      }
      return Promise.reject(error);
    } else {
      return Promise.reject(error);
    }
  },
  );

One of the cons with this approach is that it will make network calls for all APIs and even all the retry API calls for the number of times retry limit is set. Consider a scenario where you are making 100 API requests and let’s say the third party starts blocking from 50 requests. So what happens first 50 calls were a success and post that the next 50 calls failed + we made 50x2 = 100 more API request in retry. Therefore the network bandwidth consumption is high with this approach. A better approach for such scenarios would be Batch Processing.

The axios retry interceptor can be used for small scenarios wherein a max of 5 API request will be made and we want to have a fallback scenario to tackle those. Sometimes it so happens that the API might be too busy or might have rate limited our calls, so if our number of request are small it would be better to increase on the time interval, which could be equivalent to the API rate limit quotas.

Checkout the code on stackblitz

Part 2: Batch Processing

Batch processing is an efficient technique that allows to process packets of fixed pre-determined size. Consider applying batch processing when the size of the array is too large. In this approach we have split our API requests into batches of size 10 each. To prevent overload of API requests, as seen in above approach, one single batch is processed completely and then next batch is picked up for processing. This helps ensure that if the third party API is down for some reason or blocking us, we do not process further batches and can add alerts in our catch block. This process seems slower than the previous one but is much more beneficial in staying within our infrastructure capacity. With the previous approach it may so happen that the number of API instance go up drastically and may block other requests if that API is not designed properly. Therefore there a lot of factors to consider before choosing on the approach that might work best for you.

let arr = [
  {
    name: 'data plan',
    target: 'https://'
  }
];

async function higherOrderFunction(
  arr: Array<Object>,
  apiCall: Function,
  maxSize: number = 10
) {
  try {
    let n = arr.length;
    let promises = [];
    let resp = [];
    for (let idx = 0; idx < n; idx++) {
      promises.push(apiCall(arr[idx]));
      if (promises.length > maxSize) {
        console.log('Batch processing ', promises);
        resp = [...resp, ...(await Promise.all(promises))];
        promises = [];
      }
    }
    resp = [...resp, ...(await Promise.all(promises))];
    return resp;
  } catch (error) {
    console.log(error);
  }
}

// api call that responds in 3s
async function apiCall(obj: Object) {
  return new Promise((resolve, rej) => {
    setTimeout(() => resolve(obj), 3000);
  });
}

// call this function in different stages
// by passing the appropriate api call function and request body in the array
higherOrderFunction(arr, apiCall)
  .then(res => {
    console.log('test response ', res);
  })
  .catch(error => {
    console.log('test error', error);
  });

Check out the code on stackblitz