Friday, February 26, 2016

SugarCRM: Bulk API Explained

Recently I have found myself in various conversations revolving around the topic of the Bulk API and its capabilities. These conversations have helped highlight various misunderstandings relating to its functionality, which I believe are the result of a lack of documentation on the subject. Hopefully this post will help clarify some of the more common themes.

To begin with, let us discuss what the Bulk API is not.

It is not a separate interface to Sugar. It is merely another endpoint of the existing REST v10 API. In short, the moniker Bulk API is a misnomer, as the term is just referring to a component of the REST v10 API, not an alternative to it.

Quite regularly I also hear of attempts to use the Bulk API to perform mass insertions of data. For example, migrating hundreds of thousands of records from another system and into Sugar. While the Bulk API can help expedite the process, it is not specifically designed for that task and its benefits for that type of work are limited. 

What is the purpose of the Bulk API endpoint?

The Bulk API endpoint aims at helping developers reduce the quantity of web requests sent to the REST v10 API. By reducing the number of requests necessary to complete a programmatic task, the time to completion can also be reduced.

The following example helps demonstrate its value quite well.

Suppose you are developing code that requires you to create 10 new records in a Sugar module. Typically, your code would make one request per record, along the lines of:

POST /rest/v10/<SomeModule>

BODY: {JSON Payload}

Under this model, as the volume of data increases, so do the number of required web requests.

Leveraging the Bulk API allows us to reduce the number of required web requests to complete the same task by allowing us to bundle multiple requests into one. The following JSON payload extracted from the Sugar API Help system demonstrates this very point:

    "url": "/v10/Accounts", "method": "POST", "data": "{\"name\": \"Acme\"}"
    "url": "/v10/Accounts", "method": "GET"

This entire payload actually instructs Sugar to perform 2 different tasks, but through a single web request. As a result, an action that would normally take 2 web requests to complete is reduced to a single web request. If we apply this same principle to the previous example, we could potentially create all 10 records via a single request, instead of 10.

Also note that the type of requests in the payload can vary. In this example, we see a POST and a GET request being executed by the same request to the Bulk API.

Lastly, it is important to note that the Bulk API does not perform any sort of throttling. 

This is another important point to bear in mind if your intention is to use it to perform mass data inserts. The size of the body in the request is subject to limits controlled primarily by your web server. As such, they can vary from server to server, and in all cases the Bulk API assumes that the payload you are providing does not exceed the limit. If you do POST a request that exceeds the limit, you should expect to see an error and failure to complete the request.

The Bulk API endpoint can be accessed by sending a POST request to /rest/v10/bulk and is available in Sugar 7.5 and higher.


  1. Interesting, will the requests be executed in the order provided in my bulk request? And what if request 1 fails,is request 2 is still being executed?

    1. Good question. I vaguely recall there was an issue that came up with an older version of Sugar where one a failure in one of the requests would cause problems for the others, but I think that was resolved.

      I would have to test it to confirm, but I believe the current versions of Sugar allow the other requests to complete.


Your comments, feedback and suggestions are welcome, but please refrain from using offensive language and/or berating others. Thank you in advance.