Pages

Monday, January 21, 2013

SugarCRM: SOAP API Performance

It is not uncommon for discussions that revolve around the SOAP API in SugarCRM to include some measure of criticism, but how much of that criticism is actually warranted?

SOAP --the protocol-- has always been critiqued for not being an optimal method for communication between distributed systems, in short, poor performance. Thus, it is no surprise that some of that criticism should find its way into conversations about the SugarCRM SOAP API.

Some folks in the broader web development world have altogether given up on SOAP, favoring RESTful web services. True to the trends in the general web development community, many SugarCRM developers have also tossed aside any notions of using the SOAP API, and SugarCRM addresses this trend by also providing a REST API. However, this growing resentment towards all things SOAP makes me question whether it is justified, and if so, what can be done about it?

Before we go any further I should state I do not discount the point that test results can prove REST to be faster than SOAP, but I do think many performance problems attributed to the SugarCRM SOAP API can be overcome with some fine tuning of code. 

In my experience, the most common source of performance problems has come from the use of the set_relationship() method. Said method is used to create a relationship between records, for example, link an opportunity to a contact. All efforts should be made to reduce the number of times this method is called within your code.



To do this, examine the objects you are working with, whether they be accounts, contacts, opportunities or other. Depending on the type of relationship that is involved in the relationship you wish to set with your code, one of the objects involved in the relationship may already offer an alternative to using the set_relationship() method.

For example, say that you wish to link a contact to an account. To do so, one could call the set_relationship() method, or better yet, use the set_entry() method to populate the account_id field on the Contact object with the ID value of the account the contact should be linked to. This latter approach has always performed faster for me when compared to calling set_relationship()

It is also worth noting that if one is using code to create the account, the contact and then link the two records, one would naturally assume we should call set_entry() twice (once for the account and a second time for the contact), followed by a call to set_relationship() to link the two records. In this scenario, populating the account_id column in the set_entry() call for the contact would effectively eliminate the need to call set_relationship(), reducing your total number of required calls. This is especially helpful for scenarios where your code is being used to insert large volumes of records.

This principle can be applied to various modules. Thus, your first step in optimizing your use of the SOAP API should be to look for similar relationship fields that can help you eliminate the need calling set_relationship() altogether.

In addition to the above, another trick I frequently use is threading, more specifically, thread pools. Microsoft  .NET languages permit a programmer to not only create threaded applications, but also applications that use thread pools. Via these thread pools, one can write code that sends simultaneous requests to the SugarCRM API and creates a queue for the remaining calls necessary to complete your task. As the SugarCRM server completes a request, another request from the queue is automatically sent to the SugarCRM API. This process continues until all the requests are completed and the queue is clear.

These thread pools offer significant boosts in performance, especially when dealing with large volumes of data. With a bit of tuning of the SugarCRM server itself, even further gains can be achieved, as the speed at which the requests are handled is determined by the server configuration.

In general, these two techniques have repeatedly provided me with significant better performance. Hopefully these tips are of assistance to you as well.

12 comments:

  1. We are facing an issue with running sub-queries in our soap's get_entry_list call. We are trying to send the sub-query in the where clause of the params array for our soap call.

    The problem we are facing is that due to changes since 6.5.5 (where sub-queries are not allowed) of SugarCRM SOAP, there are many more SOAP call hits for doing the same thing we used to do using sub queries in a single SOAP call. The latency of multiple SOAP calls is what is causing the performance issue.

    Can you suggest anything to improve this?

    Here is the sample of the call we are trying to run.

    where id in (SELECT contact_id FROM account_contacts WHERE account_contacts.deleted=0 AND account_contacts.account_id='1475c11a-0b4d-ee40-b706-50ebee97092b')

    Sugar support guys have proposed solution to this problem by using the get_relationships call instead.

    We can not use this solution as this would take unacceptable amount of time and would badly effect our end user experience.

    ReplyDelete
    Replies
    1. The only idea that comes to mind is perhaps considering creating your own method. There is no real way to circumvent it otherwise.

      Delete
    2. If I'm understanding the purpose of that subquery, you should be able to accomplish that with the link_name_to_fields_array parameter to get_entry.

      So instead of doing a get_entry_list on Contacts, you would do a get_entry on the appropriate Account with a link_name_to_fields_array parameter of something like:
      array(
      array('name' => 'contacts',
      'value' => array('id', 'name', 'whatever', 'other', 'fields', 'you', 'want', 'from', 'contacts'),
      )
      )

      This will give you a list of all of the contacts related to that account. It's a bit weirder to parse out the resulting relationship_list, but I've had great success with it in the past.

      Delete
  2. You mention .Net, threads and pooling to create concurrent connections and an ordered update if the SOAP API is busy. Another valid option would be to use an MQ system (such as RabbitMQ or ActiveMQ amongst others) in between the systems in either a request/response or fire and forget style. That also gives you some flexibility although it does require some thinking about in terms of architecture.

    ReplyDelete
  3. I am using SugarCRM Development and one thing that I inevitably need in this time of integrating web services, platforms and devices is the ability to send data from a web form into SugarCRM and I use SOAP for this and it works well.

    ReplyDelete
  4. I am really inspired together with your writing abilities as smartly as with the structure for your weblog. Keep up the excellent quality writing, it’s uncommon to see a nice blog like this one nowadays about SugarCRM Customization.

    ReplyDelete
  5. Hi,

    Hi, having had to deal with these SOAP issues, I chose to implement interfacing with other applications by using simple HTTP connections.
    On the SugarCRM side, I get POST data and I use the internal API SugarCRM to process it, and on the client side, all that must be done is to send data over an HTTP connection.

    I found that this solution provides flexibility and performance, while being technically very simple. It can even access the database directly if necessary.

    The only problem is that SugarCRM API is not well documented. It was then that blogs like yours are a great help!

    ReplyDelete
    Replies
    1. Cool idea. Along those lines, version 6.6 and higher of SugarCRM include a truly RESTful API, so you can leverage POST/PUT/DELETE, etc., making it even easier to utilize the Sugar API.

      Delete
  6. I am using SugarCRM v.6.5.16

    Soap: http://{CRM Path}/service/v4_1/soap.php

    I have been tasked to create a program that reads values from a JSON string to input into SugarCRM.

    The object creation of Account, Contact, Opportunity etc. is working just fine. I am however unable to the the set_relationship or set_relationships from the SOAP API to work.

    If possible could someone give me a quick example of how to make this work.

    I have found **very** **little** documentation regarding coding in C# and SugarCRM SOAP functions. I can directly insert into the 'link' table but would prefer to utilize the API since somebody took the time to write it.

    If REST would work better than SOAP an example of that would also be useful.

    Thanks

    ReplyDelete
    Replies
    1. What exactly happens when you try to use set_relationship()? Do you get an error? Does it fail?

      Can you share the relevant bit of code? What objects are you attempting to link together?

      Delete

Your comments, feedback and suggestions are welcome, but please refrain from using offensive language and/or berating others. Thank you in advance.