It is not uncommon for discussions that revolve around the SOAP API in SugarCRM to include some measure of criticism, but how much of that criticism is actually warranted?
SOAP --the protocol-- has always been critiqued for not being an optimal method for communication between distributed systems, in short, poor performance. Thus, it is no surprise that some of that criticism should find its way into conversations about the SugarCRM SOAP API.
Some folks in the broader web development world have altogether given up on SOAP, favoring RESTful web services. True to the trends in the general web development community, many SugarCRM developers have also tossed aside any notions of using the SOAP API, and SugarCRM addresses this trend by also providing a REST API. However, this growing resentment towards all things SOAP makes me question whether it is justified, and if so, what can be done about it?
Before we go any further I should state I do not discount the point that test results can prove REST to be faster than SOAP, but I do think many performance problems attributed to the SugarCRM SOAP API can be overcome with some fine tuning of code.
In my experience, the most common source of performance problems has come from the use of the set_relationship() method. Said method is used to create a relationship between records, for example, link an opportunity to a contact. All efforts should be made to reduce the number of times this method is called within your code.
To do this, examine the objects you are working with, whether they be accounts, contacts, opportunities or other. Depending on the type of relationship that is involved in the relationship you wish to set with your code, one of the objects involved in the relationship may already offer an alternative to using the set_relationship() method.
For example, say that you wish to link a contact to an account. To do so, one could call the set_relationship() method, or better yet, use the set_entry() method to populate the account_id field on the Contact object with the ID value of the account the contact should be linked to. This latter approach has always performed faster for me when compared to calling set_relationship().
It is also worth noting that if one is using code to create the account, the contact and then link the two records, one would naturally assume we should call set_entry() twice (once for the account and a second time for the contact), followed by a call to set_relationship() to link the two records. In this scenario, populating the account_id column in the set_entry() call for the contact would effectively eliminate the need to call set_relationship(), reducing your total number of required calls. This is especially helpful for scenarios where your code is being used to insert large volumes of records.
This principle can be applied to various modules. Thus, your first step in optimizing your use of the SOAP API should be to look for similar relationship fields that can help you eliminate the need calling set_relationship() altogether.
In addition to the above, another trick I frequently use is threading, more specifically, thread pools. Microsoft .NET languages permit a programmer to not only create threaded applications, but also applications that use thread pools. Via these thread pools, one can write code that sends simultaneous requests to the SugarCRM API and creates a queue for the remaining calls necessary to complete your task. As the SugarCRM server completes a request, another request from the queue is automatically sent to the SugarCRM API. This process continues until all the requests are completed and the queue is clear.
These thread pools offer significant boosts in performance, especially when dealing with large volumes of data. With a bit of tuning of the SugarCRM server itself, even further gains can be achieved, as the speed at which the requests are handled is determined by the server configuration.
In general, these two techniques have repeatedly provided me with significant better performance. Hopefully these tips are of assistance to you as well.
We are facing an issue with running sub-queries in our soap's get_entry_list call. We are trying to send the sub-query in the where clause of the params array for our soap call.
ReplyDeleteThe problem we are facing is that due to changes since 6.5.5 (where sub-queries are not allowed) of SugarCRM SOAP, there are many more SOAP call hits for doing the same thing we used to do using sub queries in a single SOAP call. The latency of multiple SOAP calls is what is causing the performance issue.
Can you suggest anything to improve this?
Here is the sample of the call we are trying to run.
where id in (SELECT contact_id FROM account_contacts WHERE account_contacts.deleted=0 AND account_contacts.account_id='1475c11a-0b4d-ee40-b706-50ebee97092b')
Sugar support guys have proposed solution to this problem by using the get_relationships call instead.
We can not use this solution as this would take unacceptable amount of time and would badly effect our end user experience.
The only idea that comes to mind is perhaps considering creating your own method. There is no real way to circumvent it otherwise.
DeleteIf I'm understanding the purpose of that subquery, you should be able to accomplish that with the link_name_to_fields_array parameter to get_entry.
DeleteSo instead of doing a get_entry_list on Contacts, you would do a get_entry on the appropriate Account with a link_name_to_fields_array parameter of something like:
array(
array('name' => 'contacts',
'value' => array('id', 'name', 'whatever', 'other', 'fields', 'you', 'want', 'from', 'contacts'),
)
)
This will give you a list of all of the contacts related to that account. It's a bit weirder to parse out the resulting relationship_list, but I've had great success with it in the past.
Thanks for that tidbit Wes!
DeleteYou mention .Net, threads and pooling to create concurrent connections and an ordered update if the SOAP API is busy. Another valid option would be to use an MQ system (such as RabbitMQ or ActiveMQ amongst others) in between the systems in either a request/response or fire and forget style. That also gives you some flexibility although it does require some thinking about in terms of architecture.
ReplyDeleteVery cool! Thanks for sharing that!
DeleteI am using SugarCRM Development and one thing that I inevitably need in this time of integrating web services, platforms and devices is the ability to send data from a web form into SugarCRM and I use SOAP for this and it works well.
ReplyDeleteI am really inspired together with your writing abilities as smartly as with the structure for your weblog. Keep up the excellent quality writing, it’s uncommon to see a nice blog like this one nowadays about SugarCRM Customization.
ReplyDelete