September 22, 2010

Deep Dive - Static Context in Visualforce Apex Controllers !

Apex static is always confusing for me, I am from Java background, so it took a while for me to really understand differences between Apex vs Java static. This post discusses apex controller static behavior, i.e. in relation to Visualforce only. Here by Apex Controller static, I mean static variables/blocks declared in controller associated with a Visualforce page.

The Apex STATIC context – Variables & Code initialization blocks !

(Intermediate/Advanced Apex developers can skip this section)

Apex gives static context in form of static variables and static blocks. Most of developers from Java background understand the static block, but those who are ramping up from PHP etc, please go thru this topic on static in Salesforce Docs :

Apex VF Controller Static – Common MYTHs !

Developers, specially from Java background assume, that static variables once initialized on first class load, can be shared/updated across multiple HTTP requests. This typically goes with Java classes. But this is not true at all with Apex, never expect to set some value in Apex static variable and expect it back in next request, even next Ajax request. On every new request, you will find static variables re-initialized to the value declared or updated in static blocks.

Then, what is STATIC in Apex Controller ? – Real Story !

In general as per Salesforce docs, this is definition of static.

Static variables are only static within the scope of the request. They are not static across the server, or across the entire organization.

This definition is a one liner, but it applies well on every Apex static context. The thumb rule to understand static in Apex controller is

Static Variables and Blocks are re-executed on each User HTTP Request or Visualforce Ajax Request

Please NOTE that, the static variables/block are re-executed for Ajax requests generated via Visualforce i.e. by actionFunction, actionPoller, commandLink or commandButton tags etc. You must be wondering Whyyy ? This is because HTTP is stateless protocol, Visualforce only maintains view state(as hidden input field) to re-store all non-transient instance level variables in Apex Controller. Static variables are never stored in view state, as said in Salesforce docs clearly :

Static variables aren't transmitted as part of the view state for a Visualforce page.

So to re-store all static stuff back, force.com platform recreates all static variables and executes all static block on each normal or ajax http request. So thats why one can’t share data across the requests via static variables, they are always recreated first on each request(VF Ajax request too)

Apex Static, and the mind set change

For programmers from other backgrounds like J2EE, PHP etc, its a mindset change because:

  • JVM(Java Virtual Machine) usually don’t reload classes. A Java class is loaded once in memory usually stays till container is UP. Same stands true with static variables in Java too, all information kept in static context is available through out the application across different user sessions etc.
  • J2EE & Frameworks like Struts, create a Session per user login like Salesforce. But that session is available to multiple requests via Cookies etc and acts as good mini container to store logged in user specific information.

But these both Java approaches are memory hogs and not scalable to multiple users, that’s why J2EE containers have complex session and other clustering mechanisms to scale for huge enterprise apps.

So, having static in this manner is important for success of multitenant nature of force.com platform. As force.com is multitenant, multiple customer orgs, with multiple users share the same hardware resources like CPU, RAM(Memory). If Apex static was like Java, then I can’t imagine how much memory would be required by force.com servers to match the current load. I am sure, they can’t scale !

So, are Static variables good for anything in Apex Controllers ?

As, static variables are not part of view state. One can consider static as “C” language CONST, use them to declare constants in Apex.

Next …

In my next post, I will discuss Apex static context in Triggers.  Would love to discuss your ideas/views on Apex Controller static context, please comment !!

Why I can only follow/subscribe to 500 entities with Chatter ?

Chatter makes enterprise communication easy for sure. People can chat around standard/custom objects, share files/links, follow people and even objects etc etc. All is pretty good with Chatter, except the limitation of maximum 500 Entity Subscriptions per User.

The strange 500 limit ?

One User can only follow at max 500 objects or people or files(from winter’11). This is a weird limit, we came across this limit within a month after starting with chatter. Reaching 500 mark by manual following (hitting follow button) will take some more time, we are using triggers to do rule based auto following. But its pretty common to have some auto-follow triggers setup in chatter enabled orgs, these triggers make some users follow some interesting information based on biz rules. In such auto-follow scenario,  I am sure one will face this error pretty quickly.

Why this 500 limit hurts ?

This limit hurts, because

  • Its not documented any where, the standard Salesforce governor/limits page
  • If one has auto-follow rules/triggers setup, once this limit is reached, can block the transactions, as triggers will fail for this error. The only quick solution left in such cases is to disable any such auto-follow trigger. This requires Admin access too, so not a straight task to do for a normal non-admin user.

In my case because of this error, we have to devise another biz rule to un-follow some of the old data(that might be still important), based on time frame window etc and some certain record state. A standard example can be auto un-follow Opps that are Won/Closed, though such Opps can carry some interesting chatter conversations.

Chatter vs Facebook vs Twitter

Chatter is Enterprise Facebook/Twitter for me. From our Facebook/Twitter habits, we all are too much in habit of following, this is what we do always in Twitter OR Facebook (like/groups). We never have think the follow/like count before following any new interesting information source. Chatter unfortunately kills here, one can’t fly free or plan some cool auto-follow biz logic based on this “500” constraint.

Idea-Exchange – Remove this 500 limit – VOTE !!

I posted my pain on force.com forum here, but should have posted an idea too in parallel. Never mind, some wise person already did that, here is link to the idea. We all love Chatter, so please vote to get rid of this 500 limit ! This idea desperately needs votes.

September 15, 2010

Salesforce Winter'11 Visual Process Manager - Flows !

This post covers new Winter'11 stuff called "Visual Process Manager—Pilot", this feature seems super powerful, but somehow less discussed and marketed as of now. This might be because its just a pilot release ?
I said Visual process manager will be powerful, because it take the "point and click" configuration philosophy of force.com platform to next level. Its next level, because now business analysts and administrator (who are less of programmers) can quickly develop applications that will comprise of
  • Creation of screens with navigations
  • Ability to query Salesforce.com data, without any coding.
  • Ability to branch based on user data or input.
  • Plus ability to execute logic. Not clearly described, but hopefully this will allow Apex execution.
So with all these capablities in place, it seems easy for business analysts and admins to develop a "dynamic" application, without writing any code.
What to expect Visual Process Manager ?
Though no screenshots, details etc for Visual Process Manager are available in winter'11 release notes etc. But as it will be a BPM(Business Process Manager) application, we can expect something similar done by guys like BEA(Aqua Logic) or Savvion BPM Studio. These both applications give somehow the same abilities to develop biz flows on respective platforms. Developing flow is just like working in a paint brush application and drawing a flow chart. Here are few screen shots.
Bea Aqua Logic

Savvion BPM

Applications best for Visual Process Manager
As winter'11 release notes says, one can develop applications like
  • Call scripting in a customer-support center: Troubleshooting complex customer problems
  • Telesales: Call scripting, cross-selling, up-selling and dynamic generation of quotes in real-time
  • Insurance companies: Executing complex and dynamic insurance premium calculations
Though its too early to guess, what force.com is coming with Visual process manager. But applications not involving complex interface should be easy to develop. Though if flows, opens ability to plugin visualforce pages, just like ability to execute logic later on. Then complex interfaces should be easily possible.
Conclusion
Visual Process Manager will for sure help Business Analysts and Admins to quickly develop applications on force.com platform. On the other hand it might also hurt freelance apex/visualforce developers, as number of salesforce jobs for trivial or simple force.com screens will be for sure reduced.
Still too early to conclude more, let see what comes up. Visual process manager is not available by default, one has to request salesforce to enable it in their org.

September 9, 2010

Salesforce Winter’11 release for apex web services developers

Salesforce Winter’11 release came with many new exciting features and governor limit relaxations. These relaxations are specially cool specially for apex developers, who are creating Mash-ups & Enterprise integration projects on top of force.com platform.

This post explains

  • Correlation between existing platform limitations/issues in developing such web service projects with apex
  • And how winter’11 governor limit relaxations, make more such mash-up projects possible and easy to develop on force.com platform.

Here are few common scenarios for web service development with apex, each scenario is inline value add from winter’11 features.

  • Scenario 1: Making web service calls via Apex to other Clouds or Enterprise stacks like Google, Facebook.

    • Related Winter’11 Change: Removed governor limits in Winter’11
      • Total request time for one callout (HTTP request or Web services call) has been removed.
      • Maximum size for a callout (HTTP request or Web services call) has been removed.
    • Benefits
      • Previously “Total request time for one callout (HTTP request or Web services call)” was 60 seconds. Though 60 second is decently big time for any general web service call to compete.But we know because of some web service method requirements, internet latencies and server overload this limit might be less in some scenarios.
      • Previously this limit was 1 MB, so chances of failures exists when huge XML is returned in response. Developers have to do some hacks to make the stuff work right for them in case of big responses. 
  • Scenario 2: Parsing XML/JSON response via Apex

    • Related Winter’11 Change:
      • General heap size has been raised from 2 MB to 3 MB, with no scaling. In addition, batch Apex heap size is 6 MB.
    • Benefits
      • More heap size is important for many reasons, even previously(before winter’11) with ability to handle just 1 MB XML response.
        • 1 MB response means a String response, that will eat up 1 MB of available heap.
        • The relevant data structures that will encapsulate the parsed web service response might take some where between at least 300-500KB easily for a 1 MB response.
        • So XML + Data Structures will for sure end up you in consuming at least 1.5 MB of heap. So increase of heap size upto 3 MB, is really important.
    • Important Note : With winter’11 relaxation on “Maximum size for a callout”, one needs to watch the amount of heap available. This doesn’t really means we can handle any big web service response. For ex. one can’t parse a 4 MB big XML, with 3 MB available in heap.

  • Scenario 3: Performing WSDL2Apex, via “Generate from WSDL” button available in Salesforce Org > Setup > Develop > Apex Classes.

    • Related Winter’11 Change:
      • Apex classes and triggers have been raised from 100,000 characters and 32,000 characters, respectively, and can now be 1
        million characters long, not counting comments, test methods, and classes defined with @isTest.
      • Maximum amount of Apex code allowed in an organization has been raised from 1 MB to 2 MB.
    • Benefits
      • We can’t perform WSDL2Apex previously on WSDLs, those generate big Apex classes, for ex. we can’t do WSDL2Apex on partner WSDL. So the hacks developers do includes
        • Removing methods/operations from WSDLs to reduce the number of chars in generated Apex classes.
        • Instead of straight SOAP options available in WSDL, look for REST API and try implementing an Apex client on own.
      • So having scope of 1 million chars in Apex Class, will for sure open the ability to bring more smart Apex clients to various WSDLs.

Conclusion

I see questions(link) on force.com forums, where developers try to compare developing web service clients in Apex or Java or .Net. Developers often hesitate out of force.com platform and end up in implementing solution on Java/.Net stack to get rid of these governor limits. But now after winter’11 this scene should change a lot, developers have much more bandwidth to do on platform then ever before.

Working with Aggregate SOQL queries/results in Batch Apex !

What if you want create Batch Apex Job, that uses SOQL having aggregate functions like SUM, MAX, COUNT on top of results grouped by “GROUP BY” clause. You can’t easily create a Batch job via QueryLocator, as shown below.

global class QueryLocatorAggregator implements Database.Batchable<AggregateResult> {
}

It fails for this error

Error: Compile Error: QueryLocatorAggregator: Class must implement the global interface method: Iterable<SOBJECT:AggregateResult> start(Database.BatchableContext) from Database.Batchable<SOBJECT:AggregateResult> at line 1 column 14

So the other option that seems doable is using SObject as generic type argument in Batchable, as shown below with complete source code.

global class QueryLocatorAggregator implements Database.Batchable<Sobject> {
    global Database.QueryLocator start(Database.BatchableContext info){
        String query = 'Select Department, COUNT(Name) From Contact GROUP BY Department';
        return Database.getQueryLocator(query);
    }
    global void execute(Database.BatchableContext BC,
             List<Sobject> scope){
        for (Sobject so : scope)  {
          AggregateResult ar = (AggregateResult) so;
          Integer counter = Integer.valueOf(ar.get('expr0'));
          // process the results
        }
    }
    global void finish(Database.BatchableContext BC){}   
}

The above code/class complies(saves), but on first processing of batch it fails with this error

System.UnexpectedException: Aggregate query does not support queryMore(), use LIMIT to restrict the results to a single batch

So, how to work on queries with aggregation ? well the hint was right their in the first error message in red i.e. use Iterable<AggregateResult>

Making AggregateResult queries work in Batch Apex

The only way I found is using Iterable<??>. Iterable is an easy QueryLocator alternate to feed data to batch apex execution. Database.Batchable<?> interface gives two abstract methods to implement

  1. global (Database.QueryLocator ) start(Database.BatchableContext bc) {}
  2. global (Iterable<sObject>) start(Database.BatchableContext bc) {}

The magical part about Database.Batchable interface is you need to implement either of the above two abstract methods. Though those magic confuse me a lot, because of my Java background :-D

Here are the three steps to get rolling with Iterable<> in Batch Apex

  1. Create a class that implements contract of Iterator<AggregateResult>
  2. Create a class that implements contract of Iterable<AggregateResult>
  3. Use the instance of Iterable<AggregateResult> implementation in Database.Batchable’s “global (Iterable<sObject>) start(Database.BatchableContext bc) {}”
Create a class that implements contract of Iterator<AggregateResult>

Here is the code sample for this.

global class AggregateResultIterator implements Iterator<AggregateResult> {
   AggregateResult [] results {get;set;}
   // tracks which result item is returned
   Integer index {get; set;} 
         
   global AggregateResultIterator() {
      index = 0;
	  // Fire query here to load the results
      String query = 'Select Department, COUNT(Name) From Contact GROUP BY Department';
      results = Database.query(query);            
   } 
   
   global boolean hasNext(){ 
      return results != null && !results.isEmpty() && index < results.size(); 
   }    
   
   global AggregateResult next(){        
      return results[index++];            
   }       
}    
Create a class that implements contract of Iterable<AggregateResult>

Here is the code sample for that

global class AggregateResultIterable implements Iterable<AggregateResult> {
   global Iterator<AggregateResult> Iterator(){
      return new AggregateResultIterator();
   }
}
Use the instance of Iterable<AggregateResult> implementation in Database.Batchable’s “global (Iterable<sObject>) start(Database.BatchableContext bc) {}”

Here is the complete Database.Batchable class implementation

global class IterableAggregator implements Database.Batchable<AggregateResult> {    
    global Iterable<AggregateResult> start(Database.batchableContext info){
        // just instantiate the new iterable here and return
        return new AggregateResultIterable();
    }

    global void execute(Database.BatchableContext BC, List<Sobject> scope){
        for (Sobject so : scope)  {
          AggregateResult ar = (AggregateResult) so;
          Integer counter = Integer.valueOf(ar.get('expr0'));
          // process the results
        }        
    }

    global void finish(Database.BatchableContext BC){}   
}

When not to use Iterable<?> with AggregateResult

Database.Batchable implementations depending on Iterable<?> for data source are bound to normal governor limits, so Iterable’s can never process 50 million records like QueryLocator. Before using Iterable, you need to watch the amount of Data/Rows resulting from that Aggregated SOQL.

References

September 7, 2010

Deep dive into Visualforce <apex:actionregion /> tag !

<apex:actionRegion /> is interesting and somewhat ignored tag in Visualforce. Tag is ignored, because Visualforce documentation is not in depth, to explain the real utility of this tag. So this post explains some stuff around action region to make it easy for developers to understand.

Special thanks to @RajendraOgra for sharing his findings and information around this tag with me.

What is Region aka ActionRegion ?

The <apex:actionRegion> component specifies the components to be processed on Force.com server. If no <apex:actionRegion> is defined the whole View functions as a region.

Here components means, all visualforce tags like inputField, inputText, outputPanels etc.

And component processing includes, atleast

  • conversion : converting from user inputted strings to date, numbers etc.
  • validation : running validations like checking required fields
  • model update : updating apex bindings i.e. Controller variables binded to visualforce page merge fields.

Note, even when using <apex:actionRegion/>, the whole Form is still submitted but only part taken into region will be processed.

Example:

<apex:form>
      ...
      <apex:actionRegion>
            <apex:commandLink/>
      </apex:actionRegion>
      ...
<apex:form>

Here the complete form is submitted when request is generated by <apex:commandLink/> but only markup between <apex:actionRegion > and </apex:actionRegion> is processed on server.

Example 2: Nesting of ActionRegion(s)

<apex:form>
       ...
      <apex:actionRegion>
            <apex:commandLink value="Link 1" id="link1"/>
            <apex:actionRegion>
                 <apex:commandLink value="Link 2" id="link2"/>
            </apex:actionRegion >
      </apex:actionRegion>
      ...
<apex:form>

The regions could be nested. Server picks out and processes only the region, which contains the component that initiates the request. Here in above example, the external region is processed for link1 and the internal one is processed for link2.

ActionRegion in Action !

Now, we will go thru a simple visualforce code snippet to understand the dramatic effect of <apex:actionRegion /> tag on validations and model update.

Scenario :

  • We are having a simple page, comprising of two input fields i.e. name and job.
  • Name has keypress visualforce actionSupport handler attached to it, it just re-prints/re-renders the typed text in name field over an <apex:outputText />
  • Job input field is a “required” text field on the page.

Sample Code :

Controller

public class regioncontroller {
    public String name { get; set; }
    public String job { get; set; }
}

VF page

<apex:form >
   <apex:pageMessages id="messages1"/>    
   <apex:pageBlockSection columns="2" title="WITHOUT Action Region">
      <apex:outputText value="Name" />
      <apex:inputText value="{!name}">
         <apex:actionSupport event="onkeyup" reRender="outname,messages1" />
      </apex:inputText>
      <apex:outputText value="Job" />
      <apex:inputText required="true" id="job2" value="{!job}" />
   </apex:pageBlockSection>
<apex:outputText id="outname" style="font-weight:bold" value="Typed Name: {!name}" />
</apex:form>

When we run this code sample i.e. type something in name field we get validation error as follows

Whats this Issue !

Type the name. The outputText “outname” component should reappear after you. However, in the first case, this activity will be aborted because of the other field with required="true". You will see only the error message while the "Job" field is empty. After filling in something in Job field, you can see outputText getting updated on keypress in Name field.

How to fix this ?

To fix this, we need to use actionRegion, as of now all the visualforce components in the page are getting processed when request is generated on keypress.  So a validation error does not allow to process an ajax input(i.e. name’s keypress).  If we simply demarcate the inputText “Name” with “ActionRegion” the problem will be solved, because then only this component will be processed by the force.com server. Here is the fixed code snippet and it works well, without even filling in any job details.

<apex:form >
   <apex:pageMessages id="messages1"/>    
   <apex:pageBlockSection columns="2" title="WITH Action Region">
      <apex:outputText value="Name" />
      <apex:actionRegion >
         <apex:inputText value="{!name}">
            <apex:actionSupport event="onkeyup" reRender="outname,messages1" />
         </apex:inputText>
      </apex:actionRegion>
      <apex:outputText value="Job" />
      <apex:inputText required="true" id="job2"  value="{!job}" />
   </apex:pageBlockSection>
   <apex:outputText id="outname" style="font-weight:bold" value="Typed Name: {!name}" />
</apex:form>    

Note we just added <apex:actionRegion /> around input text, thats it.

Live Demo

Want to play and see this actionRegion’s stuff in action, here is the link to my public site vf page. You can play with both cases i.e. with / without action region on the same page.

Complete Source of the Demo VF Page

I started using http://snipplr.com/, great website for quickly sharing code snippets. Here are the links to the source code.

Other References

Visualforce internally runs on top of JSF and Rich Faces frameworks of J2EE. So many of the visualforce tags are port of rich faces JSF tags. I got this understanding around actionRegion from Rich Faces documentation. The above demo code is port of Rich Faces “region” sample itself. So if you are still hungry to learn more, please refer rich faces docs, here are the rich faces links

Other Notes

I am researching on other interesting attribute available on <apex:actionRegion /> i.e. “renderRegionOnly”. To avoid having too much information in this blog post about action-region, I will come up with more details on this in another post.

September 6, 2010

Batch Apex & First error: Attempt to de-reference a null object !

I stumbled upon a strange Batch Apex issue today. My batch job was going well in few SFDC orgs; after deploying the same to a new org it stopped working completely. I just see, “First error: Attempt to de-reference a null object” in debug and apex logs.  After adding a lot of debug statements in both start() and execute() method, I found that “execute(Database.BatchableContext BC, List<Sobject> scope)” is never called after creating QueryLocator in start().

So, finally after searching on dev forums found this solution; it says don’t query NULL columns/fields in initial SOQL i.e. the soql string given to QueryLocator in start(). This really fixed my problem.

So to put it all together in a clean way. We usually have batch jobs written in following manner

global class MyCoolBatch implements Database.Batchable<Sobject>{
     
  global Database.QueryLocator start(Database.BatchableContext BC){
    String query = 'Select X, Y, Z From Contact WHERE ...';
    Database.Querylocator qr = Database.getQueryLocator(query);
    return qr;
  }
  
  global void execute(Database.BatchableContext BC, List<Sobject> scope){
    // process the records in scope, as per biz logic
  }
  
  global void finish(Database.BatchableContext BC){}
}

Here we are querying fields X, Y and Z from Contact; in case any of X,Y or Z are NULL in any records, you will face the “First error: Attempt to de-reference a null object” issue. Solutions to this problem are

  1. Add null checks in SOQL, for fields that could go NULL for ex. X != null. This might not be possible, as this will for sure change the records returned from the SOQL.
  2. Don’t query the fields that could possibly be NULL in start() method, re-query those in “execute(Database.BatchableContext BC, List<Sobject> scope)” method. This should be safer, as execute() will always be working on smaller subset of records for ex. 200 records.

I used the later option i.e. don’t query NULLABLE fields in start() & re-query those fields in execute() method, this fixed the problem !

September 5, 2010

Sorting Apex classes Sobject using apex-lang code share project !

You must be stuck up with Apex, when sorting is required on instances of user defined classes and pre-loaded Sobjects. Unfortunately standard Apex API just gives a method List.sort(), that just works with primitive data types.

So to get the job done, developers can implement their own sorting either using standard and simple bubble sort or some other complex sorting algorithm. But this approach is not good because

  • Apex governor just gives handful of script statements and limited stack depth for your biz logic.
  • Bubble sort is never a preffered sorting algo, for performance reasons(complexity) and number of high iterations. 
  • Implementing some complex sorting algo is
    • difficult
    • error prone
    • mostly suited to one user defined type or sobject

So, what’s the solution ?

We can’t do anything about governor, best would be to let developer focus on biz logic only and let sorting taken care of by some other lib, like apex-lang.

In brief apex-lang is port of very popular Java “Apache common Lang” project to Apex. More details about apex-lang are available in my previous blog post and this google code project.

apex-lang offers many Apex utility classes. One of them is ArrayUtils, it offers APIs to quick sort instances of SObject and Object(User defined types/classes). The sorting API is very similar to Java’s Collections.sort(list, comparator), where you can pass in a Comparator to compare fields of two instances for a Class.

Sorting using Apex-Lang

First prerequisite for sorting using apex-lang is implementing Comparator. Those who are from Java background, for sure know about Comparators and their importance in sorting. Those who are not, don’t need to be panic. Comparator has just a single method

Integer compare(Object o1, Object o2)

Implementations of this method, should just “compare its two arguments for order. And return a negative integer, zero, or a positive integer as the first argument is less than, equal to, or greater than the second.

How to write create a Comparator ?

For sake of explanation lets assume we have a user defined class called “Word”, that encapsulates a single English word. As shown below

  public class Word {
    // the real word string
    public String val {get; set;}
    public Word(String theWord) {
      if (theWord == null) theWord = '';
      val = theWord;
    }
  }

Next, we will create two comparators implementations

  1. One, that sorts the words alphabetically
  2. Second, that sorts  the words based on their length.

To start we just need to extend ObjectComparator Apex class available in apex-lang library. Below is Comparator that sorts the instances of above Word class, alphabetically !, its so simple just 3 lines of code

 
  global class WordComparator implements ObjectComparator {
    global Integer compare(Object o1, Object o2) {
      // Note we used Apex String.compareTo(String1, String2) method here
      // as it does the same i.e. return Integer based on
      // String comparision
      return ((Word)o1).val.compareTo(((Word)o2).val);
    }
  }  

Next, is the comparator implementation that will be used to sort the words by length. Again a 3 liner code :)

 
  global class WordSizeComparator implements ObjectComparator {
    global Integer compare(Object o1, Object o2) {
      // Note: we can just subtract length of two Strings 
      // to compare them
      return (((Word)o1).val.length()) - (((Word)o2).val.length());
    }
  }
  

NOTE : The above comparator implementations are good for user defined apex classes only. If you want to sort instances of SObject, then instead of implementing ObjectComparator, just implement ISObjectComparator. Everything else goes same. We need two different comparators here, because unlike java in Apex List<SObject> doesn’t gels well with List<Object>.

Here is a sample Sobject comparator for Contact, that sorts it on LastName. You must be thinking we can load the Contact’s right by just querying using ORDER BY in SOQL, but trust me its not always possible :) (Governor limits !)

  global class ContactLastNameComparator implements ISObjectComparator {
    global Integer compare(SObject o1, SObject o2) {
      // Note we used Apex String.compareTo(String1, String2) method here
      // as it does the same i.e. return Integer based on
      // String comparision
      return ((Contact)o1).LastName.compareTo(((Contact)o2).LastName);
    }
  }

 

How to sort using the Comparator ?

Once Comparator is implemented, one can start sorting using apex-lang’s

  • ArrayUtils.qsort(List<Object> theList, ObjectComparator comparator), to sort ascending
  • or, ArrayUtils.qsort(List<Object> theList, ObjectComparator comparator, Boolean sortAsc), to sort descending too.

OR, to sort instances of Sobject, use

  • ArrayUtils.qsort(List<Sobject> theList, ISObjectComparator comparator), to sort ascending
  • or, ArrayUtils.qsort(List<Sobject> theList, ISObjectComparator comparator, Boolean sortAsc), to sort descending too.
Sorting Word class or User defined type/classes

for ex. to sort instances of Word class created above, all we need to do is

List<Word> wordObjects =  // Initalize list as required;
// Sort words alphabetically 
ArrayUtils.qsort(wordObjects,new WordComparator());
// Use in this manner to sort on
// the word size/length
ArrayUtils.qsort(wordObjects,new WordSizeComparator());

Note, we created WordComparator & WordSizeComparator classes above. We just instantiated it and passed it to ArrayUtils.qsort() api.

Sorting SObjects like Contact
Contact [] cons = [select Id, LastName from Contact ];
// Note we used ContactLastNameComparator
// created above
ArrayUtils.qsort(cons, new ContactLastNameComparator());

 

How to implement Comparators for Date, Number and other primitives.

apex-lang comes with a good sample Comparator called PrimitiveComparator, this class shows how to compare each primitive data type. I suggest using this class as a learning stand point, rather using it. Because this class is meant to be too generic and it does a lot of checks, instance of etc, so you might end up in giving too much share of your governor limits like script statements to it.

Many other pre-build and example Comparator implementations available as apex classes in apex-lang project. For ex. DecimalRangeComparator,  SelectOptionComparator,  SObjectSortByNameComparator. You can use these as starting point to learn how to code different comparators. In case of any issues, feel free to comment on this post. I will for sure help you in writing the correct comparator.

When apex-lang can kill you ?

Its pretty popular saying that “with great power comes great responsibility”. Same applies very well here with apex-lang, developers get a really handy way to sort almost any object(Object/SObject). But not to forget, apex-lang is not part of standard apex api, its collection of standard apex classes. So any governor limit that applies to your Apex code, is equally applicable on apex-lang. Most important of these governor limits are

  • Number of script statements used
  • Maximum stack depth allowed.

As we all know operations like sorting involve many iterations/recursions and comparisons. So its very likely, when sorting a big data set you end up hitting any of these governor limits.

So before relying super heavily on apex-lang sorting a developer needs to check following

  1. How much script statements and other governor limits quota is required to process rest of the business logic.
  2. How big will be the collection to sort via apex-lang. This is important because, the bigger the collection will be more will the governor limits used.
  3. Sorting 1000 objects roughly takes atleast 50,000+ script statements. This figure will vary hugely, depending on Comparator implementation. In above sample comparators with 1000 words
    • WordComparator consumes 51792 script statements
    • WordSizeComparator consumes 110654 script statements. With 1091 words crash occurs with reason : “System.LimitException: Maximum stack depth reached: 173

So, based on above two points. Here is the basic rule for developer to best avoid breaking governor limits in production/UAT.

  • Match the governor limits consumed by biz-logic to the sorting logic. If business logic is already consuming too governor quota like script lines + stack depth. Then one needs to limit the amount of data sorted using apex-lang (or pick some other alternate). Developer has to know, what can be the maximum collection size to be sorted. Based on this max count the biz logic should be executed, to see if we are any where close to breaking governor limits.

To see how close you are into limits you can use Limits class provided by Apex standard library. It gives pretty handy methods like getScriptStatements(), that you can use before after costly calls as shown below.

Integer scriptsBefore = Limits.getScriptStatements();
ArrayUtils.qsort(wordObjects,new WordComparator());
Integer scriptsAfter = Limits.getScriptStatements();
System.debug(LoggingLevel.info, ' ##SCRIPT STATEMENTS USED ' + (scriptsAfter - scriptsBefore));

 

How did I benchmarked “apex-lang” with big real data set.

Creating a big data set with real random words is pretty hard to achieve programmatically, unless you apply some randomization logic to generate it. For my benchmarking purposes. I did a simple trick, i.e.

  1. Hit any blog or news portal, like http://www.nytimes.com/
  2. Copy the text of a blog post or article to some text editor. This will give you many random words, easily in thousands.
  3. Split those words on whitespace, to get a Apex String array of size equal to the number of words.

I used that String array of words to create Instances of “Word” class explained above, and that big random collection was used on different sizes to bench mark apex-lang. Hope it helps.

Why Apex-Lang should be matured to standard Apex API

Because of all this governor nightmare, how cool would it be, if some of these super helpful apex-lang classes become part of Standard Apex API. So for this reason, I am posting this as an idea on Salesforce Idea Exchange.

Please promote it, if you want to see apex-lang maturing to standard apex-api. (link to idea)

References

Why apex-lang for Apex developers ?

apex-lang is an open source Apex library, its Apex port of very popular Apache Common Lang java project. Almost all java developers and open source projects in Java depend on Apache Common Lang. This is really good to see that something similar came to apex in open source. apex-lang is comprising of many Utility classes that port the Apache’s stuff to apex, plus give nice wrapper on top of standard Salesforce apex api also. Here is a list of useful Apex util classes

  • ArrayUtils:
    • APIs : To reverse, merge, subset, compare and most importantly SORT arrays. As with Apex List<Object>  and Object[] can be used interchangeably, so the same class with go well with Apex List collection too.        
    • Views : I worked a little with the sorting api’s. I liked the convenience of Apex port done for Java’s Comparator. Script statements consumed in sorting big collections requires a little care, will cover the sorting API, in a separate blog post.
  • EmailUtils:
    • APIs : On top of Apex e-mail api, had support for sending html/plain email with attachments too.
  • LanguageUtils:
    • APIs:  to deal with many language specific options like preparing list of SelectOption, parsing language information from HTTP request headers like “Accept-Language”. More details about this in Richard’s blog post
  • MapUtils:
    • APIs : To join and compare maps.
    • Views : I was expecting much more here, like support for Map-Keys as UDT(User Defined Types) etc.
  • PageUtils: A bunch of useful API’s, good stuff to get request params, set page messages and url redirects.
  • RandomUtils/RandomStringUtils:
    • APIs : For more precise control on how to generate randoms. Definitely offers flexibly APIs as compared to standard Apex Math.random().
  • TestUtils:
    • APIs : To see if their are messages of a given severity etc.
    • Views: Few more APIs could be their like, one for creating mock test users for System.runAs(), can be a good addition to this.
  • SObjectUtils:
    • APIs : 
      • To copy fields across Sobjects
      • Check if fields are null/empty
      • Get/put fields using field-name as string
    • Views : Developers beginnning with Apex, should read this class to know the mini java - reflection style support with Apex.
  • SoqlUtils:       
    • APIs: Its too rich and would need time to carefully read this blog post from Richard.
    • Views : Nice bunch of APIs for creating dynamic SOQL, and well documented in the blog post too. Excellent !
  • WordUtils :
    • APIs : Interesting API’s to capitalize, reverse case etc etc the Strings.
  • Other Util Classes like  StringUtils, SystemUtils, UrlUtils, UserUtils, SetUtils, BooleanUtils, NumberUtils, also offer decent APIs on standard Apex primitive data types and other types like User. Can be handy on many occasions.

 

Apex-Lang should be matured to standard Apex API

I said this because of a couple of reasons, like

  • Presence of Governor Limits : Because Apex always runs under control of notorious Governor. This governor just gives us small number of script statements to execute for ex. Trigger : 10,000, Test : 20,000, Anonymous Block,Visualforce Controller, or WSDL Method : 200,000. So under these small limits, it becomes risky specially if your biz logic is already on edge of these limits. Even one can easily reach these and other limits(stack depth etc) while working on big collections.
  • Loading of many Apex classes: As coming from Java background, I was habitual of following too many design patterns and creating many different Apex class files for Utility, Factory and many other relevant stuff. But sometime back I came to know that “Loading new Apex class is a big performance penalty, it triggers some validation code to ensure that dependent classes etc are valid”. So this becomes a big penalty when creating frameworks like this, its hard to keep number of Apex classes to minimal. Solution to this is “to create many related nested/inner classes inside a big Apex class file”.
  • Apex code limits: Though this is not a big reason, but as these frameworks are normal Apex classes, they share your org code limits. If you are working on big Salesforce projects, you can already be on or close to border of this limit. This salesforce community link is good example of this.

So because of these reasons, projects like apex-lang should be matured to standard Apex API. Not all utility classes but many could be moved to standard Apex API. So for this reason, I am posting this as an idea on Salesforce Idea Exchange. Please promote it, if you want to see apex-lang maturing to standard apex-api. (link to idea)

References