June 24, 2010

Won iPhone in Chatter Developer Challenge !

After a long wait and all that exciting voting action @ Gallery site, finally the Chatter Challenge results came with a good and refreshing news for me.

My challenge submission “Location Glimpse” got 3rd prize, an iPhone !

I am feeling honored to be considered by Salesforce, this is really good motivation work more hard and come up with much more powerful and better apps for AppExchange and Force.com Developer Challenges.

My sincere and hearty thanks to all who commented, liked, and voted for “Location Glimpse”. Your support and votes made the application stand tall at Chatter Gallery Leaderboard ! Many many thanks again :)

What is Next !

I am preparing “Location Glimpse” for AppExchange. It will be available as FREE app, as per chatter challenge rules.

More enhancements are in road map, like integration with other location sharing services is in the road map. Will update you guys soon. Stay connected to the “location glimpse” tag here

Related Links

June 15, 2010

Location Glimpse User Travel View in Salesforce

“Location glimpse” allows one to see a user’s travel inside salesforce. On viewing a peer’s travel salesforce users can

  • Comment on the travel
  • Suggest better route
  • Plan a pickup on the way.

Here is a video that shows this travel.

I am also trying to capture a real travel in which I will be moving and other salesforce user’s are checking my speed in real time. So in that video you can see and person travelling and his location changes are pushed in real time to salesforce.

For full details on “Location Glimpse” application check this post

June 9, 2010

Please Vote for “Location Glimpse” – Chatter Challenge !

Hello everyone, finally the Gallery site for voting Salesforce Chatter Challenge submissions is live (link). I submitted a location sharing application in this challenge (full details here).

My submission is available by Name “Location Glimpse” here is the direct link to this entry (link).

I request you all, please please vote for this application. Voting will end on Tuesday June 15th.

Thanks in Advance !!

Abhinav Gupta

June 7, 2010

Location Glimpse – Chatter Developer Challenge Submission

“Location Glimpse” is a salesforce chatter based location sharing application that I have developed for Chatter challenge and App Exchange. This application allows a salesforce user to share his location inside of salesforce using GPS based mobile phones. Later upon sharing the location in salesforce “Chatter” is used by peers to discuss and chat on that user’s location update.

Location Glimpse – Overview

  1. Its first Chatter based, REAL location sharing app. I said its real, because we are using Mobile phones with GPS to update location in salesforce, so no ip2location and reverse geocoding. Only accurate location via GPS.
  2. Integrated with a "3rd Party" Location Sharing "Mobile App" called
    "Glympse.com" to get real (GPS) location updates via Mobiles phones to Salesforce. This mobile app is available for Iphone, Android, and Windows Mobile. Blackberry app is coming soon.
  3. Salesforce users send location updates to their salesforce org via Glympse.com Mobile App.
  4. Chatter used through out this app to discuss,collaborate and plan meetings on location updates.

Location Glimpse – Presentation !

I prepared an online presentation to easily explain what the application is “in detail”. This presentation also contains the videos for various location glimpse screens in action. Please view this presentation for complete insight.

Click here to view ! 

Location Glimpse Architecture !

Location glimpse is basically made up of an inbound email handler, chatter + glympse.com integration and a couple of visualforce pages. How all that fit into a picture is shown in this post.

Click here to view !

Location Glimpse – Videos !

I have created a couple of videos for

  • Giving a full fledged Demo of the application
  • To be user manual for various screens in the application.

Click here to view !

You can also subscribe to video channel for “Location glimpse”, here is the channel link

Your views !!

Let me know your views on this, its important for me. Thanks !!!

Location Glimpse – Architecture

This post explains the architecture of Location Glimpse.

Key components

Location glimpse has 3 major components.

  1. Force.com Inbound Email handler to 
    • Parse Glympse Mobile App emails.
    • Push those Glympse details to the respective user's feed as "LinkPost" !
  2. Location Viewer, an Apex/Visualforce page to 
    • View location of user into Google Maps.
    • Chat and collaborate on the mapped location via Chatter comments.
  3. A Peer Search Visualforce Page to 
    • Search, see (maps) and chat about your peer's last locations.
    • Request latest location updates from peers.

The diagram below shows the detailed flow of how location is shared, and chatted via Location Glimpse application.

Location Glimpse – Demo & Videos

I have created a couple of videos for

  • Giving a full fledged Demo of the application
  • To be user manual for various screens in the application.

Those all videos are shared in this post

Demo Video – Full Fledged view of application

 

Video - Part 1 : How Salesforce chatter shows a Location Glimpse LINK Post

 

Video - Part 2 : View detailed location updates and chat - Location Glimpse

 

Part 3 : Location Glimpse "Peer Search" Functionality

 

Video - Part 4 : Location Glimpse Tabs

 

Location Glimpse – Youtube channel

You can also subscribe to video channel for “Location glimpse”, here is the channel link

Location Glimpse – Presentation

Pictures speak more then words, presentations speak even more. So I decided to create a presentation to help every body quickly get an insight about Location Glimpse application.

This presentation is online, that means you can view it right their in your browser. Apart from that their are some videos embedded right into the presentation, those videos can give you a detailed idea about the screen flows. Though if you want you can skip the videos, I will be covering those in separate post too.

This presentation automatically changes slides after 10 secs. If you think its too much, then you use navigation controls at the bottom.

The Presentation

June 4, 2010

Salesforce/Axis MessageElement Out Of Memory Error resolved !

If you are a java developer who’s using Apache Axis for developing web service client applications/mashups and facing “Out of Memory” issues because of “org.apache.axis.message.MessageElement”, then this post can help you in fixing this memory error. This post if also of relevance to developers creating application using Salesforce Web Services via Axis client stubs, its quite easy to go Out of memory when using salesforce partner wsdl to create a bunch of records.

“MessageElement” a nasty, badly documented class !

This class is the one that most of the developers use or even its used by Axis under the hoods at a couple of places. The sad part about this class is its very badly documented and the way developer needs to keep its state correct is not clear at all. Over that, this class eats insane amount of memory if used for bulk operations. So developers should be very careful to operate not in bulk, but chunks with this class.

Using MessageElement class in a risky way

MesageElement class mostly becomes memory hog when creating new instances of it. I faced out of memory issues for this class during one of my recent assignments, we were trying to create many records in salesforce and operation failed always because of out of memory error. We were using the salesforce sample code to create MessageElement instances like this below

	public static MessageElement newMessageElement(String name, Object value) throws Exception {
		MessageElement me = new MessageElement("", name);	
		me.setObjectValue(value);
		Element e = me.getAsDOM();
		e.removeAttribute("xsi:type");
		e.removeAttribute("xmlns:ns1");
		e.removeAttribute("xmlns:xsd");
		e.removeAttribute("xmlns:xsi");
		me = new MessageElement(e);
		return me;
	}

This is a strange but was working way for us to get the job done, we are not Axis experts unfortunately ;-)

But this stuff become unusable then we tried to research around possible solutions, so ideas that came out during that research was

  • To create a single message element and clone it rather creating from scratch.
    • This didn’t worked, MessageElement maintains a complex XML structure that is not easily to clone and then transform to new values.
  • To reduce the number of records we are working on, call Garbage collector and sleep :)

Finally one of my peer Amit, figured a way that is similar to cloning MessageElement, but not really a clone and it worked for us. He suggested to cache a Template MessageElement, and use that for creating new MessageElements. Its explained in detail below.

Alternate Solution – A Template Element

Alternate solution was to cache the Xml “org.w3c.dom.Element” used internally by MessageElement class. MessageElement also gives constructors that accept org.w3c.dom.Element. So we can cache a single Element instance as Template to create new MessageElement instances. As shown below.

private static MessageElement TEMPLATE_MESSAGE_ELEMENT = new MessageElement(
			"", "temp");
// The Template org.w3c.dom.Element instance
private static Element TEMPLATE_XML_ELEMENT;

static {
	try {
		// Create and cache this org.w3c.dom.Element instance for once here.
		TEMPLATE_XML_ELEMENT = TEMPLATE_MESSAGE_ELEMENT.getAsDOM();
	} catch (Exception e) {
		throw new RunTimeException(e);
	}
	TEMPLATE_XML_ELEMENT.removeAttribute("xsi:type");
	TEMPLATE_XML_ELEMENT.removeAttribute("xmlns:ns1");
	TEMPLATE_XML_ELEMENT.removeAttribute("xmlns:xsd");
	TEMPLATE_XML_ELEMENT.removeAttribute("xmlns:xsi");
}


public static MessageElement fromTemplateElement(String name, Object value)
		throws SOAPException {
	// Use the TEMPLATE org.w3c.dom.Element to create new Message Elements
	MessageElement me = new MessageElement(TEMPLATE_XML_ELEMENT);
	me.setObjectValue(value);
	me.setName(name);
	return me;
}

So when we started using the new fromTemplateElement() method, we never got OutOfMemory error, even for big operations that involve handling too many records.

Benchmarks – Why alternate solution “Template Element” is really better ?

For benchmarking I just setup a simple fixture, that you guys can also try quickly to get confidence. In this fixture I just iterated a 100,000 times to create 100,000 MessageElement instances using both approaches.

Here is the fixture (My Machine Core2Duo, 4 Gigs of RAM)

public class MessageElementTest {
// NOTE: Copy the above code and static block for both methods here

	public static void main(String[] args) throws SOAPException {
		List<MessageElement> elems = new ArrayList<MessageElement>();
		for (int i = 0; i < 100000; i++) {
			// Create fake name using current millis
			String name = "name" + System.currentTimeMillis();
			Object value = "val" + System.currentTimeMillis();
			// Print iteration and free memory
			System.out.println("Iteration " + i + " Free Memory :"
					+ +Runtime.getRuntime().freeMemory());
			
			// First approach
			MessageElement e = newMessageElement(name, value);			
			elems.add(e);
		}
		System.out.println(elems.size());
	}

Here are the results, this approach never completed 100,000 iterations and failed near 35,000. Here is the sample output

Iteration 0 Free Memory :15874400
Iteration 1 Free Memory :14616912
.......
.........
.......
Iteration 35642 Free Memory :5256
Iteration 35643 Free Memory :1344
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Now to try alternate approach just comment the single line and add this line as shown below

// First approach
// MessageElement e = newMessageElement(name, value);			
// Alternate Approach
MessageElement e  = fromTemplateElement(name, value);

Now I was able to complete all 100,000 iterations without any Out of memory issues etc. Even I ended with decent memory in hand. Here are the results

Iteration 0 Free Memory :15874400
Iteration 1 Free Memory :14252944
Iteration 2 Free Memory :14252944
...
....
....
Iteration 99998 Free Memory :17744408
Iteration 99999 Free Memory :17744408
100000

So its clearly visible that the original approach is just good for sample codes, if you are planning to put your app in production then must try the Alternative approach.

All the best !

June 2, 2010

Why WS-Client should RETRY Web Service Exceptions ?

Web Services are something that are either used as provider or client by almost every developer. Web services are usually exposed via SOAP, these days RESTful web services are common too. One thing common in all types web services is HTTP protocol, its the base protocol wrapped by SOAP / REST implementations. So all web services are prone to HTTP/Internet/LAN issues and other web service providers issues. This post tries to explain which of these errors/issues should be retried by client code before failing fast.

Development Mode Approach – A fairy tale !

When we are developing Client side code for a web service. We go though flows that rarely fail for Internet/LAN errors or other WS provider mistakes. So we never think about retrying on such failures, we also had a feeling when solution will run on production on A grade super fast internet wires, these issues will not be there.  But this fairy tale turns nightmare, when in production we get strange errors that are completely unexpected and hard to reproduce and debug.

Realistic Approach – Lesser Production Nightmares !

So the life is not like a fairy tale, so we should RETRY for certain errors on web service calls. Though this topic is pretty wide and its not easy to cover all known different issues so I tried to pick common error codes and failure reasons for two popular clouds web service providers i.e. Salesforce and Amazon.

Below is a table that lists these error codes and explains which of them should be retired by a web service client side code.

Error Code Retry “Why” or “Why Not” Retry ?
Unknown Host   YES Unknown host might come because of temporary network issues, we should wait and try to reconnect for those.
Service Unavailable 503 YES Pushing updates or  maintenance window is not too long and is usually known for providers so we should wait and hold on for the known period.
Temporary Redirect 307 YES As said its a temporary redirect, so we can retry on the same endpoint again.
Request Timeout 400 YES Request can timeout because of network issues or because you are querying too much with the web service. If the failure is because of network issues, we should retry, otherwise one should try tuning the web service request to reduce the queried data.

Internal Errors at Server Side

500 YES

To RETRY or NOT, depends on the web service providers. Many providers like Amazon document which internal errors to retry. For others without such documentation we should try to wait for a while and then retry.

Conflicting Request

409 NO We should try optimizing the client code to ensure proper locking, so that multiple threads don’t race against the same resource.

Slowdown

503 NO We should fix client to slow or queue requests. Another cool option provided by many providers is ability to batch multiple requests. So those options should be tried client side.

Token Refresh/Expire

400 YES

Most of the web service providers give a login token in form of Keys, Session Ids etc. Sometimes these tokens have limited life, so client code should try renewing these tokens on such errors.

Bad Request/Digest, Incomplete Body, Invalid Argument, Malformed XML,  
Malformed POST Request, Missing Content Length,
400 NO Client code should be fixed to form correct requests
Access Denied 403 NO Try different credentials
Wrong End Point, PermanentRedirect 301 / 400 NO Client code is trying incorrect end point. The URL of end point should be fixed.
MethodNotAllowed 405 NO Client code should be fixed.

In general one can follow some simple rules based on HTTP status codes too. Though these rules are not applicable to all web service providers, but most of the time you will end up in taking right RETRY or NOT decision. The table below explains this

HTTP STATUS CODE MEANING ? RETRY ? “Why” or “Why Not” Retry ?
307 Moved Temporarily YES Retry after a while.
400 Bad Request NO Needs to fix client side code to form Request correctly.
403 Forbidden NO Needs to fix the credential in client side code.
405 Method Not Allowed NO Need to fix the HTTP call to use correct method.
409 Conflict YES Client is racing for same resource, have some locking in client code to ensure not requesting conflicting actions on same resource. If locking is not possible wait for a while and retry.
411 Length Required NO Client must provide the Content-Length HTTP header.
500 Internal Server Error YES The client side code is correct, some thing on web service provider’s side failed. So retry after a while.
501 Not Implemented NO Client is trying to use a functionality that is not yet implemented
503 Service Unavailable YES You may retry here, if service is down for a while. Like for Maintenance.

Open Source Project Coming Soon !

I am about to release an open source project for Salesforce and Amazon that helps you write Retryable client side code easily. Stay connected, I will post updates.