September 29, 2011

JSON parsing in Apex, a developer’s nightmare :( Winter’12 release I badly need you !

Today I stumbled upon a requirement to parse JSON in Apex. I was happy about the fact that I know there is a cool open source library called JSONObject, that will do all the heavy lifting of JSON parsing and another more cooler native JSON parser is lined up in coming winter'12 release.

I was able to quickly setup the web service fixture using all goodness of dependency injection for callouts in apex, as discussed in this post.

“JSON response vs JSONObject Parser” the bloody war !

The real pain started when I started parsing JSON response. I met a series of awesome exceptions and errors, leaving me confused about who is wrong JSON or Parser :-?
I firstly got this exception about “Missing Value”

After doing some googling, I found this post, where the code snippet indicated that newlines might be an issue with the parser. So I removed both “\r” & “\n” chars from the JSON response, why would machine require a pretty printed JSON :) Then life moved on for a while, until I got this exception

“FATAL_ERROR|superpat.JSONObject.JSONException: Expected a , or }”

I was again thinking, now what who’s wrong here JSON or Parser :-? Again after some fighting with code, I ended up on google searching for the same and luckily found metadady already fixed this one. Many thanks to metadady, for fixing this issue and submitting a patch, I don’t know why this patch is still not applied on JSONObject since Jan 24, 2011.

Again, life moved a bit ahead, until the JSONObject started parsing some Unicode chars like “\u0026”, in the response. Again Metadady was nice enough to mention this issue with his patch, but it seems there is no clean way to handle unicode chars in Apex. So I decided to get rid of them, I really can live without these special creatures in String :(
So, I kicked off all unicode awesomeness in JSON response with something like

“jsonstring.replaceAll('\\\\u.{4}', '')”
Next, life moved really good and my JSON was parsed successfully, I was having tears of joy in my eyes :) But those tears of joy turned into tears of sorrow, when I saw the debug logs
  • it took around 10+ seconds and
  • “154574” script lines for parsing a moderate json string.
Number of script statements: 154574 out of 200000 ******* CLOSE TO LIMIT

It was a strange feeling of loosing everything after you won, words can’t express it.

I was having some hope, that I can get rid of it. But but but, it was all lost when I read this post from legendary Jeff Douglas. It was an unlucky Googling day for me, if this Jeff’s post could have appeared earlier, I could be more happy guy by EOD.

Is Game over ?

Anyway, game is not over yet. I am waiting for winter’12 release and native JSON parser i.e. SYSTEM.JSON to come. Thanks to Salesforce team for finally adding a native JSON parser to the Apex library.