Performance issue when parsing large JSON

[marinaDurkovic]

Hi,

I noticed when parsing large JSON string there is a performance issue. String that I'm parsing has around 4 000 lines, and creation of the json_doc is about 6 min . Is it possible in this case to create document in shorter time ?

json_doc = zcl_json_document=>create_with_json( my_large_json ).

Best regards,
Marina

[se38]

Hi Marina,

if you don't need the special handlings of the JSON document class (like special date format etc.) you can try the simple transformation:

"--- JSON -> ABAP ---
zcl_json_document=>transform_simple(
EXPORTING
json_in = json
IMPORTING
data_out = test
).

[marinaDurkovic]

I was using json_doc->get_next( ) to iterate through objects. When I try transform_simple I'm getting an error at this line of code

My json is structured like this

So maybe that is causing the problem with transform_simple method. In the begining I have [ and later objects that start with { . Do you have a suggestion how can I deal with this type of json structure using that method ?

Regards,
Marina

[se38]

Hi Marina,
unfortunatelly the simple transformation (SAP Standard) needs upper case variable names and has to start with { "RESULT" : ....
See https://github.com/se38/zJSON/wiki/Usage-zJSON#simple-transformation-new-in-release-0217

Performance is indeed an issue with the JSON Document class, especialy with nested loops in it. I'm still searching for a solution.

Uwe

[se38]

Starting to play around with a JavaScript parsing solution. Stay tuned, maybe a fast version 3.0 is near ;)

@marinaDurkovic Is it possible that you send me your testdata (anonymised if personel data included)? uwe.fetzer [at] se38.de

[se38]

Hi Marina,

good and bad news regarding my new JSON parser which uses the SAP Standard class CL_JAVA_SCRIPT now.

Bad news:
I've found an error in the SAP standard class. In case the JSON is too large, the class doesn't work properly.

Good news:
the parser is a loooot faster than before. I've shorten your JSON test file to around 1.600 records and the parsing process lasts 45 seconds.

Your original file:
Length 5.009.241
Lines 107.174
Records ~3.500
Doesn't work

Maximum length which worked:
Length 2.290.743
Lines 49.152
Records 1.606
45 Seconds

I think we can live with this result (and the restriction regarding the size of the file, we "just" have to split the files). Expect a beta version of the new parser end of March or in April.

Regards, Uwe