Parsing Large Json Files Using Jackson Streaming Api Example

In final twosome of JSON tutorials for Java programmers, nosotros convey learned how to parse JSON using JSON-Simple library, parsing JSON array to Java array using GSon, as well as inward this tutorial nosotros volition acquire how to parse a large JSON file inward Java using Jackson's Streaming API. Jackson is 1 of the most pop JSON processing framework as well as provides 3 primary model to parse as well as procedure JSON information including Streaming API, information binding as well as tree model. Out of these three, Streaming plant at lowest flat as well as tin sack live used to parse huge JSON reply upto fifty-fifty giga bytes of size. If y'all are familiar alongside XML parsing, thus y'all know that how hard it is to parse huge XML files alongside DOM parser because it fully loads the file inward retentiveness earlier y'all tin sack procedure it. In representative y'all convey depression retentiveness e.g. Android devices y'all can't operate that to parse XML. Thankfully, XML provides SAX as well as StAX parsers which are streaming based as well as tin sack live used to procedure huge files without loading them completely inward memory. Out of these two, StAX is fifty-fifty improve because it allows trace based processing where customer pulls information from parser instead of parser pushing data, which is the representative alongside SAX parser. Jackson's Streaming API is similar to StAX parser. You tin sack trace the information y'all desire as well as ignore what y'all don't want. Though functioning doesn't come upwards without cost, using Streaming API is trivial hard thus using other Jackson model which provides take away mapping betwixt Java as well as Jackson objects. You convey to receive got all JSON information past times yourself piece using Streaming API.

Benefits of using Jackson Streaming API

There are several advantages of using Jackson's Streaming API to parse JSON String or convert Java object to JSON, but the most of import 1 is that its real efficient. It has to the lowest degree retentiveness as well as processing overhead as well as extremely useful to parse large JSON responses, for representative a JSON reply containing thousands of guild or listing of books or listing of electronic items downloaded from e-commerce sites similar eBay or Amazon. Talking most other ii model of Jackson API, information binding model converts JSON to as well as from Java object based either notation or Java edible bean convention, piece Tree Model provides a mutable in-memory tree representation of a JSON document, similar to DOM parser. In short, Streaming API is most powerful, has less retentiveness as well as CPU overhead but tricky to use, piece information binding is ofttimes most convenient, on the other mitt Tree Model is most flexible. BTW, both of this model internally uses streaming API to parse JSON strings earlier converting it into respective models.

Library JARs as well as Dependency

In guild to endeavor next example, y'all postulate to download as well as add together Jackson streaming API inward your program's classpath. If y'all are using Maven thus y'all tin sack add together next dependency inward your pom.xml file :

<dependency>    <groupId>org.codehaus.jackson</groupId>    <artifactId>jackson-xc</artifactId>    <version>1.9.12</version> </dependency>

or only download as well as  add next JAR inward CLASSPATH of your Java application.

C:\.m2\repository\org\codehaus\jackson\jackson-xc\1.9.12\jackson-xc-1.9.12.jar C:\.m2\repository\org\codehaus\jackson\jackson-core-asl\1.9.12\jackson-core-asl-1.9.12.jar C:\.m2\repository\org\codehaus\jackson\jackson-mapper-asl\1.9.12\jackson-mapper-asl-1.9.12.jar 

It's ofttimes easier to care dependency using Maven as well as that's why I strongly advise to switch to Maven if y'all are non using it yet. You tin sack afterwards upgrade to newer version of Jackson library past times only changing 1 trouble inward Maven pom.xml file.

Parsing JSON inward Java using Jackson Streaming API

This API has ii primary module, 1 fore reading JSON as well as other for writing JSON as well as inward this tutorial nosotros volition acquire both of them. JsonGenerator is used to write JSON piece JsonParser is used to parse a JSON file. To demonstrate both reading as well as writing of JSON information inward 1 program, I convey created ii static methods, createJSON() as well as parseJSON(). As advert suggests root method creates a JSON file, which is thus read past times parseJSON() method. You tin sack run across inward the code that nosotros are dealing alongside quite depression level, nosotros convey non created whatever Java object to correspond content of JSON, instead nosotros are writing as well as reading String, numbers as well as arrays.

You tin sack acquire an instance of JsonGenerator from JsonFactory shape past times calling createJsonGenerator() method. You tin sack too supply the encoding y'all are intended to use, inward our representative I convey used "UTF-8" which is a convenient default inward most cases. You tin sack operate diverse write() methods to write contents.  Similarly, for parsing JSON, nosotros postulate to practice an instance of JsonParser, which tin sack too live obtained from JsonFactory.  We parse JSON past times calling nextToken() method of JsonParser inward a piece loop until nosotros accomplish JsonToken.END_OBJECT. Jackson API provides method to acquire advert as well as value of token which y'all tin sack operate to position data. Similarly piece parsing JSON array, y'all hold back until y'all acquire JsonToken.END_ARRAY identifier. Since nosotros never charge the whole file inward memory, this method tin sack live used to read large JSON files alongside sizes from Mega bytes to Giga bytes fifty-fifty alongside minimal retentiveness environs e.g. inward Android smartphones or Java ME enabled devices.

Here is the sample code representative to read as well as write JSON using Jackson Streaming API :

import java.io.File; import java.io.IOException;  import org.codehaus.jackson.JsonEncoding; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonGenerationException; import org.codehaus.jackson.JsonGenerator; import org.codehaus.jackson.JsonParser; import org.codehaus.jackson.JsonToken; import org.codehaus.jackson.map.JsonMappingException;  /** * Java programme to demonstrate how to operate Jackson Streaming API to read as well as * write JSON Strings efficiently as well as fast. * * @author Javin Paul */ public class JsonJacksonStreamingAPIDemo{      public static void main(String args[]) {          System.out.println("Creating JSON file past times using Jackson Streaming API inward Java");         createJSON("jacksondemo.json");         System.out.println("done");          System.out.println("Parsing JSON file past times using Jackson Streaming API");         parseJSON("jacksondemo.json");         System.out.println("done");     }      /*      * This method practice JSON String past times using Jackson Streaming API.      */     public static void createJSON(String path) {         try {             JsonFactory jsonfactory = new JsonFactory();             File jsonDoc = new File(path);             JsonGenerator generator = jsonfactory.createJsonGenerator(jsonDoc, JsonEncoding.UTF8);              generator.writeStartObject();             generator.writeStringField("firstname", "Garrison");             generator.writeStringField("lastname", "Paul");             generator.writeNumberField("phone", 847332223);              generator.writeFieldName("address");              generator.writeStartArray();             generator.writeString("Unit - 232");             generator.writeString("Sofia Streat");             generator.writeString("Mumbai");             generator.writeEndArray();              generator.writeEndObject();              generator.close();              System.out.println("JSON file created successfully");          } catch (JsonGenerationException jge) {             jge.printStackTrace();         } catch (JsonMappingException jme) {             jme.printStackTrace();         } catch (IOException ioex) {             ioex.printStackTrace();         }     }      /*      * This method parse JSON String past times using Jackson Streaming API example.      */     public static void parseJSON(String filename) {         try {             JsonFactory jsonfactory = new JsonFactory();             File source = new File(filename);              JsonParser parser = jsonfactory.createJsonParser(source);              // starting parsing of JSON String             while (parser.nextToken() != JsonToken.END_OBJECT) {                 String token = parser.getCurrentName();                  if ("firstname".equals(token)) {                     parser.nextToken();  //next token contains value                     String fname = parser.getText();  //getting text field                     System.out.println("firstname : " + fname);                  }                  if ("lastname".equals(token)) {                     parser.nextToken();                     String lname = parser.getText();                     System.out.println("lastname : " + lname);                  }                  if ("phone".equals(token)) {                     parser.nextToken();                     int telephone = parser.getIntValue();  // getting numeric field                     System.out.println("phone : " + phone);                  }                  if ("address".equals(token)) {                     System.out.println("address :");                     parser.nextToken(); // side past times side token volition live '[' which agency JSON array                      // parse tokens until y'all discovery ']'                     while (parser.nextToken() != JsonToken.END_ARRAY) {                         System.out.println(parser.getText());                     }                 }             }             parser.close();          } catch (JsonGenerationException jge) {             jge.printStackTrace();         } catch (JsonMappingException jme) {             jme.printStackTrace();         } catch (IOException ioex) {             ioex.printStackTrace();         }     } 

as well as hither is the output of our program, when y'all run it from Eclipse or straight from ascendency trouble :

Creating JSON file past times using Jackson Streaming API inward Java JSON file created successfully done Parsing JSON file past times using Jackson Streaming API firstname : Garrison lastname : Paul telephone : 847332223 address : Unit - 232 Sofia Streat Bombay done

You volition too run across file jacksondemo.json inward your projection directory alongside next JSON String :

{   "firstname":"Garrison",   "lastname":"Paul",   "phone":847332223,    "address":["Unit - 232","Sofia Streat","Mumbai"] }

That's all most how to operate Jackson Stream API to parse JSON String and to practice JSON from Java object. It's a powerful library alongside lots of characteristic but Streaming is best. I know its trivial fighting hard as well as y'all postulate to write lot of code alongside hard coded filed names, it is the fastest way to read a large JSON file inward Java alongside less retentiveness overhead. If y'all are dealing alongside normal size JSON output as well as y'all don't convey a retentiveness constraints thus y'all tin sack ever operate Jackson Data binding model to parse JSON to Java Object.

Further Learning
Master Java Web Services as well as REST API alongside Spring Boot
REST API Design, Development & Management
tutorial)

3 Ways to parse JSON String inward Java? (tutorial)

How to convert JSON array to String array inward Java? (example)

How to convert a Map to JSON inward Java? (tutorial)

How to operate Google Protocol Buffer inward Java? (tutorial)

How to operate Gson to convert JSON to Java Object? (example)

5 Books to Learn REST as well as RESTful Web Services (books)

P.S. - If y'all are looking for online grooming to acquire how to develop RESTful Web Services inward Java using Spring framework, I advise y'all joining Eugen Paraschiv's REST alongside Spring course. The course of report has diverse options depending upon your sense flat as well as how much y'all desire to acquire e.g. beginner's class, intermediate class, as well as master copy class. You tin sack bring together the 1 which suits y'all better, though I advise joining the master class if y'all are serious most becoming an skillful Java REST developer.

Share this post