By Jun Nguyen2 min read278 words
JSON and NDJSON
Technology
Technology
NDJSON (Newline Delimited JSON, also called JSON Lines or LDJSON) is a format where each line is a valid, standalone JSON object, separated by newline characters (
\\n
). This makes it ideal for streaming, logging, or processing large datasets line-by-line, since you donβt need to parse the entire file at once .
π NDJSON vs. Regular JSON
Feature | NDJSON | Regular JSON |
---|---|---|
Structure | One JSON object per line, no outer array or commas | Can be a single object, array, or nested structure |
Streaming | β Easily stream-processed line-by-line | β Must parse entire file/array at once |
Appending Data | β Just add a new line β no need to modify structure | β Must rewrite the entire array/object |
Memory Efficiency | β Low memory usage β process one line at a time | β High memory usage β entire structure loaded |
File Size | Slightly larger due to repeated keys and no compression | More compact in arrays with shared structure |
Tooling Support | β
Works with grep , awk , jq -c , etc. | β Harder to process with line-based tools |
MIME Type | application/x-ndjson | application/json |
βοΈ Example Comparison
Regular JSON:
[ {"name": "Alice", "age": 30}, {"name": "Bob", "age": 25} ]
NDJSON:
{"name":"Alice","age":30} {"name":"Bob","age":25}
β Use Cases for NDJSON
- Log aggregation (e.g., ELK stack)
- Bulk data ingestion (e.g., Elasticsearch _bulkAPI)
- Real-time event streaming
- ETL pipelines with large datasets
β οΈ Notes
- NDJSON is not valid JSON as a whole, because it contains multiple top-level JSON texts .
- Avoid pretty-printing β each line must be compact JSON to avoid breaking the format .
In short: Use NDJSON when you need to stream or append structured data efficiently. Use regular JSON when you need a single, complete data structure.