CSV to JSON Data Interchange: RFC 4180 Escaping, Delimiters & Edge Cases
Master CSV and JSON conversion mechanics. Learn how quotes, commas, CRLF line endings, and custom delimiters affect data parsing pipelines.
Key takeaways
- CSV looks deceptively simple, but RFC 4180 requires wrapping fields in double quotes whenever they contain commas, line breaks, or quotation marks.
- Quotes inside quoted fields must be escaped with double double-quotes ("").
- Windows CRLF (\r\n) versus Unix LF (\n) mismatches are the leading cause of corrupted row counts in automated ETL scripts.
- Converting large CSV files client-side requires streaming or chunking to avoid freezing the browser main thread.
Companion utility
Try CSV to JSON Converter
Use the browser utility for the same task, then return here for the implementation details.
1. The Deceptive Complexity of CSV (RFC 4180)
CSV looks like the simplest data format on the internet until you get a 50MB export from an enterprise CRM. Suddenly, lines break mid-address, unescaped commas split customer names into extra columns, and the parser crashes. The official RFC 4180 specification sets clear rules for handling these edge cases: - Any field containing commas, double quotes, or line breaks must be enclosed in double quotes. - If a double quote appears inside a field, it must be escaped by prefixing it with another double quote (""). - Line breaks should technically use Carriage Return and Line Feed (CRLF, \r\n), although Unix-style LF (\n) is widely accepted.
2. Object Mapping vs 2D Matrix Output
When transforming CSV data into JSON, developers usually choose between two structures: 1. Object Array (Header Mapping): The first row provides the property names. Each subsequent row becomes an object: [ { "id": 1, "sku": "TOOL-01" } ]. This is ideal for REST APIs and application state. 2. 2D Array (Matrix): Keeps rows and columns as nested arrays without assuming headers: [ ["id", "sku"], [1, "TOOL-01"] ]. This is much more memory-efficient for numerical data science.
import csv, json
def convert_csv_to_json(csv_path, json_path):
with open(csv_path, mode='r', encoding='utf-8') as f:
reader = csv.DictReader(f)
records = [row for row in reader]
with open(json_path, mode='w', encoding='utf-8') as f:
json.dump(records, f, indent=2)
print("Export completed successfully!")Rana Jawad
Principal Systems Architect at ToolNara. The guides focus on browser runtime behavior, standards, and engineering trade-offs.
View author profile