CSV Formats

The CSV file format has no formal definition, despite its common use as a data exchange file format. The closest thing to a formal definition is probably RFC 4180, the MIME type for CSV files. The most practical definition is probably the Microsoft Excel CSV format (this is the default format of Ricebridge CSV Manager). Although CSV nominally stands for Comma Separated Values, even the comma is not mandatory, with tabs and colons also commonly used as field separation characters.

It is more useful to consider the CSV format as a specific instance of a more general file format, which some have named DSV format, for Delimiter Separated Values. legitimate examples of DSV files then include all the variants that Excel recognises, but includes common UNIX formats such as the passwd file. The general case simply recognises that the fundamental structure is one of repeated fields within repeated rows, and that rows and fields are separated by designated characters.

For a more formal definition, Creatyvist Software have an excellent article on the finer points of the CSV format. The reference document included with this documentation also describes the formal grammar used by Ricebridge CSV Manager for parsing CSV files. On the surface CSV files seem fairly easy to parse, but there are number of issues that have to be dealt with before you can be sure of parsing any CSV file.

The critical problem with the CSV format is how to deal with commas and newlines (or any other separation characters) inside a data field. Since it is perfectly possible that data fields can contain any characters, especially for free format data such as names, addresses or short pieces of text, this is not an issue that can be ignored.

The most common way to deal with separators inside data fields, is to quote the data field. While this is not particularly problematic for commas, it does mean that you cannot parse a CSV file by simple breaking it up into input lines, as some newline characters may be inside quotes. Thus nearly all quick scripts and handwritten CSV parsers will fail in this case.

Another approach to dealing with separators is to escape them with a backslash or other character. This approach is more common on UNIX systems. While this does have the advantage that input lines does correspond directly to data rows, it still means that one cannot split data rows into data fields by simply splitting on commas, as these may in turn be escaped.

You can also add the various flavours of newline encoding to these parsing annoyances and compound that with the fact that external CSV files may combine various data escaping strategies, and you will find that CSV files are not as simple as they appear on the surface.

To help you handle any CSV file that you need to parse, CSV Manager provides you with a default set of common format variations, and also allows you to adjust each individual format variation so that you can get your data into the application. Defaults formats are provided for Excel, Mac files, passwd files, and general UNIX files.

Why bother hand-coding yet another CSV parser? Let us deal with the bugs. We make it simple and easy to focus on the real work; delivering your project on time.

Try the Online Demo Download Now for $170.00 More Licensing options Why should I buy this component?

How does CSV Manager load your data? We provide five options:

• List of String arrays
• List of Lists
• JDBC ResultSet
• Swing TableModel
• Streaming Data

Need to load your data into a different data structure? Please tell us!