CSV Manager Features

The CSV Manager component has a focused set of features that are designed for ease-of-use.

Any CSV Format

There is no standard definition of the CSV file format itself. CSV has not been deemed worthy of ISO definition, or an ECMA standard. CSV does however have a MIME type. Our thanks to Yakov Shafranovich for writing this document.

In the real world, there are a number of different implementations and informal definitions. To cover the full range of CSV file formats that you may encounter, CSV Manager allows you to control the exact type of CSV file that you need to load or save.

The most common CSV format is the Excel format. A semi-formal description can be found at the Creativyst website. This is the format that CSV Manager supports out of the box. You can read any Excel spreadsheet saved as a CSV file without any extra work.

The Excel CSV format is not as simple as it appears. The main difficulty arises from the fact that new line characters can appear inside quoted data fields. That means that each line of the CSV file may not correspond to each data record. A simple parser that splits the file into a set of lines cannot be trusted to return the correct data. CSV Manager can handle this little irritation and you can guarantee that any CSV file created by Excel will be handled correctly by your application.

CSV Format Variations

Although CSV nominally stands for Comma Separated Values, the CSV format actually includes a wide range of tabular text formats. As far as CSV Manager is concerned, any text format with data records made up of data fields, and separated by some set of characters, is a CSV format file. This flexible definition means that CSV Manager can also understand files such as the UNIX passwd file.

The following are some of the variations that CSV Manager can handle:

• Separation character: ,, TAB, ;,|, and so on.
• Quote character: " versus '
• Quote escape method: "" versus \"
• Escape codes: \n, \t, and so on. You can also define your own codes.
• Whitespace trimming: optionally remove spaces or any other characters before and after data fields.
• Intelligent newline handling: newlines are correctly handled depending on the platform. You can also specify your own data record separators.

Access Data Directly

CSV Manager makes it easy to get data into and out of CSV format files. By providing direct access to the data using standard Java objects, you can get hold of the data as a simple List of String[] arrays, or as a List of Lists. You can also access the data as a ResultSet, which makes it easy to interface with database code. And if you need to present the data as a table, then it is available as a Swing TableModel. Examples are included in the documentation for each case. As well as loading data using these objects, you can save data in CSV format using the same objects.

CSV Manager also provides more advanced access to your data. You can receive and process each data record as it is parsed. You can even receive each data field as it is parsed. And you can run the load or save process in a background thread.

For high data volumes or automated batch processing, CSV Manager can operate in a fault tolerant mode, where errors in the current data record are ignored so that processing can continue. Errors then can be collected for later review. CSV Manager also keeps timing statistics so that you can track the status of your data processing operation. When running in a background thread, these timing statistics are updated continuously.

Great Documentation

Complete and comprehensive documentation is an important part of every component. We believe that good documentation is essential when using third party software, so we make sure that all of our components are well documented. Each and every method, parameter and property of every public class is documented. The API documentation also includes simple examples. We provide a complete reference list of every setting.

Documentation must not only be complete, it must also help you learn. We include a step-by-step getting started guide, as well as simple examples that you can use right away.

Of course, real programmers always need access to the source code, so we've released selected sections of the CSV Manager component source code under a BSD-style license. This is the ultimate form of documentation and we are quite happy for you to inspect our code and see how we do things. Where source code is available, this is indicated in the API documentation for the class. For example: for class: LineProviderSupport, the source code is here: LineProviderSupport.java (in this case we provide the source of the abstract base class so that you can really understand what you are implementing).

Easy to Use

Usability is not just for user interfaces, it also applies to internal software interfaces. When it comes to getting the job done on time, the API that a component provides is just as important as the functionality. The CsvManager object provides a single direct interface to the component. Commonly used functions are provided as simple direct method calls on this object. In general, although we build components that are extensible and have sufficient layers of abstraction, we consider carefully how the software design affects ease-of-use and how difficult it is to learn.

Fast and Scalable

How fast is CSV Manager? See for yourself. We have verified that not only is CSV Manager fast, but that it is also robust. This means you can expect massive CSV files to take a reasonable amount of time to load, rather than locking up the system. The amount of time it takes to parse a CSV file is directly proportional to the size of the file, and does not increase exponentially.