Streaming Data

CSV Manager can load and save your data in a continuous stream. This allows you to handle very large CSV files.

Code Example

File       csvfile = new File( "mydata.csv" );
CsvManager csvman  = new CsvManager();
CsvLoader  loader  = csvman.makeLoader( csvfile );

loader.begin();
while( loader.hasNext() ) {
  String[] fields = loader.next();

  for( int field = 0; field < fields.length; field++ ) {
    System.out.print( fields[field]+", " );
  }
  System.out.println();
}
loader.end();

This code example can load data from a very large CSV file one line at a time. This means that the whole file is not loaded in memory which conserves resources and makes the loading process a lot faster. All you have to do is call the CsvManager.makeLoader method. This returns a CsvLoader object that you can use to pull the CSV data in, one line at a time.

Use this method when...

• you cannot load all the data into memory at once.
• you want to handle the data line-by-line.
• You need to control exactly when you get the next line of data.

What happens when some rows contain more fields than others?

Each data row is returned individually as a String[] array by the CsvLoader object. Each array always contains enough elements to hold all the data fields found in any one row, so you never lose data.

What happens when some rows are missing fields?

The String[] array returned by CsvLoader always contains a minimum number of elements. This minimum can be set using the setNumFields method. If not set, the number of fields in the first line is used as the minimum. This minimum is automatically increased (and stays increased) whenever a row containing more fields than the previous minimum is found. For any given line, if the number of data fields is less than the minimum, then the remaining fields are set to empty strings.

OK, how do I get the column headers?

The column headers are simply the first row of data returned. If the column names are not in the first line, then they are either in a predefined line (say line 3), so start loading lines from there using setStartLine, or you can set CSV Manager to ignore empty lines, until you find them.

But my data isn't in a file!

You are not restricted to loading CSV data only from files. The CsvManager.makeLoader method also accepts standard Java InputStreams and Readers. You can also load CSV data direct from in-memory Strings by using a Text object.

What about saving data to a CSV file?

You can also save data to a CSV file one line at a time by calling the CsvManager.makeSaver method. This returns a CsvSaver object that you can use to push CSV data out, one line at a time. Here's the example code:

File       csvfile = new File( "mydata.csv" );
CsvManager csvman  = new CsvManager();
CsvSaver   saver   = csvman.makeSaver( csvfile );

ArrayList data = new ArrayList();
data.add( new String[] {"1","a","AA"} );
data.add( new String[] {"2","b","BB"} );

saver.begin();
for( Iterator lineI = data.iterator(); lineI.hasNext(); ) {
  String[] fields = (String[]) lineI.next();
  saver.next( fields );
}
saver.end();

What other data structures can you handle?

We also support:

• String Arrays
• List of Lists
• ResultSets
• TableModels
• Event-driven Data

I need to see something working.

Sure, no problem. Head on over to our online demo and try out CSV Manager on your own data.

I have another question...

If you have a particular question you need answered, just ask! We're happy to explain exactly how CSV Manager works and show you how to use it.