Ricebridge Java Components - user friendly and well documented

To compile and run the Java programs in this example, either import them into your IDE (say, Eclipse or NetBeans), or follow the standard manual compilation instructions. When running the programs, run them in the same folder as this README.htm file, so that they can find their input files.

The Data

We will use a small CSV file as our example data. This file contains a list of the employees in a company, and indicates which employee is in which project, using project codes.

company.csv


Employee Number, Name, Hired, Manager, Project Names
1, John Doe, 2006-01-01, no, red
2, Jane Doe, 2006-02-02, no, red
4, Mr. Manager, 2006-04-04, yes, red:blue
5, Joe Bloggs, 2006-05-05, no, blue
6, Jane Bloggs, 2006-06-06, no, blue

The data contains some special formats that we need to deal with. First the dates are given in a format of YYYY-MM-DD, so we'll have to parse them using a DateFormat. Also, the project list for each employee is a colon-separated list of code names. These will correspond to indexed properties in our beans. Luckily, CSV Manager can handle these automatically.

We want to display this data as a set of HTML tables showing the employees and the data fields for each employee. Here is the output we want:

Number	Name	Manager	Hired	Projects
6	Jane Bloggs	NO	06/Jun/2006	Projects: blue
2	Jane Doe	NO	02/Feb/2006	Projects: red
5	Joe Bloggs	NO	05/May/2006	Projects: blue
1	John Doe	NO	01/Jan/2006	Projects: red
4	Mr. Manager	YES	04/Apr/2006	Projects: red blue

(You might be wondering about employee number 3. As you'll soon find out, he's the bad guy...)

The Beans


public class Employee {

  private int      iNumber       = 0;
  private String   iName         = "";
  private Date     iHireDate     = new Date();
  private boolean  iManager      = false;
  private String[] iProjectNames = new String[] {};


  public int getNumber() {
    return iNumber;
  }

  public void setNumber( int pNumber ) {
    iNumber = pNumber;
  }


  public String getName() {
    return iName;
  }

  public void setName( String pName ) {
    iName = pName;
  }


  public Date getHireDate() {
    return iHireDate;
  }

  public void setHireDate( Date pHireDate ) {
    iHireDate = pHireDate;
  }


  public boolean isManager() {
    return iManager;
  }

  public void setManager( boolean pManager ) {
    iManager = pManager;
  }


  public String[] getProjectNames() {
    return iProjectNames;
  }

  public void setProjectNames( String[] pProjectNames ) {
    iProjectNames = pProjectNames;
  }

  public String getProjectName( int pIndex ) {
    return iProjectNames[pIndex];
  }

  public void setProjectName( int pIndex, String pName ) {
    iProjectNames[pIndex] = pName;
  }
}

Note the ProjectNames property. This in an indexed property, which is a special type of Java Beans property that can have multiple values. We've included it in this example specially so that we can show you have to handle indexed properties.

All of these need to be converted from a textual representation to an internal Java object. Let's see how we do that...

...and actually, CSV Manager does almost all the work for us! For normal properties, like Strings and ints, we don't need to anything. They just work. For indexed properties, we rely on the convention that ':' separates the values. CSV Manager does the rest. If you want to get into the details, see TypeArrayStringConverter.

But for HireDate, we are using a custom date field. What we need is a way to tell CSV Manager how to convert between a java.util.Date and our String format, YYYY-MM-DD. To do that we use another utility class called a StringConverter. This is a very simple interface that takes a String and makes an Object, and also does the opposite. Here is our converter for dates:

DateConverter.java


public class DateConverter extends DefaultStringConverter {
  private Date       iDefault    = new Date();
  private DateFormat iDateFormat = null;

  public DateConverter( DateFormat pDateFormat ) {
    iDateFormat = pDateFormat;
  }

  protected Object makeObjectImpl( String pValue ) throws Exception {
    return iDateFormat.parse(pValue);
  }

  protected Object makeDefaultObjectImpl() {
    return new Date();
  }

  protected String makeStringImpl( Object pValue ) throws Exception {
    return iDateFormat.format((Date)pValue);
  }

  protected String makeDefaultStringImpl() {
    return new Date().toString();
  }
}

The converter actually inherits from the DefaultStringConverter class. The DefaultStringConverter handles any formatting errors for us and returns default values if required (this is how the Bean.useDefault setting is implemented).

We also have to handle the Manager field. We need to convert "yes" and "no" to true and false. We'll use a custom converter again:

YesNoConverter.java


public class YesNoConverter extends DefaultStringConverter {

  protected Object makeObjectImpl( String pValue ) throws Exception {
    return new Boolean( "yes".equalsIgnoreCase( pValue ) );
  }

  protected Object makeDefaultObjectImpl() {
    return new Boolean(false);
  }

  protected String makeStringImpl( Object pValue ) throws Exception {
    return ((Boolean)pValue).booleanValue()?"yes":"no";
  }

  protected String makeDefaultStringImpl() {
    return "no";
  }
}

As you can see, converters are pretty easy to write. You just need to think about four methods. One (makeObjectImpl) to convert a String to an Object. One (makeStringImpl) to convert an Object to a String. And two default methods ( makeDefaultObjectImpl and makeDefaultStringImpl) to create default String and Object values, for when the source value is not available.

So this is how CSV Manager lets you handle custom data. You define a StringConverter and specify exactly what the text of your CSV data field means.

Loading the Data

To load the CSV data we need a BeanSpec object and a LineSpec object. These two parameters to the CsvManager.loadBeans method provide CSV Manager with the details it needs to convert the CSV data fields in Java Bean objects.

First, the LineSpec tells CSV Manager which data fields go with which bean property methods. You provide a String[] array that lists the data fields in the CSV file in the order you expect to get them. The elements of the String[] array are the bean property method names, with the get/set/is removed. For example:


LineSpec ls_employee
  = new LineSpec(new String[]{"Number","Name","HireDate","Manager","ProjectNames"});

Second, the BeanSpec describes the Java Bean. It figures out the bean properties by looking at the method names of the bean. The easiest way to create a BeanSpec is to use the .class property of the bean object. For example:


BeanSpec bs_employee = new BeanSpec( Employee.class );

OK, now that we've got all the pieces, let's assemble them. Normally when you use CSV Manager to load Java Beans, you just call the CsvManager.loadBeans method. This loads up all the beans in your CSV document using the specified LineSpec and BeanSpec object. Here's the code:


CsvManager csvman = new CsvManager();

DateConverter  dc  = new DateConverter( sDateInputFormat );
YesNoConverter ync = new YesNoConverter();

LineSpec ls_employee
  = new LineSpec(new String[]{"Number","Name","HireDate","Manager","ProjectNames"});

HashMap employee_stringconv = new HashMap();
employee_stringconv.put( "HireDate", dc );
employee_stringconv.put( "Manager", ync );

BeanSpec bs_employee = new BeanSpec( Employee.class, employee_stringconv );

CsvSpec csvspec = csvman.getCsvSpec();
csvspec.setStartLine(2);

List employees = csvman.loadBeans( pCompanyCsvFile, ls_employee, bs_employee );

So here we've put it all together. Notice how we put the DateConverter and the YesNoConverter into a HashMap and passed this to the BeanSpec. This is how we tell CSV Manager to use our custom converters. The keys of the HashMap should match the property names in the LineSpec object.

We're also using the CsvSpec object. This is used to control various options when loading CSV. In this case, the first line of data is just a header line, so we want to ignore it. So we tell CSV Manager to start loading at line two using the CsvSpec.setStartLine method. (Hey, why so many 'Spec' objects!? Well normally you'd only really use CsvSpec. It's just that Java Beans need a few more bits and pieces of information. Sorry!)

Creating the HTML Output


FileOutputStream fos = new FileOutputStream( "company.htm" ) );
PrintWriter      pw  = new PrintWriter( fos );

pw.print("<div><table border=\"1\"><tr>"
  +"<th>Number</th><th>Name</th>"
  +"<th>Manager</th><th>Hired</th><th>Projects</th></tr>\n");

for( Iterator eI = pEmployees.iterator(); eI.hasNext(); ) {
  Employee emp = (Employee) eI.next();

  pw.print( "<tr><td>"+emp.getNumber()+"</a></td>\n" );
  pw.print( "<td>"+emp.getName()+"</a></td>\n" );
  pw.print( "<td>"+(emp.isManager()?"YES":"NO")+"</td>\n" );
  pw.print( "<td>"+sDateDisplayFormat.format(emp.getHireDate())+"</td>\n" );
  pw.print( "<td>Projects: " );
  String[] prjs = emp.getProjectNames();
  for( int i = 0; i < prjs.length; i++ ) { 
    pw.print( "<i>"+prjs[i]+"</i> " );
  }
  pw.print( "</td></tr>\n" );
}

pw.print( "</table></div>\n" );

And that's pretty much it! All this code is put together in the MakeTable.java class, if you want to see it in action.

Dealing With Comments and Errors

company.csv


# This CSV file contains a list of employees and
# the names of the projects ( project 'red' or project 'blue') 
# they are assigned to.
# 
# Also demonstrated is the use of comments in a CSV file; lines
# beginnning with '#' are comments.
# (requires CsvSpec.setUseComment(true))
#
# And it also demonstrates CSV Manager's built-in fault
# tolerance; the third line is badly formatted (the quoted field 
# has an unescaped quote) and will be ignored.
# (requires CsvSpec.setIgnoreBadLines(true))
#
Employee Number, Name, Hired, Manager, Project Names
1, John Doe, 2006-01-01, no, red
2, Jane Doe, 2006-02-02, no, red
3, "Bad" Bob", 2006-03-03, no, red
4, Mr. Manager, 2006-04-04, yes, red:blue
5, Joe Bloggs, 2006-05-05, no, blue
6, Jane Bloggs, 2006-06-06, no, blue

First, we've added employee number 3, "Bad Bob", into the CSV data. Second, we've added a load of comments. Now let's sort out this mess.

Ok. We've got a syntax error. "Bad" Bob" is not a properly quoted data field. CSV expects quotes inside quoted data fields to be doubled up, like so: "Bad"" Bob". Normally, CSV Manager will halt as soon as this line is encountered and throw a CsvManagerException explaining the problem. But part of the reason for using CSV Manager is fault tolerance and error handling. So let's look at that.


csvspec.setIgnoreBadLines(true);

Now, when you run MakeTable, the invalid line will be ignored, and loading will continue as normal. Of course, you need to know about errors, so you can use this code to get them:


System.out.println( "Ignored bad lines were: "+csvman.getBadLines() );

The CsvManager.getBadLines method returns a List of BadLine objects. These contain all the details of the bad lines that caused problems. You can use them for data auditing or manual correction.

The other nice thing about CsvSpec.setIgnoreBadLines is that you get to keep sleeping at 4:00AM because your pager doesn't go off — instead of just falling over, the nightly batch CSV import just keeps on truckin' and you can sort out any problems in the morning...

What about all those pesky comments as well? They're easy to get rid of. Just use:


csvspec.setUseComment(true);
csvspec.setIgnoreEmptyLines(true);

The method CsvSpec.setUseComment turns comment support on. By default, '#' is the comment character. Any text following an unquoted '#' in a line is ignored. You can change the comment character using CsvSpec.setComment if your CSV file uses a different character for comment lines.

We also use CsvSpec.setIgnoreEmptyLines. Why? Well the commented text is ignored, but the lines are still there. They're just empty as far as CSV Manager is concerned (that's how comments work). Now maybe we should combine these two options, but we like to leave it up to you. You never know with CSV. It comes in all shapes and sizes.

All these extra options allow you to deal with "real world" CSV. There are lots more in the CsvSpec object.

Source Code

Here is a list of all the files used in this example. Note that the actual source code is slightly longer than the examples above, which have been abridged for clarity.

Feel free to experiment with these classes and see what happens. You can also use them as the basis for your own Java Beans solution.

Questions and Comments

Please feel free to email us at examples@ricebridge.com if you have any questions or comments about this example.

XML Manager

Making Java Beans From CSV Files

Summary

A Note on Compiling and Running

The Data

The Beans

Loading the Data

Creating the HTML Output

Dealing With Comments and Errors

Source Code

Questions and Comments

Tutorials

XML Manager

Questions

Quality

Bookmarks