Copyright 2023 Mark Rotteveel and all contributing authors under the Apache 2.0 license.

1. Introduction

The “Firebird External Table Generator” or ext-table-gen is a commandline tool to convert RFC 4180 CSV files and CSV files with custom formats to Firebird external tables (a binary format for Firebird external table data).

External tables are a good way for bulk-loading data into Firebird. Unfortunately, external tables use a fixed-width binary format[1], and not a (more) standard format like CSV. It is not always easy to create an appropriate external table file, and this is where ext-table-gen can be used.

At a high level, ext-table-gen provides the following features:

  • Derive a CHAR-based external table definition (i.e. a CREATE TABLE statement and a configuration for ext-table-gen) from a CSV file

  • Transform a CSV file to an external table file (either based on the CSV file itself, or based on a configuration file)

By default, ext-table-gen derives tables with only CHAR columns. This makes the generated file essentially a fixed-width text format. However, you can modify the configuration file to make ext-table-gen generate columns of different types, in which case the file becomes a binary format. ext-table-gen supports the following types of columns:

  • String types:

    • char

  • Integral number types:

    • smallint

    • integer

    • bigint

    • int128

  • Exact numeric types (fixed point):

    • numeric

    • decimal

  • Datetime types:

    • date

    • time (without time zone)

    • timestamp (without time zone)

2. About this version

Firebird External Table Generator version 2.0 is the second release of this application.

This version introduces two major new features:

  • Support for additional data types:

    • Integral number types:

      • smallint

      • integer

      • bigint

      • int128

    • Exact numeric types (fixed point):

      • numeric

      • decimal

    • Datetime types:

      • date

      • time (without time zone)

      • timestamp (without time zone)

  • Customizing the CSV parser configuration

This application is licensed under the Apache 2.0 license.

Consult the User Manual for installation, usage examples and reference documentation.

For more information, or to report issues or submit pull requests, go to https://github.com/mrotteveel/ext-table-gen

2.1. Version history

2.1.1. ext-table-gen 2.0

The following changes were made in ext-table-gen 2.0

  • Add support for integer types (SMALLINT, INTEGER, BIGINT, INT128) (#11)

  • Support custom conversions (#17)

  • Add DATE support (#8)

  • Add TIME and TIMESTAMP (without time zone) support (#9)

  • Add support for NUMERIC/DECIMAL types (#12)

  • Support custom CSV formats (#18)

  • Define module-info.java (#16)

  • The XML schema is now versioned. (#14)
    Where possible, backwards compatible changes are performed while retaining the same namespace. See 2023-05 XML Schema Versioning for details.

    • We introduced a minor breaking change to XSD, while not changing the namespace. This will result in validation errors when a config file from version 1.0 is validated against the XSD of version 2.0. We considered this change too minor to go through the trouble of changing the namespace.

      To address this breaking change, let ext-table-gen upgrade the file for you (this example uses the Windows Command Prompt continuation character ^):

      ext-table-gen --config-in=path-to-config.xml ^
  --config-out=path-to-config.xml ^
  --overwrite-config

      Alternatively, edit the XML to add the attribute schemaVersion with value 1.0 (i.e. schemaVersion="1.0") to the extTableGenConfig element. For example:

      XML from version 1.0
      <extTableGenConfig xmlns="https://www.lawinegevaar.nl/xsd/ext-table-gen-1.0.xsd">
    <!-- ... -->
</extTableGenConfig>
      Modified XML
      <extTableGenConfig xmlns="https://www.lawinegevaar.nl/xsd/ext-table-gen-1.0.xsd" schemaVersion="1.0">
    <!-- ... -->
</extTableGenConfig>

2.1.2. ext-table-gen 1.0

Initial release

1. The format is essentially the in-memory layout of rows which Firebird also uses internally