GitHub Perl Weekly Trending

glpi-project/glpi-agent

GLPI Agent

Summary

The GLPI Agent is a generic management agent. It can perform a certain number of tasks, according to its own execution plan, or on behalf of a GLPI server acting as a control point.

Description

This agent is based on a fork of FusionInventory agent and so works mainly like FusionInventory agent. It introduces new features and a new protocol to communicate directly with a GLPI server and its native inventory feature. Anyway it also keeps the compatibility with FusionInventory for GLPI plugin.

Download

Release: See our github releases for official win32, MacOSX & linux packages.
Development builds:
- nightly builds for last 'develop' branch commits: GLPI-Agent nightly builds
- with a github account, you can also access artifacts for any other branches supporting "GLPI Agent Packaging" workflow

Documentation

The GLPI Agent has its dedicated documentation project where any contribution will also be appreciated.

The documentation itself is readable online.

Dependencies

Core

Minimum perl version: 5.8

Mandatory Perl modules:

File::Which
LWP::UserAgent
Net::IP
Text::Template
UNIVERSAL::require
XML::LibXML
Cpanel::JSON::XS

Optional Perl modules:

Compress::Zlib, for message compression
HTTP::Daemon, for web interface
IO::Socket::SSL, for HTTPS support
LWP::Protocol::https, for HTTPS support
Proc::Daemon, for daemon mode (Unix only)
Proc::PID::File, for daemon mode (Unix only)

Inventory task

Optional Perl modules:

Net::CUPS, for printers detection
Parse::EDID, for EDID data parsing
DateTime, for reliable timezone name extraction

Optional programs:

dmidecode, for DMI data retrieval
lspci, for PCI bus scanning
hdparm, for additional disk drive info retrieval
monitor-get-edid-using-vbe, monitor-get-edid or get-edid, for EDID data access
ssh-keyscan, for host SSH public key retrieval

Network discovery tasks

Mandatory Perl modules:

Thread::Queue

Optional Perl modules:

Net::NBName, for NetBios method support
Net::SNMP, for SNMP method support

Optional programs:

arp, for arp table lookup method support

Network inventory tasks

Mandatory Perl modules:

Net::SNMP
Thread::Queue

Optional Perl modules:

Crypt::DES, for SNMPv3 support

Wake on LAN task

Optional Perl modules:

Net::Write::Layer2, for ethernet method support

Deploy task

Mandatory Perl modules:

Digest::SHA
File::Copy::Recursive
Cpanel::JSON::XS
URI::Escape

Mandatory Perl modules for P2P Support:

Net::Ping
Parallel::ForkManager

MSI Packaging

Tools:

dmidecode modified to be built with mingw32
hdparm
7zip

Mandatory Perl modules:

Perl::Dist::Strawberry

MacOSX Packaging

Tools:

dmidecode modified to be built on macosx
munkipkg
Xcode
productbuild
hdiutil

Public databases

Pci.ids
Usb.ids
SysObject.ids: sysobject.ids

Forking/Contributors

See CONTRIB to find references to GLPI Agent related scritps/files

We would like to ask you few things in the context of a fork:

Do not change/remove the LICENSE file
Do not change/remove any lines in the Changes file that do not relate to your own code. This is really important to keep the changelog coherent between forks and will facilitate your life if you need to follow our own changes.

Don't hesitate to add any entry in Changes or CONTRIB to describe the purpose of your changes.

Contacts

Project websites:

main site: https://glpi-project.org/
forum: https://forum.glpi-project.org/
github: http://github.com/glpi-project/glpi-agent

Project Telegram channel:

https://t.me/glpien

Please report any issues on project github issue tracker.

Active authors

Guillaume Bougard gbougard@teclib.com

License

This software is licensed under the terms of GPLv2+, see LICENSE file for details.

When forking, please, don't delete this file.

Additional pieces of software

The glpi-injector script is based on fusioninventory-injector script:

author: Pascal Danek
copyright: 2005 Pascal Danek

GLPI::Agent::Task::Inventory::Vmsystem contains code from imvirt:

url: http://micky.ibh.net/~liske/imvirt.html
author: Thomas Liske liske@ibh.de
copyright: 2008 IBH IT-Service GmbH http://www.ibh.de/
License: GPLv2+

exiftool/exiftool

ExifTool meta information reader/writer

ExifTool by Phil Harvey (philharvey66 at gmail.com)

ExifTool is a customizable set of Perl modules plus a full-featured command-line application for reading and writing meta information in a wide variety of files, including the maker note information of many digital cameras by various manufacturers such as Canon, Casio, DJI, FLIR, FujiFilm, GE, HP, JVC/Victor, Kodak, Leaf, Minolta/Konica-Minolta, Nikon, Nintendo, Olympus/Epson, Panasonic/Leica, Pentax/Asahi, Phase One, Reconyx, Ricoh, Samsung, Sanyo, Sigma/Foveon and Sony.

Below is a list of file types and meta information formats currently supported by ExifTool (r = read, w = write, c = create):

See html/index.html for more details about ExifTool features.

ExifTool can be downloaded from

https://exiftool.org/

RUNNING

The exiftool script can be run right away without the need to install Image::ExifTool. For example, from within the exiftool directory you can extract the information from one of the included test files by typing:

./exiftool t/images/ExifTool.jpg

If you move the exiftool script to a different directory, you must also either move the contents of the lib directory or install the Image::ExifTool package so the script can find the necessary libraries.

Note: If you are using the Windows cmd shell, you may need to rename 'exiftool' to 'exiftool.pl' to run it directly from the command line. Alternatively, you can run exiftool with the command 'perl exiftool'.

IF YOU ARE STILL CONFUSED

The exiftool script is a command line application. You run it by typing commands in a terminal window. The first step is to determine the name of the directory where you downloaded the ExifTool distribution package. Assuming, for example, you downloaded it to a folder called "Desktop" in your home directory, then you would type the following commands in a terminal window to extract and run ExifTool:

cd ~/Desktop gzip -dc Image-ExifTool-13.52.tar.gz | tar -xf - cd Image-ExifTool-13.52 ./exiftool t/images/ExifTool.jpg

Note: These commands extract meta information from one of the test images. To use one of your images instead, enter the full path name of your file in place of "t/images/ExifTool.jpg".

INSTALLATION

You can install the Image::ExifTool package to make it available for use by other Perl scripts by typing the following:

perl Makefile.PL make make test make install

Notes: i) You need root access for the last step above.

ii) Some Perl installations (like the standard OSX installation) may not contain the necessary files to complete the first step above. But no worries: You can install ExifTool manually by moving 'exiftool' and the 'lib' directory to any directory in your current PATH (ie. /usr/bin).

iii) In Windows, "dmake" or "nmake" may be used if "make" is not available.

(Also see html/install.html for more help with installation.)

DEPENDENCIES

Requires Perl version 5.004 or later. No other special libraries are required, however the following modules are recommended for decoding compressed and/or encrypted information from the indicated file types, and for calculating digest values and providing other features listed below:

Archive::Zip (ZIP, DOCX, PPTX, XLSX, ODP, ODS, ODT, EIP, iWork) Compress::Zlib (DNG, PNG, PDF, DCM, MIE and SWF files) Digest::MD5 (PDF files, IPTC information, and JPG Extended XMP) Digest::SHA (PDF with AES-256 encryption) IO::Compress::Bzip2 (RWZ files) Time::HiRes (for generating ProcessingTime tag) POSIX::strptime (for inverse date/time conversion) Time::Piece (alternative to POSIX::strptime) Unicode::LineBreak (for column alignment of alternate-language output) Win32::API (for proper handling of Windows file names/times) Win32::FindFile (for Windows Unicode directory support, app only) Win32API::File (for Windows Unicode file names and file times) Compress::Raw::Lzma (for reading encoded 7z files) IO::Compress::RawDeflate (for writing FLIF images) IO::Uncompress::RawInflate (for reading FLIF images) IO::Compress::Brotli (for writing compressed JXL metadata) IO::Uncompress::Brotli (for reading compressed JXL metadata)

This is free software; you can redistribute it and/or modify it under the same terms as Perl itself (either the Perl Artistic License or GPL).

DISTRIBUTION FILES

Below is a list of the files/directories included in the full ExifTool distribution package:

Changes - Revision history MANIFEST - Full list of distribution files META.json - Standard CPAN dependency file (JSON format) META.yml - Standard CPAN dependency file (YAML format) Makefile.PL - Makefile for installation README - This file arg_files/ - Argument files to convert metadata formats: exif2iptc.args - Arguments for converting EXIF to IPTC exif2xmp.args - Arguments for converting EXIF to XMP gps2xmp.args - Arguments for converting GPS to XMP iptc2exif.args - Arguments for converting IPTC to EXIF iptc2xmp.args - Arguments for converting IPTC to XMP iptcCore.args - Complete list of IPTC Core XMP tags pdf2xmp.args - Arguments for converting PDF to XMP xmp2exif.args - Arguments for converting XMP to EXIF xmp2gps.args - Arguments for converting XMP to GPS xmp2iptc.args - Arguments for converting XMP to IPTC xmp2pdf.args - Arguments for converting XMP to PDF build_geolocation - Build custom Geolocation database config_files/ - Sample ExifTool configuration files: acdsee.config - Definitions for writing ACDSee XMP regions age.config - Calculate Age of person in photo bibble.config - Definitions for writing Bibble XMP tags convert_regions.config - Convert between MWG, MP and IPTC regions cuepointlist.config - Extract cue points and labels as a list depthmap.config - Extract Google DepthMap images example.config - General example showing config features fotoware.config - Definitions for writing Fotoware XMP tags frameCount.config - Extract FrameCount from MP4 videos gps2utm.config - Generate UTM coordinate tags from GPS info guano.config - Decode individual tags from Guano metadata local_time.config - Determine local time from GPS information nksc.config - Decode tags in Nikon ViewNX NKSC files onone.config - Definitions for writing On1 XMP tags photoshop_paths.config - For extracting or copying Photoshop paths picasa_faces.config - Convert from Picasa to MWG/MP face regions pix4d.config - Definitions for writing Pix4D XMP tags rotate_regions.config - Rotate MWG and MP region tags tiff_version.config - Determine the version of a TIFF file time_zone.config - Calculate time zone from EXIF tags exiftool - The exiftool application (Perl script) fmt_files/ - Output formatting example files: gpx.fmt - Format file for creating GPX track gpx_wpt.fmt - Format file for creating GPX waypoints kml.fmt - Format file for creating KML placemarks kml_track.fmt - Format file for creating KML track html/ - HTML documentation html/TagNames/ - HTML tag name documentation lib/ - ExifTool Perl library modules perl-Image-ExifTool.spec - Red Hat Packaging Manager specification file t/ - Verification test code t/images/ - Verification test images

ADDITIONAL INFORMATION

Read the following files included in the full distribution for more information:

html/index.html - Main ExifTool documentation html/install.html - Installation instructions html/history.html - Revision history html/ExifTool.html - API documentation html/TagNames/index.html - Tag name documentation html/geotag.html - Geotag feature html/geolocation.html - Geolocation feature html/faq.html - Frequently asked questions html/filename.html - Renaming/moving files html/metafiles.html - Working with metadata sidecar files html/struct.html - Working with structured XMP information lib/Image/ExifTool/README - ExifTool library modules documentation

and if you have installed Image::ExifTool, you can also consult perldoc or the man pages:

perldoc exiftool perldoc Image::ExifTool perldoc Image::ExifTool::TagNames

man exiftool man Image::ExifTool man Image::ExifTool::TagNames

Note: If the man pages don't work, it is probably because your man path is not set to include the installed documentation. See "man man" for information about how to set the man path.

Perl/perl5

🐪 The Perl programming language

ABOUT PERL

Perl is a general-purpose programming language originally developed for text manipulation and now used for a wide range of tasks including system administration, web development, network programming, GUI development, and more.

The language is intended to be practical (easy to use, efficient, complete) rather than beautiful (tiny, elegant, minimal). Its major features are that it's easy to use, supports both procedural and object-oriented (OO) programming, has powerful built-in support for text processing, and has one of the world's most impressive collections of third-party modules.

For an introduction to the language's features, see pod/perlintro.pod.

For a discussion of the important changes in this release, see pod/perldelta.pod.

There are also many Perl books available, covering a wide variety of topics, from various publishers. See pod/perlbook.pod for more information.

INSTALLATION

If you're using a relatively modern operating system and want to install this version of Perl locally, run the following commands:

./Configure -des -Dprefix=$HOME/localperl
make test
make install

This will configure and compile perl for your platform, run the regression tests, and install perl in a subdirectory "localperl" of your home directory.

If you run into any trouble whatsoever or you need to install a customized version of Perl, you should read the detailed instructions in the "INSTALL" file that came with this distribution. Additionally, there are a number of "README" files with hints and tips about building and using Perl on a wide variety of platforms, some more common than others.

Once you have Perl installed, a wealth of documentation is available to you through the 'perldoc' tool. To get started, run this command:

perldoc perl

IF YOU RUN INTO TROUBLE

Perl is a large and complex system that's used for everything from knitting to rocket science. If you run into trouble, it's quite likely that someone else has already solved the problem you're facing. Once you've exhausted the documentation, please report bugs to us at the GitHub issue tracker at https://github.com/Perl/perl5/issues

While it was current when we made it available, Perl is constantly evolving and there may be a more recent version that fixes bugs you've run into or adds new features that you might find useful.

You can always find the latest version of perl on a CPAN (Comprehensive Perl Archive Network) site near you at https://www.cpan.org/src/

If you want to submit a simple patch to the perl source, see the "SUPER QUICK PATCH GUIDE" in pod/perlhack.pod.

Just a personal note: I want you to know that I create nice things like this because it pleases the Author of my story. If this bothers you, then your notion of Authorship needs some revision. But you can use perl anyway. :-)

The author.

LICENSING

This program is free software; you can redistribute it and/or modify it under the terms of either:

a. the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or

b. the "Artistic License" which comes with this Kit.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See either the GNU General Public License or the Artistic License for more details.

You should have received a copy of the Artistic License with this Kit, in the file named "Artistic". If not, I'll be glad to provide one.

You should also have received a copy of the GNU General Public License along with this program in the file named "Copying". If not, see https://www.gnu.org/licenses/.

For those of you that choose to use the GNU General Public License, my interpretation of the GNU General Public License is that no Perl script falls under the terms of the GPL unless you explicitly put said script under the terms of the GPL yourself. Furthermore, any object code linked with perl does not automatically fall under the terms of the GPL, provided such object code only adds definitions of subroutines and variables, and does not otherwise impair the resulting interpreter from executing any standard Perl script. I consider linking in C subroutines in this manner to be the moral equivalent of defining subroutines in the Perl language itself. You may sell such an object file as proprietary provided that you provide or offer to provide the Perl source, as specified by the GNU General Public License. (This is merely an alternate way of specifying input to the program.) You may also sell a binary produced by the dumping of a running Perl script that belongs to you, provided that you provide or offer to provide the Perl source as specified by the GPL. (The fact that a Perl interpreter and your code are in the same binary file is, in this case, a form of mere aggregation.) This is my interpretation of the GPL. If you still have concerns or difficulties understanding my intent, feel free to contact me. Of course, the Artistic License spells all this out for your protection, so you may prefer to use that.

darold/ora2pg

Ora2Pg is a free tool used to migrate an Oracle database to a PostgreSQL compatible schema. It connects your Oracle database, scan it automatically and extracts its structure or data, it then generates SQL scripts that you can load into PostgreSQL.

NAME Ora2Pg - Oracle to PostgreSQL database schema converter

DESCRIPTION Ora2Pg is a free tool used to migrate an Oracle database to a PostgreSQL compatible schema. It connects to your Oracle database, scans it automatically and extracts its structure or data, then generates SQL scripts that you can load into your PostgreSQL database.

Ora2Pg can be used for anything from reverse engineering an Oracle
database to huge enterprise database migration or simply replicating
some Oracle data into a PostgreSQL database. It is really easy to use
and doesn't require any Oracle database knowledge other than providing
the parameters needed to connect to the Oracle database.

FEATURES Ora2Pg consists of a Perl script (ora2pg) and a Perl module (Ora2Pg.pm). The only thing you have to modify is the configuration file ora2pg.conf by setting the DSN to the Oracle database and optionally the name of a schema. Once that's done, you just have to set the type of export you want: TABLE with constraints, VIEW, MVIEW, TABLESPACE, SEQUENCE, INDEXES, TRIGGER, GRANT, FUNCTION, PROCEDURE, PACKAGE, PARTITION, TYPE, INSERT or COPY, FDW, QUERY, KETTLE, SYNONYM.

By default, Ora2Pg exports to a file that you can load into PostgreSQL
with the psql client, but you can also import directly into a PostgreSQL
database by setting its DSN into the configuration file. With all
configuration options of ora2pg.conf, you have full control of what
should be exported and how.

Features included:

        - Export full database schema (tables, views, sequences, indexes), with
          unique, primary, foreign key and check constraints.
        - Export grants/privileges for users and groups.
        - Export range/list partitions and sub partitions.
        - Export a table selection (by specifying the table names).
        - Export Oracle schema to a PostgreSQL 8.4+ schema.
        - Export predefined functions, triggers, procedures, packages and
          package bodies.
        - Export full data or following a WHERE clause.
        - Full support of Oracle BLOB object as PG BYTEA.
        - Export Oracle views as PG tables.
        - Export Oracle user defined types.
        - Provide some basic automatic conversion of PLSQL code to PLPGSQL.
        - Works on any platform.
        - Export Oracle tables as foreign data wrapper tables.
        - Export materialized view.
        - Show a  report of an Oracle database content.
        - Migration cost assessment of an Oracle database.
        - Migration difficulty level assessment of an Oracle database.
        - Migration cost assessment of PL/SQL code from a file.
        - Migration cost assessment of Oracle SQL queries stored in a file.
        - Generate XML ktr files to be used with Pentaho Data Integrator (Kettle)
        - Export Oracle locator and spatial geometries into PostGIS.
        - Export DBLINK as Oracle FDW.
        - Export SYNONYMS as views.
        - Export DIRECTORY as external table or directory for external_file extension.
        - Dispatch a list of SQL orders over multiple PostgreSQL connections
        - Perform a diff between Oracle and PostgreSQL database for test purposes.
        - MySQL/MariaDB and Microsoft SQL Server migration.

Ora2Pg does its best to automatically convert your Oracle database to
PostgreSQL but there's still manual work to do. The Oracle specific
PL/SQL code generated for functions, procedures, packages and triggers
has to be reviewed to match the PostgreSQL syntax. You will find some
useful recommendations on porting Oracle PL/SQL code to PostgreSQL
PL/PGSQL at "Converting from other Databases to PostgreSQL", section:
Oracle (http://wiki.postgresql.org/wiki/Main_Page).

See http://ora2pg.darold.net/report.html for an HTML sample of an Oracle
database migration report.

INSTALLATION All Perl modules can always be found at CPAN (http://search.cpan.org/). Just type the full name of the module (ex: DBD::Oracle) into the search input box, it will bring you to the page for download.

Releases of Ora2Pg are published at SF.net
(https://sourceforge.net/projects/ora2pg/).

On Windows(TM) you should install Strawberry Perl
(http://strawberryperl.com/) and the OSes corresponding Oracle clients.
Since version 5.32, the Perl distribution includes pre-compiled driver
for DBD::Oracle and DBD::Pg.

Required The Oracle Instant Client or a full Oracle installation must be installed on the system. You can download the RPM from Oracle download center:

    rpm -ivh oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm
    rpm -ivh oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm
    rpm -ivh oracle-instantclient12.2-jdbc-12.2.0.1.0-1.x86_64.rpm
    rpm -ivh oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm

or simply download the corresponding ZIP archives from Oracle download
center and install them where you want, for example:
/opt/oracle/instantclient_12_2/

You also need a modern Perl distribution (Perl 5.10 or later). To
connect to a database and proceed with its migration, you need the DBI
Perl module > 1.614. To migrate an Oracle database, you need the
DBD::Oracle Perl module to be installed.

To install DBD::Oracle and have it working, you need to have the Oracle
client libraries installed and the ORACLE_HOME environment variable must
be defined.

If you plan to export a MySQL database, you need to install the Perl
module DBD::MySQL which requires that the MySQL client libraries are
installed.

If you plan to export a SQL Server database, you need to install the
Perl module DBD::ODBC which requires that the unixODBC package is
installed.

On some Perl distributions, you may need to install the Time::HiRes Perl
module.

If your distribution doesn't include these Perl modules, you can install
them using CPAN:

        perl -MCPAN -e 'install DBD::Oracle'
        perl -MCPAN -e 'install DBD::MySQL'
        perl -MCPAN -e 'install DBD::ODBC'
        perl -MCPAN -e 'install Time::HiRes'

otherwise, use the packages provided by your distribution.

Optional By default, Ora2Pg dumps exports to flat files. To load them into your PostgreSQL database, you need the PostgreSQL client (psql). If you don't have it on the host running Ora2Pg, you can always transfer these files to a host with the psql client installed. If you prefer to load exports 'on the fly', the Perl module DBD::Pg is required.

Ora2Pg allows you to dump all output in a compressed gzip file. To do
this, you need the Compress::Zlib Perl module or, if you prefer using
bzip2 compression, the program bzip2 must be available in your PATH.

If your distribution doesn't include these Perl modules, you can install
them using CPAN:

        perl -MCPAN -e 'install DBD::Pg'
        perl -MCPAN -e 'install Compress::Zlib'

otherwise, use the packages provided by your distribution.

Instruction for SQL Server For SQL Server, you need to install the unixodbc package and the Perl DBD::ODBC driver:

        sudo apt install unixodbc
        sudo apt install libdbd-odbc-perl

or

        sudo yum install unixodbc
        sudo yum install perl-DBD-ODBC
        sudo yum install perl-DBD-Pg

Then install the Microsoft ODBC Driver for SQL Server. Follow the
instructions for to your operating system from here:

        https://docs.microsoft.com/fr-fr/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16

Once done, set the following in the /etc/odbcinst.ini file by adjusting
the SQL Server ODBC driver version:

        [msodbcsql18]
        Description=Microsoft ODBC Driver 18 for SQL Server
        Driver=/opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.0.so.1.1
        UsageCount=1

See ORACLE_DSN to learn how to use the driver to connect to your MSSQL
database.

Installing Ora2Pg Like any other Perl Module, Ora2Pg can be installed with the following commands:

        tar xjf ora2pg-x.x.tar.bz2
        cd ora2pg-x.x/
        perl Makefile.PL
        make && make install

This will install Ora2Pg.pm into your site Perl repository, ora2pg into
/usr/local/bin/ and ora2pg.conf into /etc/ora2pg/.

On Windows(TM), you may use instead:

        perl Makefile.PL
        gmake && gmake install

This will install scripts and libraries into your Perl site installation
directory and the ora2pg.conf file as well as all documentation files
into C:\ora2pg\

To install ora2pg in a different directory than the default one, simply
use this command:

        perl Makefile.PL PREFIX=<your_install_dir>
        make && make install

then set PERL5LIB to the path to your installation directory before
using Ora2Pg.

        export PERL5LIB=<your_install_dir>
        ora2pg -c config/ora2pg.conf -t TABLE -b outdir/

Packaging If you want to build binary packages for your preferred Linux distribution, take a look at the packaging/ directory of the source tarball. It contains everything needed to build RPM, Slackware and Debian packages. See the README file in that directory.

Installing DBD::Oracle Ora2Pg needs the Perl module DBD::Oracle for connectivity to an Oracle database from Perl DBI. You can get DBD::Oracle from CPAN, a Perl module repository.

After setting ORACLE_HOME and LD_LIBRARY_PATH environment variables as
root user, install DBD::Oracle. Proceed as follows:

        export LD_LIBRARY_PATH=/usr/lib/oracle/12.2/client64/lib
        export ORACLE_HOME=/usr/lib/oracle/12.2/client64
        perl -MCPAN -e 'install DBD::Oracle'

If you are running for the first time, it will ask many questions; you
can keep defaults by pressing ENTER key, but you need to provide one
appropriate mirror site for CPAN to download the modules. Install
through CPAN manually if the above doesn't work:

        #perl -MCPAN -e shell
        cpan> get DBD::Oracle
        cpan> quit
        cd ~/.cpan/build/DBD-Oracle*
        export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client64/lib
        export ORACLE_HOME=/usr/lib/oracle/11.2/client64
        perl Makefile.PL
        make
        make install

Installing DBD::Oracle requires that the three Oracle packages:
instant-client, SDK and SQLplus are installed as well as the libaio1
library.

If you are using Instant Client from ZIP archives, the LD_LIBRARY_PATH
and ORACLE_HOME will be the same and must be set to the directory where
you have installed the files. For example:
/opt/oracle/instantclient_12_2/

CONFIGURATION Configuring Ora2Pg can be as simple as choosing the Oracle database to export and choosing the export type. This can be done in a minute.

By reading this documentation you will also be able to:

        - Select only certain tables and/or columns for export.
        - Rename some tables and/or columns during export.
        - Select data to export following a WHERE clause per table.
        - Delay database constraints during data loading.
        - Compress exported data to save disk space.
        - and much more.

The Oracle database migration is fully controlled through a single
configuration file named ora2pg.conf. The format of this file consists
of a directive name in upper case followed by a tab character and a
value. Comments are lines beginning with a #.

There's no specific order to place the configuration directives, they
are set at the time they are read in the configuration file.

For configuration directives that just take a single value, you can use
them multiple times in the configuration file but only the last
occurrence found in the file will be used. For configuration directives
that allow a list of values, you can use them multiple times, the values
will be appended to the list. If you use the IMPORT directive to load a
custom configuration file, directives defined in this file will be
stored from the place the IMPORT directive is found, so it is better to
put it at the end of the configuration file.

Values set in command line options will override values from the
configuration file.

Ora2Pg usage First of all be sure that libraries and binaries paths include the Oracle Instant Client installation:

        export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client64/lib
        export PATH="/usr/lib/oracle/11.2/client64/bin:$PATH"

By default, Ora2Pg will look for /etc/ora2pg/ora2pg.conf configuration
file. If the file exists, you can simply execute:

        /usr/local/bin/ora2pg

or under Windows(TM) run ora2pg.bat file, located in your Perl bin
directory. Windows(TM) users may also find a template configuration file
in C:\ora2pg

If you want to call another configuration file, just give the path as a
command line argument:

        /usr/local/bin/ora2pg -c /etc/ora2pg/new_ora2pg.conf

Here are all command line parameters available when using ora2pg:

Usage: ora2pg [-dhpqv --estimate_cost --dump_as_html] [--option value]

    -a | --allow str  : Comma separated list of objects to allow from export.
                        Can be used with SHOW_COLUMN too.
    -b | --basedir dir: Set the default output directory, where files
                        resulting from exports will be stored.
    -c | --conf file  : Set an alternate configuration file other than the
                        default /etc/ora2pg/ora2pg.conf.
    -C | --cdc_file file: File used to store/read SCN per table during export.
                        default: TABLES_SCN.log in the current directory. This
                        is the file written by the --cdc_ready option.
    -d | --debug      : Enable verbose output.
    -D | --data_type str : Allow custom type replacement at command line.
    -e | --exclude str: Comma separated list of objects to exclude from export.
                        Can be used with SHOW_COLUMN too.
    -h | --help       : Print this short help.
    -g | --grant_object type : Extract privilege from the given object type.
                        See possible values with GRANT_OBJECT configuration.
    -i | --input file : File containing Oracle PL/SQL code to convert with
                        no Oracle database connection initiated.
    -j | --jobs num   : Number of parallel process to send data to PostgreSQL.
    -J | --copies num : Number of parallel connections to extract data from Oracle.
    -l | --log file   : Set a log file. Default is stdout.
    -L | --limit num  : Number of tuples extracted from Oracle and stored in
                        memory before writing, default: 10000.
    -m | --mysql      : Export a MySQL database instead of an Oracle schema.
    -M | --mssql      : Export a Microsoft SQL Server database.
    -n | --namespace schema : Set the Oracle schema to extract from.
    -N | --pg_schema schema : Set PostgreSQL's search_path.
    -o | --out file   : Set the path to the output file where SQL will
                        be written. Default: output.sql in running directory.
    -O | --options    : Used to override any configuration parameter, it can
                        be used multiple time. Syntax: -O "PARAM_NAME=value"
    -p | --plsql      : Enable PLSQL to PLPGSQL code conversion.
    -P | --parallel num: Number of parallel tables to extract at the same time.
    -q | --quiet      : Disable progress bar.
    -r | --relative   : use \ir instead of \i in the psql scripts generated.
    -s | --source DSN : Allow to set the Oracle DBI datasource.
    -S | --scn    SCN : Allow to set the Oracle System Change Number (SCN) to
                        use to export data. It will be used in the WHERE clause
                        to get the data. It is used with action COPY or INSERT.
    -t | --type export: Set the export type. It will override the one
                        given in the configuration file (TYPE).
    -T | --temp_dir dir: Set a distinct temporary directory when two
                        or more ora2pg are run in parallel.
    -u | --user name  : Set the Oracle database connection user.
                        ORA2PG_USER environment variable can be used instead.
    -v | --version    : Show Ora2Pg Version and exit.
    -w | --password pwd : Set the password of the Oracle database user.
                        ORA2PG_PASSWD environment variable can be used instead.
    -W | --where clause : Set the WHERE clause to apply to the Oracle query to
                        retrieve data. Can be used multiple time.
    --forceowner      : Force ora2pg to set tables and sequences owner like in
                  Oracle database. If the value is set to a username this one
                  will be used as the objects owner. By default it's the user
                  used to connect to the Pg database that will be the owner.
    --nls_lang code: Set the Oracle NLS_LANG client encoding.
    --client_encoding code: Set the PostgreSQL client encoding.
    --view_as_table str: Comma separated list of views to export as table.
    --estimate_cost   : Activate the migration cost evaluation with SHOW_REPORT
    --cost_unit_value minutes: Number of minutes for a cost evaluation unit.
                  default: 5 minutes, corresponds to a migration conducted by a
                  PostgreSQL expert. Set it to 10 if this is your first migration.
   --dump_as_html     : Force ora2pg to dump report in HTML, used only with
                        SHOW_REPORT. Default is to dump report as simple text.
   --dump_as_csv      : As above but force ora2pg to dump report in CSV.
   --dump_as_json     : As above but force ora2pg to dump report in JSON.
   --dump_as_sheet    : Report migration assessment with one CSV line per database.
   --init_project name: Initialise a typical ora2pg project tree. Top directory
                           dump_as_* selected switches, suffixes
                           will be .html, .csv, .json.
   --init_project name: Initialise a typical ora2pg project tree. Top directory
                        will be created under project base dir.
   --project_base dir : Define the base dir for ora2pg project trees. Default
                        is current directory.
   --print_header     : Used with --dump_as_sheet to print the CSV header
                        especially for the first run of ora2pg.
   --human_days_limit num : Set the number of person-days limit where the migration
                        assessment level switch from B to C. Default is set to
                        5 person-days.
   --audit_user list  : Comma separated list of usernames to filter queries in
                        the DBA_AUDIT_TRAIL table. Used only with SHOW_REPORT
                        and QUERY export type.
   --pg_dsn DSN       : Set the datasource to PostgreSQL for direct import.
   --pg_user name     : Set the PostgreSQL user to use.
   --pg_pwd password  : Set the PostgreSQL password to use.
   --count_rows       : Force ora2pg to perform a real row count in TEST,
                        TEST_COUNT and SHOW_TABLE actions.
   --no_header        : Do not append Ora2Pg header to output file
   --oracle_speed     : Use to know at which speed Oracle is able to send
                        data. No data will be processed or written.
   --ora2pg_speed     : Use to know at which speed Ora2Pg is able to send
                        transformed data. Nothing will be written.
   --blob_to_lo       : export BLOB as large objects, can only be used with
                        action SHOW_COLUMN, TABLE and INSERT.
   --cdc_ready        : use current SCN per table to export data and register
                        them into a file named TABLES_SCN.log per default. It
                        can be changed using -C | --cdc_file.
   --lo_import        : use psql \lo_import command to import BLOB as large
                        object. Can be use to import data with COPY and import
                        large object manually in a second pass. It is recquired
                        for BLOB > 1GB. See documentation for more explanation.
   --mview_as_table str: Comma separated list of materialized views to export
                        as regular table.
   --drop_if_exists   : Drop the object before creation if it exists.
   --delete clause    : Set the DELETE clause to apply to the Oracle query to
                        be applied before importing data. Can be used multiple
                        time.
   --oracle_fdw_prefetch: Set the oracle_fdw prefetch value. Larger values
                        generally result in faster data transfer at the cost
                        of greater memory utilisation at the destination.

See full documentation at https://ora2pg.darold.net/ for more help or
see manpage with 'man ora2pg'.

ora2pg will return 0 on success, 1 on error. It will return 2 when a
child process has been interrupted and you've gotten the warning
message: "WARNING: an error occured during data export. Please check
what's happened." Most of the time this is an OOM issue, so first try
reducing DATA_LIMIT value.

For developers, it is possible to add your own custom option(s) in the
Perl script ora2pg as any configuration directive from ora2pg.conf can
be passed in lower case to the new Ora2Pg object instance. See ora2pg
code on how to add your own option.

Note that performance might be improved by updating stats on Oracle:

        BEGIN
        DBMS_STATS.GATHER_SCHEMA_STATS
        DBMS_STATS.GATHER_DATABASE_STATS 
        DBMS_STATS.GATHER_DICTIONARY_STATS
        END;

Generate a migration template The two options --project_base and --init_project indicate to ora2pg that it should create a project template with a work tree, a configuration file and a script to export all objects from the Oracle database. Here is a sample of the command usage:

        ora2pg --project_base /app/migration/ --init_project test_project
        Creating project test_project.
        /app/migration/test_project/
                schema/
                        dblinks/
                        directories/
                        functions/
                        grants/
                        mviews/
                        packages/
                        partitions/
                        procedures/
                        sequences/
                        synonyms/
                        tables/
                        tablespaces/
                        triggers/
                        types/
                        views/
                sources/
                        functions/
                        mviews/
                        packages/
                        partitions/
                        procedures/
                        triggers/
                        types/
                        views/
                data/
                config/
                reports/

        Generating generic configuration file
        Creating script export_schema.sh to automate all exports.
        Creating script import_all.sh to automate all imports.

It creates a generic config file where you just have to define the
Oracle database connection and a shell script called export_schema.sh.
The sources/ directory will contain the Oracle code, the schema/
directory will contain the code ported to PostgreSQL. The reports/
directory will contain the HTML and JSON reports with the migration cost
assessment.

If you want to use your own default config file, use the -c option to
give the path to that file. Rename it with .dist suffix if you want
ora2pg to apply the generic configuration values; otherwise, the
configuration file will be copied untouched.

Once you have set the connection to the Oracle Database you can execute
the script export_schema.sh that will export all object types from your
Oracle database and output DDL files into the schema's subdirectories.
At end of the export it will give you the command to export data later
when the import of the schema is done and verified.

You can choose to load the DDL files generated manually or use the
second script import_all.sh to import those files interactively. If this
kind of migration is not something common for you, it's recommended that
you use those scripts.

Oracle database connection There are 5 configuration directives to control the access to the Oracle database.

ORACLE_HOME
    Used to set the ORACLE_HOME environment variable for the Oracle
    libraries required by the DBD::Oracle Perl module.

ORACLE_DSN
    This directive is used to set the data source name in the standard
    DBI DSN form. For example:

            dbi:Oracle:host=oradb_host.myhost.com;sid=DB_SID;port=1521

    or

            dbi:Oracle:DB_SID

    On 18c this could be for example:

            dbi:Oracle:host=192.168.1.29;service_name=pdb1;port=1521

    For the second notation, the SID should be declared in the
    well-known file $ORACLE_HOME/network/admin/tnsnames.ora or in the
    path given to the TNS_ADMIN environment variable.

    For MySQL the DSN will look like this:

            dbi:mysql:host=192.168.1.10;database=sakila;port=3306

    The 'sid' part is replaced by 'database'.

    For MS SQL Server it will look like this:

            dbi:ODBC:driver=msodbcsql18;server=mydb.database.windows.net;database=testdb;TrustServerCertificate=yes

ORACLE_USER and ORACLE_PWD
    These two directives are used to define the user and password for
    the Oracle database connection. Note that if possible, it is better
    to login as Oracle super admin to avoid grant problems during the
    database scan and ensure nothing is missing.

    If you do not supply credentials with ORACLE_PWD and you have
    installed the Term::ReadKey Perl module, Ora2Pg will ask for the
    password interactively. If ORACLE_USER is not set it will also be
    asked interactively.

    To connect to a local ORACLE instance with connections "as sysdba"
    you have to set ORACLE_USER to "/" and an empty password.

    To make a connection using an Oracle Secure External Password Store
    (SEPS), first configure the Oracle Wallet and then set both the
    ORACLE_USER and ORACLE_PWD directives to the special value of
    "__SEPS__" (without the quotes but with the double underscore).

USER_GRANTS
    Set this directive to 1 if you connect to Oracle database as a
    simple user and do not have enough grants to extract things from the
    DBA_... tables. It will use ALL_... tables instead.

    Warning: if you use export type GRANT, you must set this
    configuration option to 0 or it will not work.

TRANSACTION
    This directive may be used if you want to change the default
    isolation level of the data export transaction. Default is now to
    set the level to a serializable transaction to ensure data
    consistency. The allowed values for this directive are:

            readonly: 'SET TRANSACTION READ ONLY',
            readwrite: 'SET TRANSACTION READ WRITE',
            serializable: 'SET TRANSACTION ISOLATION LEVEL SERIALIZABLE'
            committed: 'SET TRANSACTION ISOLATION LEVEL READ COMMITTED',

    Releases before 6.2 used to set the isolation level to READ ONLY
    transaction but in some cases this was breaking data consistency so
    now default is set to SERIALIZABLE.

INPUT_FILE
    This directive does not control the Oracle database connection but
    rather it purely disables the use of any Oracle database by
    accepting a file as argument. Set this directive to a file
    containing PL/SQL Oracle Code like function, procedure or full
    package body to prevent Ora2Pg from connecting to an Oracle database
    and just apply its conversion tool to the content of the file. This
    can be used with most export types: TABLE, TRIGGER, PROCEDURE, VIEW,
    FUNCTION or PACKAGE, etc.

ORA_INITIAL_COMMAND
    This directive can be used to send an initial command to Oracle,
    just after the connection. For example to unlock a policy before
    reading objects or to set some session parameters. This directive
    can be used multiple times.

Data encryption with Oracle server If your Oracle Client config file already includes the encryption method, then DBD::Oracle uses those settings to encrypt the connection while extracting data. For example, if you have configured the Oracle Client config file (sqlnet.ora or .sqlnet) with the following information:

        # Configure encryption of connections to Oracle
        SQLNET.ENCRYPTION_CLIENT = required
        SQLNET.ENCRYPTION_TYPES_CLIENT = (AES256, RC4_256)
        SQLNET.CRYPTO_SEED = 'should be 10-70 random characters'

Any tool that uses the Oracle client to communicate with the database
will have encrypted connections if you setup session encryption as shown
above.

For example, Perl's DBI uses DBD::Oracle, which uses the Oracle client
for actual database communication. If the Oracle client installation
used by Perl is setup to request encrypted connections, then your Perl
connection to an Oracle database will also be encrypted.

Full details at
https://kb.berkeley.edu/jivekb/entry.jspa?externalID=1005

Testing connection Once you have set the Oracle database DSN, you can execute ora2pg to see if it works:

        ora2pg -t SHOW_VERSION -c config/ora2pg.conf

will show the Oracle database server version. Take some time here to
test your installation as most problems occur here. The other
configuration steps are more technical.

Troubleshooting If the output.sql file hasn't exported anything other than the PostgreSQL transaction header and footer, there are two possible reasons: 1) The perl script ora2pg dumps an ORA-XXX error, which means that your DSN or login information is wrong - check the error and your settings and try again. 2) The perl script says nothing and the output file is empty: the user lacks permissions to extract something from the database. Try to connect to Oracle as super user or review the USER_GRANTS directive above and the next section, especially the SCHEMA directive.

LOGFILE
    By default, all messages are sent to the standard output. If you
    provide a file path to this directive, all output will be appended
    to this file.

Oracle schema to export The Oracle database export can be limited to a specific Schema or Namespace; this may be mandatory depending on the database connection user.

SCHEMA
    This directive is used to set the schema name to use during export.
    For example:

            SCHEMA  APPS

    will extract objects associated with the APPS schema.

    When no schema name is provided and EXPORT_SCHEMA is enabled, Ora2Pg
    will export all objects from all schemas of the Oracle instance with
    their names prefixed with the schema name.

EXPORT_SCHEMA
    By default, the Oracle schema is not exported into the PostgreSQL
    database and all objects are created under the default Pg namespace.
    If you want to also export this schema and create all objects under
    this namespace, set the EXPORT_SCHEMA directive to 1. This will set
    the schema search_path at the top of the export SQL file to the
    schema name set in the SCHEMA directive with the default pg_catalog
    schema. If you want to change this path, use the directive
    PG_SCHEMA.

CREATE_SCHEMA
    Enable/disable the CREATE SCHEMA SQL order at the start of the
    output file. It is enabled by default and concerns the TABLE export
    type.

COMPILE_SCHEMA
    By default, Ora2Pg will only export valid PL/SQL code. You can force
    Oracle to compile again the invalidated code to get a chance to have
    it obtain the valid status and then be able to export it.

    Enable this directive to force Oracle to compile schema before
    exporting code. When this directive is enabled and SCHEMA is set to
    a specific schema name, only invalid objects in this schema will be
    recompiled. If SCHEMA is not set then all schema will be recompiled.
    To force recompilation of invalid object in a specific schema, set
    COMPILE_SCHEMA to the schema name you want to recompile.

    This will ask Oracle to validate the PL/SQL that could have been
    invalidated after an export/import for example. The 'VALID' or
    'INVALID' status applies to functions, procedures, packages and user
    defined types. It also concerns disabled triggers.

EXPORT_INVALID
    If the above configuration directive is not enough to validate your
    PL/SQL code, enable this configuration directive to allow export of
    all PL/SQL code even if it is marked as invalid. The 'VALID' or
    'INVALID' status applies to functions, procedures, packages,
    triggers and user defined types.

PG_SCHEMA
    Allows you to define/force the PostgreSQL schema to use. By default,
    if you set EXPORT_SCHEMA to 1, the PostgreSQL search_path will be
    set to the schema name exported set as value of the SCHEMA
    directive.

    The value can be a comma-delimited list of schema names but not when
    using TABLE export type because in this case it will generate the
    CREATE SCHEMA statement and it doesn't support multiple schema
    names. For example, if you set PG_SCHEMA to something like
    "user_schema, public", the search path will be set like this:

            SET search_path = user_schema, public;

    forcing the use of an other schema (here user_schema) than the one
    from Oracle schema set in the SCHEMA directive.

    You can also set the default search_path for the PostgreSQL user you
    are using to connect to the destination database by using:

            ALTER ROLE username SET search_path TO user_schema, public;

    in this case you don't have to set PG_SCHEMA.

SYSUSERS
    Without explicit schema, Ora2Pg will export all objects that do not
    belong to system schemas or roles:

            SYSTEM,CTXSYS,DBSNMP,EXFSYS,LBACSYS,MDSYS,MGMT_VIEW,
            OLAPSYS,ORDDATA,OWBSYS,ORDPLUGINS,ORDSYS,OUTLN,
            SI_INFORMTN_SCHEMA,SYS,SYSMAN,WK_TEST,WKSYS,WKPROXY,
            WMSYS,XDB,APEX_PUBLIC_USER,DIP,FLOWS_020100,FLOWS_030000,
            FLOWS_040100,FLOWS_010600,FLOWS_FILES,MDDATA,ORACLE_OCM,
            SPATIAL_CSW_ADMIN_USR,SPATIAL_WFS_ADMIN_USR,XS$NULL,PERFSTAT,
            SQLTXPLAIN,DMSYS,TSMSYS,WKSYS,APEX_040000,APEX_040200,
            DVSYS,OJVMSYS,GSMADMIN_INTERNAL,APPQOSSYS,DVSYS,DVF,
            AUDSYS,APEX_030200,MGMT_VIEW,ODM,ODM_MTR,TRACESRV,MTMSYS,
            OWBSYS_AUDIT,WEBSYS,WK_PROXY,OSE$HTTP$ADMIN,
            AURORA$JIS$UTILITY$,AURORA$ORB$UNAUTHENTICATED,
            DBMS_PRIVILEGE_CAPTURE,CSMIG,MGDSYS,SDE,DBSFWUSER

    Depending on your Oracle installation, you may have several other
    system roles defined. To append these users to the schema exclusion
    list, just set the SYSUSERS configuration directive to a
    comma-separated list of system users to exclude. For example:

            SYSUSERS        INTERNAL,SYSDBA,BI,HR,IX,OE,PM,SH

    will add users INTERNAL and SYSDBA to the schema exclusion list.

FORCE_OWNER
    By default, the owner of the database objects is the one you're
    using to connect to PostgreSQL using the psql command. If you use an
    other user (postgres for example), you can force Ora2Pg to set the
    object owner to be the one used in the Oracle database by setting
    the directive to 1, or to a completely different username by setting
    the directive value to that username.

FORCE_SECURITY_INVOKER
    Ora2Pg uses the function's security privileges set in Oracle and it
    is often defined as SECURITY DEFINER. If you want to override those
    security privileges for all functions and use SECURITY DEFINER
    instead, enable this directive.

USE_TABLESPACE
    When enabled this directive forces ora2pg to export all tables and
    indexes using the tablespace name defined in Oracle database. This
    works only with tablespaces that are not TEMP, USERS or SYSTEM.

WITH_OID
    Activating this directive will force Ora2Pg to add WITH (OIDS) when
    creating tables or views as tables. Default is same as PostgreSQL,
    disabled.

LOOK_FORWARD_FUNCTION
    List of schemas to get functions/procedures meta information that
    are used in the current schema export. When replacing calls to
    functions with OUT parameters, if a function is declared in an other
    package, then the function call rewriting can not be done because
    Ora2Pg only knows about functions declared in the current schema. By
    setting a comma-separated list of schemas as value of this
    directive, Ora2Pg will look forward in these packages for all
    functions/procedures/packages declarations before proceeding to
    current schema export.

NO_FUNCTION_METADATA
    Forces Ora2Pg to not look for function declarations. Note that this
    will prevent Ora2Pg from rewriting function replacement calls if
    needed. Do not enable it unless looking forward at functions breaks
    other exports.

Export type The export action is performed following a single configuration directive 'TYPE', some others add more control over what should be exported.

TYPE
    Here are the different values of the TYPE directive, default is
    TABLE:

            - TABLE: Extract all tables with indexes, primary keys, unique keys,
              foreign keys and check constraints.
            - VIEW: Extract only views.
            - GRANT: Extract roles converted to Pg groups, users and grants on all
              objects.
            - SEQUENCE: Extract all sequences and their last positions.
            - TABLESPACE: Extract storage spaces for tables and indexes (Pg >= v8).
            - TRIGGER: Extract triggers defined on actions.
            - FUNCTION: Extract functions.
            - PROCEDURE: Extract procedures.
            - PACKAGE: Extract packages and package bodies.
            - INSERT: Extract data as INSERT statements.
            - COPY: Extract data as COPY statements.
            - PARTITION: Extract range and list Oracle partitions with subpartitions.
            - TYPE: Extract user defined Oracle types.
            - FDW: Export Oracle tables as foreign tables for Oracle, MySQL and SQL Server FDW.
            - MVIEW: Export materialized views.
            - QUERY: Try to automatically convert Oracle SQL queries.
            - KETTLE: Generate XML ktr template files for use by Kettle.
            - DBLINK: Generate Oracle foreign data wrapper server to use as dblink.
            - SYNONYM: Export Oracle's synonyms as views on other schema's objects.
            - DIRECTORY: Export Oracle's directories as external_file extension objects.
            - LOAD: Dispatch a list of queries over multiple PostgreSQL connections.
            - TEST: Perform a diff between Oracle and PostgreSQL databases.
            - TEST_COUNT: Perform a row count diff between Oracle and PostgreSQL tables.
            - TEST_VIEW: Perform a row count diff between Oracle and PostgreSQL views.
            - TEST_DATA: Perform data validation check on rows on both sides.
            - SEQUENCE_VALUES: Export DDL to set the last values of sequences

    Only one type of export can be performed at a time so the TYPE
    directive must be unique. If you have more than one only the last
    found in the file will be registered.

    Some export types cannot or should not be loaded directly into the
    PostgreSQL database and still require little manual editing. This is
    the case for GRANT, TABLESPACE, TRIGGER, FUNCTION, PROCEDURE, TYPE,
    QUERY and PACKAGE export types especially if you have PL/SQL code or
    Oracle specific SQL in them.

    For TABLESPACE you must ensure that file paths exist on the system
    and for SYNONYM you may need to ensure that the object's owners and
    schemas correspond to the new PostgreSQL database design.

    Note that you can chain multiple exports by giving to the TYPE
    directive a comma-separated list of export types, but in this case
    you must not use COPY or INSERT with other export types.

    Ora2Pg will convert Oracle partitions using table inheritance,
    triggers and functions. See documentation at PostgreSQL:
    http://www.postgresql.org/docs/current/interactive/ddl-partitioning.
    html

    The TYPE export allows export of user defined Oracle types. If you
    don't use the --plsql command line parameter it simply dumps Oracle
    user type as-is else Ora2Pg will try to convert it to PostgreSQL
    syntax.

    The KETTLE export type requires that the Oracle and PostgreSQL DNS
    are defined.

    Since Ora2Pg v8.1 there are three new export types:

            SHOW_VERSION : display Oracle version
            SHOW_SCHEMA  : display the list of schemas available in the database.
            SHOW_TABLE   : display the list of tables available.
            SHOW_COLUMN  : display the list of tables columns available and the
                    Ora2PG conversion type from Oracle to PostgreSQL that will be
                    applied. It will also warn you if there are PostgreSQL reserved
                    words in Oracle object names.

    Here is an example of the SHOW_COLUMN output:

            [2] TABLE CURRENT_SCHEMA (1 rows) (Warning: 'CURRENT_SCHEMA' is a reserved word in PostgreSQL)
                    CONSTRAINT : NUMBER(22) => bigint (Warning: 'CONSTRAINT' is a reserved word in PostgreSQL)
                    FREEZE : VARCHAR2(25) => varchar(25) (Warning: 'FREEZE' is a reserved word in PostgreSQL)
            ...
            [6] TABLE LOCATIONS (23 rows)
                    LOCATION_ID : NUMBER(4) => smallint
                    STREET_ADDRESS : VARCHAR2(40) => varchar(40)
                    POSTAL_CODE : VARCHAR2(12) => varchar(12)
                    CITY : VARCHAR2(30) => varchar(30)
                    STATE_PROVINCE : VARCHAR2(25) => varchar(25)
                    COUNTRY_ID : CHAR(2) => char(2)

    These extraction keywords are used to only display the requested
    information and exit. This allows you to quickly explore on what you
    are going to work with.

    The SHOW_COLUMN allows another ora2pg command line option: '--allow
    relname' or '-a relname' to limit the displayed information to the
    given table.

    The SHOW_ENCODING export type will display the NLS_LANG and
    CLIENT_ENCODING values that Ora2Pg will use and the real encoding of
    the Oracle database with the corresponding client encoding that
    could be used with PostgreSQL.

    Ora2Pg allows you to export your Oracle, MySQL or MSSQL table
    definitions to be used with the oracle_fdw, mysql_fdw or tds_fdw
    foreign data wrapper. By using type FDW your tables will be exported
    as follows:

            CREATE FOREIGN TABLE oratab (
                    id        integer           NOT NULL,
                    text      character varying(30),
                    floating  double precision  NOT NULL
            ) SERVER oradb OPTIONS (table 'ORATAB');

    Now you can use the table like a regular PostgreSQL table.

    Release 10 adds a new export type designed to evaluate the content
    of the database to migrate, in terms of objects and cost to complete
    the migration:

            SHOW_REPORT  : show a detailed report of the Oracle database content.

    Here is a sample report: http://ora2pg.darold.net/report.html

    There is also a more advanced report with migration cost. See the
    dedicated chapter about Migration Cost Evaluation.

ESTIMATE_COST
    Activate the migration cost evaluation. Must only be used with
    SHOW_REPORT, FUNCTION, PROCEDURE, PACKAGE and QUERY export types.
    Default is disabled. You may want to use the --estimate_cost command
    line option instead to activate this functionality. Note that
    enabling this directive will force PLSQL_PGSQL activation.

COST_UNIT_VALUE
    Sets the value in minutes of the migration cost evaluation unit.
    Default is five minutes per unit. See --cost_unit_value to change
    the unit value at command line.

DUMP_AS_HTML
    By default when using SHOW_REPORT the migration report is generated
    as simple text. Enabling this directive will force ora2pg to create
    a report in HTML format.

    See http://ora2pg.darold.net/report.html for a sample report.

DUMP_AS_JSON
    By default when using SHOW_REPORT the migration report is generated
    as simple text. Enabling this directive will force ora2pg to create
    a report in JSON format.

    See http://ora2pg.darold.net/report.html for a sample report.

DUMP_AS_CSV
    By default when using SHOW_REPORT the migration report is generated
    as simple text, enabling this directive will force ora2pg to create
    a report in CSV format.

    See http://ora2pg.darold.net/report.html for a sample report.

DUMP_AS_FILE_PREFIX
    By default when using SHOW_REPORT the migration report is generated
    to stout. Enabling this directive in conjunction with DUMP_AS_*
    directives will force ora2pg to create a report files with the given
    extensions and formats. This option allows you to combine multiple
    DUMP_AS_* formats.

    See http://ora2pg.darold.net/report.html for a sample report.

HUMAN_DAYS_LIMIT
    Use this directive to redefine the number of person-days limit where
    the migration assessment level must switch from B to C. Default is
    set to 10 person-days.

JOBS
    This configuration directive adds multiprocess support to COPY,
    FUNCTION and PROCEDURE export types. The value is the number of
    processes to use. Default is to disable multiprocessing.

    This directive is used to set the number of cores to use to
    parallelize data import into PostgreSQL. During FUNCTION or
    PROCEDURE export type each function will be translated to plpgsql
    using a new process. The performance gain can be very important when
    you have tons of functions to convert.

    There's no limitation in parallel processing other than the number
    of cores and the PostgreSQL I/O performance capabilities.

    Doesn't work under Windows Operating System, it is simply disabled.

ORACLE_COPIES
    This configuration directive adds multiprocess support to extract
    data from Oracle. The value is the number of processes to use to
    parallelize the select query. Default is parallel query disabled.

    The parallelism is built on splitting the query following the number
    of cores given as value to ORACLE_COPIES as follows:

            SELECT * FROM MYTABLE WHERE ABS(MOD(COLUMN, ORACLE_COPIES)) = CUR_PROC

    where COLUMN is a technical key like a primary or unique key where
    split will be based and the current core used by the query
    (CUR_PROC). You can also force the column name to use using the
    DEFINED_PK configuration directive.

    Doesn't work under Windows Operating System, it is simply disabled.

DEFINED_PK
    This directive is used to define the technical key to use to split
    the query between number of cores set with the ORACLE_COPIES
    variable. For example:

            DEFINED_PK      EMPLOYEES:employee_id

    The parallel query that will be used supposing that -J or
    ORACLE_COPIES is set to 8:

            SELECT * FROM EMPLOYEES WHERE ABS(MOD(employee_id, 8)) = N

    where N is the current process forked starting from 0.

PARALLEL_TABLES
    This directive is used to define the number of tables that will be
    processed in parallel for data extraction. The limit is the number
    of cores on your machine. Ora2Pg will open one database connection
    for each parallel table extraction. This directive, when higher than
    1, will invalidate ORACLE_COPIES but not JOBS, so the real number of
    processes that will be used is PARALLEL_TABLES * JOBS.

    Note that this directive when set higher than 1 will also
    automatically enable the FILE_PER_TABLE directive if you are
    exporting to files. This is used to export tables and views in
    separate files.

    Use PARALLEL_TABLES to use parallelism with COPY, INSERT and
    TEST_DATA actions. It is also useful with TEST, TEST_COUNT, and
    SHOW_TABLE if --count_rows is used for real row count.

DEFAULT_PARALLELISM_DEGREE
    You can force Ora2Pg to use /*+ PARALLEL(tbname, degree) */ hint in
    each query used to export data from Oracle by setting a value higher
    than 1 to this directive. A value of 0 or 1 disables the use of
    parallel hint. Default is disabled.

FDW_SERVER
    This directive is used to set the name of the foreign data server
    that is used in the "CREATE SERVER name FOREIGN DATA WRAPPER
    <fdw_extension> ..." command. This name will then be used in the
    "CREATE FOREIGN TABLE ..." SQL commands and to import data using
    oracle_fdw. By default, no foreign server is defined. This only
    concerns export types FDW, COPY and INSERT. For export type FDW, the
    default value is orcl.

FDW_IMPORT_SCHEMA
    Schema where foreign tables for data migration will be created. If
    you use several instances of ora2pg for data migration through the
    foreign data wrapper, you might need to change the name of the
    schema for each instance. Default: ora2pg_fdw_import

ORACLE_FDW_PREFETCH
    The default behavior of Ora2Pg is to NOT set the "prefetch" option
    for oracle_fdw when used for COPY and INSERT. This directive allows
    the prefetch to be set. See oracle_fdw documentation for the current
    default.

ORACLE_FDW_COPY_MODE
    When using Ora2Pg COPY with oracle_fdw, it is possible to use two
    different modes: 1) "local", which uses psql on the host running
    Ora2Pg for the "TO" binary stream; 2) "server", which uses
    PostgreSQL server-side COPY for the "TO" binary stream. Both modes
    use psql for the "FROM STDIN BINARY". However, "local" runs the psql
    "FROM STDIN BINARY" on the host Ora2Pg is run from, whereas "server"
    runs the psql "FROM STDIN BINARY" on the PostgreSQL server. "local"
    mode should work on any PostgreSQL-based system, including managed
    offerings, which are not expected to support use of "server" mode
    due to permissions. The default is "local" as this is compatible
    with more configurations.

ORACLE_FDW_COPY_FORMAT
    When using Ora2Pg COPY with oracle_fdw, it is possible to use either
    BINARY or CSV data format. BINARY provides better performance,
    however, requires exact data type matching between the FDW and
    destination table. CSV provides greater flexibility with respect to
    data type matching: if the FDW and destination data types are
    functionally-compatible, the columns can be copied. The default is
    "binary".

DROP_FOREIGN_SCHEMA
    By default, Ora2Pg drops the temporary schema ora2pg_fdw_import used
    to import the Oracle foreign schema before each new import. If you
    want to preserve the existing schema because of modifications or the
    use of a third-party server, disable this directive.

EXTERNAL_TO_FDW
    This directive, enabled by default, allows exporting Oracle's
    External Tables as file_fdw foreign tables. To not export these
    tables at all, set the directive to 0.

INTERNAL_DATE_MAX
    Internal timestamps retrieved from custom types are extracted in the
    following format: 01-JAN-77 12.00.00.000000 AM. It is impossible to
    know the exact century that must be used, so by default any year
    below 49 will be added to 2000 and others to 1900. You can use this
    directive to change the default value 49. This is only relevant if
    you have a user-defined type with a timestamp column.

AUDIT_USER
    Set the comma-separated list of usernames that must be used to
    filter queries from the DBA_AUDIT_TRAIL table. Default is to not
    scan this table and to never look for queries. This parameter is
    used only with SHOW_REPORT and QUERY export type with no input file
    for queries. Note that queries will be normalized before output
    unlike when a file is given at input using the -i option or INPUT
    directive.

FUNCTION_CHECK
    Disable this directive if you want to disable check_function_bodies.

            SET check_function_bodies = false;

    It disables validation of the function body string during CREATE
    FUNCTION. Default is to use the postgresql.conf setting, which
    enables it by default.

ENABLE_BLOB_EXPORT
    Exporting BLOBs takes time; in some circumstances you may want to
    export all data except the BLOB columns. In this case, disable this
    directive and the BLOB columns will not be included into data
    export. Take care that the target bytea column does not have a NOT
    NULL constraint.

ENABLE_CLOB_EXPORT
    Same behavior as ENABLE_BLOB_EXPORT but for CLOB.

DATA_EXPORT_ORDER
    By default, data export order will be done by sorting on table name.
    If you have huge tables at the end of alphabetic order and you are
    using multiprocess, it can be better to set the sort order on size
    so that multiple small tables can be processed before the largest
    tables finish. In this case set this directive to size. Possible
    values are name and size. Note that export types SHOW_TABLE and
    SHOW_COLUMN will use this sort order too, not only COPY or INSERT
    export type. If you want to give your custom export order, just give
    a filename as value that contains the ordered list tables to export.
    Must be a list of one table per line, in uppercase for Oracle.

Limiting objects to export You may want to export only a part of an Oracle database. Here is a set of configuration directives that will allow you to control which parts of the database should be exported.

ALLOW
    This directive allows you to set a list of objects to which the
    export must be limited, excluding all other objects of the same type
    of export. The value is a space or comma-separated list of object
    names to export. You can include valid regex into the list. For
    example:

            ALLOW           EMPLOYEES SALE_.* COUNTRIES .*_GEOM_SEQ

    will export objects with names EMPLOYEES, COUNTRIES, all objects
    beginning with 'SALE_' and all objects with a name ending in
    '_GEOM_SEQ'. The object depends of the export type. Note that regex
    will not work with 8i database, you must use the % placeholder
    instead, Ora2Pg will use the LIKE operator.

    This is the way to declare global filters that will be used with the
    current export type. You can also use extended filters that will be
    applied to specific objects or only on their related export type.
    For example:

            ora2pg -p -c ora2pg.conf -t TRIGGER -a 'TABLE[employees]'

    will limit export of triggers to those defined on table employees.
    If you want to extract all triggers but not some INSTEAD OF
    triggers:

            ora2pg -c ora2pg.conf -t TRIGGER -e 'VIEW[trg_view_.*]'

    Or a more complex form:

            ora2pg -p -c ora2pg.conf -t TABLE -a 'TABLE[EMPLOYEES]' \
                    -e 'INDEX[emp_.*];CKEY[emp_salary_min]'

    This command will export the definition of the employee table but
    will exclude all indexes beginning with 'emp_' and the CHECK
    constraint called 'emp_salary_min'.

    When exporting partitions you can exclude some partition tables by
    using

            ora2pg -p -c ora2pg.conf -t PARTITION -e 'PARTITION[PART_199.* PART_198.*]'

    This will exclude partitioned tables for years 1980 to 1999 from the
    export but not the main partition table. The trigger will also be
    adapted to exclude those tables.

    With GRANT export you can use this extended form to exclude some
    users from the export or limit the export to some others:

            ora2pg -p -c ora2pg.conf -t GRANT -a 'USER1 USER2'

    or

            ora2pg -p -c ora2pg.conf -t GRANT -a 'GRANT[USER1 USER2]'

    will limit export grants to users USER1 and USER2. But if you don't
    want to export grants on some functions for these users, for
    example:

            ora2pg -p -c ora2pg.conf -t GRANT -a 'USER1 USER2' -e 'FUNCTION[adm_.*];PROCEDURE[adm_.*]'

    Advanced filters may need some learning.

    Oracle doesn't allow the use of lookahead expressions so you may
    want to exclude some objects that match the ALLOW regexp you have
    defined. For example if you want to export all tables starting with
    E but not those starting with EXP it is not possible to do that in a
    single expression. This is why you can start a regular expression
    with the ! character to exclude objects matching the regexp given
    just after. Our previous example can be written as follows:

            ALLOW   E.* !EXP.*

    it will be translated into:

             REGEXP_LIKE(..., '^E.*$') AND NOT REGEXP_LIKE(..., '^EXP.*$')

    in the object search expression.

EXCLUDE
    This directive is the opposite of the previous. It allows you to
    define a space or comma-separated list of object names to exclude
    from the export. You can include valid regex in the list. For
    example:

            EXCLUDE         EMPLOYEES TMP_.* COUNTRIES

    will exclude objects with names EMPLOYEES, COUNTRIES and all tables
    beginning with 'tmp_'.

    For example, you can ban some unwanted functions from export with
    this directive:

            EXCLUDE         write_to_.* send_mail_.*

    This example will exclude all functions, procedures or functions in
    a package with names beginning with those regex. Note that regex
    will not work with 8i database, you must use the % placeholder
    instead, Ora2Pg will use the NOT LIKE operator.

    See above (directive 'ALLOW') for the extended syntax.

NO_EXCLUDED_TABLE
    By default, Ora2Pg excludes from export some Oracle "garbage" tables
    from export that should never be part of an export. This behavior
    generates a lot of REGEXP_LIKE expressions which slow down the
    export when looking at tables. To disable this behavior enable this
    directive, you will have to exclude or clean up later by yourself
    the unwanted tables. The regexps used to exclude the tables are
    defined in the array @EXCLUDED_TABLES in lib/Ora2Pg.pm. Note this
    behavior is independent of the EXCLUDE configuration directive.

VIEW_AS_TABLE
    Set which views to export as tables. By default none. Value must be
    a list of view names or regexps separated by space or comma. If the
    object name is a view and the export type is TABLE, the view will be
    exported as a create table statement. If export type is COPY or
    INSERT, the corresponding data will be exported.

    See chapter "Exporting views as PostgreSQL table" for more details.

MVIEW_AS_TABLE
    Set which materialized views to export as tables. By default none.
    Value must be a list of materialized view names or regexps separated
    by space or comma. If the object name is a materialized view and the
    export type is TABLE, the view will be exported as a create table
    statement. If export type is COPY or INSERT, the corresponding data
    will be exported.

NO_VIEW_ORDERING
    By default, Ora2Pg tries to order views to avoid errors at import
    time with nested views. With a huge number of views this can take a
    very long time, you can bypass this ordering by enabling this
    directive.

GRANT_OBJECT
    When exporting GRANTs you can specify a comma separated list of
    objects for which privileges will be exported. Default is export for
    all objects. Here are the possible values: TABLE, VIEW, MATERIALIZED
    VIEW, SEQUENCE, PROCEDURE, FUNCTION, PACKAGE BODY, TYPE, SYNONYM,
    DIRECTORY. Only one object type is allowed at a time. For example
    set it to TABLE if you just want to export privileges on tables. You
    can use the -g option to overwrite it.

    When used this directive prevents the export of users unless it is
    set to USER. In this case only user definitions are exported.

WHERE
    This directive allows you to specify a WHERE clause filter when
    dumping the contents of tables. The value is constructed as follows:
    TABLE_NAME[WHERE_CLAUSE], or if you have only one where clause for
    all tables just put the where clause as the value. Both are possible
    too. Here are some examples:

            # Global where clause applying to all tables included in the export
            WHERE  1=1

            # Apply the where clause only on table TABLE_NAME
            WHERE  TABLE_NAME[ID1='001']

            # Applies two different clauses on tables TABLE_NAME and OTHER_TABLE
            # and a generic where clause on DATE_CREATE to all other tables
            WHERE  TABLE_NAME[ID1='001' OR ID1='002] DATE_CREATE > '2001-01-01' OTHER_TABLE[NAME='test']

    Any WHERE clause not included in a table name bracket clause will be
    applied to all exported tables including the tables defined in the
    WHERE clause. These WHERE clauses are very useful if you want to
    archive some data or only export some recent data.

    To be able to quickly test data import it is useful to limit data
    export to the first thousand tuples of each table. For Oracle define
    the following clause:

            WHERE   ROWNUM < 1000

    and for MySQL, use the following:

            WHERE   1=1 LIMIT 1,1000

    This can also be restricted to some tables' data export.

    Command line option -W or --where will override this directive for
    the global part and per table if the table names are the same.

TOP_MAX
    This directive is used to limit the number of items shown in the top
    N lists like the top list of tables per number of rows and the top
    list of largest tables in megabytes. By default it is set to 10
    items.

LOG_ON_ERROR
    Enable this directive if you want to continue direct data import on
    error. When Ora2Pg receives an error in the COPY or INSERT statement
    from PostgreSQL it will log the statement to a file called
    TABLENAME_error.log in the output directory and continue to next
    bulk of data. Like this you can try to fix the statement and
    manually reload the error log file. Default is disabled: abort
    import on error.

REPLACE_QUERY
    Sometimes you may want to extract data from an Oracle table but you
    need a custom query for that. Not just a "SELECT * FROM table" like
    Ora2Pg does but a more complex query. This directive allows you to
    overwrite the query used by Ora2Pg to extract data. The format is
    TABLENAME[SQL_QUERY]. If you have multiple tables to extract by
    replacing the Ora2Pg query, you can define multiple REPLACE_QUERY
    lines.

            REPLACE_QUERY   EMPLOYEES[SELECT e.id,e.firstname,lastname FROM EMPLOYEES e JOIN EMP_UPDT u ON (e.id=u.id AND u.cdate>'2014-08-01 00:00:00')]

Control of Full Text Search export Several directives can be used to control how Ora2Pg exports the Oracle's Text search indexes. By default, CONTEXT indexes will be exported to PostgreSQL FTS indexes, while CTXCAT indexes will be exported as indexes using the pg_trgm extension.

CONTEXT_AS_TRGM
    Forces Ora2Pg to translate Oracle Text indexes into PostgreSQL
    indexes using the pg_trgm extension. By default, CONTEXT indexes are
    translated into FTS indexes and CTXCAT indexes use pg_trgm. Most of
    the time using pg_trgm is sufficient, which is why this directive
    exists. You need to create the pg_trgm extension in the destination
    database before importing the objects:

            CREATE EXTENSION pg_trgm;

FTS_INDEX_ONLY
    By default, Ora2Pg creates a function-based index to translate
    Oracle Text indexes:

            CREATE INDEX ON t_document
                    USING gin(to_tsvector('pg_catalog.french', title));

    You will have to rewrite the CONTAINS() clause using to_tsvector(),
    for example:

            SELECT id,title FROM t_document
                    WHERE to_tsvector(title) @@ to_tsquery('search_word');

    To force Ora2Pg to create an extra tsvector column with dedicated
    triggers for FTS indexes, disable this directive. In this case,
    Ora2Pg will add the column as follows: ALTER TABLE t_document ADD
    COLUMN tsv_title tsvector; Then update the column to compute FTS
    vectors if data have been loaded before: UPDATE t_document SET
    tsv_title = to_tsvector('pg_catalog.french', coalesce(title,'')); To
    automatically update the column when a modification in the title
    column occurs, Ora2Pg adds the following trigger:

            CREATE FUNCTION tsv_t_document_title() RETURNS trigger AS $$
            BEGIN
                   IF TG_OP = 'INSERT' OR new.title != old.title THEN
                           new.tsv_title :=
                           to_tsvector('pg_catalog.french', coalesce(new.title,''));
                   END IF;
                   return new;
            END
            $$ LANGUAGE plpgsql;
            CREATE TRIGGER trig_tsv_t_document_title BEFORE INSERT OR UPDATE
             ON t_document
             FOR EACH ROW EXECUTE PROCEDURE tsv_t_document_title();

    When the Oracle text index is defined over multiple columns, Ora2Pg
    will use setweight() to set weights in the order of the column
    declarations.

FTS_CONFIG
    Use this directive to force which text search configuration to use.
    When it is not set, Ora2Pg will autodetect the stemmer used by
    Oracle for each index and use pg_catalog.english if the information
    is not found.

USE_UNACCENT
    If you want to perform text searches in an accent-insensitive way,
    enable this directive. Ora2Pg will create a helper function using
    unaccent() and create the pg_trgm indexes using this function. With
    FTS, Ora2Pg will redefine your text search configuration, for
    example:

          CREATE TEXT SEARCH CONFIGURATION fr (COPY = french); 
          ALTER TEXT SEARCH CONFIGURATION fr
                  ALTER MAPPING FOR hword, hword_part, word WITH unaccent, french_stem;

    then set the FTS_CONFIG ora2pg.conf directive to fr instead of
    pg_catalog.english.

    When enabled, Ora2pg will create the wrapper function:

          CREATE OR REPLACE FUNCTION unaccent_immutable(text)
          RETURNS text AS
          $$
              SELECT public.unaccent('public.unaccent', $1);
          $$ LANGUAGE sql IMMUTABLE
             COST 1;

    The indexes are exported as follows:

          CREATE INDEX t_document_title_unaccent_trgm_idx ON t_document 
              USING gin (unaccent_immutable(title) gin_trgm_ops);

    In your queries, you will need to use the same function in the
    search to be able to use the function-based index. Example:

            SELECT * FROM t_document
                    WHERE unaccent_immutable(title) LIKE '%donnees%';

USE_LOWER_UNACCENT
    Same as above but calls lower() in the unaccent_immutable()
    function:

          CREATE OR REPLACE FUNCTION unaccent_immutable(text)
          RETURNS text AS
          $$
              SELECT lower(public.unaccent('public.unaccent', $1));
          $$ LANGUAGE sql IMMUTABLE;

Modifying object structure One of the great uses of Ora2Pg is its flexibility to replicate an Oracle database into a PostgreSQL database with a different structure or schema. There are three configuration directives that allow you to map these differences.

REORDERING_COLUMNS
    Enable this directive to reorder columns and minimize the footprint
    on disk, so that more rows fit on a data page, which is the most
    important factor for speed. Default is disabled, meaning the same
    order as in Oracle tables definition, which should be enough for
    most uses. This directive is only used with TABLE export.

MODIFY_STRUCT
    This directive allows you to limit the columns to extract for a
    given table. The value consists of a space-separated list of table
    names with a set of columns between parentheses as follows:

            MODIFY_STRUCT   TABLE_NAME(colname1,colname2,...) ...

    for example:

            MODIFY_STRUCT   T_TEST1(id,dossier) T_TEST2(id,fichier)

    This will only extract columns 'id' and 'dossier' from table T_TEST1
    and columns 'id' and 'fichier' from the T_TEST2 table. This
    directive can only be used with TABLE, COPY or INSERT export. With
    TABLE export create table DDL will respect the new list of columns
    and all indexes or foreign keys pointing to or from a removed column
    will not be exported.

EXCLUDE_COLUMNS
    Instead of redefining the table structure with MODIFY_STRUCT you may
    want to exclude some columns from the table export. The value
    consists of a space-separated list of table names with a set of
    column between parentheses as follows:

            EXCLUDE_COLUMNS TABLE_NAME(colname1,colname2,...) ...

    for example:

            EXCLUDE_COLUMNS T_TEST1(id,dossier) T_TEST2(id,fichier)

    This will exclude columns 'id' and 'dossier' from table T_TEST1 and
    columns 'id' and 'fichier' from the T_TEST2 table from the export.
    This directive can only be used with TABLE, COPY or INSERT export.
    With TABLE export create table DDL will respect the new list of
    columns and all indexes or foreign keys pointing to or from a
    removed column will not be exported.

REPLACE_TABLES
    This directive allows you to remap a list of Oracle table names to a
    PostgreSQL table names during export. The value is a list of
    space-separated values with the following structure:

            REPLACE_TABLES  ORIG_TBNAME1:DEST_TBNAME1 ORIG_TBNAME2:DEST_TBNAME2

    Oracle tables ORIG_TBNAME1 and ORIG_TBNAME2 will be respectively
    renamed to DEST_TBNAME1 and DEST_TBNAME2

REPLACE_COLS
    Like table names, column names can be remapped to different names
    using the following syntax:

            REPLACE_COLS    ORIG_TBNAME(ORIG_COLNAME1:NEW_COLNAME1,ORIG_COLNAME2:NEW_COLNAME2)

    For example:

            REPLACE_COLS    T_TEST(dico:dictionary,dossier:folder)

    will rename Oracle columns 'dico' and 'dossier' from table T_TEST to
    new names 'dictionary' and 'folder'.

REPLACE_AS_BOOLEAN
    If you want to change the type of some Oracle columns to PostgreSQL
    boolean during the export you can define here a list of tables and
    columns separated by spaces as follows.

            REPLACE_AS_BOOLEAN     TB_NAME1:COL_NAME1 TB_NAME1:COL_NAME2 TB_NAME2:COL_NAME2

    The values set in the boolean columns list will be replaced with 't'
    and 'f' following the default replacement values and those
    additionally set in directive BOOLEAN_VALUES.

    Note that if you have modified the table name with REPLACE_TABLES
    and/or the column's name, you need to use the name of the original
    table and/or column.

            REPLACE_COLS            TB_NAME1(OLD_COL_NAME1:NEW_COL_NAME1)
            REPLACE_AS_BOOLEAN      TB_NAME1:OLD_COL_NAME1

    You can also give a type and precision to automatically convert all
    fields of that type to boolean. For example:

            REPLACE_AS_BOOLEAN      NUMBER:1 CHAR:1 TB_NAME1:COL_NAME1 TB_NAME1:COL_NAME2

    will also replace any field of type number(1) or char(1) as a
    boolean in all exported tables.

BOOLEAN_VALUES
    Use this to add additional definitions of the possible boolean
    values used in Oracle fields. You must set a space-separated list of
    TRUE:FALSE values. By default, here are the values recognized by
    Ora2Pg:

            BOOLEAN_VALUES          yes:no y:n 1:0 true:false enabled:disabled

    Any values defined here will be added to the default list.

REPLACE_ZERO_DATE
    When Ora2Pg finds a "zero" date: 0000-00-00 00:00:00 it is replaced
    by a NULL. This could be a problem if your column is defined with a
    NOT NULL constraint. If you can not remove the constraint, use this
    directive to set an arbitrary date that will be used instead. You
    can also use -INFINITY if you don't want to use a fake date.

INDEXES_SUFFIX
    Add the given value as suffix to index names. Useful if you have
    indexes with the same name as tables. For example:

            INDEXES_SUFFIX          _idx

    will add _idx at the end of all index names. Not very common but
    helpful.

INDEXES_RENAMING
    Enable this directive to rename all indexes using
    tablename_columns_names. Could be very useful for databases that
    have multiple instances of the same index name or that use the same
    name as a table, which is not allowed by PostgreSQL. Disabled by
    default.

USE_INDEX_OPCLASS
    Operator classes text_pattern_ops, varchar_pattern_ops, and
    bpchar_pattern_ops support B-tree indexes on the corresponding
    types. The difference from the default operator classes is that the
    values are compared strictly character by character rather than
    according to locale-specific collation rules. This makes these
    operator classes suitable for use by queries involving pattern
    matching expressions (LIKE or POSIX regular expressions) when the
    database does not use the standard "C" locale. If enabled with value
    1, this will force Ora2Pg to export all indexes defined on
    varchar2() and char() columns using those operators. If you set it
    to a value greater than 1, it will only change indexes on columns
    where the character limit is greater than or equal to this value.
    For example, set it to 128 to create these kinds of indexes on
    columns of type varchar2(N) where N >= 128.

RENAME_PARTITION
    Enable this directive if you want your partition tables to be
    renamed. Disabled by default. If you have multiple partitioned
    tables, when exported to PostgreSQL some partitions could have the
    same name but different parent tables. This is not allowed - table
    names must be unique. In this case, enable this directive. A
    partition will be renamed following the rule: "tablename"_part"pos"
    where "pos" is the partition number. For subpartition this is:
    "tablename"_part"pos"_subpart"pos" If this is partition/subpartition
    default: "tablename"_part_default
    "tablename"_part"pos"_subpart_default

DISABLE_PARTITION
    If you don't want to reproduce the partitioning like in Oracle and
    want to export all partitioned Oracle data into the main single
    table in PostgreSQL, enable this directive. Ora2Pg will export all
    data into the main table name. Default is to use partitioning -
    Ora2Pg will export data from each partition and import them into the
    PostgreSQL dedicated partition table.

PARTITION_BY_REFERENCE
    How to export partition by reference. Possible values are none,
    duplicate or the number of hash partitions to create. Default is
    none to not export the partitions by reference.

    Value 'none' means no translation and export of partition by
    reference like before. Value 'duplicate' will duplicate the
    referenced column in the partitioned table and apply the same
    partitioning from the referenced table to the partitioned table. If
    the value is a number, the table will be partitioned with the HASH
    method using the value as the modulo. For example, if you set it to
    4 it will create 4 HASH partitions.

DISABLE_UNLOGGED
    By default, Ora2Pg exports Oracle tables with the NOLOGGING
    attribute as UNLOGGED tables. You may want to fully disable this
    feature because you will lose all data from unlogged tables in case
    of a PostgreSQL crash. Set it to 1 to export all tables as normal
    tables.

DOUBLE_MAX_VARCHAR
    Increase varchar max character constraints to support PostgreSQL two
    bytes character encoding when the source database applies the length
    constraint on characters not bytes. Default disabled.

Oracle Spatial to PostGIS Ora2Pg fully exports Spatial objects from Oracle database. There are some configuration directives that can be used to control the export.

AUTODETECT_SPATIAL_TYPE
    By default, Ora2Pg looks at indexes to see the spatial constraint
    type and dimensions defined under Oracle. Those constraints are
    passed at index creation using for example:

            CREATE INDEX ... INDEXTYPE IS MDSYS.SPATIAL_INDEX
            PARAMETERS('sdo_indx_dims=2, layer_gtype=point');

    If those Oracle constraint parameters are not set, the default is to
    export those columns as generic type GEOMETRY to be able to receive
    any spatial type.

    The AUTODETECT_SPATIAL_TYPE directive allows Ora2Pg to autodetect
    the real spatial type and dimension used in a spatial column;
    otherwise a non- constrained "geometry" type is used. Enabling this
    feature will force Ora2Pg to scan a sample of 50,000 columns to look
    at the GTYPE used. You can increase or reduce the sample size by
    setting the value of AUTODETECT_SPATIAL_TYPE to the desired number
    of lines to scan. The directive is enabled by default.

    For example, in the case of a column named shape and defined with
    Oracle type SDO_GEOMETRY, with AUTODETECT_SPATIAL_TYPE disabled it
    will be converted as:

        shape geometry(GEOMETRY) or shape geometry(GEOMETRYZ, 4326)

    and if the directive is enabled and the column just contains a
    single geometry type that uses a single dimension:

        shape geometry(POLYGON, 4326) or shape geometry(POLYGONZ, 4326)

    with a two or three dimensional polygon.

CONVERT_SRID
    This directive allows you to control the automatic conversion of
    Oracle SRID to standard EPSG. If enabled, Ora2Pg will use the Oracle
    function sdo_cs.map_oracle_srid_to_epsg() to convert all SRIDs.
    Enabled by default.

    If the SDO_SRID returned by Oracle is NULL, it will be replaced by
    the default value 8307 converted to its EPSG value: 4326 (see
    DEFAULT_SRID).

    If the value is higher than 1, all SRIDs will be forced to this
    value. In this case, DEFAULT_SRID will not be used when Oracle
    returns a null value, and the value will be forced to CONVERT_SRID.

    Note that it is also possible to set the EPSG value on the Oracle
    side when sdo_cs.map_oracle_srid_to_epsg() returns NULL if you want
    to force the value:

      system@db> UPDATE sdo_coord_ref_sys SET legacy_code=41014 WHERE srid = 27572;

DEFAULT_SRID
    Use this directive to override the default EPSG SRID to use: 4326.
    Can be overwritten by CONVERT_SRID, see above.

GEOMETRY_EXTRACT_TYPE
    This directive can take three values: WKT (default), WKB and
    INTERNAL. When it is set to WKT, Ora2Pg will use
    SDO_UTIL.TO_WKTGEOMETRY() to extract the geometry data. When it is
    set to WKB, Ora2Pg will use the binary output using
    SDO_UTIL.TO_WKBGEOMETRY(). If these two extract types are called at
    the Oracle side, they are slow and you can easily reach Out Of
    Memory when you have lots of rows. Also, WKB is not able to export
    3D geometry and some geometries like CURVEPOLYGON. In this case, you
    may use the INTERNAL extraction type. It will use a Pure Perl
    library to convert the SDO_GEOMETRY data into a WKT representation,
    the translation is done on Ora2Pg side. This is a work in progress,
    please validate your exported data geometries before use. Default
    spatial object extraction type is INTERNAL.

POSTGIS_SCHEMA
    Use this directive to add a specific schema to the search path to
    look for PostGIS functions.

ST_SRID_FUNCTION
    Oracle function to use to extract the SRID from ST_Geometry meta
    information. Default: ST_SRID, for example it should be set to
    sde.st_srid for ArcSDE.

ST_DIMENSION_FUNCTION
    Oracle function to use to extract the dimension from ST_Geometry
    meta information. Default: ST_DIMENSION, for example it should be
    set to sde.st_dimension for ArcSDE.

ST_GEOMETRYTYPE_FUNCTION
    Oracle function to use to extract the geometry type from an
    ST_Geometry column. Default: ST_GEOMETRYTYPE, for example it should
    be set to sde.st_geometrytype for ArcSDE.

ST_ASBINARY_FUNCTION
    Oracle function used to convert an ST_Geometry value into WKB
    format. Default: ST_ASBINARY, for example it should be set to
    sde.st_asbinary for ArcSDE.

ST_ASTEXT_FUNCTION
    Oracle function used to convert an ST_Geometry value into WKT
    format. Default: ST_ASTEXT, for example it should be set to
    sde.st_astext for ArcSDE.

PostgreSQL Import By default, conversion to PostgreSQL format is written to a file named 'output.sql'. The command:

        psql mydb < output.sql

will import the contents of file output.sql into the PostgreSQL mydb
database.

DATA_LIMIT
    When performing INSERT/COPY export, Ora2Pg processes data in chunks
    of DATA_LIMIT tuples for speed improvement. Tuples are stored in
    memory before being written to disk, so if you want speed and have
    enough system resources you can increase this limit to a higher
    value, for example: 100000 or 1000000. Before release 7.0, a value
    of 0 meant no limit so that all tuples were stored in memory before
    being flushed to disk. In the 7.x branch this has been removed and
    chunks will be set to the default: 10000

BLOB_LIMIT
    When Ora2Pg detects a table with BLOB data, it will automatically
    reduce the value of this directive by dividing it by 10 until its
    value is below 1000. You can control this value by setting
    BLOB_LIMIT. Exporting BLOBs uses lot of resources; setting it to a
    too high value can produce OOM errors.

CLOB_AS_BLOB
    Applies same behavior on CLOBs than BLOBs with BLOB_LIMIT settings.
    This is especially useful if you have large CLOB data. Default:
    enabled

OUTPUT
    The Ora2Pg output filename can be changed with this directive.
    Default value is output.sql. If you set the file name with extension
    .gz or .bz2 the output will be automatically compressed. This
    requires that the Compress::Zlib Perl module is installed if the
    filename extension is .gz and that the bzip2 system command is
    installed for the .bz2 extension.

OUTPUT_DIR
    Since release 7.0, you can define a base directory where the files
    will be written. The directory must exist.

BZIP2
    This directive allows you to specify the full path to the bzip2
    program if it can not be found in the PATH environment variable.

FILE_PER_CONSTRAINT
    Allows object constraints to be saved in a separate file during
    schema export. The file will be named CONSTRAINTS_OUTPUT, where
    OUTPUT is the value of the corresponding configuration directive.
    You can use .gz or .bz2 extension to enable compression. Default is
    to save all data in the OUTPUT file. This directive is usable only
    with TABLE export type.

    The constraints can be imported quickly into PostgreSQL using the
    LOAD export type to parallelize their creation over multiple (-j or
    JOBS) connections.

FILE_PER_INDEX
    Allows indexes to be saved in a separate file during schema export.
    The file will be named INDEXES_OUTPUT, where OUTPUT is the value of
    the corresponding configuration directive. You can use .gz or .bz2
    file extension to enable compression. Default is to save all data in
    the OUTPUT file. This directive is usable only with TABLE AND
    TABLESPACE export type. With the TABLESPACE export, it is used to
    write "ALTER INDEX ... TABLESPACE ..." into a separate file named
    TBSP_INDEXES_OUTPUT that can be loaded at the end of the migration
    after the indexes creation to move the indexes.

    The indexes can be imported quickly into PostgreSQL using the LOAD
    export type to parallelize their creation over multiple (-j or JOBS)
    connections.

FILE_PER_FKEYS
    Allows foreign key declarations to be saved in a separate file
    during schema export. By default foreign keys are exported into the
    main output file or in the CONSTRAINT_output.sql file. When enabled,
    foreign keys will be exported into a file named FKEYS_output.sql

FILE_PER_TABLE
    Allows data export to be saved in one file per table/view. The files
    will be named as tablename_OUTPUT, where OUTPUT is the value of the
    corresponding configuration directive. You can still use .gz or .bz2
    extension in the OUTPUT directive to enable compression. Default 0
    will save all data in one file, set it to 1 to enable this feature.
    This is usable only during INSERT or COPY export type.

FILE_PER_FUNCTION
    Allows functions, procedures and triggers to be saved in one file
    per object. The files will be named as objectname_OUTPUT, where
    OUTPUT is the value of the corresponding configuration directive.
    You can still use .gz or .bz2 extension in the OUTPUT directive to
    enable compression. Default 0 will save all in one single file, set
    it to 1 to enable this feature. This is usable only during the
    corresponding export type; the package body export has a special
    behavior.

    When export type is PACKAGE and you've enabled this directive,
    Ora2Pg will create a directory per package, named with the lower
    case name of the package, and will create one file per
    function/procedure in that directory. If the configuration directive
    is not enabled, it will create one file per package as
    packagename_OUTPUT, where OUTPUT is the value of the corresponding
    directive.

TRUNCATE_TABLE
    If this directive is set to 1, a TRUNCATE TABLE instruction will be
    added before loading data. This is usable only during INSERT or COPY
    export types.

    When activated, the instruction will be added only if there's no
    global DELETE clause or no specific one for to the current table
    (see below).

DELETE
    Supports including a DELETE FROM ... WHERE clause filter before
    importing data to perform a delete of some lines instead of
    truncating tables. Value is constructed as follows:
    TABLE_NAME[DELETE_WHERE_CLAUSE], or if you have only one where
    clause for all tables just put the delete clause as a single value.
    Both are possible too. Here are some examples:

            DELETE  1=1    # Apply to all tables and delete all tuples
            DELETE  TABLE_TEST[ID1='001']   # Apply only on table TABLE_TEST
            DELETE  TABLE_TEST[ID1='001' OR ID1='002] DATE_CREATE > '2001-01-01' TABLE_INFO[NAME='test']

    The last example applies two different delete where clauses on
    tables TABLE_TEST and TABLE_INFO and a generic delete where clause
    on DATE_CREATE to all other tables. If TRUNCATE_TABLE is enabled it
    will be applied to all tables not covered by the DELETE definition.

    These DELETE clauses might be useful with regular "updates".

STOP_ON_ERROR
    Set this parameter to 0 to not include the call to \set
    ON_ERROR_STOP ON in all SQL scripts generated by Ora2Pg. By default
    this order is always present so that the script will immediately
    abort when an error is encountered.

COPY_FREEZE
    Enable this directive to use COPY FREEZE instead of a simple COPY to
    export data with rows already frozen. This is intended as a
    performance option for initial data loading. Rows will be frozen
    only if the table being loaded has been created or truncated in the
    current sub-transaction. This will only work with export to file and
    when -J or ORACLE_COPIES is not set or defaults to 1. It can be used
    with direct import into PostgreSQL under the same condition but -j
    or JOBS must also be unset or default to 1.

CREATE_OR_REPLACE
    By default Ora2Pg uses CREATE OR REPLACE in functions and views DDL.
    If you need not to override existing functions or views, disable
    this configuration directive - DDL will not include OR REPLACE.

DROP_IF_EXISTS
    To add a DROP <OBJECT> IF EXISTS before creating the object, enable
    this directive. Can be useful in iterative work. Default is
    disabled.

EXPORT_GTT
    PostgreSQL does not support Global Temporary Tables natively but you
    can use the pgtt extension to emulate this behavior. Enable this
    directive to export global temporary tables.

PGTT_NOSUPERUSER
    By default the pgtt extension is loaded using superuser privileges.
    Enable it if you run the SQL scripts generated using a non superuser
    user. It will use:

        LOAD '$libdir/plugins/pgtt';

    instead of default:

        LOAD 'pgtt';

NO_HEADER
    Enabling this directive will prevent Ora2Pg from printing its header
    into output files. Only the translated code will be written.

PSQL_RELATIVE_PATH
    By default, Ora2Pg uses \i psql command to execute generated SQL
    files. If you want to use a relative path following the script
    execution file, enabling this option will use \ir. See psql help for
    more information.

DATA_VALIDATION_ROWS
    Number of rows that must be retrieved on both sides for data
    validation. Default is to compare the first 10000 rows. A value of 0
    means compare all rows.

DATA_VALIDATION_ORDERING
    Order of rows between both sides is different once the data has been
    modified. In this case data must be ordered using a primary key or a
    unique index, meaning that a table without such object cannot be
    compared. If the validation is done just after the data migration
    without any data modification, the validation can be done on all
    tables without any ordering.

DATA_VALIDATION_ERROR
    Stop validating data from a table after a certain amount of row
    mismatches. Default is to stop after 10 rows validation errors.

TRANSFORM_VALUE
    Use this directive to specify which transformation should be applied
    to a column when exporting data. Value must be a semicolon-separated
    list of

       TABLE[COLUMN_NAME, <replace code in SELECT target list>]

    For example, to replace the string 'Oracle' with 'PostgreSQL' in a
    varchar2 column, use the following.

       TRANSFORM_VALUE   ERROR_LOG_SAMPLE[DBMS_TYPE:regexp_replace("DBMS_TYPE",'Oracle','PostgreSQL')]

    or to replace all Oracle char(0) in a string with a space character:

        TRANSFORM_VALUE   CLOB_TABLE[CHARDATA:translate("CHARDATA", chr(0), ' ')]

    The expression will be applied in the SQL statement used to extract
    data from the source database.

NO_START_SCN
    Enable this directive if you don't want to export all data based on
    current SCN. By default Ora2Pg get first the current SCN and then
    retrieve all table data using this SCN to be consistant in case of
    data modification.

When using Ora2Pg export type INSERT or COPY to dump data to a file and
FILE_PER_TABLE is enabled, you will be warned that Ora2Pg will not
export data again if the file already exists. This is to prevent
downloading table data twice when dealing with huge amount of data. To
force the download of data from these tables you have to remove the
existing output file first.

If you want to import data on the fly to the PostgreSQL database, you
have three configuration directives to set the PostgreSQL database
connection. This is only possible with COPY or INSERT export type as for
database schema there's no real benefit to do that.

PG_DSN
    Use this directive to set the PostgreSQL data source namespace using
    DBD::Pg Perl module as follows:

            dbi:Pg:dbname=pgdb;host=localhost;port=5432

    will connect to database 'pgdb' on localhost at tcp port 5432.

    Note that this directive is only used for data export, other exports
    need to be imported manually through the use of psql or any other
    PostgreSQL client.

    To use SSL encrypted connection you must add sslmode=require to the
    connection string as follows:

            dbi:Pg:dbname=pgdb;host=localhost;port=5432;sslmode=require

PG_USER and PG_PWD
    These two directives are used to set the login user and password.

    If you do not supply credentials with PG_PWD and you have installed
    the Term::ReadKey Perl module, Ora2Pg will ask for the password
    interactively. If PG_USER is not set it will be asked interactively
    too.

SYNCHRONOUS_COMMIT
    Specifies whether transaction commit will wait for WAL records to be
    written to disk before the command returns a "success" indication to
    the client. This is equivalent to setting the synchronous_commit
    directive in the postgresql.conf file. This is only used when you
    load data directly to PostgreSQL; the default is off to disable
    synchronous commit to gain speed at writing data. Some modified
    versions of PostgreSQL, like Greenplum, do not have this setting, so
    in this case set this directive to 1, and ora2pg will not try to
    change the setting.

PG_INITIAL_COMMAND
    This directive can be used to send an initial command to PostgreSQL,
    just after the connection. For example to set some session
    parameters. This directive can be used multiple times.

INSERT_ON_CONFLICT
    When enabled this instructs Ora2Pg to add an ON CONFLICT DO NOTHING
    clause to all INSERT statements generated for this type of data
    export.

Column type control PG_NUMERIC_TYPE If set to 1, replace portable numeric type with PostgreSQL internal type. Oracle data type NUMBER(p,s) is approximately converted to real and float PostgreSQL data types. If you have monetary fields or don't want rounding issues with the extra decimals, you should preserve the same numeric(p,s) PostgreSQL data type. Do this only if you need exactness because using numeric(p,s) is slower than using real or double.

PG_INTEGER_TYPE
    If set to 1, replace portable numeric type with PostgreSQL internal
    type. Oracle data types NUMBER(p) or NUMBER are converted to
    smallint, integer or bigint PostgreSQL data types following the
    value of the precision. If NUMBER without precision is set to
    DEFAULT_NUMERIC (see below).

DEFAULT_NUMERIC
    NUMBER without precision is converted by default to bigint only if
    PG_INTEGER_TYPE is true. You can override this value to any PG type,
    like integer or float.

DATA_TYPE
    If you're experiencing any problems in data type schema conversion,
    with this directive you can take full control of the correspondence
    between Oracle and PostgreSQL types to redefine data type
    translation used in Ora2pg. The syntax is a comma-separated list of
    "Oracle datatype:PostgreSQL datatype". Here is the default list
    used:

            DATA_TYPE       VARCHAR2:varchar,NVARCHAR2:varchar,NVARCHAR:varchar,NCHAR:char,DATE:timestamp(0),LONG:text,LONG RAW:bytea,CLOB:text,NCLOB:text,BLOB:bytea,BFILE:bytea,RAW(16):uuid,RAW(32):uuid,RAW:bytea,UROWID:oid,ROWID:oid,FLOAT:double precision,DEC:decimal,DECIMAL:decimal,DOUBLE PRECISION:double precision,INT:integer,INTEGER:integer,REAL:real,SMALLINT:smallint,BINARY_FLOAT:double precision,BINARY_DOUBLE:double precision,TIMESTAMP:timestamp,XMLTYPE:xml,BINARY_INTEGER:integer,PLS_INTEGER:integer,TIMESTAMP WITH TIME ZONE:timestamp with time zone,TIMESTAMP WITH LOCAL TIME ZONE:timestamp with time zone

    The directive and the list definition must be a single line.

    Note that when RAW(16) or RAW(32) columns are found or when the RAW
    column has "SYS_GUID()" as default value, Ora2Pg will automatically
    translate the type of the column into uuid which might be the right
    translation in most cases. In this case data will be automatically
    migrated as PostgreSQL uuid data type provided by the "uuid-ossp"
    extension.

    If you want to replace a type with a precision and scale you need to
    escape the comma with a backslash. For example, if you want to
    replace all NUMBER(*,0) with bigint instead of numeric(38) add the
    following:

           DATA_TYPE       NUMBER(*\,0):bigint

    You don't have to repeat all default type conversions. Instead just
    specify the ones you want to rewrite.

    There's a special case with BFILE when they are converted to type
    TEXT - they will just contain the full path to the external file. If
    you set the destination type to BYTEA (the default), Ora2Pg will
    export the content of the BFILE as bytea. The third case is when you
    set the destination type to EFILE - in this case, Ora2Pg will export
    it as an EFILE record: (DIRECTORY, FILENAME). Use the DIRECTORY
    export type to export the existing directories as well as privileges
    on those directories.

    There's no SQL function available to retrieve the path to the BFILE.
    Ora2Pg has to create one using the DBMS_LOB package.

            CREATE OR REPLACE FUNCTION ora2pg_get_bfilename( p_bfile IN BFILE )
            RETURN VARCHAR2
            AS
                l_dir   VARCHAR2(4000);
                l_fname VARCHAR2(4000);
                l_path  VARCHAR2(4000);
            BEGIN
                dbms_lob.FILEGETNAME( p_bfile, l_dir, l_fname );
                SELECT directory_path INTO l_path FROM all_directories
                    WHERE directory_name = l_dir;
                l_dir := rtrim(l_path,'/');
                RETURN l_dir || '/' || l_fname;
            END;

    This function is only created if Ora2Pg found a table with a BFILE
    column and that the destination type is TEXT. The function is
    dropped at the end of the export. This concern both, COPY and INSERT
    export type.

    There's no SQL function available to retrieve BFILE as an EFILE
    record, therefore Ora2Pg needs to create one using the DBMS_LOB
    package.

            CREATE OR REPLACE FUNCTION ora2pg_get_efile( p_bfile IN BFILE )
            RETURN VARCHAR2
            AS
                l_dir   VARCHAR2(4000);
                l_fname VARCHAR2(4000);
            BEGIN
                dbms_lob.FILEGETNAME( p_bfile, l_dir, l_fname );
                RETURN '(' || l_dir || ',' || l_fnamei || ')';
            END;

    This function is only created if Ora2Pg finds a table with a BFILE
    column and that the destination type is EFILE. The function is
    dropped at the end of the export. This concerns both COPY and INSERT
    export types.

    To set the destination type, use the DATA_TYPE configuration
    directive:

            DATA_TYPE       BFILE:EFILE

    for example.

    The EFILE type is a user defined type created by the PostgreSQL
    extension external_file that can be found here:
    https://github.com/darold/external_file This is a port of the BFILE
    Oracle type to PostgreSQL.

    There's no SQL function available to retrieve the content of a
    BFILE. Ora2Pg needs to create one using the DBMS_LOB package.

            CREATE OR REPLACE FUNCTION ora2pg_get_bfile( p_bfile IN BFILE ) RETURN
            BLOB
              AS
                    filecontent BLOB := NULL;
                    src_file BFILE := NULL;
                    l_step PLS_INTEGER := 12000;
                    l_dir   VARCHAR2(4000);
                    l_fname VARCHAR2(4000);
                    offset NUMBER := 1;
              BEGIN
                IF p_bfile IS NULL THEN
                  RETURN NULL;
                END IF;

                DBMS_LOB.FILEGETNAME( p_bfile, l_dir, l_fname );
                src_file := BFILENAME( l_dir, l_fname );
                IF src_file IS NULL THEN
                    RETURN NULL;
                END IF;

                DBMS_LOB.FILEOPEN(src_file, DBMS_LOB.FILE_READONLY);
                DBMS_LOB.CREATETEMPORARY(filecontent, true);
                DBMS_LOB.LOADBLOBFROMFILE (filecontent, src_file, DBMS_LOB.LOBMAXSIZE, offset, offset);
                DBMS_LOB.FILECLOSE(src_file);
                RETURN filecontent;
            END;

    This function is only created if Ora2Pg finds a table with a BFILE
    column and that the destination type is bytea (the default). The
    function is dropped at the end of the export. This applies to both
    COPY and INSERT export types.

    Regarding ROWID and UROWID, they are converted into OID by "logical"
    default, but this will throw an error during data import. There is
    no equivalent data type so you might want to use the DATA_TYPE
    directive to change the corresponding type in PostgreSQL. You should
    consider replacing this data type with a bigserial (autoincremented
    sequence), text, or uuid data type.

MODIFY_TYPE
    Sometimes you need to force the destination type. For example, a
    column exported as timestamp by Ora2Pg can be forced into type date.
    Value is a comma-separated list of TABLE:COLUMN:TYPE structures. If
    you need to use commas or spaces inside type definitions, you will
    have to escape them with backslashes.

            MODIFY_TYPE     TABLE1:COL3:varchar,TABLE1:COL4:decimal(9\,6)

    The type of table1.col3 will be replaced by varchar and table1.col4
    by decimal with precision and scale.

    If the column's type is a user-defined type, Ora2Pg will autodetect
    the composite type and will export its data using ROW(). Some Oracle
    user-defined types are just arrays of a native types. In this case,
    you may want to transform this column into a simple array of a
    PostgreSQL native type. To do so, just redefine the destination type
    as wanted, and Ora2Pg will also transform the data as an array. For
    example, with the following definition in Oracle:

            CREATE OR REPLACE TYPE mem_type IS VARRAY(10) of VARCHAR2(15);
            CREATE TABLE club (Name VARCHAR2(10),
                    Address VARCHAR2(20),
                    City VARCHAR2(20),
                    Phone VARCHAR2(8),
                    Members mem_type
            );

    custom type "mem_type" is just a string array and can be translated
    into the following in PostgreSQL:

            CREATE TABLE club (
                    name varchar(10),
                    address varchar(20),
                    city varchar(20),
                    phone varchar(8),
                    members text[]
            ) ;

    To do so, just use the directive as follows:

            MODIFY_TYPE     CLUB:MEMBERS:text[]

    Ora2Pg will take care to transform all data of this column into the
    correct format. Only arrays of characters and numeric types are
    supported.

TO_NUMBER_CONVERSION
    By default, Oracle's call to function TO_NUMBER will be translated
    as a cast into numeric. For example, TO_NUMBER('10.1234') is
    converted into PostgreSQL call to_number('10.1234')::numeric. If you
    want, you can cast the call to integer or bigint by changing the
    value of the configuration directive. If you need better control of
    the format, just set it as value, for example: TO_NUMBER_CONVERSION
    99999999999999999999.9999999999 will convert the code above as:
    TO_NUMBER('10.1234', '99999999999999999999.9999999999') Any value of
    the directive that is not numeric, integer or bigint will be taken
    as a mask format. If set to none, no conversion will be done.

VARCHAR_TO_TEXT
    By default varchar2 without size constraint are translated into
    text. If you want to keep the varchar name, disable this directive.

FORCE_IDENTITY_BIGINT
    Usually identity columns must be bigint to correspond to an auto
    increment sequence so Ora2Pg always forces it to be a bigint. If,
    for any reason you want Ora2Pg to respect the DATA_TYPE you have set
    for identity columns then disable this directive.

TO_CHAR_NOTIMEZONE
    If you want Ora2Pg to remove any timezone information from the
    format part of the TO_CHAR() function, enable this directive.
    Disabled by default.

Taking export under control The following other configuration directives interact directly with the export process and give you fine granularity in database export control.

SKIP
    For TABLE export you may not want to export all schema constraints,
    the SKIP configuration directive allows you to specify a
    space-separated list of constraints that should not be exported.
    Possible values are:

            - fkeys: turn off foreign key constraints
            - pkeys: turn off primary keys
            - ukeys: turn off unique column constraints
            - indexes: turn off all other index types
            - checks: turn off check constraints

    For example:

            SKIP    indexes,checks

    will remove indexes and check constraints from export.

PKEY_IN_CREATE
    Enable this directive if you want to add primary key definition
    inside the create table statement. If disabled (the default) primary
    key definition will be added with an alter table statement. Enable
    it if you are exporting to GreenPlum PostgreSQL database.

KEEP_PKEY_NAMES
    By default names of the primary and unique keys in the source Oracle
    database are ignored and key names are autogenerated in the target
    PostgreSQL database with the PostgreSQL internal default naming
    rules. If you want to preserve Oracle primary and unique key names
    set this option to 1.

FKEY_ADD_UPDATE
    This directive allows you to add an ON UPDATE CASCADE option to a
    foreign key when an ON DELETE CASCADE is defined or always. Oracle
    does not support this feature, you have to use triggers to operate
    the ON UPDATE CASCADE. As PostgreSQL has this feature, you can
    choose how to add the foreign key option. There are three values to
    this directive: never, the default that means that foreign keys will
    be declared exactly like in Oracle. The second value is delete, that
    mean that the ON UPDATE CASCADE option will be added only if the ON
    DELETE CASCADE is already defined on the foreign Keys. The last
    value, always, will force all foreign keys to be defined using the
    update option.

FKEY_DEFERRABLE
    When exporting tables, Ora2Pg normally exports constraints as they
    are, if they are non-deferrable they are exported as non-deferrable.
    However, non-deferrable constraints will probably cause problems
    when attempting to import data to Pg. The FKEY_DEFERRABLE option set
    to 1 will cause all foreign key constraints to be exported as
    deferrable.

DEFER_FKEY
    In addition to exporting data when the DEFER_FKEY option is set to
    1, it will add a command to defer all foreign key constraints during
    data export and the import will be done in a single transaction.
    This will work only if foreign keys have been exported as deferrable
    and you are not using direct import to PostgreSQL (PG_DSN is not
    defined). Constraints will then be checked at the end of the
    transaction.

    This directive can also be enabled if you want to force all foreign
    keys to be created as deferrable and initially deferred during
    schema export (TABLE export type).

DROP_FKEY
    If deferring foreign keys is not possible due to the amount of data
    in a single transaction, you've not exported foreign keys as
    deferrable or you are using direct import to PostgreSQL, you can use
    the DROP_FKEY directive.

    It will drop all foreign keys before all data import and recreate
    them at the end of the import.

DROP_INDEXES
    This directive allows you to gain a lot of speed during data import
    by removing all indexes that are not an automatic index (indexes of
    primary keys) and recreate them at the end of data import. Of course
    it is far better to not import indexes and constraints before having
    imported all data.

DISABLE_TRIGGERS
    This directive is used to disable triggers on all tables in COPY or
    INSERT export modes. Available values are USER (disable user-defined
    triggers only) and ALL (includes RI system triggers). Default is 0:
    do not add SQL statements to disable triggers before data import.

    If you want to disable triggers during data migration, set the value
    to USER if you are connected as a non-superuser and ALL if you are
    connected as a PostgreSQL superuser. A value of 1 is equal to USER.

DISABLE_SEQUENCE
    If set to 1, it disables alter of sequences on all tables during
    COPY or INSERT export mode. This is used to prevent the update of
    sequences during data migration. Default is 0, alter sequences.

NOESCAPE
    By default, all data that are not of type date or time are escaped.
    If you experience any problems with that, you can set it to 1 to
    disable character escaping during data export. This directive is
    only used during a COPY export. See STANDARD_CONFORMING_STRINGS for
    enabling/disabling escape with INSERT statements.

STANDARD_CONFORMING_STRINGS
    This controls whether ordinary string literals ('...') treat
    backslashes literally, as specified in the SQL standard. This was
    the default before Ora2Pg v8.5 so that all strings were escaped
    first; now this is currently on, causing Ora2Pg to use the escape
    string syntax (E'...') if this parameter is not set to 0. This is
    the exact behavior of the same option in PostgreSQL. This directive
    is only used during data export to build INSERT statements. See
    NOESCAPE for enabling/disabling escape in COPY statements.

TRIM_TYPE
    If you want to convert CHAR(n) from Oracle into varchar(n) or text
    in PostgreSQL using directive DATA_TYPE, you might want to do some
    trimming on the data. By default, Ora2Pg will auto-detect this
    conversion and remove any whitespace at both leading and trailing
    positions. If you just want to remove the leading characters, set
    the value to LEADING. If you just want to remove the trailing
    characters, set the value to TRAILING. Default value is BOTH.

TRIM_CHAR
    The default trimming character is space; use this directive if you
    need to change the character that will be removed. For example, set
    it to - if you have leading - in the char(n) field. To use space as
    trimming character, comment this directive, this is the default
    value.

PRESERVE_CASE
    If you want to preserve the case of Oracle object names, set this
    directive to 1. By default, Ora2Pg will convert all Oracle object
    names to lower case. I do not recommend enabling this unless you
    will always have to double-quote object names in all your SQL
    scripts.

ORA_RESERVED_WORDS
    Allow escaping of column names using Oracle reserved words. Value is
    a list of comma-separated reserved words. Default:
    audit,comment,references.

USE_RESERVED_WORDS
    Enable this directive if you have table or column names that are
    reserved words for PostgreSQL. Ora2Pg will double quote the name of
    the object.

GEN_USER_PWD
    Set this directive to 1 to replace default passwords with random
    passwords for all extracted users during a GRANT export.

PG_SUPPORTS_MVIEW
    Since PostgreSQL 9.3, materialized views are supported with the SQL
    syntax 'CREATE MATERIALIZED VIEW'. To force Ora2Pg to use the native
    PostgreSQL support, you must enable this configuration - enabled by
    default. If you want to use the old style with table and a set of
    functions, you should disable it.

PG_SUPPORTS_IFEXISTS
    PostgreSQL versions below 9.x do not support IF EXISTS in DDL
    statements. Disabling the directive with value 0 will prevent Ora2Pg
    to from adding those keywords in all generated statements. Default
    value is 1, enabled.

PG_VERSION
    Set the PostgreSQL major version number of the target database. Ex:
    9.6 or 13. Default is current major version at time of a new
    release. This replaces the old and deprecated PG_SUPPORTS_*
    configuration directives described below.

PG_SUPPORTS_ROLE (Deprecated)
    This option is deprecated since Ora2Pg release v7.3.

    By default Oracle roles are translated into PostgreSQL groups. If
    you have PostgreSQL 8.1 or higher, consider the use of ROLES and set
    this directive to 1 to export roles.

PG_SUPPORTS_INOUT (Deprecated)
    This option is deprecated since Ora2Pg release v7.3.

    If set to 0, all IN, OUT or INOUT parameters will not be used in the
    generated PostgreSQL function declarations (disable it for
    PostgreSQL database versions lower than 8.1). This is now enabled by
    default.

PG_SUPPORTS_DEFAULT
    This directive enables or disables the use of default parameter
    values in function exports. Prior to PostgreSQL 8.4, such default
    values were not supported. This feature is now enabled by default.

PG_SUPPORTS_WHEN (Deprecated)
    Adds support for WHEN clauses on triggers as PostgreSQL v9.0 now
    supports them. This directive is enabled by default; set it to 0 to
    disable this feature.

PG_SUPPORTS_INSTEADOF (Deprecated)
    Adds support for INSTEAD OF usage on triggers (used with PG >= 9.1).
    If this directive is disabled, the INSTEAD OF triggers will be
    rewritten as Pg rules.

PG_SUPPORTS_CHECKOPTION
    When enabled, exports views with CHECK OPTION. Disable it if you
    have a PostgreSQL version prior to 9.4. Default: 1, enabled.

PG_SUPPORTS_IFEXISTS
    If disabled, do not export objects with IF EXISTS statements.
    Enabled by default.

PG_SUPPORTS_PARTITION
    PostgreSQL versions prior to 10.0 do not have native partitioning.
    Enable this directive if you want to use declarative partitioning.
    Enabled by default.

PG_SUPPORTS_SUBSTR
    Some versions of PostgreSQL like Redshift don't support substr() and
    need to be replaced by a call to substring(). In this case, disable
    it.

PG_SUPPORTS_NAMED_OPERATOR
    Disable this directive if you are using PG < 9.5. PL/SQL operators
    used in named parameters => will be replaced by PostgreSQL's
    proprietary operator := Enabled by default.

PG_SUPPORTS_IDENTITY
    Enable this directive if you have PostgreSQL >= 10 to use IDENTITY
    columns instead of serial or bigserial data types. If
    PG_SUPPORTS_IDENTITY is disabled and there is an IDENTITY column in
    the Oracle table, they are exported as serial or bigserial columns.
    When it is enabled they are exported as IDENTITY columns like:

          CREATE TABLE identity_test_tab (
                  id bigint GENERATED ALWAYS AS IDENTITY,
                  description varchar(30)
          ) ;

    If there are non-default sequence options set in Oracle, they will
    be appended after the IDENTITY keyword. Additionally in both cases,
    Ora2Pg will create a file AUTOINCREMENT_output.sql with an embedded
    function to update the associated sequences with the restart value
    set to "SELECT max(colname)+1 FROM tablename". Of course this file
    must be imported after data import otherwise sequence will be kept
    at start value. Enabled by default.

PG_SUPPORTS_PROCEDURE
    PostgreSQL v11 adds support for PROCEDURE, enable it if you use such
    version.

BITMAP_AS_GIN
    Use btree_gin extension to create bitmap-like index with pg >= 9.4.
    You will need to create the extension yourself: create extension
    btree_gin; Default is to create GIN index, when disabled, a btree
    index will be created.

PG_BACKGROUND
    Use pg_background extension to create an autonomous transaction
    instead of using a dblink wrapper. With pg >= 9.5 only. Default is
    to use dblink. See https://github.com/vibhorkum/pg_background about
    this extension.

DBLINK_CONN
    By default if you have an autonomous transaction translated using
    dblink extension instead of pg_background, the connection is defined
    using the values set with PG_DSN, PG_USER and PG_PWD. If you want to
    fully override the connection string, use this directive to set the
    connection in the autonomous transaction wrapper function. For
    example:

            DBLINK_CONN    port=5432 dbname=pgdb host=localhost user=pguser password=pgpass

LONGREADLEN
    Use this directive to set the database handle's 'LongReadLen'
    attribute to a value that will be larger than the expected size of
    the LOBs. The default is 1MB which may not be enough to extract
    BLOBs or CLOBs. If the size of the LOB exceeds the 'LongReadLen'
    DBD::Oracle will return an 'ORA-24345: A Truncation' error. Default:
    1023*1024 bytes.

    Take a look at this page to learn more:
    http://search.cpan.org/~pythian/DBD-Oracle-1.22/Oracle.pm#Data_Inter
    face_for_Persistent_LOBs

    Important note: If you increase the value of this directive take
    care that DATA_LIMIT will probably need to be reduced. Even if you
    only have a 1MB blob, trying to read 10000 of them (the default
    DATA_LIMIT) all at once will require 10GB of memory. You may extract
    data from those tables separately and set a DATA_LIMIT to 500 or
    lower, otherwise you may experience some out of memory issues.

LONGTRUNKOK
    If you want to bypass the 'ORA-24345: A Truncation' error, set this
    directive to 1. It will truncate the data extracted to the
    LongReadLen value. Disabled by default so that you will be warned if
    your LongReadLen value is not high enough.

USE_LOB_LOCATOR
    Disable this if you want to load the full content of BLOB and CLOB
    and not use LOB locators. In this case, you will have to set
    LONGREADLEN to the right value. Note that this will not improve the
    speed of BLOB export as most of the time is always consumed by the
    bytea escaping and in this case, export is done line by line and not
    by chunk of DATA_LIMIT rows. For more information on how it works,
    see
    http://search.cpan.org/~pythian/DBD-Oracle-1.74/lib/DBD/Oracle.pm#Da
    ta_Interface_for_LOB_Locators

    Default is enabled; it uses LOB locators.

LOB_CHUNK_SIZE
    Oracle recommends reading from and writing to a LOB in batches using
    a multiple of the LOB chunk size. This chunk size defaults to 8k
    (8192). Recent tests have shown that the best performance can be
    reached with higher values like 512K or 4Mb.

    A quick benchmark with 30120 rows with different size of BLOB
    (200x5Mb, 19800x212k, 10000x942K, 100x17Mb, 20x156Mb), with
    DATA_LIMIT=100, LONGREADLEN=170Mb and a total table size of 20GB
    gives:

           no lob locator  : 22m46,218s (1365 sec., avg: 22 recs/sec)
           chunk size 8k   : 15m50,886s (951 sec., avg: 31 recs/sec)
           chunk size 512k : 1m28,161s (88 sec., avg: 342 recs/sec)
           chunk size 4Mb  : 1m23,717s (83 sec., avg: 362 recs/sec)

    In conclusion, it can be more than 10 times faster with
    LOB_CHUNK_SIZE set to 4Mb. Depending on the size of most BLOBs, you
    may want to adjust the value here. For example, if you have a
    majority of small lobs below 8K, using 8192 is better to not waste
    space. Default value for LOB_CHUNK_SIZE is 512000.

XML_PRETTY
    Forces the use of getStringVal() instead of getClobVal() for XML
    data export. Default is 1, enabled for backward compatibility. Set
    it to 0 to use extract method like CLOB. Note that XML values
    extracted with getStringVal() must not exceed VARCHAR2 size limit
    (4000); otherwise, it will return an error.

ENABLE_MICROSECOND
    Set it to 0 if you want to disable export of milliseconds from
    Oracle timestamp columns. By default, milliseconds are exported by
    using the following format:

            'YYYY-MM-DD HH24:MI:SS.FF'

    Disabling will force the use of the following Oracle format:

            to_char(..., 'YYYY-MM-DD HH24:MI:SS')

    By default, milliseconds are exported.

DISABLE_COMMENT
    Set this to 1 if you don't want to export comments associated with
    tables and columns definition. Default is enabled.

Control MySQL export behavior MYSQL_PIPES_AS_CONCAT Enable this if double pipe and double ampersand (|| and &&) should not be taken as equivalent to OR and AND. It depends on the variable @sql_mode. Use it only if Ora2Pg fails on auto-detecting this behavior.

MYSQL_INTERNAL_EXTRACT_FORMAT
    Enable this directive if you want EXTRACT() replacement to use the
    internal format returned as an integer, for example DD HH24:MM:SS
    will be replaced with format; DDHH24MMSS::bigint, this depends on
    your apps usage.

Control SQL Server export behavior DROP_ROWVERSION PostgreSQL has no equivalent to rowversion datatype and feature. If you want to remove these useless columns, enable this directive. Columns of datatype 'rowversion' or 'timestamp' will not be exported.

CASE_INSENSITIVE_SEARCH
    Emulate the same behavior of MSSQL with case-insensitive search. If
    the value is citext, it will use the citext data type instead of
    char/varchar/text in tables DDL (Ora2Pg will add a CHECK constraint
    for columns with a precision). Instead of citext, you can also set a
    collation name that will be used in the column definitions. To
    disable case-insensitive search set it to: none.

SELECT_TOP
    Appends a TOP N clause to the SELECT command used to extract the
    data from SQL Server. This is equivalent to a WHERE ROWNUM < 1000
    clause for Oracle.

Special options to handle character encoding NLS_LANG and NLS_NCHAR By default, Ora2Pg will set NLS_LANG to AMERICAN_AMERICA.AL32UTF8 and NLS_NCHAR to AL32UTF8. It is not recommended to change these settings, but in some cases it could be useful. Using your own settings with these configuration directives will change the client encoding on the Oracle side by setting the environment variables $ENV{NLS_LANG} and $ENV{NLS_NCHAR}.

BINMODE
    By default, Ora2Pg will force Perl to use UTF8 encoding. This is
    done through a call to the Perl pragma:

            use open ':utf8';

    You can override this encoding by using the BINMODE directive. For
    example, you can set it to :locale to use your locale or iso-8859-7.
    It will respectively use:

            use open ':locale';
            use open ':encoding(iso-8859-7)';

    If you have changed the NLS_LANG to non-UTF8 encoding, you might
    want to set this directive. See
    http://perldoc.perl.org/5.14.2/open.html for more information. Most
    of the time, leave this directive commented.

CLIENT_ENCODING
    By default, PostgreSQL client encoding is automatically set to UTF8
    to avoid encoding issues. If you have changed the value of NLS_LANG,
    you might have to change the encoding of the PostgreSQL client.

    You can take a look at the PostgreSQL supported character sets here:
    http://www.postgresql.org/docs/9.0/static/multibyte.html

FORCE_PLSQL_ENCODING
    Enable this directive to force UTF8 encoding of the PL/SQL code
    exported. Could be helpful in some rare conditions.

PLSQL to PLPGSQL conversion Automatic code conversion from Oracle PL/SQL to PostgreSQL PL/PGSQL is a work in progress in Ora2Pg and you will likely have manual work. The Perl code used for automatic conversion is stored in a specific Perl Module named Ora2Pg/PLSQL.pm. Feel free to modify/add your own code and send me patches. The main work is on function, procedure, package and package body headers and parameter rewrites.

PLSQL_PGSQL
    Enable/disable PLSQL to PLPGSQL conversion. Enabled by default.

NULL_EQUAL_EMPTY
    Ora2Pg can replace all conditions with a test on NULL by calling the
    coalesce() function to mimic the Oracle behavior where empty strings
    are considered equal to NULL.

            (field1 IS NULL) is replaced by (coalesce(field1::text, '') = '')
            (field2 IS NOT NULL) is replaced by (field2 IS NOT NULL AND field2::text <> '')

    You might want this replacement to ensure your application will have
    the same behavior, but if you have control over your application, a
    better way is to transform empty strings into NULL because
    PostgreSQL differentiates between them.

EMPTY_LOB_NULL
    Force empty_clob() and empty_blob() to be exported as NULL instead
    of an empty string for the first one and '\x' for the second. If
    NULL is allowed in your column, this might improve data export speed
    if you have lots of empty lobs. Default is to preserve the exact
    data from Oracle.

PACKAGE_AS_SCHEMA
    If you don't want to export packages as schemas but as simple
    functions, you might also want to replace all calls to
    package_name.function_name. If you disable the PACKAGE_AS_SCHEMA
    directive then Ora2Pg will replace all calls to
    package_name.function_name() with package_name_function_name().
    Default is to use a schema to emulate packages.

    The replacement will be done in all kinds of DDL or code that is
    parsed by the PLSQL to PLPGSQL converter. PLSQL_PGSQL must be
    enabled or -p used in command line.

REWRITE_OUTER_JOIN
    Enable this directive if the rewrite of Oracle native syntax (+) of
    OUTER JOIN is broken. This will force Ora2Pg to not rewrite such
    code. Default is to try to rewrite simple forms of right outer joins
    for now.

UUID_FUNCTION
    By default, Ora2Pg will convert calls to SYS_GUID() Oracle function
    with a call to uuid_generate_v4 from the uuid-ossp extension. You
    can redefine it to use the gen_random_uuid function from the
    pgcrypto extension by changing the function name. Default is
    uuid_generate_v4.

    Note that when RAW(16) or RAW(32) columns are found or when the RAW
    column has "SYS_GUID()" as default value, Ora2Pg will automatically
    translate the type of the column into uuid which might be the right
    translation in most cases. In this case data will be automatically
    migrated as PostgreSQL uuid data type provided by the "uuid-ossp"
    extension.

FUNCTION_STABLE
    By default, Oracle functions are marked as STABLE as they can not
    modify data unless when used in PL/SQL with variable assignment or
    as conditional expressions. You can force Ora2Pg to create these
    functions as VOLATILE by disabling this configuration directive.

COMMENT_COMMIT_ROLLBACK
    By default, calls to COMMIT/ROLLBACK are kept untouched by Ora2Pg to
    force the user to review the logic of the function. Once it is fixed
    in Oracle source code or you want to comment these calls, enable the
    following directive.

COMMENT_SAVEPOINT
    It is common to see SAVEPOINT calls inside PL/SQL procedures
    together with a ROLLBACK TO savepoint_name. When
    COMMENT_COMMIT_ROLLBACK is enabled, you may want to also comment
    SAVEPOINT calls; in this case enable it.

STRING_CONSTANT_REGEXP
    Ora2Pg replaces all string constants during the PL/SQL to PL/PGSQL
    translation. String constants are all text included between single
    quotes. If you have some string placeholders used in dynamic calls
    to queries, you can set a list of regexps to be temporarily replaced
    to not break the parser. For example:

            STRING_CONSTANT_REGEXP         <placeholder value=".*">

    The list of regexps must use the semi colon as separator.

ALTERNATIVE_QUOTING_REGEXP
    To support the Alternative Quoting Mechanism ('Q' or 'q') for String
    Literals, set the regexp with the text capture to use to extract the
    text part. For example, with a variable declared as:

            c_sample VARCHAR2(100 CHAR) := q'{This doesn't work.}';

    the regexp to use must be:

            ALTERNATIVE_QUOTING_REGEXP     q'{(.*)}'

    Ora2pg will use the $$ delimiter; for the example the result will
    be:

            c_sample varchar(100) := $$This doesn't work.$$;

    The value of this configuration directive can be a list of regexps
    separated by a semicolon. The capture part (between parentheses) is
    mandatory in each regexp if you want to restore the string constant.

USE_ORAFCE
    If you want to use functions defined in the Orafce library and
    prevent Ora2Pg from translating calls to these functions, enable
    this directive. The Orafce library can be found here:
    https://github.com/orafce/orafce

    By default, Ora2pg rewrites add_month(), add_year(), date_trunc()
    and to_char() functions, but you may prefer to use the orafce
    version of these functions that do not need any code transformation.

AUTONOMOUS_TRANSACTION
    Enable translation of autonomous transactions into wrapper functions
    using dblink or pg_background extension. If you don't want to use
    this translation and just want the function to be exported as a
    normal one without the pragma call, disable this directive.

Materialized View Materialized views are exported as snapshot "Snapshot Materialized Views" as PostgreSQL only supports full refresh.

If you want to import materialized views in PostgreSQL prior to 9.3, you
have to set configuration directive PG_SUPPORTS_MVIEW to 0. In this case
Ora2Pg will export all materialized views as explained in this document:

        http://tech.jonathangardner.net/wiki/PostgreSQL/Materialized_Views.

When exporting materialized views, Ora2Pg will first add the SQL code to
create the "materialized_views" table:

        CREATE TABLE materialized_views (
                mview_name text NOT NULL PRIMARY KEY,
                view_name text NOT NULL,
                iname text,
                last_refresh TIMESTAMP WITH TIME ZONE
        );

All materialized views will have an entry in this table. It then adds
the plpgsql code to create three functions:

        create_materialized_view(text, text, text) used to create a materialized view
        drop_materialized_view(text) used to delete a materialized view
        refresh_full_materialized_view(text) used to refresh a view

Then it adds the SQL code to create the view and the materialized view:

        CREATE VIEW mviewname_mview AS
        SELECT ... FROM ...;

        SELECT create_materialized_view('mviewname','mviewname_mview', change with the name of the column to be used for the index);

The first argument is the name of the materialized view, the second is
the name of the view on which the materialized view is based, and the
third is the column name on which the index should be built (typically
the primary key). This column is not automatically deduced so you need
to replace its name.

As mentioned above, Ora2Pg only supports snapshot materialized views so
the table will be entirely refreshed by first truncating the table and
then loading all data again from the view:

         refresh_full_materialized_view('mviewname');

To drop the materialized view, you just have to call the
drop_materialized_view() function with the name of the materialized view
as a parameter.

Other configuration directives DEBUG Set it to 1 to enable verbose output.

IMPORT
    You can define common Ora2Pg configuration directives in a single
    file that can be imported into other configuration files with the
    IMPORT configuration directive as follows:

            IMPORT  commonfile.conf

    This will import all configuration directives defined in
    commonfile.conf into the current configuration file.

Exporting views as PostgreSQL tables You can export any Oracle view as a PostgreSQL table simply by setting the TYPE configuration option to TABLE to get the corresponding create table statement. Or use type COPY or INSERT to export the corresponding data. To allow this, you have to specify your views in the VIEW_AS_TABLE configuration option.

Then if Ora2Pg finds the view, it will extract its schema (if
TYPE=TABLE) into a PG create table form, then it will extract the data
(if TYPE=COPY or INSERT) following the view schema.

For example, with the following view:

        CREATE OR REPLACE VIEW product_prices (category_id, product_count, low_price, high_price) AS
        SELECT  category_id, COUNT(*) as product_count,
            MIN(list_price) as low_price,
            MAX(list_price) as high_price
         FROM   product_information
        GROUP BY category_id;

Setting VIEW_AS_TABLE to product_prices and using export type TABLE,
will force Ora2Pg to detect columns' returned types and to generate a
create table statement:

        CREATE TABLE product_prices (
                category_id bigint,
                product_count integer,
                low_price numeric,
                high_price numeric
        );

Data will be loaded following the COPY or INSERT export type and the
view declaration.

You can use the ALLOW and EXCLUDE directives in addition to filter other
objects to export.

Export as Kettle transformation XML files The KETTLE export type is useful if you want to use Pentaho Data Integrator (Kettle) to import data to PostgreSQL. With this type of export, Ora2Pg will generate one XML Kettle transformation file (.ktr) per table and add a line to manually execute the transformation in the output.sql file. For example:

        ora2pg -c ora2pg.conf -t KETTLE -j 12 -a MYTABLE -o load_mydata.sh

will generate one file called 'HR.MYTABLE.ktr' and add a line to the
output file (load_mydata.sh):

        #!/bin/sh

        KETTLE_TEMPLATE_PATH='.'

        JAVAMAXMEM=4096 ./pan.sh -file $KETTLE_TEMPLATE_PATH/HR.MYTABLE.ktr -level Detailed

The -j 12 option will create a template with 12 processes to insert data
into PostgreSQL. It is also possible to specify the number of parallel
queries used to extract data from Oracle with the -J command line option
as follows:

        ora2pg -c ora2pg.conf -t KETTLE -J 4 -j 12 -a EMPLOYEES -o load_mydata.sh

This is only possible if there is a unique key defined on a numeric
column or if you have defined the technical key to be used to split the
query between cores in the DEFINED_PKEY configuration directive. For
example:

        DEFINED_PK      EMPLOYEES:employee_id

This will force the number of Oracle connection copies to 4 and define
the SQL query as follows in the Kettle XML transformation file:

        <sql>SELECT * FROM HR.EMPLOYEES WHERE ABS(MOD(employee_id,${Internal.Step.Unique.Count}))=${Internal.Step.Unique.Number}</sql>

The KETTLE export type requires that the Oracle and PostgreSQL DSN are
defined. You can also activate the TRUNCATE_TABLE directive to force a
truncation of the table before data import.

The KETTLE export type is an original work by Marc Cousin.

Migration Cost Assessment Estimating the cost of migrating from Oracle to PostgreSQL is not easy. To obtain a good assessment of this migration cost, Ora2Pg will inspect all database objects, all functions and stored procedures to detect if there are any objects and PL/SQL code that can not be automatically converted by Ora2Pg.

Ora2Pg has a content analysis mode that inspects the Oracle database to
generate a text report on what the Oracle database contains and what
cannot be exported.

To activate the "analysis and report" mode, you have to use the export
type SHOW_REPORT with in the following command:

        ora2pg -t SHOW_REPORT

Here is a sample report obtained with this command:

        --------------------------------------
        Ora2Pg: Oracle Database Content Report
        --------------------------------------
        Version Oracle Database 10g Enterprise Edition Release 10.2.0.1.0
        Schema  HR
        Size  880.00 MB
     
        --------------------------------------
        Object  Number  Invalid Comments
        --------------------------------------
        CLUSTER   2 0 Clusters are not supported and will not be exported.
        FUNCTION  40  0 Total size of function code: 81992.
        INDEX     435 0 232 index(es) are concerned by the export, others are automatically generated and will
                                        do so on PostgreSQL. 1 bitmap index(es). 230 b-tree index(es). 1 reversed b-tree index(es)
                                        Note that bitmap index(es) will be exported as b-tree index(es) if any. Cluster, domain,
                                        bitmap join and IOT indexes will not be exported at all. Reverse indexes are not exported
                                        too, you may use a trigram-based index (see pg_trgm) or a reverse() function based index
                                        and search. You may also use 'varchar_pattern_ops', 'text_pattern_ops' or 'bpchar_pattern_ops'
                                        operators in your indexes to improve search with the LIKE operator respectively into
                                        varchar, text or char columns.
        MATERIALIZED VIEW 1 0 All materialized view will be exported as snapshot materialized views, they
                                        are only updated when fully refreshed.
        PACKAGE BODY  2 1 Total size of package code: 20700.
        PROCEDURE 7 0 Total size of procedure code: 19198.
        SEQUENCE  160 0 Sequences are fully supported, but all call to sequence_name.NEXTVAL or sequence_name.CURRVAL
                                        will be transformed into NEXTVAL('sequence_name') or CURRVAL('sequence_name').
        TABLE     265 0 1 external table(s) will be exported as standard table. See EXTERNAL_TO_FDW configuration
                                        directive to export as file_fdw foreign tables or use COPY in your code if you just
                                        want to load data from external files. 2 binary columns. 4 unknown types.
        TABLE PARTITION 8 0 Partitions are exported using table inheritance and check constraint. 1 HASH partitions.
                                        2 LIST partitions. 6 RANGE partitions. Note that Hash partitions are not supported.
        TRIGGER   30  0 Total size of trigger code: 21677.
        TYPE      7 1 5 type(s) are concerned by the export, others are not supported. 2 Nested Tables.
                                        2 Object type. 1 Subtype. 1 Type Boby. 1 Type inherited. 1 Varrays. Note that Type
                                        inherited and Subtype are converted as table, type inheritance is not supported.
        TYPE BODY 0 3 Export of type with member method are not supported, they will not be exported.
        VIEW      7 0 Views are fully supported, but if you have updatable views you will need to use
                                        INSTEAD OF triggers.
        DATABASE LINK 1 0 Database links will not be exported. You may try the dblink perl contrib module or use
                                        the SQL/MED PostgreSQL features with the different Foreign Data Wrapper (FDW) extensions.
                                    
        Note: Invalid code will not be exported unless the EXPORT_INVALID configuration directive is activated.

Once the database has been analysed, Ora2Pg, through its ability to
convert SQL and PL/SQL code from Oracle syntax to PostgreSQL, can go
further by estimating the code complexity and time necessary to perform
a full database migration.

To estimate the migration cost in person-days, Ora2Pg allows you to use
a configuration directive called ESTIMATE_COST that you can also enable
at command line:

        --estimate_cost

This feature can only be used with the SHOW_REPORT, FUNCTION, PROCEDURE,
PACKAGE and QUERY export types.

        ora2pg -t SHOW_REPORT  --estimate_cost

The generated report is the same as above but with a new 'Estimated
cost' column as follows:

        --------------------------------------
        Ora2Pg: Oracle Database Content Report
        --------------------------------------
        Version Oracle Database 10g Express Edition Release 10.2.0.1.0
        Schema  HR
        Size  890.00 MB
     
        --------------------------------------
        Object  Number  Invalid Estimated cost  Comments
        --------------------------------------
        DATABASE LINK  3 0 9 Database links will be exported as SQL/MED PostgreSQL's Foreign Data Wrapper (FDW) extensions
                                        using oracle_fdw.
        FUNCTION  2 0 7 Total size of function code: 369 bytes. HIGH_SALARY: 2, VALIDATE_SSN: 3.
        INDEX 21  0 11  11 index(es) are concerned by the export, others are automatically generated and will do so
                                        on PostgreSQL. 11 b-tree index(es). Note that bitmap index(es) will be exported as b-tree
                                        index(es) if any. Cluster, domain, bitmap join and IOT indexes will not be exported at all.
                                        Reverse indexes are not exported too, you may use a trigram-based index (see pg_trgm) or a
                                        reverse() function based index and search. You may also use 'varchar_pattern_ops', 'text_pattern_ops'
                                        or 'bpchar_pattern_ops' operators in your indexes to improve search with the LIKE operator
                                        respectively into varchar, text or char columns.
        JOB 0 0 0 Job are not exported. You may set external cron job with them.
        MATERIALIZED VIEW 1 0 3 All materialized view will be exported as snapshot materialized views, they
                                                are only updated when fully refreshed.
        PACKAGE BODY  0 2 54  Total size of package code: 2487 bytes. Number of procedures and functions found
                                                inside those packages: 7. two_proc.get_table: 10, emp_mgmt.create_dept: 4,
                                                emp_mgmt.hire: 13, emp_mgmt.increase_comm: 4, emp_mgmt.increase_sal: 4,
                                                emp_mgmt.remove_dept: 3, emp_mgmt.remove_emp: 2.
        PROCEDURE 4 0 39  Total size of procedure code: 2436 bytes. TEST_COMMENTAIRE: 2, SECURE_DML: 3,
                                                PHD_GET_TABLE: 24, ADD_JOB_HISTORY: 6.
        SEQUENCE  3 0 0 Sequences are fully supported, but all call to sequence_name.NEXTVAL or sequence_name.CURRVAL
                                                will be transformed into NEXTVAL('sequence_name') or CURRVAL('sequence_name').
        SYNONYM   3 0 4 SYNONYMs will be exported as views. SYNONYMs do not exists with PostgreSQL but a common workaround
                                                is to use views or set the PostgreSQL search_path in your session to access
                                                object outside the current schema.
                                                user1.emp_details_view_v is an alias to hr.emp_details_view.
                                                user1.emp_table is an alias to hr.employees@other_server.
                                                user1.offices is an alias to hr.locations.
        TABLE 17  0 8.5 1 external table(s) will be exported as standard table. See EXTERNAL_TO_FDW configuration
                                        directive to export as file_fdw foreign tables or use COPY in your code if you just want to
                                        load data from external files. 2 binary columns. 4 unknown types.
        TRIGGER 1 1 4 Total size of trigger code: 123 bytes. UPDATE_JOB_HISTORY: 2.
        TYPE  7 1 5 5 type(s) are concerned by the export, others are not supported. 2 Nested Tables. 2 Object type.
                                        1 Subtype. 1 Type Boby. 1 Type inherited. 1 Varrays. Note that Type inherited and Subtype are
                                        converted as table, type inheritance is not supported.
        TYPE BODY 0 3 30  Export of type with member method are not supported, they will not be exported.
        VIEW  1 1 1 Views are fully supported, but if you have updatable views you will need to use INSTEAD OF triggers.
        --------------------------------------
        Total 65  8 162.5 162.5 cost migration units means approximatively 2 man day(s).

The last line shows the total estimated migration cost in person-days
following the number of migration units estimated for each object. Each
migration unit represents approximately five minutes for a PostgreSQL
expert. If this is your first migration, you can increase it with the
configuration directive COST_UNIT_VALUE or the --cost_unit_value command
line option:

        ora2pg -t SHOW_REPORT  --estimate_cost --cost_unit_value 10

Ora2Pg is also able to give you a migration difficulty level assessment.
Here's a sample:

Migration level: B-5

    Migration levels:
        A - Migration that might be run automatically
        B - Migration with code rewrite and a person-days cost up to 5 days
        C - Migration with code rewrite and a person-days cost above 5 days
    Technical levels:
        1 = trivial: no stored functions and no triggers
        2 = easy: no stored functions but with triggers, no manual rewriting
        3 = simple: stored functions and/or triggers, no manual rewriting
        4 = manual: no stored functions but with triggers or views with code rewriting
        5 = difficult: stored functions and/or triggers with code rewriting

This assessment consists of a letter A or B to specify whether the
migration needs manual rewriting or not, and a number from 1 up to 5 to
indicate a technical difficulty level. You have an additional option
--human_days_limit to specify the number of person-days limit where the
migration level should be set to C to indicate that it needs a huge
amount of work and full project management with migration support.
Default is 10 person-days. You can use the configuration directive
HUMAN_DAYS_LIMIT to change this default value permanently.

This feature has been developed to help you or your boss to decide which
database to migrate first and the team that must be mobilized to conduct
the migration.

Global Oracle and MySQL migration assessment Ora2Pg comes with a script ora2pg_scanner that can be used when you have a huge number of instances and schemas to scan for migration assessment.

Usage: ora2pg_scanner -l CSVFILE [-o OUTDIR]

   -b | --binpath DIR: full path to directory where the ora2pg binary resides.
                Might be useful only on Windows OS.
   -c | --config FILE: set custom configuration file to use, otherwise ora2pg
                will use the default: /etc/ora2pg/ora2pg.conf.
   -l | --list FILE : CSV file containing a list of databases to scan with
                all required information. The first line of the file
                can contain the following header that describes the
                format that must be used:

                "type","schema/database","dsn","user","password"

   -o | --outdir DIR : (optional) by default all reports will be dumped to a
                directory named 'output', it will be created automatically.
                If you want to change the name of this directory, set the name
                at second argument.

   -t | --test : just try all connections by retrieving the required schema
                 or database name. Useful to validate your CSV list file.
   -u | --unit MIN : redefine globally the migration cost unit value in minutes.
                 Default is taken from the ora2pg.conf (default 5 minutes).

   Here is a full example of a CSV databases list file:

        "type","schema/database","dsn","user","password"
        "MYSQL","sakila","dbi:mysql:host=192.168.1.10;database=sakila;port=3306","root","secret"
        "ORACLE","HR","dbi:Oracle:host=192.168.1.10;sid=XE;port=1521","system","manager"
        "MSSQL","HR","dbi:ODBC:driver=msodbcsql18;server=srv.database.windows.net;database=testdb","system","manager"

   The CSV field separator must be a comma.

   Note that if you want to scan all schemas from an Oracle instance, you just
   have to leave the schema field empty. Ora2Pg will automatically detect all
   available schemas and generate a report for each one. Of course, you need to
   use a connection user with enough privileges to be able to scan all schemas.
   For example:

        "ORACLE","","dbi:Oracle:host=192.168.1.10;sid=XE;port=1521","system","manager"
        "MSSQL","","dbi:ODBC:driver=msodbcsql18;server=srv.database.windows.net;database=testdb","usrname","passwd"

   will generate a report for all schemas in the XE instance. Note that in this
   case the SCHEMA directive in ora2pg.conf must not be set.

It will generate a CSV file with the assessment result, one line per
schema or database and a detailed HTML report for each database scanned.

Hint: Use the -t | --test option beforehand to test all your connections
in your CSV file.

For Windows users, you must use the -b command line option to set the
directory where ora2pg_scanner resides, otherwise the ora2pg command
calls will fail.

In the migration assessment details about functions, Ora2Pg always
includes by default 2 migration units for TEST and 1 unit for SIZE per
1000 characters in the code. This means that by default it will add 15
minutes to the migration assessment per function. Obviously, if you have
unit tests or very simple functions this will not represent the actual
migration time.

Migration assessment method Migration unit scores given to each type of Oracle database object are defined in the Perl library lib/Ora2Pg/PLSQL.pm in the %OBJECT_SCORE variable definition.

The number of PL/SQL lines associated with a migration unit is also
defined in this file in the $SIZE_SCORE variable value.

The number of migration units associated with each PL/SQL code
difficulty can be found in the same Perl library lib/Ora2Pg/PLSQL.pm in
the hash %UNCOVERED_SCORE initialization.

This assessment method is a work in progress, so I'm expecting feedback
on migration experiences to refine the scores/units attributed in these
variables.

Improving indexes and constraints creation speed Using the LOAD export type and a file containing SQL orders to perform, it is possible to dispatch those orders over multiple PostgreSQL connections. To be able to use this feature, the PG_DSN, PG_USER and PG_PWD must be set. Then:

        ora2pg -t LOAD -c config/ora2pg.conf -i schema/tables/INDEXES_table.sql -j 4

will dispatch index creation over 4 simultaneous PostgreSQL connections.

This will considerably accelerate this part of the migration process
with huge data sizes.

Exporting LONG RAW If you still have columns defined as LONG RAW, Ora2Pg will not be able to export these kinds of data. The OCI library fails to export them and always returns the same first record. To be able to export the data you need to transform the field as BLOB by creating a temporary table before migrating data. For example, the Oracle table:

        SQL> DESC TEST_LONGRAW
         Name                 NULL ?   Type
         -------------------- -------- ----------------------------
         ID                            NUMBER
         C1                            LONG RAW

needs to be "translated" into a table using BLOB as follows:

        CREATE TABLE test_blob (id NUMBER, c1 BLOB);

And then copy the data with the following INSERT query:

        INSERT INTO test_blob SELECT id, to_lob(c1) FROM test_longraw;

Then you just have to exclude the original table from the export (see
EXCLUDE directive) and to rename the new temporary table on the fly
using the REPLACE_TABLES configuration directive.

Global variables Oracle allows the use of global variables defined in packages. Ora2Pg will export these variables for PostgreSQL as user-defined custom variables available in a session. Oracle variable assignments are exported as calls to:

    PERFORM set_config('pkgname.varname', value, false);

Use of these variables in the code is replaced by:

    current_setting('pkgname.varname')::global_variables_type;

where global_variables_type is the type of the variable extracted from
the package definition.

If the variable is a constant or has a default value assigned at
declaration, Ora2Pg will create a file global_variables.conf with the
definition to include in the postgresql.conf file so that their values
will already be set at database connection. Note that the value can
always be modified by the user so you can not have exactly a constant.

Hints Converting your queries with Oracle style outer join (+) syntax to ANSI standard SQL at the Oracle side can save you a lot of time for the migration. You can use TOAD Query Builder to re-write these using the proper ANSI syntax, see: http://www.toadworld.com/products/toad-for-oracle/f/10/t/9518.aspx

There's also an alternative with SQL Developer Data Modeler, see
http://www.thatjeffsmith.com/archive/2012/01/sql-developer-data-modeler-
quick-tip-use-oracle-join-syntax-or-ansi/

Toad is also able to rewrite the native Oracle DECODE() syntax into ANSI
standard SQL CASE statement. You can find some slides about this in a
presentation given at PgConf.RU:
http://ora2pg.darold.net/slides/ora2pg_the_hard_way.pdf

Test the migration The type of action called s you to check that all objects from Oracle database have been created under PostgreSQL. Of course PG_DSN must be set to be able to check PostgreSQL side.

Note that this feature respects the schema name limitation if
EXPORT_SCHEMA and SCHEMA or PG_SCHEMA are defined. If only EXPORT_SCHEMA
is set all schemas from Oracle and PostgreSQL are scanned. You can
filter to a single schema using SCHEMA and/or PG_SCHEMA but you can not
filter on a list of schemas. To test a list of schemas you will have to
repeat the calls to Ora2Pg by specifying a single schema each time.

For example command:

        ora2pg -t TEST -c config/ora2pg.conf > migration_diff.txt

Will create a file containing the report of all objects and row count on
both sides, Oracle and PostgreSQL, with an error section giving you the
details of the differences for each kind of object. Here is a sample
result:

        [TEST INDEXES COUNT]
        ORACLEDB:DEPARTMENTS:2
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:6
        POSTGRES:employees:6
        [ERRORS INDEXES COUNT]
        Table departments doesn't have the same number of indexes in Oracle (2) and in PostgreSQL (1).

        [TEST UNIQUE CONSTRAINTS COUNT]
        ORACLEDB:DEPARTMENTS:1
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS UNIQUE CONSTRAINTS COUNT]
        OK, Oracle and PostgreSQL have the same number of unique constraints.

        [TEST PRIMARY KEYS COUNT]
        ORACLEDB:DEPARTMENTS:1
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS PRIMARY KEYS COUNT]
        OK, Oracle and PostgreSQL have the same number of primary keys.

        [TEST CHECK CONSTRAINTS COUNT]
        ORACLEDB:DEPARTMENTS:1
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS CHECK CONSTRAINTS COUNT]
        OK, Oracle and PostgreSQL have the same number of check constraints.

        [TEST NOT NULL CONSTRAINTS COUNT]
        ORACLEDB:DEPARTMENTS:1
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS NOT NULL CONSTRAINTS COUNT]
        OK, Oracle and PostgreSQL have the same number of not null constraints.

        [TEST COLUMN DEFAULT VALUE COUNT]
        ORACLEDB:DEPARTMENTS:1
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS COLUMN DEFAULT VALUE COUNT]
        OK, Oracle and PostgreSQL have the same number of column default value.

        [TEST IDENTITY COLUMN COUNT]
        ORACLEDB:DEPARTMENTS:1
        POSTGRES:departments:1
        ORACLEDB:EMPLOYEES:0
        POSTGRES:employees:0
        [ERRORS IDENTITY COLUMN COUNT]
        OK, Oracle and PostgreSQL have the same number of identity column.

        [TEST FOREIGN KEYS COUNT]
        ORACLEDB:DEPARTMENTS:0
        POSTGRES:departments:0
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS FOREIGN KEYS COUNT]
        OK, Oracle and PostgreSQL have the same number of foreign keys.

        [TEST TABLE COUNT]
        ORACLEDB:TABLE:2
        POSTGRES:TABLE:2
        [ERRORS TABLE COUNT]
        OK, Oracle and PostgreSQL have the same number of TABLE.

        [TEST TABLE TRIGGERS COUNT]
        ORACLEDB:DEPARTMENTS:0
        POSTGRES:departments:0
        ORACLEDB:EMPLOYEES:1
        POSTGRES:employees:1
        [ERRORS TABLE TRIGGERS COUNT]
        OK, Oracle and PostgreSQL have the same number of table triggers.

        [TEST TRIGGER COUNT]
        ORACLEDB:TRIGGER:2
        POSTGRES:TRIGGER:2
        [ERRORS TRIGGER COUNT]
        OK, Oracle and PostgreSQL have the same number of TRIGGER.

        [TEST VIEW COUNT]
        ORACLEDB:VIEW:1
        POSTGRES:VIEW:1
        [ERRORS VIEW COUNT]
        OK, Oracle and PostgreSQL have the same number of VIEW.

        [TEST MVIEW COUNT]
        ORACLEDB:MVIEW:0
        POSTGRES:MVIEW:0
        [ERRORS MVIEW COUNT]
        OK, Oracle and PostgreSQL have the same number of MVIEW.

        [TEST SEQUENCE COUNT]
        ORACLEDB:SEQUENCE:1
        POSTGRES:SEQUENCE:0
        [ERRORS SEQUENCE COUNT]
        SEQUENCE does not have the same count in Oracle (1) and in PostgreSQL (0).

        [TEST TYPE COUNT]
        ORACLEDB:TYPE:1
        POSTGRES:TYPE:0
        [ERRORS TYPE COUNT]
        TYPE does not have the same count in Oracle (1) and in PostgreSQL (0).

        [TEST FDW COUNT]
        ORACLEDB:FDW:0
        POSTGRES:FDW:0
        [ERRORS FDW COUNT]
        OK, Oracle and PostgreSQL have the same number of FDW.

        [TEST FUNCTION COUNT]
        ORACLEDB:FUNCTION:3
        POSTGRES:FUNCTION:3
        [ERRORS FUNCTION COUNT]
        OK, Oracle and PostgreSQL have the same number of functions.

        [TEST SEQUENCE VALUES]
        ORACLEDB:EMPLOYEES_NUM_SEQ:1285
        POSTGRES:employees_num_seq:1285
        [ERRORS SEQUENCE VALUES COUNT]
        OK, Oracle and PostgreSQL have the same values for sequences

        [TEST ROWS COUNT]
        ORACLEDB:DEPARTMENTS:27
        POSTGRES:departments:27
        ORACLEDB:EMPLOYEES:854
        POSTGRES:employees:854
        [ERRORS ROWS COUNT]
        OK, Oracle and PostgreSQL have the same number of rows.

Data validation Data validation consists of comparing data retrieved from a foreign table pointing to the source Oracle table and a local PostgreSQL table resulting from the data export.

To run data validation you can use a direct connection like any other
Ora2Pg action but you can also use the oracle_fdw, mysql_fdw or tds_fdw
extension provided that FDW_SERVER and PG_DSN configuration directives
are set.

By default, Ora2Pg will extract the first 10000 rows from both sides,
you can change this value using directive DATA_VALIDATION_ROWS. When it
is set to zero all rows of the tables will be compared.

Data validation requires that the table has a primary key or unique
index and that the key column is not a LOB. Rows will be sorted using
this unique key. Due to differences in sort behavior between Oracle and
PostgreSQL, if the collation of unique key columns in PostgreSQL is not
'C', the sort order can be different compared to Oracle. In this case
the data validation will fail.

Data validation must be done before any data is modified.

Ora2Pg will stop comparing two tables after DATA_VALIDATION_ROWS is
reached or after 10 errors have been encountered, results are dumped in
a file named "data_validation.log" written in the current directory by
default. The number of errors before stopping the diff between rows can
be controlled using the configuration directive DATA_VALIDATION_ERROR.
All rows with errors are printed to the output file for your analysis.

It is possible to parallelize data validation by using -P option or the
corresponding configuration directive PARALLEL_TABLES in ora2pg.conf.

Use of System Change Number (SCN) Ora2Pg is able to export data as of a specific SCN. You can set it at command line using the -S or --scn option. You can give a specific SCN or if you want to use the current SCN at first connection time set the value to 'current'. In this last case if connection user has the "SELECT ANY DICTIONARY" or the "SELECT_CATALOG_ROLE" role, the current SCN is looked up in the v$database view.

Example of use:

    ora2pg -c ora2pg.conf -t COPY --scn 16605281

This adds the following clause to the query used to retrieve data for
example:

    AS OF SCN 16605281

You can also use the --scn option to use the Oracle flashback capability
by specifying a timestamp expression instead of a SCN. For example:

    ora2pg -c ora2pg.conf -t COPY --scn "TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')"

This will add the following clause to the query used to retrieve data:

    AS OF TIMESTAMP TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')

or for example to only retrieve yesterday's data:

    ora2pg -c ora2pg.conf -t COPY --scn "SYSDATE - 1"

Change Data Capture (CDC) Ora2Pg does not have a feature which allows importing data and only applying changes after the first import. But you can use the --cdc_ready option to export data with registration of the SCN at the time of the table export. All SCNs per table are written to a file named TABLES_SCN.log by default, it can be changed using -C | --cdc_file option.

These SCNs registered per table during COPY or INSERT export can be used
with a CDC tool. The format of the file is tablename:SCN per line.

Importing BLOB as large objects By default Ora2Pg imports Oracle BLOB as bytea, the destination column is created using the bytea data type. If you want to use large objects instead of bytea, just add the --blob_to_lo option to the ora2pg command. It will create the destination column as data type Oid and will save the BLOB as a large object using the lo_from_bytea() function. The Oid returned by the call to lo_from_bytea() is inserted in the destination column instead of a bytea. Because of the use of the function this option can only be used with actions SHOW_COLUMN, TABLE and INSERT. Action COPY is not allowed.

If you want to use COPY or have huge size BLOBs ( > 1GB) than cannot be
imported using lo_from_bytea() you can add option --lo_import to the
ora2pg command. This will allow importing data in two passes.

1) Export data using COPY or INSERT will set the Oid destination column
for BLOB to value 0 and save the BLOB value into a dedicated file. It
will also create a Shell script to import the BLOB files into the
database using psql command \lo_import and to update the table Oid
column to the returned large object Oid. The script is named
lo_import-TABLENAME.sh

2) Execute all scripts lo_import-TABLENAME.sh after setting the
environment variables PGDATABASE and optionally PGHOST, PGPORT, PGUSER,
etc. if they do not correspond to the default values for libpq.

You might also execute manually a VACUUM FULL on the table to remove the
bloat created by the table update.

Limitation: the table must have a primary key, it is used to set the
WHERE clause to update the Oid column after the large object import.
Importing BLOB using this second method (--lo_import) is very slow so it
should be reserved for rows where the BLOB > 1GB. For all other rows use
the option --blob_to_lo. To filter the rows you can use the WHERE
configuration directive in ora2pg.conf.

SUPPORT Author / Maintainer Gilles Darold

Please report any bugs, patches, help, etc. to <gilles AT darold DOT
net>.

Feature Requests If you need new features, please let me know at . This helps a lot in developing a better/more useful tool.

How To Contribute Any contribution to build a better tool is welcome. Just send me your ideas, features requests or patches and they will be applied.

        This program is free software: you can redistribute it and/or modify
        it under the terms of the GNU General Public License as published by
        the Free Software Foundation, either version 3 of the License, or
        any later version.

        This program is distributed in the hope that it will be useful,
        but WITHOUT ANY WARRANTY; without even the implied warranty of
        MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
        GNU General Public License for more details.

        You should have received a copy of the GNU General Public License
        along with this program.  If not, see < http://www.gnu.org/licenses/ >.

ACKNOWLEDGEMENTS Many thanks to all the great contributors. See changelog for all acknowledgments.

AlDanial/cloc

cloc counts blank lines, comment lines, and physical lines of source code in many programming languages.

cloc

Count Lines of Code

cloc counts blank lines, comment lines, and physical lines of source code in many programming languages.

Latest release: v2.08 (Jan. 24, 2026)

cloc moved to GitHub in September 2015 after being hosted at http://cloc.sourceforge.net/ since August 2006.

Quick Start

Step 1: Install cloc (see Install from Github Releases and Install via package manager) or run cloc's docker image. The Windows executable has no requirements. The source version of cloc requires a Perl interpreter, and the Docker version of cloc requires a Docker installation.

Step 2: Open a terminal (cmd.exe on Windows).

Step 3: Invoke cloc to count your source files, directories, archives, or git commits. The executable name differs depending on whether you use the development source version (cloc), source for a released version (cloc-2.08.pl) or a Windows executable (cloc-2.08.exe).

On this page, cloc is the generic term used to refer to any of these.

Include Security has a YouTube video showing the steps in action.

a file

prompt> cloc hello.c
       1 text file.
       1 unique file.
       0 files ignored.

https://github.com/AlDanial/cloc v 1.65  T=0.04 s (28.3 files/s, 340.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                                1              0              7              5
-------------------------------------------------------------------------------

a directory

prompt> cloc gcc-5.2.0/gcc/c
      16 text files.
      15 unique files.
       3 files ignored.

https://github.com/AlDanial/cloc v 1.65  T=0.23 s (57.1 files/s, 188914.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                               10           4680           6621          30812
C/C++ Header                     3             99            286            496
-------------------------------------------------------------------------------
SUM:                            13           4779           6907          31308
-------------------------------------------------------------------------------

an archive

We'll pull cloc's source zip file from GitHub, then count the contents:

prompt> wget https://github.com/AlDanial/cloc/archive/master.zip

prompt> cloc master.zip
https://github.com/AlDanial/cloc v 1.65  T=0.07 s (26.8 files/s, 141370.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Perl                             2            725           1103           8713
-------------------------------------------------------------------------------
SUM:                             2            725           1103           8713
-------------------------------------------------------------------------------

a git repository, using a specific commit

This example uses code from PuDB, a fantastic Python debugger.

prompt> git clone https://github.com/inducer/pudb.git

prompt> cd pudb

prompt> cloc 6be804e07a5db
      48 text files.
      41 unique files.                              
       8 files ignored.

github.com/AlDanial/cloc v 1.99  T=0.04 s (1054.9 files/s, 189646.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          28           1519            728           4659
reStructuredText                 6            102             20            203
YAML                             2              9              2             75
Bourne Shell                     3              6              0             17
Text                             1              0              0             11
make                             1              4              6             10
-------------------------------------------------------------------------------
SUM:                            41           1640            756           4975
-------------------------------------------------------------------------------

each subdirectory of a particular directory

Say you have a directory with three different git-managed projects, Project0, Project1, and Project2. You can use your shell's looping capability to count the code in each. This example uses bash (scroll down for cmd.exe example):

prompt> for d in ./*/ ; do (cd "$d" && echo "$d" && cloc --vcs git); done
./Project0/
7 text files.
       7 unique files.
       1 file ignored.

github.com/AlDanial/cloc v 1.71  T=0.02 s (390.2 files/s, 25687.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
D                                4             61             32            251
Markdown                         1              9              0             38
make                             1              0              0              4
-------------------------------------------------------------------------------
SUM:                             6             70             32            293
-------------------------------------------------------------------------------
./Project1/
       7 text files.
       7 unique files.
       0 files ignored.

github.com/AlDanial/cloc v 1.71  T=0.02 s (293.0 files/s, 52107.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Go                               7            165            282            798
-------------------------------------------------------------------------------
SUM:                             7            165            282            798
-------------------------------------------------------------------------------
./Project2/
      49 text files.
      47 unique files.
      13 files ignored.

github.com/AlDanial/cloc v 1.71  T=0.10 s (399.5 files/s, 70409.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          33           1226           1026           3017
C                                4            327            337            888
Markdown                         1             11              0             28
YAML                             1              0              2             12
-------------------------------------------------------------------------------
SUM:                            39           1564           1365           3945
-------------------------------------------------------------------------------

each subdirectory of a particular directory (Windows/cmd.exe)

for /D %I in (.\*) do cd %I && cloc --vcs git && cd ..

Overview

cloc counts blank lines, comment lines, and physical lines of source code in many programming languages. Given two versions of a code base, cloc can compute differences in blank, comment, and source lines. It is written entirely in Perl with no dependencies outside the standard distribution of Perl v5.6 and higher (code from some external modules is embedded within cloc) and so is quite portable. cloc is known to run on many flavors of Linux, FreeBSD, NetBSD, OpenBSD, macOS, AIX, HP-UX, Solaris, IRIX, z/OS, and Windows. (To run the Perl source version of cloc on Windows one needs ActiveState Perl 5.6.1 or higher, Strawberry Perl, Windows Subsystem for Linux, Cygwin, MobaXTerm with the Perl plug-in installed, or a mingw environment and terminal such as provided by Git for Windows. Alternatively one can use the Windows binary of cloc generated with PAR::Packer to run on Windows computers that have neither Perl nor Cygwin.)

In addition to counting code in individual text files, directories, and git repositories, cloc can also count code in archive files such as .tar (including compressed versions), .zip, Python wheel .whl, Jupyter notebook .ipynb, source RPMs .rpm or .src (requires rpm2cpio), and Debian .deb files (requires dpkg-deb).

cloc contains code from David Wheeler's SLOCCount, Damian Conway and Abigail's Perl module Regexp::Common, Sean M. Burke's Perl module Win32::Autoglob, and Tye McQueen's Perl module Algorithm::Diff. Language scale factors were derived from Mayes Consulting, LLC web site http://softwareestimator.com/IndustryData2.htm.

New releases nominally appear every six months.

Install from Github Releases

Grab the latest release of cloc from the Releases section of this repository.

Source version on Linux/macOS

Save the latest source file, for example cloc-2.08.pl, as cloc (if you prefer the shorter command name) somewhere on your PATH. After downloading, make the file executable:

cd ~/Downloads
mv cloc-2.08.pl cloc
chmod a+rx cloc
./cloc --version     # make sure it runs
mv cloc ~/bin        # for example, if ~/bin is in $PATH

Source version on Windows

You'll need a Perl interpreter such as Strawberry Perl installed to run the source version of cloc. After downloading the cloc source file, open a command prompt or PowerShell window, navigate to the download directory (C:\TEMP in the example below), then test cloc with:

cd C:\TEMP>
C:TEMP\> perl cloc-2.08.pl --version

Binary version on Windows

Download the latest released Windows executable, for example cloc-2.08.exe and save it as cloc.exe (if desired) in a directory on your PATH.

There is no binary version for Linux or macOS.

Install via package manager

Depending your operating system, one of these installation methods may work for you (all but the last two entries for Windows require a Perl interpreter):

npm install -g cloc              # https://www.npmjs.com/package/cloc
sudo apt install cloc            # Debian, Ubuntu
sudo yum install cloc            # Red Hat, Fedora
sudo dnf install cloc            # Fedora 22 or later
sudo pacman -S cloc              # Arch
yay -S cloc-git                  # Arch AUR (latest git version)
sudo emerge -av dev-util/cloc    # Gentoo https://packages.gentoo.org/packages/dev-util/cloc
sudo apk add cloc                # Alpine Linux
doas pkg_add cloc                # OpenBSD
sudo pkg install cloc            # FreeBSD
sudo port install cloc           # macOS with MacPorts
brew install cloc                # macOS with Homebrew
winget install AlDanial.Cloc     # Windows with winget (might not work, ref https://github.com/AlDanial/cloc/issues/849)
choco install cloc               # Windows with Chocolatey
scoop install cloc               # Windows with Scoop

Note: I don't control any of these packages. If you encounter a bug in cloc using one of the above packages, try with cloc pulled from the latest stable release here on GitHub (link follows below) before submitting a problem report.

Run via docker

These docker commands count lines of code in and below the current directory:

docker run --rm -v $PWD:/tmp aldanial/cloc .

Run via docker on git-bash

docker run --rm -v "/$(pwd -W)":/tmp aldanial/cloc .

Development version

Download the cloc source code at https://github.com/AlDanial/cloc/raw/master/cloc and save it as the file cloc (or cloc.pl, or whatever executable name you wish). The next step depends on the operating system you're using.

License

cloc is licensed under the GNU General Public License, v 2, excluding portions which are copied from other sources. Code copied from the Regexp::Common, Win32::Autoglob, and Algorithm::Diff Perl modules is subject to the Artistic License.

Why Use cloc?

cloc has many features that make it easy to use, thorough, extensible, and portable:

Exists as a single, self-contained file that requires minimal installation effort---just download the file and run it.
Can read language comment definitions from a file and thus potentially work with computer languages that do not yet exist.
Allows results from multiple runs to be summed together by language and by project.
Can produce results in a variety of formats: plain text, Markdown, SQL, JSON, XML, YAML, comma separated values.
Can count code within compressed archives (tar balls, Zip files, Java .ear files).
Has numerous troubleshooting options.
Handles file and directory names with spaces and other unusual characters.
Has no dependencies outside the standard Perl distribution.
Runs on Linux, FreeBSD, NetBSD, OpenBSD, macOS, AIX, HP-UX, Solaris, IRIX, and z/OS systems that have Perl 5.6 or higher. The source version runs on Windows with either ActiveState Perl, Strawberry Perl, Cygwin, or MobaXTerm+Perl plugin. Alternatively on Windows one can run the Windows binary which has no dependencies.

Other Counters

If cloc does not suit your needs here are other freely available counters to consider:

Other references:

QSM's directory of code counting tools.
The Wikipedia entry for source code line counts.

Regexp::Common, Digest::MD5, Win32::Autoglob, Algorithm::Diff

Although cloc does not need Perl modules outside those found in the standard distribution, cloc does rely on a few external modules. Code from three of these external modules--Regexp::Common, Win32::Autoglob, and Algorithm::Diff--is embedded within cloc. A fourth module, Digest::MD5, is used only if it is available. If cloc finds Regexp::Common or Algorithm::Diff installed locally it will use those installation. If it doesn't, cloc will install the parts of Regexp::Common and/or Algorithm:Diff it needs to temporary directories that are created at the start of a cloc run then removed when the run is complete. The necessary code from Regexp::Common v2.120 and Algorithm::Diff v1.1902 are embedded within the cloc source code (see subroutines Install_Regexp_Common() and Install_Algorithm_Diff() ). Only three lines are needed from Win32::Autoglob and these are included directly in cloc.

Additionally, cloc will use Digest::MD5 to validate uniqueness among equally-sized input files if Digest::MD5 is installed locally.

A parallel processing option, --processes=*N*, was introduced with cloc version 1.76 to enable faster runs on multi-core machines. However, to use it, one must have the module Parallel::ForkManager installed. This module does not work reliably on Windows so parallel processing will only work on Unix-like operating systems.

The Windows binary is built on a computer that has both Regexp::Common and Digest::MD5 installed locally.

Building a Windows Executable

Create your own executable

The most robust option for creating a Windows executable of cloc is to use ActiveState's Perl Development Kit. It includes a utility, perlapp, which can build stand-alone Windows, Mac, and Linux binaries of Perl source code.

perl2exe will also do the trick. If you do have perl2exe, modify lines 84-87 in the cloc source code for a minor code modification that is necessary to make a cloc Windows executable.

Otherwise, to build a Windows executable with pp from PAR::Packer, first install a Windows-based Perl distribution (for example Strawberry Perl or ActivePerl) following their instructions. Next, open a command prompt, aka a DOS window and install the PAR::Packer module. Finally, invoke the newly installed pp command with the cloc source code to create an .exe file:

C:> cpan -i Digest::MD5
C:> cpan -i Regexp::Common
C:> cpan -i Algorithm::Diff
C:> cpan -i PAR::Packer
C:> cpan -i Win32::LongPath
C:> pp -M Win32::LongPath -M Encode::Unicode -M Digest::MD5 -c -x -o cloc-2.08.exe cloc-2.08.pl

A variation on the instructions above is if you installed the portable version of Strawberry Perl, you will need to run portableshell.bat first to properly set up your environment.

The Windows executable in the Releases section, cloc-2.08.exe, was built on a 64 bit Windows 11 computer using Strawberry Perl 5.32.1 and PAR::Packer to build the .exe.

Is the Windows executable safe to run? Does it have malware?

Ideally, no one would need the Windows executable because they have a Perl interpreter installed on their machines and can run the cloc source file. On centrally-managed corporate Windows machines, however, this this may be difficult or impossible.

The Windows executable distributed with cloc is provided as a best-effort of a virus and malware-free .exe. You are encouraged to run your own virus scanners against the executable and also check sites such https://www.virustotal.com/ . The entries for recent versions are:

cloc-2.08.exe: https://www.virustotal.com/gui/file/4529557d957ade0dd45746eae10e9c51ee01061bb617eeeab256672faf6e42c6?nocache=1

cloc-2.06.exe: https://www.virustotal.com/gui/file/bbe48de9102d0f2520d292d65897001c1d068340eb7cd74dd1ee30c1a9091c4a?nocache=1

cloc-2.04.exe: https://www.virustotal.com/gui/file/89cda0038bf4e13c6c13ebc1e60bec4dfad362e69ac8a5b8e2d5ebe3020359e1

cloc-2.02-winget.exe: (includes PR 850 to allow running from a symlink on Windows) https://www.virustotal.com/gui/file/be033061e091fea48a5bc9e8964cee0416ddd5b34bd5226a1c9aa4b30bdba66a?nocache=1

cloc-2.02.exe: https://www.virustotal.com/gui/file/369ed76125f7399cd582d169adf39a2e08ae5066031fea0cc8b2836ea50e7ce2?nocache=1

cloc-2.00.exe: https://www.virustotal.com/gui/file/7a234ef0cb495de1b5776acf88c5554e2bab1fb02725a5fb85756a6db3121c1f

Why is the Windows executable so large?

Windows executables of cloc versions 1.60 and earlier, created with perl2exe as noted above, are about 1.6 MB, while versions 1.62 and 1.54, created with PAR::Packer, are 11 MB. Version 1.66, built with a newer version of PAR::Packer, is about 5.5 MB. Why are the PAR::Packer, executables so much larger than those built with perl2exe? My theory is that perl2exe uses smarter tree pruning logic than PAR::Packer, but that's pure speculation.

Basic Use

cloc is a command line program that takes file, directory, and/or archive names as inputs. Here's an example of running cloc against the Perl v5.22.0 source distribution:

prompt> cloc perl-5.22.0.tar.gz
    5605 text files.
    5386 unique files.
    2176 files ignored.

https://github.com/AlDanial/cloc v 1.65  T=25.49 s (134.7 files/s, 51980.3 lines/s)
-----------------------------------------------------------------------------------
Language                         files          blank        comment           code
-----------------------------------------------------------------------------------
Perl                              2892         136396         184362         536445
C                                  130          24676          33684         155648
C/C++ Header                       148           9766          16569         147858
Bourne Shell                       112           4044           6796          42668
Pascal                               8            458           1603           8592
XML                                 33            142              0           2410
YAML                                49             20             15           2078
C++                                 10            313            277           2033
make                                 4            426            488           1986
Prolog                              12            438              2           1146
JSON                                14              1              0           1037
yacc                                 1             85             76            998
Windows Message File                 1            102             11            489
DOS Batch                           14             92             41            389
Windows Resource File                3             10              0             85
D                                    1              5              7              8
Lisp                                 2              0              3              4
-----------------------------------------------------------------------------------
SUM:                              3434         176974         243934         903874
-----------------------------------------------------------------------------------

To run cloc on Windows computers, open up a command (aka DOS) window and invoke cloc.exe from the command line there. Alternatively, try ClocViewer, the GUI wrapper around cloc found at https://github.com/Roemer/ClocViewer.

See also https://github.com/jmensch1/codeflower for a graphical rendering of cloc results.

GUI Frontends

Several GUI frontends to cloc are available:

Options

Full options list (cloc --help) (click the triangle to see all option)

prompt> cloc --help

Usage: cloc [options] <file(s)/dir(s)/git hash(es)> | <set 1> <set 2> | <report files>

 Count, or compute differences of, physical lines of source code in the
 given files (may be archives such as compressed tarballs or zip files,
 or git commit hashes or branch names) and/or recursively below the
 given directories.

 Input Options
   --extract-with=<cmd>      This option is only needed if cloc is unable
                             to figure out how to extract the contents of
                             the input file(s) by itself.
                             Use <cmd> to extract binary archive files (e.g.:
                             .tar.gz, .zip, .Z).  Use the literal '>FILE<' as
                             a stand-in for the actual file(s) to be
                             extracted.  For example, to count lines of code
                             in the input files
                                gcc-4.2.tar.gz  perl-5.8.8.tar.gz
                             on Unix use
                               --extract-with='gzip -dc >FILE< | tar xf -'
                             or, if you have GNU tar,
                               --extract-with='tar zxf >FILE<'
                             and on Windows use, for example:
                               --extract-with="\"c:\Program Files\WinZip\WinZip32.exe\" -e -o >FILE< ."
                             (if WinZip is installed there).
   --list-file=<file>        Take the list of file and/or directory names to
                             process from <file>, which has one file/directory
                             name per line.  Only exact matches are counted;
                             relative path names will be resolved starting from
                             the directory where cloc is invoked.  Set <file>
                             to - to read file names from a STDIN pipe.
                             See also --exclude-list-file.
   --diff-list-file=<file>   Take the pairs of file names to be diff'ed from
                             <file>, whose format matches the output of
                             --diff-alignment.  (Run with that option to
                             see a sample.)  The language identifier at the
                             end of each line is ignored.  This enables --diff
                             mode and bypasses file pair alignment logic.
   --vcs=<VCS>               Invoke a system call to <VCS> to obtain a list of
                             files to work on.  If <VCS> is 'git', then will
                             invoke 'git ls-files' to get a file list and
                             'git submodule status' to get a list of submodules
                             whose contents will be ignored.  See also --git
                             which accepts git commit hashes and branch names.
                             If <VCS> is 'svn' then will invoke 'svn list -R'.
                             The primary benefit is that cloc will then skip
                             files explicitly excluded by the versioning tool
                             in question, ie, those in .gitignore or have the
                             svn:ignore property.
                             Alternatively <VCS> may be any system command
                             that generates a list of files.
                             Note:  cloc must be in a directory which can read
                             the files as they are returned by <VCS>.  cloc will
                             not download files from remote repositories.
                             'svn list -R' may refer to a remote repository
                             to obtain file names (and therefore may require
                             authentication to the remote repository), but
                             the files themselves must be local.
                             Setting <VCS> to 'auto' selects between 'git'
                             and 'svn' (or neither) depending on the presence
                             of a .git or .svn subdirectory below the directory
                             where cloc is invoked.  --files-from is a synonym
                             for --vcs.
   --unicode                 Check binary files to see if they contain Unicode
                             expanded ASCII text.  This causes performance to
                             drop noticeably.

 Processing Options
   --autoconf                Count .in files (as processed by GNU autoconf) of
                             recognized languages.  See also --no-autogen.
   --by-file                 Report results for every source file encountered.
   --by-file-by-lang         Report results for every source file encountered
                             in addition to reporting by language.
   --config <file>           Read command line switches from <file> instead of
                             the default location of /home/al/.config/cloc/options.txt.
                             The file should contain one switch, along with
                             arguments (if any), per line.  Blank lines and lines
                             beginning with '#' are skipped.  Options given on
                             the command line take priority over entries read from
                             the file.
   --count-and-diff <set1> <set2>
                             First perform direct code counts of source file(s)
                             of <set1> and <set2> separately, then perform a diff
                             of these.  Inputs may be pairs of files, directories,
                             or archives.  If --out or --report-file is given,
                             three output files will be created, one for each
                             of the two counts and one for the diff.  See also
                             --diff, --diff-alignment, --diff-timeout,
                             --ignore-case, --ignore-whitespace.
   --diff <set1> <set2>      Compute differences in code and comments between
                             source file(s) of <set1> and <set2>.  The inputs
                             may be any mix of files, directories, archives,
                             or git commit hashes.  Use --diff-alignment to
                             generate a list showing which file pairs where
                             compared.  When comparing git branches, only files
                             which have changed in either commit are compared.
                             See also --git, --count-and-diff, --diff-alignment,
                             --diff-list-file, --diff-timeout, --ignore-case,
                             --ignore-whitespace.
   --diff-timeout <N>        Ignore files which take more than <N> seconds
                             to process.  Default is 10 seconds.  Setting <N>
                             to 0 allows unlimited time.  (Large files with many
                             repeated lines can cause Algorithm::Diff::sdiff()
                             to take hours.) See also --timeout.
   --docstring-as-code       cloc considers docstrings to be comments, but this is
                             not always correct as docstrings represent regular
                             strings when they appear on the right hand side of an
                             assignment or as function arguments.  This switch
                             forces docstrings to be counted as code.
   --follow-links            [Unix only] Follow symbolic links to directories
                             (sym links to files are always followed).
                             See also --stat.
   --force-lang=<lang>[,<ext>]
                             Process all files that have a <ext> extension
                             with the counter for language <lang>.  For
                             example, to count all .f files with the
                             Fortran 90 counter (which expects files to
                             end with .f90) instead of the default Fortran 77
                             counter, use
                               --force-lang="Fortran 90,f"
                             If <ext> is omitted, every file will be counted
                             with the <lang> counter.  This option can be
                             specified multiple times (but that is only
                             useful when <ext> is given each time).
                             See also --script-lang, --lang-no-ext.
   --force-lang-def=<file>   Load language processing filters from <file>,
                             then use these filters instead of the built-in
                             filters.  Note:  languages which map to the same
                             file extension (for example:
                             MATLAB/Mathematica/Objective-C/MUMPS/Mercury;
                             Pascal/PHP; Lisp/OpenCL; Lisp/Julia; Perl/Prolog)
                             will be ignored as these require additional
                             processing that is not expressed in language
                             definition files.  Use --read-lang-def to define
                             new language filters without replacing built-in
                             filters (see also --write-lang-def,
                             --write-lang-def-incl-dup).
   --git                     Forces the inputs to be interpreted as git targets
                             (commit hashes, branch names, et cetera) if these
                             are not first identified as file or directory
                             names.  This option overrides the --vcs=git logic
                             if this is given; in other words, --git gets its
                             list of files to work on directly from git using
                             the hash or branch name rather than from
                             'git ls-files'.  This option can be used with
                             --diff to perform line count diffs between git
                             commits, or between a git commit and a file,
                             directory, or archive.  Use -v/--verbose to see
                             the git system commands cloc issues.
   --git-diff-rel            Same as --git --diff, or just --diff if the inputs
                             are recognized as git targets.  Only files which
                             have changed in either commit are compared.
   --git-diff-all            Git diff strategy #2:  compare all files in the
                             repository between the two commits.
   --ignore-whitespace       Ignore horizontal white space when comparing files
                             with --diff.  See also --ignore-case.
   --ignore-case             Ignore changes in case within file contents;
                             consider upper- and lowercase letters equivalent
                             when comparing files with --diff.  See also
                             --ignore-whitespace.
   --ignore-case-ext         Ignore case of file name extensions.  This will
                             cause problems counting some languages
                             (specifically, .c and .C are associated with C and
                             C++; this switch would count .C files as C rather
                             than C++ on *nix operating systems).  File name
                             case insensitivity is always true on Windows.
   --lang-no-ext=<lang>      Count files without extensions using the <lang>
                             counter.  This option overrides internal logic
                             for files without extensions (where such files
                             are checked against known scripting languages
                             by examining the first line for #!).  See also
                             --force-lang, --script-lang.
   --max-file-size=<MB>      Skip files larger than <MB> megabytes when
                             traversing directories.  By default, <MB>=100.
                             cloc's memory requirement is roughly twenty times
                             larger than the largest file so running with
                             files larger than 100 MB on a computer with less
                             than 2 GB of memory will cause problems.
                             Note:  this check does not apply to files
                             explicitly passed as command line arguments.
   --no-autogen[=list]       Ignore files generated by code-production systems
                             such as GNU autoconf.  To see a list of these files
                             (then exit), run with --no-autogen list
                             See also --autoconf.
   --original-dir            [Only effective in combination with
                             --strip-comments]  Write the stripped files
                             to the same directory as the original files.
   --read-binary-files       Process binary files in addition to text files.
                             This is usually a bad idea and should only be
                             attempted with text files that have embedded
                             binary data.
   --read-lang-def=<file>    Load new language processing filters from <file>
                             and merge them with those already known to cloc.
                             If <file> defines a language cloc already knows
                             about, cloc's definition will take precedence.
                             Use --force-lang-def to over-ride cloc's
                             definitions (see also --write-lang-def,
                             --write-lang-def-incl-dup).
   --script-lang=<lang>,<s>  Process all files that invoke <s> as a #!
                             scripting language with the counter for language
                             <lang>.  For example, files that begin with
                                #!/usr/local/bin/perl5.8.8
                             will be counted with the Perl counter by using
                                --script-lang=Perl,perl5.8.8
                             The language name is case insensitive but the
                             name of the script language executable, <s>,
                             must have the right case.  This option can be
                             specified multiple times.  See also --force-lang,
                             --lang-no-ext.
   --sdir=<dir>              Use <dir> as the scratch directory instead of
                             letting File::Temp chose the location.  Files
                             written to this location are not removed at
                             the end of the run (as they are with File::Temp).
   --skip-uniqueness         Skip the file uniqueness check.  This will give
                             a performance boost at the expense of counting
                             files with identical contents multiple times
                             (if such duplicates exist).
   --stat                    Some file systems (AFS, CD-ROM, FAT, HPFS, SMB)
                             do not have directory 'nlink' counts that match
                             the number of its subdirectories.  Consequently
                             cloc may undercount or completely skip the
                             contents of such file systems.  This switch forces
                             File::Find to stat directories to obtain the
                             correct count.  File search speed will decrease.
                             See also --follow-links.
   --stdin-name=<file>       Give a file name to use to determine the language
                             for standard input.  (Use - as the input name to
                             receive source code via STDIN.)
   --strip-comments=<ext>    For each file processed, write to the current
                             directory a version of the file which has blank
                             and commented lines removed (in-line comments
                             persist).  The name of each stripped file is the
                             original file name with .<ext> appended to it.
                             It is written to the current directory unless
                             --original-dir is on.
   --strip-str-comments      Replace comment markers embedded in strings with
                             'xx'.  This attempts to work around a limitation
                             in Regexp::Common::Comment where comment markers
                             embedded in strings are seen as actual comment
                             markers and not strings, often resulting in a
                             'Complex regular subexpression recursion limit'
                             warning and incorrect counts.  There are two
                             disadvantages to using this switch:  1/code count
                             performance drops, and 2/code generated with
                             --strip-comments will contain different strings
                             where ever embedded comments are found.
   --sum-reports             Input arguments are report files previously
                             created with the --report-file option in plain
                             format (eg. not JSON, YAML, XML, or SQL).
                             Makes a cumulative set of results containing the
                             sum of data from the individual report files.
   --timeout <N>             Ignore files which take more than <N> seconds
                             to process at any of the language's filter stages.
                             The default maximum number of seconds spent on a
                             filter stage is the number of lines in the file
                             divided by one thousand.  Setting <N> to 0 allows
                             unlimited time.  See also --diff-timeout.
   --processes=NUM           [Available only on systems with a recent version
                             of the Parallel::ForkManager module.  Not
                             available on Windows.] Sets the maximum number of
                             cores that cloc uses.  The default value of 0
                             disables multiprocessing.
   --unix                    Override the operating system autodetection
                             logic and run in UNIX mode.  See also
                             --windows, --show-os.
   --use-sloccount           If SLOCCount is installed, use its compiled
                             executables c_count, java_count, pascal_count,
                             php_count, and xml_count instead of cloc's
                             counters.  SLOCCount's compiled counters are
                             substantially faster than cloc's and may give
                             a performance improvement when counting projects
                             with large files.  However, these cloc-specific
                             features will not be available: --diff,
                             --count-and-diff, --strip-comments, --unicode.
   --windows                 Override the operating system autodetection
                             logic and run in Microsoft Windows mode.
                             See also --unix, --show-os.

 Filter Options
   --include-content=<regex> Only count files containing text that matches the
                             given regular expression.
   --exclude-content=<regex> Exclude files containing text that matches the given
                             regular expression.
   --exclude-dir=<D1>[,D2,]  Exclude the given comma separated directories
                             D1, D2, D3, et cetera, from being scanned.  For
                             example  --exclude-dir=.cache,test  will skip
                             all files and subdirectories that have /.cache/
                             or /test/ as their parent directory.
                             Directories named .bzr, .cvs, .hg, .git, .svn,
                             and .snapshot are always excluded.
                             This option only works with individual directory
                             names so including file path separators is not
                             allowed.  Use --fullpath and --not-match-d=<regex>
                             to supply a regex matching multiple subdirectories.
   --exclude-ext=<ext1>[,<ext2>[...]]
                             Do not count files having the given file name
                             extensions.
   --exclude-lang=<L1>[,L2[...]]
                             Exclude the given comma separated languages
                             L1, L2, L3, et cetera, from being counted.
   --exclude-list-file=<file>  Ignore files and/or directories whose names
                             appear in <file>.  <file> should have one file
                             name per line.  Only exact matches are ignored;
                             relative path names will be resolved starting from
                             the directory where cloc is invoked.
                             See also --list-file.
   --fullpath                Modifies the behavior of --match-f, --not-match-f,
                             and --not-match-d to include the file's path
                             in the regex, not just the file's basename.
                             (This does not expand each file to include its
                             absolute path, instead it uses as much of
                             the path as is passed in to cloc.)
                             Note:  --match-d always looks at the full
                             path and therefore is unaffected by --fullpath.
   --include-ext=<ext1>[,ext2[...]]
                             Count only languages having the given comma
                             separated file extensions.  Use --show-ext to
                             see the recognized extensions.
   --include-lang=<L1>[,L2[...]]
                             Count only the given comma separated languages
                             L1, L2, L3, et cetera.  Use --show-lang to see
                             the list of recognized languages.
   --match-d=<regex>         Only count files in directories matching the Perl
                             regex.  For example
                               --match-d='/(src|include)/'
                             only counts files in directories containing
                             /src/ or /include/.  Unlike --not-match-d,
                             --match-f, and --not-match-f, --match-d always
                             compares the fully qualified path against the
                             regex.
   --not-match-d=<regex>     Count all files except those in directories
                             matching the Perl regex.  Only the trailing
                             directory name is compared, for example, when
                             counting in /usr/local/lib, only 'lib' is
                             compared to the regex.
                             Add --fullpath to compare parent directories to
                             the regex.
                             Do not include file path separators at the
                             beginning or end of the regex.
   --match-f=<regex>         Only count files whose basenames match the Perl
                             regex.  For example
                               --match-f='^[Ww]idget'
                             only counts files that start with Widget or widget.
                             Add --fullpath to include parent directories
                             in the regex instead of just the basename.
   --not-match-f=<regex>     Count all files except those whose basenames
                             match the Perl regex.  Add --fullpath to include
                             parent directories in the regex instead of just
                             the basename.
   --skip-archive=<regex>    Ignore files that end with the given Perl regular
                             expression.  For example, if given
                               --skip-archive='(zip|tar(.(gz|Z|bz2|xz|7z))?)'
                             the code will skip files that end with .zip,
                             .tar, .tar.gz, .tar.Z, .tar.bz2, .tar.xz, and
                             .tar.7z.
   --skip-win-hidden         On Windows, ignore hidden files.

 Debug Options
   --categorized=<file>      Save file sizes in bytes, identified languages
                             and names of categorized files to <file>.
   --counted=<file>          Save names of processed source files to <file>.
   --diff-alignment=<file>   Write to <file> a list of files and file pairs
                             showing which files were added, removed, and/or
                             compared during a run with --diff.  This switch
                             forces the --diff mode on.
   --explain=<lang>          Print the filters used to remove comments for
                             language <lang> and exit.  In some cases the
                             filters refer to Perl subroutines rather than
                             regular expressions.  An examination of the
                             source code may be needed for further explanation.
   --help                    Print this usage information and exit.
   --found=<file>            Save names of every file found to <file>.
   --ignored=<file>          Save names of ignored files and the reason they
                             were ignored to <file>.
   --print-filter-stages     Print processed source code before and after
                             each filter is applied.
   --show-ext[=<ext>]        Print information about all known (or just the
                             given) file extensions and exit.
   --show-lang[=<lang>]      Print information about all known (or just the
                             given) languages and exit.
   --show-os                 Print the value of the operating system mode
                             and exit.  See also --unix, --windows.
   -v[=<n>]                  Verbose switch (optional numeric value).
   -verbose[=<n>]            Long form of -v.
   --version                 Print the version of this program and exit.
   --write-lang-def=<file>   Writes to <file> the language processing filters
                             then exits.  Useful as a first step to creating
                             custom language definitions. Note: languages which
                             map to the same file extension will be excluded.
                             (See also --force-lang-def, --read-lang-def).
   --write-lang-def-incl-dup=<file>
                             Same as --write-lang-def, but includes duplicated
                             extensions.  This generates a problematic language
                             definition file because cloc will refuse to use
                             it until duplicates are removed.

 Output Options
   --3                       Print third-generation language output.
                             (This option can cause report summation to fail
                             if some reports were produced with this option
                             while others were produced without it.)
   --by-percent  X           Instead of comment and blank line counts, show
                             these values as percentages based on the value
                             of X in the denominator:
                                X = 'c'   -> # lines of code
                                X = 'cm'  -> # lines of code + comments
                                X = 'cb'  -> # lines of code + blanks
                                X = 'cmb' -> # lines of code + comments + blanks
                             For example, if using method 'c' and your code
                             has twice as many lines of comments as lines
                             of code, the value in the comment column will
                             be 200%.  The code column remains a line count.
   --csv                     Write the results as comma separated values.
   --csv-delimiter=<C>       Use the character <C> as the delimiter for comma
                             separated files instead of ,.  This switch forces --csv to be on.
   --file-encoding=<E>       Write output files using the <E> encoding instead of
                             the default ASCII (<E> = 'UTF-7').  Examples: 'UTF-16',
                             'euc-kr', 'iso-8859-16'.  Known encodings can be
                             printed with
                               perl -MEncode -e 'print join("\n", Encode->encodings(":all")), "\n"'
   --hide-rate               Do not show line and file processing rates in the
                             output header. This makes output deterministic.
   --json                    Write the results as JavaScript Object Notation
                             (JSON) formatted output.
   --md                      Write the results as Markdown-formatted text.
   --out=<file>              Synonym for --report-file=<file>.
   --percent                 Show counts as percentages of sums for each column.
                             Same as '--by-percent t'.
   --progress-rate=<n>       Show progress update after every <n> files are
                             processed (default <n>=100).  Set <n> to 0 to
                             suppress progress output (useful when redirecting
                             output to STDOUT).
   --quiet                   Suppress all information messages except for
                             the final report.
   --report-file=<file>      Write the results to <file> instead of STDOUT.
   --sql=<file>              Write results as SQL create and insert statements
                             which can be read by a database program such as
                             SQLite.  If <file> is -, output is sent to STDOUT.
   --sql-append              Append SQL insert statements to the file specified
                             by --sql and do not generate table creation
                             statements.  Only valid with the --sql option.
   --sql-project=<name>      Use <name> as the project identifier for the
                             current run.  Only valid with the --sql option.
   --sql-style=<style>       Write SQL statements in the given style instead
                             of the default SQLite format.  Styles include
                             'Oracle' and 'Named_Columns'.
   --sum-one                 For plain text reports, show the SUM: output line
                             even if only one input file is processed.
   --thousands-delimiter=<C> Divides numbers with many digits (i.e. numbers
                             over 999) into groups using the character <C> as
                             delimiter (e.g. for <C> = '.': 12345 -> 12.345).
                             Only works with the '--fmt' option.
                             Sample values: '.', ',', '_', ' '
                             Synonym:  --ksep
   --xml                     Write the results in XML.
   --xsl=<file>              Reference <file> as an XSL stylesheet within
                             the XML output.  If <file> is 1 (numeric one),
                             writes a default stylesheet, cloc.xsl (or
                             cloc-diff.xsl if --diff is also given).
                             This switch forces --xml on.
   --yaml                    Write the results in YAML.

Recognized Languages

Full language list (cloc --show-lang) (click the triangle to see all languages)

prompt> cloc --show-lang

Language                   Extension(s)
--------------------       ----------------------------------------
ABAP                       (abap)
ActionScript               (as)
Activiti Business Process  (bpmn)
Ada                        (ada, adb, ads, pad)
ADSO/IDSM                  (adso)
Agda                       (agda, lagda)
AMPLE                      (ample, dofile, startup)
AnsProlog                  (lp)
Ant                        (build.xml, build.xml)
ANTLR Grammar              (g, g4)
Apex Class                 (cls)
Apex Trigger               (trigger)
APL                        (apl, apla, aplc, aplf, apli, apln, aplo, dyalog, dyapp, mipage)
AppleScript                (applescript)
Arduino Sketch             (ino)
Aria                       (aria)
ArkTs                      (ets)
Arturo                     (art)
AsciiDoc                   (adoc, asciidoc)
ASP                        (asa, ashx, asp, axd)
ASP.NET                    (asax, ascx, asmx, aspx, master, sitemap, webinfo)
AspectJ                    (aj)
Assembly                   (a51, asm, nasm, S, s)
Astro                      (astro)
Asymptote                  (asy)
AutoHotkey                 (ahk, ahkl)
awk                        (auk, awk, gawk, mawk, nawk)
AXAML                      (axaml)
Bazel                      (BUILD)
Beluga                     (bel)
Bicep                      (bicep, bicepparam)
BitBake                    (bb, bbappend, bbclass)
BizTalk Orchestration      (odx)
BizTalk Pipeline           (btp)
Blade                      (blade, blade.php)
Blueprint                  (blp)
Bourne Again Shell         (bash)
Bourne Shell               (sh)
BrightScript               (brs)
builder                    (xml.builder)
C                          (c, cats, ec, idc, pgc)
C Shell                    (csh, tcsh)
C#                         (cs)
C# Designer                (designer.cs)
C++                        (C, c++, c++m, cc, ccm, CPP, cpp, cppm, cxx, cxxm, h++, inl, ipp,
                            ixx, pcc, tcc, tpp)
C/C++ Header               (H, h, hh, hpp, hxx)
Cadence                    (cdc)
Cairo                      (cairo)
Cake Build Script          (cake)
Cangjie                    (cj)
Carbon                     (carbon)
CCS                        (ccs)
Chapel                     (chpl)
Circom                     (circom)
Civet                      (civet, cvt, cvtx)
Clarity                    (clar)
Clean                      (dcl, icl)
Clojure                    (boot, cl2, clj, cljs.hl, cljscm, cljx, hic, riemann.config, cj)
ClojureC                   (cljc)
ClojureScript              (cljs)
CMake                      (cmake, cmake.in, CMakeLists.txt)
COBOL                      (CBL, cbl, ccp, COB, cob, cobol, cpy)
CoCoA 5                    (c5, cocoa5, cocoa5server, cpkg5)
CodeQL                     (ql, qll)
CoffeeScript               (_coffee, cakefile, cjsx, coffee, iced)
ColdFusion                 (cfm, cfml)
ColdFusion CFScript        (cfc)
Constraint Grammar         (cg3, rlx)
Containerfile              (Containerfile)
Coq                        (v)
Crystal                    (cr)
CSON                       (cson)
CSS                        (css)
CSV                        (csv)
Cucumber                   (feature)
CUDA                       (cu, cuh)
Cython                     (pxd, pxi, pyx)
D                          (d)
Dafny                      (dfy)
DAL                        (da)
Dart                       (dart)
Delphi Form                (dfm)
DenizenScript              (dsc)
Derw                       (derw)
dhall                      (dhall)
DIET                       (dt)
diff                       (diff, patch)
DITA                       (dita)
Dockerfile                 (Dockerfile, dockerfile)
DOORS Extension Language   (dxl)
DOS Batch                  (BAT, bat, BTM, btm, CMD, cmd)
Drools                     (drl)
DTD                        (dtd)
dtrace                     (d)
ECPP                       (ecpp)
EEx                        (eex)
EJS                        (ejs)
Elixir                     (ex)
Elixir Script              (exs)
Elm                        (elm)
Embedded Crystal           (ecr)
ERB                        (ERB, erb)
Erlang                     (app.src, emakefile, erl, hrl, rebar.config, rebar.config.lock,
                            rebar.lock, xrl, yrl)
Expect                     (exp)
F#                         (fsi, fs, fs)
F# Script                  (fsx)
Fennel                     (fnl)
Finite State Language      (fsl, jssm)
Fish Shell                 (fish)
Flatbuffers                (fbs)
Focus                      (focexec)
Forth                      (4th, e4, f83, fb, forth, fpm, fr, frt, ft, fth, rx, fs, f, for)
Fortran 2003               (F03, f03)
Fortran 77                 (F, F77, f77, FOR, FTN, ftn, pfo, f, for)
Fortran 90                 (F90, f90)
Fortran 95                 (F95, f95)
Freemarker Template        (ftl)
Futhark                    (fut)
FXML                       (fxml)
GDScript                   (gd)
Gencat NLS                 (msg)
Glade                      (glade, ui)
Gleam                      (gleam)
Glimmer JavaScript         (gjs)
Glimmer TypeScript         (gts)
GLSL                       (comp, fp, frag, frg, fsh, fshader, geo, geom, glsl, glslv, gshader,
                            tesc, tese, vert, vrx, vsh, vshader)
Go                         (go, ʕ◔ϖ◔ʔ)
Godot Resource             (tres)
Godot Scene                (tscn)
Godot Shaders              (gdshader)
Gradle                     (gradle, gradle.kts)
Grails                     (gsp)
GraphQL                    (gql, graphql, graphqls)
Groovy                     (gant, groovy, grt, gtpl, gvy, jenkinsfile)
Haml                       (haml, haml.deface)
Handlebars                 (handlebars, hbs)
Harbour                    (hb)
Hare                       (ha)
Haskell                    (hs, hsc, lhs)
Haskell Boot               (hs-boot, lhs-boot)
Haxe                       (hx, hxsl)
HCL                        (hcl, nomad, tf, tfvars)
Hibernate                  (hbm.xml)
HLSL                       (cg, cginc, fxh, hlsl, hlsli, shader)
HolyC                      (HC)
Hoon                       (hoon)
HTML                       (htm, html, html.hl, xht)
HTML EEx                   (heex)
IDL                        (dlm, idl, pro)
Idris                      (idr)
Igor Pro                   (ipf)
Imba                       (imba)
INI                        (buildozer.spec, editorconfig, ini, lektorproject, prefs)
InstallShield              (ism)
IPL                        (ipl)
Jai                        (jai)
Janet                      (janet)
Jasper Report XML/Template (jrxml)
Java                       (java)
JavaScript                 (_js, bones, cjs, es6, jake, jakefile, js, jsb, jscad, jsfl, jsm,
                            jss, mjs, njs, pac, sjs, ssjs, xsjs, xsjslib)
JavaServer Faces           (jsf)
JCL                        (jcl)
Jinja Template             (j2, jinja, jinja2)
JSON                       (arcconfig, avsc, composer.lock, geojson, gltf, har, htmlhintrc,
                            json, json-tmlanguage, jsonl, mcmeta, mcmod.info, tern-config,
                            tern-project, tfstate, tfstate.backup, topojson, watchmanconfig,
                            webapp, webmanifest, yyp)
JSON5                      (json5)
Jsonnet                    (jsonnet)
JSP                        (jsp, jspf)
JSP Tag Library Definition (tld)
JSX                        (jsx)
Julia                      (jl)
Juniper Junos              (junos)
Jupyter Notebook           (ipynb)
Justfile                   (just)
Kermit                     (ksc)
Korn Shell                 (ksh)
Kotlin                     (kt, ktm, kts)
kvlang                     (kv)
Lean                       (hlean, lean)
Lem                        (lem)
LESS                       (less)
lex                        (l, lex)
LFE                        (lfe)
Linker Script              (ld)
Liquibase                  (lb.xml)
liquid                     (liquid)
Lisp                       (asd, el, lisp, lsp, cl, jl)
Literate Idris             (lidr)
LiveLink OScript           (oscript)
LLVM IR                    (ll)
Logos                      (x, xm)
Logtalk                    (lgt, logtalk)
Lua                        (lua, nse, p8, pd_lua, rbxs, wlua)
Luau                       (luau)
m4                         (ac, m4)
Magik                      (magik)
make                       (am, Gnumakefile, gnumakefile, Makefile, makefile, mk)
Mako                       (mako, mao)
Markdown                   (contents.lr, markdown, md, mdown, mdwn, mdx, mkd, mkdn, mkdown,
                            ronn, workbook)
Mathematica                (cdf, ma, mathematica, mt, nbp, wl, wlt, m)
MATLAB                     (m)
Maven                      (pom, pom.xml)
Meson                      (meson.build)
Metal                      (metal)
Modelica                   (mo)
Modula3                    (i3, ig, m3, mg)
Mojo                       (mojo, 🔥)
Mojom                      (mojom)
MoonBit                    (mbt, mbti, mbtx, mbty)
MSBuild script             (btproj, csproj, msbuild, vcproj, wdproj, wixproj)
MUMPS                      (mps, m)
Mustache                   (mustache)
MXML                       (mxml)
NAnt script                (build)
NASTRAN DMAP               (dmap)
Nemerle                    (n)
NetLogo                    (nlogo, nls)
Nextflow                   (nf)
Nickel                     (ncl)
Nim                        (nim, nim.cfg, nimble, nimrod, nims)
Nix                        (nix)
Nunjucks                   (njk)
Nushell                    (nu)
Nushell Object Notation    (nuon)
Objective-C                (m)
Objective-C++              (mm)
OCaml                      (eliom, eliomi, ml, ml4, mli, mll, mly)
Odin                       (odin)
OpenCL                     (cl)
OpenSCAD                   (scad)
Oracle Forms               (fmt)
Oracle PL/SQL              (bdy, bod, fnc, prc, spc, trg)
Oracle Reports             (rex)
Org Mode                   (org)
P4                         (p4)
Pascal                     (dpr, lpr, pas, pascal)
Pascal/Pawn                (p)
Pascal/Puppet              (pp)
Patran Command Language    (pcl, ses)
Pawn                       (pawn, pwn)
PEG                        (peg)
peg.js                     (pegjs)
peggy                      (peggy)
Pek                        (pek)
Perl                       (ack, al, cpanfile, makefile.pl, perl, ph, plh, plx, pm, psgi,
                            rexfile, pl, p6)
Pest                       (pest)
PHP                        (aw, ctp, phakefile, php, php3, php4, php5, php_cs, php_cs.dist, phps,
                            phpt, phtml)
PHP/Pascal/Fortran/Pawn/BitBake (inc)
Pig Latin                  (pig)
Pkl                        (pkl)
PL/I                       (pl1)
PL/M                       (lit, plm)
PlantUML                   (iuml, plantuml, pu, puml, wsd)
PO File                    (po)
Pony                       (pony)
PowerBuilder               (pbt, sra, srf, srm, srs, sru, srw)
PowerShell                 (ps1, psd1, psm1)
Prisma Schema              (prisma)
Processing                 (pde)
ProGuard                   (pro)
Prolog                     (P, prolog, yap, pl, p6, pro)
Properties                 (properties)
Protocol Buffers           (proto)
PRQL                       (prql)
Pug                        (jade, pug)
PureScript                 (purs)
Python                     (buck, build.bazel, gclient, gyp, gypi, lmi, py, py3, pyde, pyi, pyp,
                            pyt, pyw, sconscript, sconstruct, snakefile, tac, workspace, wscript,
                            wsgi, xpy)
QML                        (qbs, qml)
Qt Linguist                (ts)
Qt Project                 (pro)
R                          (expr-dist, R, r, rd, rprofile, rsx)
Racket                     (rkt, rktd, rktl, scrbl)
Raku                       (pm6, raku, rakumod)
Raku/Prolog                (P6, p6)
RAML                       (raml)
RapydScript                (pyj)
Razor                      (cshtml, razor)
ReasonML                   (re, rei)
Rego                       (rego)
ReScript                   (res, resi)
reStructuredText           (rest, rest.txt, rst, rst.txt)
Rexx                       (pprx, rexx)
Ring                       (rform, rh, ring)
Rmd                        (Rmd)
RobotFramework             (robot)
Ruby                       (appraisals, berksfile, brewfile, builder, buildfile, capfile,
                            dangerfile, deliverfile, eye, fastfile, gemfile, gemfile.lock,
                            gemspec, god, guardfile, irbrc, jarfile, jbuilder, mavenfile,
                            mspec, podfile, podspec, pryrc, puppetfile, rabl, rake, rb,
                            rbuild, rbw, rbx, ru, snapfile, thor, thorfile, vagrantfile, watchr)
Ruby HTML                  (rhtml)
Rust                       (rs, rs.in)
SaltStack                  (sls)
SAS                        (sas)
Sass                       (sass)
Scala                      (kojo, sbt, scala)
Scheme                     (sc, sch, scm, sld, sps, ss, sls)
SCSS                       (scss)
sed                        (sed)
SKILL++                    (ils)
SKILL/.NET IL              (il)
Slice                      (ice)
Slim                       (slim)
Slint                      (slint)
Smalltalk                  (st, cs)
Smarty                     (smarty, tpl)
Snakemake                  (rules, smk)
Softbridge Basic           (SBL, sbl)
Solidity                   (sol)
SparForte                  (sp)
Specman e                  (e)
SQL                        (cql, mysql, psql, SQL, sql, tab, udf, viw)
SQL Data                   (data.sql)
SQL Stored Procedure       (spc.sql, spoc.sql, sproc.sql, udf.sql)
Squirrel                   (nut)
Standard ML                (fun, sig, sml)
Starlark                   (bazel, bzl)
Stata                      (ado, DO, do, doh, ihlp, mata, matah, sthlp)
Stylus                     (styl)
SugarSS                    (sss)
SurrealQL                  (surql)
Svelte                     (svelte)
SVG                        (SVG, svg)
Swift                      (swift)
SWIG                       (i)
TableGen                   (td)
Tcl/Tk                     (itk, tcl, tk)
TEAL                       (teal)
Teamcenter met             (met)
Teamcenter mth             (mth)
Templ                      (templ)
TeX                        (aux, bbx, bib, bst, cbx, dtx, ins, lbx, ltx, mkii, mkiv, mkvi,
                            sty, tex, cls)
Text                       (text, txt)
Thrift                     (thrift)
TITAN Project File Information (tpd)
Titanium Style Sheet       (tss)
TLA+                       (tla)
TNSDL                      (cii, cin, in1, in2, in3, in4, inf, interface, rou, sdl, sdt,
                            spd, ssc, sst)
TOML                       (toml)
tspeg                      (jspeg, tspeg)
TTCN                       (ttcn, ttcn2, ttcn3, ttcnpp)
Twig                       (twig)
TypeScript                 (cts, mts, tsx, ts)
Typst                      (typ)
Umka                       (um)
Unity-Prefab               (mat, prefab)
Unknown/BitBake            (conf)
USS                        (uss)
UXML                       (uxml)
Vala                       (vala)
Vala Header                (vapi)
VBA                        (VBA, vba)
VBScript                   (VBS, vbs)
Velocity Template Language (vm)
Verilog-SystemVerilog      (sv, svh, v)
VHDL                       (VHD, vhd, VHDL, vhdl, vhf, vhi, vho, vhs, vht, vhw)
vim script                 (vim)
Visual Basic               (BAS, bas, ctl, Dsr, dsr, frm, FRX, frx, vbp, vbw, cls)
Visual Basic .NET          (VB, vb, VBHTML, vbhtml, vbproj)
Visual Fox Pro             (SCA, sca)
Visual Studio Solution     (sln)
Visualforce Component      (component)
Visualforce Page           (page)
VSCode Workspace           (code-workspace)
Vuejs Component            (vue)
Vyper                      (vy)
Web Services Description   (wsdl)
WebAssembly                (wast, wat)
WGSL                       (wgsl)
Windows Message File       (mc)
Windows Module Definition  (def)
Windows Resource File      (rc, rc2)
WiX include                (wxi)
WiX source                 (wxs)
WiX string localization    (wxl)
WXML                       (wxml)
WXSS                       (wxss)
X++                        (xpo)
XAML                       (xaml)
xBase                      (prg, prw)
xBase Header               (ch)
XHTML                      (xhtml)
XMI                        (XMI, xmi)
XML                        (adml, admx, ant, app.config, axml, builds, ccproj, ccxml,
                            classpath, clixml, cproject, cscfg, csdef, csl, ct, depproj,
                            ditamap, ditaval, dll.config, dotsettings, filters, fsproj,
                            gmx, grxml, iml, ivy, jelly, jsproj, kml, launch, mdpolicy,
                            mjml, natvis, ndproj, nproj, nuget.config, nuspec, odd, osm,
                            packages.config, pkgproj, plist, proj, project, props, ps1xml,
                            psc1, pt, rdf, resx, rss, scxml, settings.stylecop, sfproj,
                            shproj, srdf, storyboard, sttheme, sublime-snippet, targets,
                            tmcommand, tml, tmlanguage, tmpreferences, tmsnippet, tmtheme,
                            urdf, ux, vcxproj, vsixmanifest, vssettings, vstemplate, vxml,
                            web.config, web.debug.config, web.release.config, wsf, x3d,
                            xacro, xib, xlf, xliff, XML, xml, xml.dist, xproj, xspec, xul,
                            zcml)
XML-Qt-GTK                 (ui)
XQuery                     (xq, xql, xqm, xquery, xqy)
XSD                        (XSD, xsd)
XSLT                       (XSL, xsl, XSLT, xslt)
Xtend                      (xtend)
yacc                       (y, yacc)
YAML                       (clang-format, clang-tidy, gemrc, glide.lock, mir, reek, rviz,
                            sublime-syntax, syntax, yaml, yaml-tmlanguage, yml, yml.mysql)
Yang                       (yang)
Yarn                       (yarn)
Zig                        (zig)
zsh                        (zsh)

The above list can be customized by reading language definitions from a file with the --read-lang-def or --force-lang-def options.

These file extensions map to multiple languages:

cj files could be Clojure or Cangjie
cl files could be Lisp or OpenCL
cls files could be Visual Basic, TeX or Apex Class
conf files could be BitBake or plain text
cs files could be C# or Smalltalk
d files could be D or dtrace
f files could be Fortran 77 or Forth
fnc files could be Oracle PL or SQL
for files could be Fortran 77 or Forth
fs files could be F# or Forth
inc files could be PHP, Pascal or BitBake
itk files could be Tcl or Tk
jl files could be Lisp or Julia
lit files could be PL or M
m files could be MATLAB, Mathematica, Objective-C, MUMPS or Mercury
p6 files could be Perl or Prolog
pl files could be Perl or Prolog
PL files could be Perl or Prolog
pp files could be Pascal or Puppet
pro files could be IDL, Qt Project, Prolog or ProGuard
ts files could be TypeScript or Qt Linguist
ui files could be Qt or Glade
v files could be Verilog-SystemVerilog or Coq

cloc has subroutines that attempt to identify the correct language based on the file's contents for these special cases. Language identification accuracy is a function of how much code the file contains; .m files with just one or two lines for example, seldom have enough information to correctly distinguish between MATLAB, Mercury, MUMPS, or Objective-C.

Languages with file extension collisions are difficult to customize with --read-lang-def or --force-lang-def as they have no mechanism to identify languages with common extensions. In this situation one must modify the cloc source code.

How It Works

cloc's method of operation resembles SLOCCount's: First, create a list of files to consider. Next, attempt to determine whether or not found files contain recognized computer language source code. Finally, for files identified as source files, invoke language-specific routines to count the number of source lines.

A more detailed description:

If the input file is an archive (such as a .tar.gz or .zip file), create a temporary directory and expand the archive there using a system call to an appropriate underlying utility (tar, bzip2, unzip, etc) then add this temporary directory as one of the inputs. (This works more reliably on Unix than on Windows.)
Use File::Find to recursively descend the input directories and make a list of candidate file names. Ignore binary and zero-sized files.
Make sure the files in the candidate list have unique contents (first by comparing file sizes, then, for similarly sized files, compare MD5 hashes of the file contents with Digest::MD5). For each set of identical files, remove all but the first copy, as determined by a lexical sort, of identical files from the set. The removed files are not included in the report. (The --skip-uniqueness switch disables the uniqueness tests and forces all copies of files to be included in the report.) See also the --ignored= switch to see which files were ignored and why.
Scan the candidate file list for file extensions which cloc associates with programming languages (see the --show-lang and --show-ext options). Files which match are classified as containing source code for that language. Each file without an extension is opened and its first line read to see if it is a Unix shell script (anything that begins with #!). If it is shell script, the file is classified by that scripting language (if the language is recognized). If the file does not have a recognized extension or is not a recognized scripting language, the file is ignored.
All remaining files in the candidate list should now be source files for known programming languages. For each of these files:
1. Read the entire file into memory.
2. Count the number of lines (= L_original).
3. Remove blank lines, then count again (= L_{non_blank}).
4. Loop over the comment filters defined for this language. (For example, C++ has two filters: (1) remove lines that start with optional whitespace followed by // and (2) remove text between /* and */) Apply each filter to the code to remove comments. Count the left over lines (= L_code).
5. Save the counts for this language:
  - blank lines = L_original - L_{non_blank}
  - comment lines = L_{non_blank} - L_code
  - code lines = L_code

The options modify the algorithm slightly. The --read-lang-def option for example allows the user to read definitions of comment filters, known file extensions, and known scripting languages from a file. The code for this option is processed between Steps 2 and 3.

Advanced Use

Remove Comments from Source Code

How can you tell if cloc correctly identifies comments? One way to convince yourself cloc is doing the right thing is to use its --strip-comments option to remove comments and blank lines from files, then compare the stripped-down files to originals.

Let's try this out with the SQLite amalgamation, a C file containing all code needed to build the SQLite library along with a header file:

prompt> tar zxf sqlite-amalgamation-3.5.6.tar.gz
prompt> cd sqlite-3.5.6/
prompt> cloc --strip-comments=nc sqlite.c
       1 text file.
       1 unique file.
Wrote sqlite3.c.nc
       0 files ignored.

http://cloc.sourceforge.net v 1.03  T=1.0 s (1.0 files/s, 82895.0 lines/s)
-------------------------------------------------------------------------------
Language          files     blank   comment      code    scale   3rd gen. equiv
-------------------------------------------------------------------------------
C                     1      5167     26827     50901 x   0.77 =       39193.77
-------------------------------------------------------------------------------

The extension argument given to --strip-comments is arbitrary; here nc was used as an abbreviation for "no comments".

cloc removed over 31,000 lines from the file:

prompt> wc -l sqlite3.c sqlite3.c.nc
  82895 sqlite3.c
  50901 sqlite3.c.nc
 133796 total
prompt> echo "82895 - 50901" | bc
31994

We can now compare the original file, sqlite3.c and the one stripped of comments, sqlite3.c.nc with tools like diff or vimdiff and see what exactly cloc considered comments and blank lines. A rigorous proof that the stripped-down file contains the same C code as the original is to compile these files and compare checksums of the resulting object files.

First, the original source file:

prompt> gcc -c sqlite3.c
prompt> md5sum sqlite3.o
cce5f1a2ea27c7e44b2e1047e2588b49  sqlite3.o

Next, the version without comments:

prompt> mv sqlite3.c.nc sqlite3.c
prompt> gcc -c sqlite3.c
prompt> md5sum sqlite3.o
cce5f1a2ea27c7e44b2e1047e2588b49  sqlite3.o

cloc removed over 31,000 lines of comments and blanks but did not modify the source code in any significant way since the resulting object file matches the original.

Work with Compressed Archives

Versions of cloc before v1.07 required an --extract-with=CMD option to tell cloc how to expand an archive file. Beginning with v1.07 this is extraction is attempted automatically. At the moment the automatic extraction method works reasonably well on Unix-type OS's for the following file types: .tar.gz, .tar.bz2, .tar.xz, .tgz, .zip, .ear, .deb. Some of these extensions work on Windows if one has WinZip installed in the default location (C:\Program Files\WinZip\WinZip32.exe). Additionally, with newer versions of WinZip, the [http://www.winzip.com/downcl.htm](command line add-on) is needed for correct operation; in this case one would invoke cloc with something like

 --extract-with="\"c:\Program Files\WinZip\wzunzip\" -e -o >FILE< ."
 </code>

Ref. http://sourceforge.net/projects/cloc/forums/forum/600963/topic/4021070?message=8938196

In situations where the automatic extraction fails, one can try the --extract-with=CMD option to count lines of code within tar files, Zip files, or other compressed archives for which one has an extraction tool. cloc takes the user-provided extraction command and expands the archive to a temporary directory (created with File::Temp), counts the lines of code in the temporary directory, then removes that directory. While not especially helpful when dealing with a single compressed archive (after all, if you're going to type the extraction command anyway why not just manually expand the archive?) this option is handy for working with several archives at once.

For example, say you have the following source tarballs on a Unix machine

perl-5.8.5.tar.gz
Python-2.4.2.tar.gz

and you want to count all the code within them. The command would be

cloc --extract-with='gzip -dc >FILE< | tar xf -' perl-5.8.5.tar.gz Python-2.4.2.tar.gz

If that Unix machine has GNU tar (which can uncompress and extract in one step) the command can be shortened to

cloc --extract-with='tar zxf >FILE<' perl-5.8.5.tar.gz Python-2.4.2.tar.gz

On a Windows computer with WinZip installed in c:\Program Files\WinZip the command would look like

cloc.exe --extract-with="\"c:\Program Files\WinZip\WinZip32.exe\" -e -o >FILE< ." perl-5.8.5.tar.gz Python-2.4.2.tar.gz

Java .ear files are Zip files that contain additional Zip files. cloc can handle nested compressed archives without difficulty--provided all such files are compressed and archived in the same way. Examples of counting a Java .ear file in Unix and Windows:

Unix> cloc --extract-with="unzip -d . >FILE< " Project.ear
DOS> cloc.exe --extract-with="\"c:\Program Files\WinZip\WinZip32.exe\" -e -o >FILE< ." Project.ear

Differences

The --diff switch allows one to measure the relative change in source code and comments between two versions of a file, directory, or archive. Differences reveal much more than absolute code counts of two file versions. For example, say a source file has 100 lines and its developer delivers a newer version with 102 lines. Did the developer add two comment lines, or delete seventeen source lines and add fourteen source lines and five comment lines, or did the developer do a complete rewrite, discarding all 100 original lines and adding 102 lines of all new source? The diff option tells how many lines of source were added, removed, modified or stayed the same, and how many lines of comments were added, removed, modified or stayed the same.

Differences in blank lines are handled much more coarsely because these are stripped by cloc early on. Unless a file pair is identical, cloc will report only differences in absolute counts of blank lines. In other words, one can expect to see only entries for 'added' if the second file has more blanks than the first, and 'removed' if the situation is reversed. The entry for 'same' will be non-zero only when the two files are identical.

In addition to file pairs, one can give cloc pairs of directories, or pairs of file archives, or a file archive and a directory. cloc will try to align file pairs within the directories or archives and compare diffs for each pair. For example, to see what changed between GCC 4.4.0 and 4.5.0 one could do

cloc --diff gcc-4.4.0.tar.bz2  gcc-4.5.0.tar.bz2

Be prepared to wait a while for the results though; the --diff option runs much more slowly than an absolute code count.

To see how cloc aligns files between the two archives, use the --diff-alignment option

cloc --diff-alignment=align.txt gcc-4.4.0.tar.bz2  gcc-4.5.0.tar.bz2

to produce the file align.txt which shows the file pairs as well as files added and deleted. The symbols == and != before each file pair indicate if the files are identical (==) or if they have different content (!=).

Here's sample output showing the difference between the Python 2.6.6 and 2.7 releases:

prompt> cloc --diff Python-2.7.9.tgz Python-2.7.10.tar.xz
    4315 text files.
    4313 text files.s
    2173 files ignored.

4 errors:
Diff error, exceeded timeout:  /tmp/8ToGAnB9Y1/Python-2.7.9/Mac/Modules/qt/_Qtmodule.c
Diff error, exceeded timeout:  /tmp/M6ldvsGaoq/Python-2.7.10/Mac/Modules/qt/_Qtmodule.c
Diff error (quoted comments?):  /tmp/8ToGAnB9Y1/Python-2.7.9/Mac/Modules/qd/qdsupport.py
Diff error (quoted comments?):  /tmp/M6ldvsGaoq/Python-2.7.10/Mac/Modules/qd/qdsupport.py

https://github.com/AlDanial/cloc v 1.65  T=298.59 s (0.0 files/s, 0.0 lines/s)
-----------------------------------------------------------------------------
Language                   files          blank        comment           code
-----------------------------------------------------------------------------
Visual Basic
 same                          2              0              1             12
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
make
 same                         11              0            340           2952
 modified                      1              0              0              1
 added                         0              0              0              0
 removed                       0              0              0              0
diff
 same                          1              0             87            105
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
CSS
 same                          0              0             19            327
 modified                      1              0              0              1
 added                         0              0              0              0
 removed                       0              0              0              0
Objective-C
 same                          7              0             61            635
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
NAnt script
 same                          2              0              0             30
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
XML
 same                          3              0              2             72
 modified                      1              0              0              1
 added                         0              0              0              1
 removed                       0              1              0              0
Windows Resource File
 same                          3              0             56            206
 modified                      1              0              0              1
 added                         0              0              0              0
 removed                       0              0              0              0
Expect
 same                          6              0            161            565
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
HTML
 same                         14              0             11           2344
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
vim script
 same                          1              0              7            106
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
C++
 same                          2              0             18            128
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
Windows Module Definition
 same                          7              0            187           2080
 modified                      2              0              0              0
 added                         0              0              0              1
 removed                       0              1              0              2
Prolog
 same                          1              0              0             24
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
Javascript
 same                          3              0             49            229
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
Assembly
 same                         51              0           6794          12298
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
Bourne Shell
 same                         41              0           7698          45024
 modified                      1              0              0              3
 added                         0             13              2             64
 removed                       0              0              0              0
DOS Batch
 same                         29              0            107            494
 modified                      1              0              0              9
 added                         0              1              0              3
 removed                       0              0              0              0
MSBuild script
 same                         77              0              3          38910
 modified                      0              0              0              0
 added                         0              0              0              0
 removed                       0              0              0              0
Python
 same                       1947              0         109012         430335
 modified                    192              0             94            950
 added                         2            323            283           2532
 removed                       2             55             58            646
m4
 same                         18              0            191          15352
 modified                      1              0              0              2
 added                         1             31              0            205
 removed                       0              0              0              0
C
 same                        505              0          37439         347837
 modified                     45              0             13            218
 added                         0             90             33            795
 removed                       0              9              2            148
C/C++ Header
 same                        255              0          10361          66635
 modified                      5              0              5              7
 added                         0              1              3            300
 removed                       0              0              0              0
---------------------------------------------------------------------
SUM:
 same                       2986              0         172604         966700
 modified                    251              0            112           1193
 added                         3            459            321           3901
 removed                       2             66             60            796
---------------------------------------------------------------------

A pair of errors occurred. The first pair was caused by timing out when computing diffs of the file Python-X/Mac/Modules/qt/_Qtmodule.c in each Python version. This file has > 26,000 lines of C code and takes more than 10 seconds--the default maximum duration for diff'ing a single file--on my slow computer. (Note: this refers to performing differences with the sdiff() function in the Perl Algorithm::Diff module, not the command line diff utility.) This error can be overcome by raising the time to, say, 20 seconds with --diff-timeout 20.

The second error is more problematic. The files Python-X/Mac/Modules/qd/qdsupport.py include Python docstring (text between pairs of triple quotes) containing C comments. cloc treats docstrings as comments and handles them by first converting them to C comments, then using the C comment removing regular expression. Nested C comments yield erroneous results however.

Create Custom Language Definitions

cloc can write its language comment definitions to a file or can read comment definitions from a file, overriding the built-in definitions. This can be useful when you want to use cloc to count lines of a language not yet included, to change association of file extensions to languages, or to modify the way existing languages are counted.

The easiest way to create a custom language definition file is to make cloc write its definitions to a file, then modify that file:

Unix> cloc --write-lang-def=my_definitions.txt

creates the file my_definitions.txt which can be modified then read back in with either the --read-lang-def or --force-lang-def option. The difference between the options is former merges language definitions from the given file in with cloc's internal definitions with cloc's taking precedence if there are overlaps. The --force-lang-def option, on the other hand, replaces cloc's definitions completely. This option has a disadvantage in preventing cloc from counting languages whose extensions map to multiple languages as these languages require additional logic that is not easily expressed in a definitions file.

Unix> cloc --read-lang-def=my_definitions.txt  file1 file2 dir1 ...

Each language entry has four parts:

The language name starting in column 1.
One or more comment filters starting in column 5.
One or more filename extensions starting in column 5.
A 3rd generation scale factor starting in column 5. This entry must be provided but its value is not important unless you want to compare your language to a hypothetical third generation programming language.

A filter defines a method to remove comment text from the source file. For example the entry for C++ looks like this

C++
    filter call_regexp_common C++
    filter remove_inline //.*$
    extension C
    extension c++
    extension cc
    extension cpp
    extension cxx
    extension pcc
    3rd_gen_scale 1.51
    end_of_line_continuation \\$

C++ has two filters: first, remove lines matching Regexp::Common's C++ comment regex. The second filter using remove_inline is currently unused. Its intent is to identify lines with both code and comments and it may be implemented in the future.

A more complete discussion of the different filter options may appear here in the future. The output of cloc's --write-lang-def option should provide enough examples for motivated individuals to modify or extend cloc's language definitions.

Combine Reports

If you manage multiple software projects you might be interested in seeing line counts by project, not just by language. Say you manage three software projects called MariaDB, PostgreSQL, and SQLite. The teams responsible for each of these projects run cloc on their source code and provide you with the output. For example, the MariaDB team does

cloc --out mariadb-10.1.txt mariadb-server-10.1.zip

and provides you with the file mariadb-10.1.txt. The contents of the three files you get are

Unix> cat mariadb-10.1.txt
https://github.com/AlDanial/cloc v 1.65  T=45.36 s (110.5 files/s, 66411.4 lines/s)
-----------------------------------------------------------------------------------
Language                         files          blank        comment           code
-----------------------------------------------------------------------------------
C++                               1613         225338         290077         983026
C                                  853          62442          73017         715018
C/C++ Header                      1327          48300         114577         209394
Bourne Shell                       256          10224          10810          61943
Perl                               147          10342           8305          35562
Pascal                             107           4907           5237          32541
HTML                                56            195              6          16489
Javascript                           5           3309           3019          15540
m4                                  30           1599            359          14215
CMake                              190           1919           4097          12206
XML                                 35            648             56           5210
Ruby                                59            619            184           4998
Puppet                              10              0              1           3848
make                               134            724            360           3631
SQL                                 23            306            377           3405
Python                              34            371            122           2545
Bourne Again Shell                  27            299            380           1604
Windows Module Definition           37             27             13           1211
lex                                  4            394            166            991
yacc                                 2            152             64            810
DOS Batch                           19             89             82            700
Prolog                               1              9             40            448
RobotFramework                       1              0              0            441
CSS                                  2             33            155            393
JSON                                 5              0              0            359
dtrace                               9             59            179            306
Windows Resource File               10             61             89            250
Assembly                             2             70            284            237
WiX source                           1             18             10            155
Visual Basic                         6              0              0             88
YAML                                 2              4              4             65
PHP                                  1             11              2             24
SKILL                                1              8             15             16
sed                                  2              0              0             16
Windows Message File                 1              2              8              6
diff                                 1              1              4              4
D                                    1              4             11              4
-----------------------------------------------------------------------------------
SUM:                              5014         372484         512110        2127699
-----------------------------------------------------------------------------------

Unix> cat sqlite-3081101.txt
https://github.com/AlDanial/cloc v 1.65  T=1.22 s (3.3 files/s, 143783.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                                2          11059          53924         101454
C/C++ Header                     2            211           6630           1546
-------------------------------------------------------------------------------
SUM:                             4          11270          60554         103000
-------------------------------------------------------------------------------

Unix> cat postgresql-9.4.4.txt
https://github.com/AlDanial/cloc v 1.65  T=22.46 s (172.0 files/s, 96721.6 lines/s)
-----------------------------------------------------------------------------------
Language                         files          blank        comment           code
-----------------------------------------------------------------------------------
HTML                              1254           3725              0         785991
C                                 1139         139289         244045         736519
C/C++ Header                       667          12277          32488          57014
SQL                                410          13400           8745          51926
yacc                                 8           3163           2669          28491
Bourne Shell                        41           2647           2440          17170
Perl                                81           1702           1308           9456
lex                                  9            792           1631           4285
make                               205           1525           1554           4114
m4                                  12            218             25           1642
Windows Module Definition           13              4             17           1152
XSLT                                 5             76             55            294
DOS Batch                            7             29             30             92
CSS                                  1             20              7             69
Assembly                             3             17             38             69
D                                    1             14             14             66
Windows Resource File                3              4              0             62
Lisp                                 1              1              1             16
sed                                  1              1              7             15
Python                               1              5              0             13
Bourne Again Shell                   1              8              6             10
Windows Message File                 1              0              0              5
-----------------------------------------------------------------------------------
SUM:                              3864         178917         295080        1698471
-----------------------------------------------------------------------------------

While these three files are interesting, you also want to see the combined counts from all projects. That can be done with cloc's --sum_reports option:

Unix> cloc --sum-reports --out=databases mariadb-10.1.txt  sqlite-3081101.txt  postgresql-9.4.4.txt
Wrote databases.lang
Wrote databases.file

The report combination produces two output files, one for sums by programming language (databases.lang) and one by project (databases.file). Their contents are

Unix> cat databases.lang
https://github.com/AlDanial/cloc v 1.65
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
C                              1994         212790         370986        1552991
C++                            1613         225338         290077         983026
HTML                           1310           3920              6         802480
C/C++ Header                   1996          60788         153695         267954
Bourne Shell                    297          12871          13250          79113
SQL                             433          13706           9122          55331
Perl                            228          12044           9613          45018
Pascal                          107           4907           5237          32541
yacc                             10           3315           2733          29301
m4                               42           1817            384          15857
Javascript                        5           3309           3019          15540
CMake                           190           1919           4097          12206
make                            339           2249           1914           7745
lex                              13           1186           1797           5276
XML                              35            648             56           5210
Ruby                             59            619            184           4998
Puppet                           10              0              1           3848
Python                           35            376            122           2558
Windows Module Definition        50             31             30           2363
Bourne Again Shell               28            307            386           1614
DOS Batch                        26            118            112            792
CSS                               3             53            162            462
Prolog                            1              9             40            448
RobotFramework                    1              0              0            441
JSON                              5              0              0            359
Windows Resource File            13             65             89            312
Assembly                          5             87            322            306
dtrace                            9             59            179            306
XSLT                              5             76             55            294
WiX source                        1             18             10            155
Visual Basic                      6              0              0             88
D                                 2             18             25             70
YAML                              2              4              4             65
sed                               3              1              7             31
PHP                               1             11              2             24
SKILL                             1              8             15             16
Lisp                              1              1              1             16
Windows Message File              2              2              8             11
diff                              1              1              4              4
--------------------------------------------------------------------------------
SUM:                           8882         562671         867744        3929170
--------------------------------------------------------------------------------

Unix> cat databases.file
----------------------------------------------------------------------------------
File                            files          blank        comment           code
----------------------------------------------------------------------------------
mariadb-10.1.txt                 5014         372484         512110        2127699
postgresql-9.4.4.txt             3864         178917         295080        1698471
sqlite-3081101.txt                  4          11270          60554         103000
----------------------------------------------------------------------------------
SUM:                             8882         562671         867744        3929170
----------------------------------------------------------------------------------

Report files themselves can be summed together. Say you also manage development of Perl and Python and you want to keep track of those line counts separately from your database projects. First create reports for Perl and Python separately:

cloc --out perl-5.22.0.txt   perl-5.22.0.tar.gz
cloc --out python-2.7.10.txt Python-2.7.10.tar.xz

then sum these together with

Unix> cloc --sum-reports --out script_lang perl-5.22.0.txt python-2.7.10.txt
Wrote script_lang.lang
Wrote script_lang.file

Unix> cat script_lang.lang
https://github.com/AlDanial/cloc v 1.65
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Perl                          2892         136396         184362         536445
C                              680          75566          71211         531203
Python                        2141          89642         109524         434015
C/C++ Header                   408          16433          26938         214800
Bourne Shell                   154          11088          14496          87759
MSBuild script                  77              0              3          38910
m4                              20           1604            191          15559
Assembly                        51           3775           6794          12298
Pascal                           8            458           1603           8592
make                            16            897            828           4939
XML                             37            198              2           2484
HTML                            14            393             11           2344
C++                             12            338            295           2161
Windows Module Definition        9            171            187           2081
YAML                            49             20             15           2078
Prolog                          12            438              2           1146
JSON                            14              1              0           1037
yacc                             1             85             76            998
DOS Batch                       44            199            148            895
Objective-C                      7             98             61            635
Expect                           6            104            161            565
Windows Message File             1            102             11            489
CSS                              1             98             19            328
Windows Resource File            7             55             56            292
Javascript                       3             31             49            229
vim script                       1             36              7            106
diff                             1             17             87            105
NAnt script                      2              1              0             30
IDL                              1              0              0             24
Visual Basic                     2              1              1             12
D                                1              5              7              8
Lisp                             2              0              3              4
-------------------------------------------------------------------------------
SUM:                          6674         338250         417148        1902571
-------------------------------------------------------------------------------

Unix> cat script_lang.file
-------------------------------------------------------------------------------
File                         files          blank        comment           code
-------------------------------------------------------------------------------
python-2.7.10.txt             3240         161276         173214         998697
perl-5.22.0.txt               3434         176974         243934         903874
-------------------------------------------------------------------------------
SUM:                          6674         338250         417148        1902571
-------------------------------------------------------------------------------

Finally, combine the combination files:

Unix> cloc --sum-reports --report_file=everything databases.lang script_lang.lang
Wrote everything.lang
Wrote everything.file

Unix> cat everything.lang
https://github.com/AlDanial/cloc v 1.65
---------------------------------------------------------------------------------
Language                       files          blank        comment           code
---------------------------------------------------------------------------------
C                               2674         288356         442197        2084194
C++                             1625         225676         290372         985187
HTML                            1324           4313             17         804824
Perl                            3120         148440         193975         581463
C/C++ Header                    2404          77221         180633         482754
Python                          2176          90018         109646         436573
Bourne Shell                     451          23959          27746         166872
SQL                              433          13706           9122          55331
Pascal                           115           5365           6840          41133
MSBuild script                    77              0              3          38910
m4                                62           3421            575          31416
yacc                              11           3400           2809          30299
Javascript                         8           3340           3068          15769
make                             355           3146           2742          12684
Assembly                          56           3862           7116          12604
CMake                            190           1919           4097          12206
XML                               72            846             58           7694
lex                               13           1186           1797           5276
Ruby                              59            619            184           4998
Windows Module Definition         59            202            217           4444
Puppet                            10              0              1           3848
YAML                              51             24             19           2143
DOS Batch                         70            317            260           1687
Bourne Again Shell                28            307            386           1614
Prolog                            13            447             42           1594
JSON                              19              1              0           1396
CSS                                4            151            181            790
Objective-C                        7             98             61            635
Windows Resource File             20            120            145            604
Expect                             6            104            161            565
Windows Message File               3            104             19            500
RobotFramework                     1              0              0            441
dtrace                             9             59            179            306
XSLT                               5             76             55            294
WiX source                         1             18             10            155
diff                               2             18             91            109
vim script                         1             36              7            106
Visual Basic                       8              1              1            100
D                                  3             23             32             78
sed                                3              1              7             31
NAnt script                        2              1              0             30
IDL                                1              0              0             24
PHP                                1             11              2             24
Lisp                               3              1              4             20
SKILL                              1              8             15             16
---------------------------------------------------------------------------------
SUM:                           15556         900921        1284892        5831741
---------------------------------------------------------------------------------

Unix> cat everything.file
-------------------------------------------------------------------------------
File                         files          blank        comment           code
-------------------------------------------------------------------------------
databases.lang                8882         562671         867744        3929170
script_lang.lang              6674         338250         417148        1902571
-------------------------------------------------------------------------------
SUM:                         15556         900921        1284892        5831741
-------------------------------------------------------------------------------

One limitation of the --sum-reports feature is that the individual counts must be saved in the plain text format. Counts saved as XML, JSON, YAML, or SQL will produce errors if used in a summation.

SQL

Cloc can write results in the form of SQL table create and insert statements for use with relational database programs such as SQLite, MySQL, PostgreSQL, Oracle, or Microsoft SQL. Once the code count information is in a database, the information can be interrogated and displayed in interesting ways.

A database created from cloc SQL output has two tables, metadata and t:

Table metadata:

Field	Type
id	integer primary key
timestamp	text
project	text
elapsed_s	text

Table t:

Field	Type
project	text
language	text
file	text
nBlank	integer
nComment	integer
nCode	integer
nScaled	real
foreign key (id)	references metadata (id)

The metadata table contains information about when the cloc run was made. Run time is stored two ways: as Unix epoch seconds in id and as an ISO 8601 formatted text string in the local time zone (for example 2024-03-01 14:19:41) in timestamp. The --sql-append switch allows one to combine many runs in a single database; each run adds a row to the metadata table. The code count information resides in table t. The id key makes it easy to associate a run's code count with its metadata.

Let's repeat the code count examples of Perl, Python, SQLite, MySQL and PostgreSQL tarballs shown in the Combine Reports example above, this time using the SQL output options and the SQLite database engine.

The --sql switch tells cloc to generate output in the form of SQL table create and insert commands. The switch takes an argument of a file name to write these SQL statements into, or, if the argument is 1 (numeric one), streams output to STDOUT. Since the SQLite command line program, sqlite3, can read commands from STDIN, we can dispense with storing SQL statements to a file and use --sql 1 to pipe data directly into the SQLite executable:

cloc --sql 1 --sql-project mariadb mariadb-server-10.1.zip | sqlite3 code.db

The --sql-project mariadb part is optional; there's no need to specify a project name when working with just one code base. However, since we'll be adding code counts from four other tarballs, we'll only be able to identify data by input source if we supply a project name for each run.

Now that we have a database we will need to pass in the --sql-append switch to tell cloc not to wipe out this database but instead add more data:

cloc --sql 1 --sql-project postgresql --sql-append postgresql-9.4.4.tar.bz2        | sqlite3 code.db
cloc --sql 1 --sql-project sqlite     --sql-append sqlite-amalgamation-3081101.zip | sqlite3 code.db
cloc --sql 1 --sql-project python     --sql-append Python-2.7.10.tar.xz            | sqlite3 code.db
cloc --sql 1 --sql-project perl       --sql-append perl-5.22.0.tar.gz              | sqlite3 code.db

Now the fun begins--we have a database, code.db, with lots of information about the five projects and can query it for all manner of interesting facts.

Which is the longest file over all projects?

prompt> sqlite3 code.db 'select project,file,nBlank+nComment+nCode as nL from t
                                 where nL = (select max(nBlank+nComment+nCode) from t)'

sqlite|sqlite-amalgamation-3081101/sqlite3.c|161623

sqlite3's default output format leaves a bit to be desired. We can add an option to the program's rc file, ~/.sqliterc, to show column headers:

  .header on

One might be tempted to also include

  .mode column

in ~/.sqliterc but this causes problems when the output has more than one row since the widths of entries in the first row govern the maximum width for all subsequent rows. Often this leads to truncated output--not at all desirable. One option is to write a custom SQLite output formatter such as sqlite_formatter, included with cloc.

To use it, simply pass sqlite3's STDOUT into sqlite_formatter via a pipe:

prompt> sqlite3 code.db 'select project,file,nBlank+nComment+nCode as nL from t
                         where nL = (select max(nBlank+nComment+nCode) from t)' | ./sqlite_formatter
  <font color="darkgreen">
  -- Loading resources from ~/.sqliterc
  Project File                                  nL
  _______ _____________________________________ ______
  sqlite  sqlite-amalgamation-3081101/sqlite3.c 161623
  </font>

If the "Project File" line doesn't appear, add .header on to your ~/.sqliterc file as explained above.

What is the longest file over all projects?

prompt> sqlite3 code.db 'select project,file,nBlank+nComment+nCode as nL from t
                         where nL = (select max(nBlank+nComment+nCode) from t)' | sqlite_formatter

Project File                                  nL
_______ _____________________________________ ______
sqlite  sqlite-amalgamation-3081101/sqlite3.c 161623

What is the longest file in each project?

prompt> sqlite3 code.db 'select project,file,max(nBlank+nComment+nCode) as nL from t
                          group by project order by nL;' | sqlite_formatter

Project    File                                                             nL
__________ ________________________________________________________________ ______
python     Python-2.7.10/Mac/Modules/qt/_Qtmodule.c                          28091
postgresql postgresql-9.4.4/src/interfaces/ecpg/preproc/preproc.c            54623
mariadb    server-10.1/storage/mroonga/vendor/groonga/lib/nfkc.c             80246
perl       perl-5.22.0/cpan/Locale-Codes/lib/Locale/Codes/Language_Codes.pm 100747
sqlite     sqlite-amalgamation-3081101/sqlite3.c                            161623

Which files in each project have the most code lines?

prompt> sqlite3 code.db 'select project,file,max(nCode) as nL from t
                         group by project order by nL desc;' | sqlite_formatter

Project    File                                                             nL
__________ ________________________________________________________________ ______
perl       perl-5.22.0/cpan/Locale-Codes/lib/Locale/Codes/Language_Codes.pm 100735
sqlite     sqlite-amalgamation-3081101/sqlite3.c                             97469
mariadb    server-10.1/storage/mroonga/vendor/groonga/lib/nfkc.c             80221
postgresql postgresql-9.4.4/src/interfaces/ecpg/preproc/preproc.c            45297
python     Python-2.7.10/Mac/Modules/qt/_Qtmodule.c                          26705

Which C source files with more than 300 lines have a comment ratio below 1%?

prompt> sqlite3 code.db 'select project, file, nCode, nComment,
                         (100.0*nComment)/(nComment+nCode) as comment_ratio from t
                         where language="C" and nCode > 300 and
                         comment_ratio < 1 order by comment_ratio;' | sqlite_formatter

Project    File                                                                                            nCode nComment comment_ratio
__________ _______________________________________________________________________________________________ _____ ________ __________________
mariadb    server-10.1/storage/mroonga/vendor/groonga/lib/nfkc.c                                           80221       14 0.0174487443135789
python     Python-2.7.10/Python/graminit.c                                                                  2175        1 0.0459558823529412
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_turkish.c                            2095        1 0.0477099236641221
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_french.c                             1211        1 0.0825082508250825
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_french.c                        1201        1 0.0831946755407654
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_hungarian.c                          1182        1 0.084530853761623
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_hungarian.c                     1178        1 0.0848176420695505
mariadb    server-10.1/strings/ctype-eucjpms.c                                                             67466       60 0.0888546633889169
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_english.c                            1072        1 0.0931966449207828
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_english.c                       1064        1 0.0938967136150235
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_spanish.c                            1053        1 0.094876660341556
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_spanish.c                       1049        1 0.0952380952380952
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_italian.c                            1031        1 0.0968992248062016
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_italian.c                       1023        1 0.09765625
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_portuguese.c                          981        1 0.10183299389002
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_portuguese.c                     975        1 0.102459016393443
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_romanian.c                            967        1 0.103305785123967
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_2_romanian.c                       961        1 0.103950103950104
mariadb    server-10.1/strings/ctype-ujis.c                                                                67177       79 0.117461639110265
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_finnish.c                             720        1 0.13869625520111
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_porter.c                              717        1 0.139275766016713
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_finnish.c                        714        1 0.13986013986014
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_porter.c                         711        1 0.140449438202247
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_KOI8_R_russian.c                            660        1 0.151285930408472
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_russian.c                             654        1 0.152671755725191
python     Python-2.7.10/Mac/Modules/qt/_Qtmodule.c                                                        26705       42 0.157026956294164
python     Python-2.7.10/Mac/Modules/icn/_Icnmodule.c                                                       1521        3 0.196850393700787
mariadb    server-10.1/strings/ctype-extra.c                                                                8282       18 0.216867469879518
postgresql postgresql-9.4.4/src/bin/psql/sql_help.c                                                         3576        8 0.223214285714286
mariadb    server-10.1/strings/ctype-sjis.c                                                                34006       86 0.252258594391646
python     Python-2.7.10/Python/Python-ast.c                                                                6554       17 0.258712524729874
mariadb    server-10.1/strings/ctype-cp932.c                                                               34609       92 0.265122042592432
perl       perl-5.22.0/keywords.c                                                                           2815        8 0.283386468296139
python     Python-2.7.10/Mac/Modules/menu/_Menumodule.c                                                     3263       10 0.305530094714329
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_dutch.c                               596        2 0.334448160535117
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_dutch.c                          586        2 0.340136054421769
mariadb    server-10.1/strings/ctype-gbk.c                                                                 10684       38 0.354411490393583
python     Python-2.7.10/Mac/Modules/qd/_Qdmodule.c                                                         6694       24 0.357249181303959
python     Python-2.7.10/Mac/Modules/win/_Winmodule.c                                                       3056       11 0.358656667753505
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_german.c                              476        2 0.418410041841004
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_german.c                         470        2 0.423728813559322
mariadb    server-10.1/strings/ctype-euc_kr.c                                                               9956       44 0.44
postgresql postgresql-9.4.4/src/backend/utils/fmgrtab.c                                                     4815       23 0.475403059115337
python     Python-2.7.10/Mac/Modules/ctl/_Ctlmodule.c                                                       5442       28 0.511882998171846
python     Python-2.7.10/Mac/Modules/ae/_AEmodule.c                                                         1347        7 0.51698670605613
python     Python-2.7.10/Mac/Modules/app/_Appmodule.c                                                       1712        9 0.52295177222545
mariadb    server-10.1/strings/ctype-gb2312.c                                                               6377       35 0.54585152838428
mariadb    server-10.1/storage/tokudb/ft-index/third_party/xz-4.999.9beta/src/liblzma/lzma/fastpos_table.c   516        3 0.578034682080925
python     Python-2.7.10/Mac/Modules/evt/_Evtmodule.c                                                        504        3 0.591715976331361
python     Python-2.7.10/Modules/expat/xmlrole.c                                                            1256        8 0.632911392405063
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_UTF_8_danish.c                              312        2 0.636942675159236
postgresql postgresql-9.4.4/src/backend/snowball/libstemmer/stem_ISO_8859_1_danish.c                         310        2 0.641025641025641
python     Python-2.7.10/Mac/Modules/res/_Resmodule.c                                                       1621       12 0.734843845682792
python     Python-2.7.10/Mac/Modules/drag/_Dragmodule.c                                                     1046        8 0.759013282732448
python     Python-2.7.10/Mac/Modules/list/_Listmodule.c                                                     1021        8 0.777453838678329
python     Python-2.7.10/Mac/Modules/te/_TEmodule.c                                                         1198       10 0.827814569536424
python     Python-2.7.10/Mac/Modules/cg/_CGmodule.c                                                         1190       10 0.833333333333333
python     Python-2.7.10/Modules/clmodule.c                                                                 2379       23 0.957535387177352
python     Python-2.7.10/Mac/Modules/folder/_Foldermodule.c                                                  306        3 0.970873786407767

What are the ten longest files (based on code lines) that have no comments at all? Exclude header, .html, and YAML files.

prompt> sqlite3 code.db 'select project, file, nCode from t
                         where nComment = 0 and
                         language not in ("C/C++ Header", "YAML", "HTML")
                         order by nCode desc limit 10;' | sqlite_formatter

Project File                                                                 nCode
_______ ____________________________________________________________________ _____
perl    perl-5.22.0/cpan/Unicode-Collate/Collate/Locale/ja.pl                 1938
python  Python-2.7.10/PCbuild/pythoncore.vcproj                               1889
python  Python-2.7.10/PC/VS8.0/pythoncore.vcproj                              1889
mariadb server-10.1/mysql-test/extra/binlog_tests/mysqlbinlog_row_engine.inc  1862
perl    perl-5.22.0/cpan/Unicode-Collate/Collate/Locale/zh_strk.pl            1589
perl    perl-5.22.0/cpan/Unicode-Collate/Collate/Locale/zh_zhu.pl             1563
mariadb server-10.1/storage/mroonga/vendor/groonga/configure.ac               1526
perl    perl-5.22.0/cpan/Unicode-Collate/Collate/Locale/zh_pin.pl             1505
mariadb server-10.1/mysql-test/suite/funcs_1/storedproc/storedproc_02.inc     1465
python  Python-2.7.10/PC/VS8.0/_bsddb.vcproj                                  1463

What are the most popular languages (in terms of lines of code) in each project?

prompt> sqlite3 code.db 'select project, language, sum(nCode) as SumCode from t
                         group by project,language
                         order by project,SumCode desc;' | sqlite_formatter
Project    Language                  SumCode
__________ _________________________ _______
mariadb    C++                        983026
mariadb    C                          715018
mariadb    C/C++ Header               209394
mariadb    Bourne Shell                61943
mariadb    Perl                        35562
mariadb    Pascal                      32541
mariadb    HTML                        16489
mariadb    Javascript                  15540
mariadb    m4                          14215
mariadb    CMake                       12206
mariadb    XML                          5210
mariadb    Ruby                         4998
mariadb    Puppet                       3848
mariadb    make                         3631
mariadb    SQL                          3405
mariadb    Python                       2545
mariadb    Bourne Again Shell           1604
mariadb    Windows Module Definition    1211
mariadb    lex                           991
mariadb    yacc                          810
mariadb    DOS Batch                     700
mariadb    Prolog                        448
mariadb    RobotFramework                441
mariadb    CSS                           393
mariadb    JSON                          359
mariadb    dtrace                        306
mariadb    Windows Resource File         250
mariadb    Assembly                      237
mariadb    WiX source                    155
mariadb    Visual Basic                   88
mariadb    YAML                           65
mariadb    PHP                            24
mariadb    SKILL                          16
mariadb    sed                            16
mariadb    Windows Message File            6
mariadb    D                               4
mariadb    diff                            4
perl       Perl                       536445
perl       C                          155648
perl       C/C++ Header               147858
perl       Bourne Shell                42668
perl       Pascal                       8592
perl       XML                          2410
perl       YAML                         2078
perl       C++                          2033
perl       make                         1986
perl       Prolog                       1146
perl       JSON                         1037
perl       yacc                          998
perl       Windows Message File          489
perl       DOS Batch                     389
perl       Windows Resource File          85
perl       D                               8
perl       Lisp                            4
postgresql HTML                       785991
postgresql C                          736519
postgresql C/C++ Header                57014
postgresql SQL                         51926
postgresql yacc                        28491
postgresql Bourne Shell                17170
postgresql Perl                         9456
postgresql lex                          4285
postgresql make                         4114
postgresql m4                           1642
postgresql Windows Module Definition    1152
postgresql XSLT                          294
postgresql DOS Batch                      92
postgresql Assembly                       69
postgresql CSS                            69
postgresql D                              66
postgresql Windows Resource File          62
postgresql Lisp                           16
postgresql sed                            15
postgresql Python                         13
postgresql Bourne Again Shell             10
postgresql Windows Message File            5
python     Python                     434015
python     C                          375555
python     C/C++ Header                66942
python     Bourne Shell                45091
python     MSBuild script              38910
python     m4                          15559
python     Assembly                    12298
python     make                         2953
python     HTML                         2344
python     Windows Module Definition    2081
python     Objective-C                   635
python     Expect                        565
python     DOS Batch                     506
python     CSS                           328
python     Javascript                    229
python     Windows Resource File         207
python     C++                           128
python     vim script                    106
python     diff                          105
python     XML                            74
python     NAnt script                    30
python     Prolog                         24
python     Visual Basic                   12
sqlite     C                          101454
sqlite     C/C++ Header                 1546

Custom Column Output

Cloc's default output is a text table with five columns: language, file count, number of blank lines, number of comment lines and number of code lines. The switches --by-file, --3, and --by-percent generate additional information but sometimes even those are insufficient.

The --sql option described in the previous section offers the ability to create custom output. This section has a pair of examples that show how to create custom columns. The first example includes an extra column, Total, which is the sum of the numbers of blank, comment, and code lines. The second shows how to include the language name when running with --by-file.

Example 1: Add a "Totals" column.

The first step is to run cloc and save the output to a relational database, SQLite in this case:

cloc --sql 1 --sql-project x yaml-cpp-yaml-cpp-0.5.3.tar.gz | sqlite3 counts.db

(the tar file comes from the YAML-C++ project).

Second, we craft an SQL query that returns the regular cloc output plus an extra column for totals, then save the SQL statement to a file, query_with_totals.sql:

-- file query_with_totals.sql
select Language, count(File)   as files                       ,
                 sum(nBlank)   as blank                       ,
                 sum(nComment) as comment                     ,
                 sum(nCode)    as code                        ,
                 sum(nBlank)+sum(nComment)+sum(nCode) as Total
    from t group by Language order by code desc;

Third, we run this query through SQLite using the counts.db database. We'll include the -header switch so that SQLite prints the column names:

> cat query_with_totals.sql | sqlite3 -header counts.db
Language|files|blank|comment|code|Total
C++|141|12786|17359|60378|90523
C/C++ Header|110|8566|17420|51502|77488
Bourne Shell|10|6351|6779|38264|51394
m4|11|2037|260|17980|20277
Python|30|1613|2486|4602|8701
MSBuild script|11|0|0|1711|1711
CMake|7|155|285|606|1046
make|5|127|173|464|764
Markdown|2|30|0|39|69

The extra column for Total is there but the format is unappealing. Running the output through sqlite_formatter yields the desired result:

> cat query_with_totals.sql | sqlite3 -header counts.db | sqlite_formatter
Language       files blank comment code  Total
______________ _____ _____ _______ _____ _____
C++              141 12786   17359 60378 90523
C/C++ Header     110  8566   17420 51502 77488
Bourne Shell      10  6351    6779 38264 51394
m4                11  2037     260 17980 20277
Python            30  1613    2486  4602  8701
MSBuild script    11     0       0  1711  1711
CMake              7   155     285   606  1046
make               5   127     173   464   764
Markdown           2    30       0    39    69

The next section, Wrapping cloc in other scripts, shows one way these commands can be combined into a new utility program.

Example 2: Include a column for "Language" when running with --by-file.

Output from --by-file omits each file's language to save screen real estate; file paths for large projects can be long and including an extra 20 or so characters for a Language column can be excessive.

As an example, here are the first few lines of output using the same code base as in Example 1:

> cloc --by-file yaml-cpp-yaml-cpp-0.5.3.tar.gz
github.com/AlDanial/cloc v 1.81  T=1.14 s (287.9 files/s, 221854.9 lines/s)
--------------------------------------------------------------------------------------------------------------------------------------------
File                                                                                                     blank        comment           code
--------------------------------------------------------------------------------------------------------------------------------------------
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/configure                                                        2580           2264          13691
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/configure                                                  2541           2235          13446
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/fused-src/gtest/gtest.h                                    1972           4681          13408
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/fused-src/gmock/gmock.h                                          1585           3397           9216
yaml-cpp-yaml-cpp-0.5.3/test/integration/gen_emitter_test.cpp                                              999              0           8760
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/aclocal.m4                                                        987            100           8712
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/m4/libtool.m4                                               760             65           7176
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/build-aux/ltmain.sh                                         959           1533           7169
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/fused-src/gmock-gtest-all.cc                                     1514           3539           6390
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/fused-src/gtest/gtest-all.cc                               1312           2896           5384
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/test/gtest_unittest.cc                                     1226           1091           5098
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/include/gtest/internal/gtest-param-util-generated.h         349            235           4559

The absence of language identification for each file is a bit disappointing, but this can be remedied with a custom column solution.

The first step, creating a database, matches that from Example 1 so we'll go straight to the second step of creating the desired SQL query. We'll store this one in the file by_file_with_language.sql:

-- file by_file_with_language.sql
select File, Language, nBlank   as blank  ,
                       nComment as comment,
                       nCode    as code
    from t order by code desc;

Our desired extra column appears when we pass this custom SQL query through our database:

> cat by_file_with_language.sql | sqlite3 -header counts.db | sqlite_formatter
File                                                                                               Language       blank comment code
__________________________________________________________________________________________________ ______________ _____ _______ _____
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/configure                                                 Bourne Shell    2580    2264 13691
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/configure                                           Bourne Shell    2541    2235 13446
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/fused-src/gtest/gtest.h                             C/C++ Header    1972    4681 13408
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/fused-src/gmock/gmock.h                                   C/C++ Header    1585    3397  9216
yaml-cpp-yaml-cpp-0.5.3/test/integration/gen_emitter_test.cpp                                      C++              999       0  8760
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/aclocal.m4                                                m4               987     100  8712
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/m4/libtool.m4                                       m4               760      65  7176
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/build-aux/ltmain.sh                                 Bourne Shell     959    1533  7169
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/fused-src/gmock-gtest-all.cc                              C++             1514    3539  6390
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/fused-src/gtest/gtest-all.cc                        C++             1312    2896  5384
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/test/gtest_unittest.cc                              C++             1226    1091  5098
yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/include/gtest/internal/gtest-param-util-generated.h C/C++ Header     349     235  4559

Wrapping cloc in other scripts

More complex code counting solutions are possible by wrapping cloc in scripts or programs. The "total lines" column from example 1 of Custom Column Output could be simplified to a single command with this shell script (on Linux):

#!/bin/sh
#
# These commands must be in the user's $PATH:
#   cloc
#   sqlite3
#   sqlite_formatter
#
if test $# -eq 0 ; then
    echo "Usage: $0  [cloc arguments]"
    echo "       Run cloc to count lines of code with an additional"
    echo "       output column for total lines (code+comment+blank)."
    exit
fi
DBFILE=`tempfile`
cloc --sql 1 --sql-project x $@ | sqlite3 ${DBFILE}
SQL="select Language, count(File)   as files                       ,
                      sum(nBlank)   as blank                       ,
                      sum(nComment) as comment                     ,
                      sum(nCode)    as code                        ,
                      sum(nBlank)+sum(nComment)+sum(nCode) as Total
         from t group by Language order by code desc;
"
echo ${SQL} | sqlite3 -header ${DBFILE} | sqlite_formatter
rm ${DBFILE}

Saving the lines above to total_columns.sh and making it executable (chmod +x total_columns.sh) would let us do

./total_columns.sh yaml-cpp-yaml-cpp-0.5.3.tar.gz

to directly get

Language       files blank comment code  Total
______________ _____ _____ _______ _____ _____
C++              141 12786   17359 60378 90523
C/C++ Header     110  8566   17420 51502 77488
Bourne Shell      10  6351    6779 38264 51394
m4                11  2037     260 17980 20277
Python            30  1613    2486  4602  8701
MSBuild script    11     0       0  1711  1711
CMake              7   155     285   606  1046
make               5   127     173   464   764
Markdown           2    30       0    39    69

Other examples:

Count code from a specific branch of a web-hosted git repository and send the results as a .csv email attachment: https://github.com/dannyloweatx/checkmarx

git and UTF8 pathnames

cloc's --git option may fail if you work with directory or file names with UTF-8 characters (for example, see issue 457). The solution, https://stackoverflow.com/questions/22827239/how-to-make-git-properly-display-utf-8-encoded-pathnames-in-the-console-window, is to apply this git configuration command:

git config --global core.quotepath off

Your console's font will need to be capable of displaying Unicode characters.

Third Generation Language Scale Factors

cloc versions before 1.50 by default computed, for the provided inputs, a rough estimate of how many lines of code would be needed to write the same code in a hypothetical third-generation computer language. To produce this output one must now use the --3 switch.

Scale factors were derived from the 2006 version of language gearing ratios listed at Mayes Consulting web site, http://softwareestimator.com/IndustryData2.htm, using this equation:

cloc scale factor for language X = 3rd generation default gearing ratio / language X gearing ratio

For example, cloc 3rd generation scale factor for DOS Batch = 80 / 128 = 0.625.

The biggest flaw with this approach is that gearing ratios are defined for logical lines of source code not physical lines (which cloc counts). The values in cloc's 'scale' and '3rd gen. equiv.' columns should be taken with a large grain of salt.

options.txt configuration file

If you find yourself using the same command line switches every time you invoke cloc, you can save some typing by adding those switches to the options.txt runtime configuration file. cloc will look for this file in the following default locations:

# Linux, NetBSD, FreeBSD, macOS:
/home/USERNAME/.config/cloc/options.txt

# Windows
C:\Users\USERNAME\AppData\Roaming\cloc

If you run cloc with --help, cloc will tell you where it expects to find this config file file. The information appears by the explanation of the --config switch after the text the default location of. On Unix-like operating systems, this can be simplified to

> cloc --help | grep "default location"
             the default location of /home/al/.config/cloc/options.txt.

and in a Windows cmd terminal with

> cloc --help | findstr default | findstr location
             the default location of C:\Users\al\AppData\Roaming\cloc

Place each switch and arguments, if any, on a line by itself. Lines prefixed with # symbol are ignored as comments and blank lines are skipped. Leading hyphens on the switches are optional. Here's a sample file:

# options.txt
--vcs git
v      # verbose level 1
exclude-ext svg,html

The path to the options.txt file can also be specified with the --config FILE switch.

Finally, if cloc finds an options.txt file in the same directory as files given by any of these switches (in the listed priority), it will use that configuration file from that location:

--list-file
--exclude-list-file
--read-lang-def
--force-lang-def
--diff-list-file

Run with --verbose to have cloc tell you which, if any, options.txt file it uses.

Python Programmatic Interface

Stefano Campanella created a Python programmatic interface to cloc. It is available at https://github.com/pycloc/pycloc

Java Programmatic Interface

Ozren Dabić created a Java programmatic interface to cloc. It is available at https://github.com/seart-group/jcloc

Complex regular subexpression recursion limit

cloc relies on the Regexp::Common module's regular expressions to remove comments from source code. If comments are malformed, for example the /* start comment marker appears in a C program without a corresponding */ marker, the regular expression engine could enter a recursive loop, eventually triggering the warning Complex regular subexpression recursion limit.

The most common cause for this warning is the existence of comment markers in string literals. While language compilers and interpreters are smart enough to recognize that "/*" (for example) is a string and not a comment, cloc is fooled. File path globs, as in this line of JavaScript

var paths = globArray("**/*.js", {cwd: srcPath});

are frequent culprits.

In an attempt to overcome this problem, a different algorithm which removes comment markers in strings can be enabled with the --strip-str-comments switch. Doing so, however, has drawbacks: cloc will run more slowly and the output of --strip-comments will contain strings that no longer match the input source.

Limitations

Identifying comments within source code is trickier than one might expect. Many languages would need a complete parser to be counted correctly. cloc does not attempt to parse any of the languages it aims to count and therefore is an imperfect tool. The following are known problems:

Lines containing both source code and comments are counted as lines of code.
Comment markers within strings or here-documents are treated as actual comment markers and not string literals. For example the following lines of C code ``` printf(" /* "); for (i = 0; i < 100; i++) { a += i; } printf(" */ "); ``` look to cloc like this: ``` printf(" xxxxxxx xxxxxxx xxxxxxx xxxxxxx xxxxxxx "); ``` where `xxxxxxx` represents cloc's view of commented text. Therefore cloc counts the five lines as two lines of C code and three lines of comments (lines with both code and comment are counted as code).
If you suspect your code has such strings, use the switch --strip-str-comments to switch to the algorithm which removes embedded comment markers. Its use will render the five lines above as
```
printf("  ");
for (i = 0; i < 100; i++) {
    a += i;
}
printf("  ");
```
and therefore return a count of five lines of code. See the previous section on drawbacks to using --strip-str-comments.
Embedded languages are not recognized. For example, an HTML file containing JavaScript will be counted entirely as HTML.
Python docstrings can serve several purposes. They may contain documentation, comment out blocks of code, or they can be regular strings (when they appear on the right hand side of an assignment or as a function argument). cloc is unable to infer the meaning of docstrings by context; by default cloc treats all docstrings as comments. The switch `--docstring-as-code` treats all docstrings as code.
Language definition files read with `--read-lang-def` or `--force-lang-def` must be plain ASCII text files.
cloc treats compiler pragma's, for example `#if` / `#endif`, as code even if these are used to block lines of source from being compiled; the blocked lines still contribute to the code count.
On Windows, cloc will fail with Can't cd to ... No such file or directory at /File/Find.pm if the code being scanned has file paths longer than 255 characters. A work-around is to run cloc from the Windows Subsystem for Linux (WSL).
cloc's comment match code uses regular expressions which cannot properly account for nested comments using the same comment markers (such as `/* /* */ */`).
XML comments embedded within `CDATA` blocks are counted as comments rather than code.

Support for Additional Languages

If cloc does not recognize a language you are interested in counting, you can either

Implement an Additional Language

In short, you will need to:

edit the cloc file:
- look for %{$rh_Language_by_Extension} = ( # {{{1 and add, respecting the alphabetical order,
```
'<ext>' => '<Language Name>'           ,
```
  where <ext> is the extension used by your language, and <Language Name> is its name. Repeat for each extension used by your language.
- look for %{$rhaa_Filters_by_Language} = ( # {{{1, and add, respecting the alphabetical order,
```
 '<Language Name>'     => [
                              <list>
                          ],
```
where <Language Name> is the name of the language you are implementing, and <list> is a list of filter options (whose documentation remains to be written -- take inspiration from existing languages).
- look for %{$rh_Scale_Factor} = ( # {{{1, and add, respecting the alphabetical order,
```
'<Language Name>'             =>   <factor>,
```
  where <Language Name> is the name of the language you are implementing, and <factor> is a third generation language scale factor.
edit the Readme.md file, adding under "Recognized Languages"
```
<Language Name>                    (<list of extensions>)
```
for <Language Name> the name of the language you are implementing, and <list of extensions> the list of extensions you declared in rh_Language_by_Extension.
(optional, but recommended) Add a test case:
- Create a tests/inputs/<Language name>.<extension> file, containing some code written in <Language name>,
- Generate a ‎tests/outputs/<Language name>.<extension>.yaml file, using
```
cloc --yaml tests/inputs/<Language name>.<extension> --out ‎tests/outputs/<Language name>.<extension>.yaml
```
- Add your case in Unix/t/00_C.t: under my @Tests = (, add
```
{
  'name' => '<Language name>',
  'ref'  => '../tests/outputs/<Language name>.<extension>.yaml',
  'args' => '../tests/inputs/<Language name>.<extension>',
},
```
- Make sure you examine the output of ‎tests/outputs/<Language name>.<extension>.yaml and that it is correct, then run make test from the Unix folder to make sure your test was successful.

You can use this pull request or this one as guidelines.

Request an Additional Language

Create a GitHub issue requesting support for your language. Include this information:

File extensions associated with the language. If the language does not rely on file extensions and instead works with fixed file names or with `#!` style program invocations, explain what those are.
A description of how comments are defined.
Links to sample code.

Reporting Problems

If you encounter a problem with cloc, first check to see if you're running with the latest version of the tool:

  cloc --version

If the version is older than the most recent release at https://github.com/AlDanial/cloc/releases, download the latest version and see if it solves your problem.

If the problem happens with the latest release, submit a new issue at https://github.com/AlDanial/cloc/issues only if you can supply enough information for anyone reading the issue report to reproduce the problem. That means providing

the operating system you're running on
the cloc command with all options
the code you are counting (URL to a public git repo or zip file or tar file, et cetera)

The last item is generally problematic. If the code base is proprietary or amounts to more than a few dozen kilobytes, you'll need to try to reconstruct similar inputs or demonstrate the problem with an existing public code base.

Problem reports that cannot be reproduced will be ignored and eventually closed.

Citation

Please use the following bibtex entry to cite cloc in a publication:

@software{adanial_cloc,
  author       = {Albert Danial},
  title        = {cloc: v2.08},
  month        = dec,
  year         = 2026,
  publisher    = {Zenodo},
  version      = {v2.08},
  doi          = {10.5281/zenodo.5760077},
  url          = {https://doi.org/10.5281/zenodo.5760077}
}

(Update the version number and corresponding year if this entry is outdated.)

Acknowledgments

Wolfram Rösler provided most of the code examples in the test suite. These examples come from his Hello World collection.

Ismet Kursunoglu found errors with the MUMPS counter and provided access to a computer with a large body of MUMPS code to test cloc.

Tod Huggins gave helpful suggestions for the Visual Basic filters.

Anton Demichev found a flaw with the JSP counter in cloc v0.76 and wrote the XML output generator for the --xml option.

Reuben Thomas pointed out that ISO C99 allows // as a comment marker, provided code for the --no3 and --stdin-name options, counting the m4 language, and suggested several user-interface enhancements.

Michael Bello provided code for the --opt-match-f, --opt-not-match-f, --opt-match-d, and --opt-not-match-d options.

Mahboob Hussain inspired the --original-dir and --skip-uniqueness options, found a bug in the duplicate file detection logic and improved the JSP filter.

Randy Sharo found and fixed an uninitialized variable bug for shell scripts having only one line.

Steven Baker found and fixed a problem with the YAML output generator.

Greg Toth provided code to improve blank line detection in COBOL.

Joel Oliveira provided code to let --exclude-list-file handle directory name exclusion.

Blazej Kroll provided code to produce an XSLT file, cloc-diff.xsl, when producing XML output for the --diff option.

Denis Silakov enhanced the code which generates cloc.xsl when using --by-file and --by-file-by-lang options, and provided an XSL file that works with --diff output.

Andy (awalshe@sf.net) provided code to fix several bugs: correct output of --counted so that only files that are used in the code count appear and that results are shown by language rather than file name; allow --diff output from multiple runs to be summed together with --sum-reports.

Jari Aalto created the initial version of cloc.1.pod and maintains the Debian package for cloc.

Mikkel Christiansen (mikkels@gmail.com) provided counter definitions for Clojure and ClojureScript.

Vera Djuraskovic from Webhostinggeeks.com provided the Serbo-Croatian translation.

Gill Ajoft of Ajoft Software provided the Bulgarian translation.

The Knowledge Team provided the Slovakian translation.

Erik Gooven Arellano Casillas provided an update to the MXML counter to recognize ActionScript comments.

Gianluca Casati created the cloc CPAN package.

Ryan Lindeman implemented the --by-percent feature.

Kent C. Dodds, @kentcdodds, created and maintains the npm package of cloc.

Viktoria Parnak provided the Ukrainian translation.

Natalie Harmann provided the Belarussian translation.

Nithyal at Healthcare Administration Portal provided the Tamil translation.

Patricia Motosan provided the Romanian translation.

Gajk Melikyan provided the provided the Armenian translation for http://studybay.com.

Hungarian translation courtesy of Zsolt Boros.

Sietse Snel implemented the parallel processing capability available with the --processes=*N* switch.

The development of cloc was partially funded by the Northrop Grumman Corporation.

Copyright

tseemann/prokka

⚡ ♒ Rapid prokaryotic genome annotation

Prokka: rapid prokaryotic genome annotation

Introduction

Whole genome annotation is the process of identifying features of interest in a set of genomic DNA sequences, and labelling them with useful information. Prokka is a software tool to annotate bacterial, archaeal and viral genomes quickly and produce standards-compliant output files.

Bakta is the next generation of Prokka

Prokka has served the community well for over a decade. I can no longer maintain it, so, with my blessing and gratitude, Oliver Schwengers has taken the bacterial annotation toruch and developed Bakta which is a modern version of Prokka with better tooling and maintained databases, including db-light which is in the spirit of Prokka's "fast and lean" approach. I recommend replacing Prokka with Bakta in your analysis pipelines going forward.

Installation

Bioconda

The best way to install Prokka is using Conda (miniforge).

conda create -n prokka bioconda::prokka
conda activate prokka
prokka --version

Docker

Maintained by STAPHB:


docker pull staphb/prokka:latest
docker run staphb/prokka:latest prokka --version

Singularity

singularity build prokka.sif docker://staphb/prokka:latest
singularity exec prokka.sif prokka -h

Test

Type prokka and it should output its help screen.
Type prokka --version and you should see an output like prokka 1.x
Type prokka --listdb and it will show you what databases it has installed to use.

Invoking Prokka

Beginner

# Vanilla (but with free toppings)
% prokka contigs.fa

# Look for a folder called PROKKA_yyyymmdd (today's date) and look at stats
% cat PROKKA_yyyymmdd/*.txt

Moderate

# Choose the names of the output files
% prokka --outdir mydir --prefix mygenome contigs.fa

# Visualize it in Artemis
% art mydir/mygenome.gff

Specialist

# Have curated genomes I want to use to annotate from
% prokka --proteins MG1655.gbk --outdir mutant --prefix K12_mut contigs.fa

# Look at tabular features
% less -S mutant/K12_mut.tsv

Expert

# It's not just for bacteria, people
% prokka --kingdom Archaea --outdir mydir --genus Pyrococcus --locustag PYCC

# Search for your favourite gene
% exonerate --bestn 1 zetatoxin.fasta mydir/PYCC_06072012.faa | less

Wizard

# Watch and learn
% prokka --outdir mydir --locustag EHEC --proteins NewToxins.faa --evalue 0.001 --gram neg --addgenes contigs.fa

# Check to see if anything went really wrong
% less mydir/EHEC_06072012.err

# Add final details using Sequin
% sequin mydir/EHEC_0607201.sqn

NCBI Genbank submitter

# Register your BioProject (e.g. PRJNA123456) and your locus_tag prefix (e.g. EHEC) first!
% prokka --compliant --centre UoN --outdir PRJNA123456 --locustag EHEC --prefix EHEC-Chr1 contigs.fa

# Check to see if anything went really wrong
% less PRJNA123456/EHEC-Chr1.err

# Add final details using Sequin
% sequin PRJNA123456/EHEC-Chr1.sqn

European Nucleotide Archive (ENA) submitter

# Register your BioProject (e.g. PRJEB12345) and your locus_tag (e.g. EHEC) prefix first!
% prokka --compliant --centre UoN --outdir PRJEB12345 --locustag EHEC --prefix EHEC-Chr1 contigs.fa

# Check to see if anything went really wrong
% less PRJNA123456/EHEC-Chr1.err

# Install and run Sanger Pathogen group's Prokka GFF3 to EMBL converter
# available from https://github.com/sanger-pathogens/gff3toembl
# Find the closest NCBI taxonomy id (e.g. 562 for Escherichia coli)
% gff3_to_embl -i "Submitter, A." \
    -m "Escherichia coli EHEC annotated using Prokka." \
    -g linear -c PROK -n 11 -f PRJEB12345/EHEC-Chr1.embl \
    "Escherichia coli" 562 PRJEB12345 "Escherichia coli strain EHEC" PRJEB12345/EHEC-Chr1.gff

# Download and run the latest EMBL validator prior to submitting the EMBL flat file
# from http://central.maven.org/maven2/uk/ac/ebi/ena/sequence/embl-api-validator/
# which at the time of writing is v1.1.129
% curl -L -O http://central.maven.org/maven2/uk/ac/ebi/ena/sequence/embl-api-validator/1.1.129/embl-api-validator-1.1.129.jar
% java -jar embl-api-validator-1.1.129.jar -r PRJEB12345/EHEC-Chr1.embl

# Compress the file ready to upload to ENA, and calculate MD5 checksum
% gzip PRJEB12345/EHEC-Chr1.embl
% md5sum PRJEB12345/EHEC-Chr1.embl.gz

Crazy Person

# No stinking Perl script is going to control me
% prokka \
        --outdir $HOME/genomes/Ec_POO247 --force \
        --prefix Ec_POO247 --addgenes --locustag ECPOOp \
        --increment 10 --gffver 2 --centre CDC  --compliant \
        --genus Escherichia --species coli --strain POO247 --plasmid pECPOO247 \
        --kingdom Bacteria --gcode 11 --usegenus \
        --proteins /opt/prokka/db/trusted/Ecocyc-17.6 \
        --evalue 1e-9 --rfam \
        plasmid-closed.fna

Output Files

Extension	Description
.gff	This is the master annotation in GFF3 format, containing both sequences and annotations. It can be viewed directly in Artemis or IGV.
.gbk	This is a standard Genbank file derived from the master .gff. If the input to prokka was a multi-FASTA, then this will be a multi-Genbank, with one record for each sequence.
.fna	Nucleotide FASTA file of the input contig sequences.
.faa	Protein FASTA file of the translated CDS sequences.
.ffn	Nucleotide FASTA file of all the prediction transcripts (CDS, rRNA, tRNA, tmRNA, misc_RNA)
.sqn	An ASN1 format "Sequin" file for submission to Genbank. It needs to be edited to set the correct taxonomy, authors, related publication etc.
.fsa	Nucleotide FASTA file of the input contig sequences, used by "tbl2asn" to create the .sqn file. It is mostly the same as the .fna file, but with extra Sequin tags in the sequence description lines.
.tbl	Feature Table file, used by "tbl2asn" to create the .sqn file.
.err	Unacceptable annotations - the NCBI discrepancy report.
.log	Contains all the output that Prokka produced during its run. This is a record of what settings you used, even if the --quiet option was enabled.
.txt	Statistics relating to the annotated features found.
.tsv	Tab-separated file of all features: locus_tag,ftype,len_bp,gene,EC_number,COG,product

Command line options

General:
  --help            This help
  --version         Print version and exit
  --citation        Print citation for referencing Prokka
  --quiet           No screen output (default OFF)
  --debug           Debug mode: keep all temporary files (default OFF)
Setup:
  --listdb          List all configured databases
  --setupdb         Index all installed databases
  --cleandb         Remove all database indices
  --depends         List all software dependencies
Outputs:
  --outdir [X]      Output folder [auto] (default '')
  --force           Force overwriting existing output folder (default OFF)
  --prefix [X]      Filename output prefix [auto] (default '')
  --addgenes        Add 'gene' features for each 'CDS' feature (default OFF)
  --locustag [X]    Locus tag prefix (default 'PROKKA')
  --increment [N]   Locus tag counter increment (default '1')
  --gffver [N]      GFF version (default '3')
  --compliant       Force Genbank/ENA/DDJB compliance: --genes --mincontiglen 200 --centre XXX (default OFF)
  --centre [X]      Sequencing centre ID. (default '')
Organism details:
  --genus [X]       Genus name (default 'Genus')
  --species [X]     Species name (default 'species')
  --strain [X]      Strain name (default 'strain')
  --plasmid [X]     Plasmid name or identifier (default '')
Annotations:
  --kingdom [X]     Annotation mode: Archaea|Bacteria|Mitochondria|Viruses (default 'Bacteria')
  --gcode [N]       Genetic code / Translation table (set if --kingdom is set) (default '0')
  --prodigaltf [X]  Prodigal training file (default '')
  --gram [X]        Gram: -/neg +/pos (default '')
  --usegenus        Use genus-specific BLAST databases (needs --genus) (default OFF)
  --proteins [X]    Fasta file of trusted proteins to first annotate from (default '')
  --hmms [X]        Trusted HMM to first annotate from (default '')
  --metagenome      Improve gene predictions for highly fragmented genomes (default OFF)
  --rawproduct      Do not clean up /product annotation (default OFF)
Computation:
  --fast            Fast mode - skip CDS /product searching (default OFF)
  --cpus [N]        Number of CPUs to use [0=all] (default '8')
  --mincontiglen [N] Minimum contig size [NCBI needs 200] (default '1')
  --evalue [n.n]    Similarity e-value cut-off (default '1e-06')
  --rfam            Enable searching for ncRNAs with Infernal+Rfam (SLOW!) (default '0')
  --norrna          Don't run rRNA search (default OFF)
  --notrna          Don't run tRNA search (default OFF)
  --rnammer         Prefer RNAmmer over Barrnap for rRNA prediction (default OFF)

Option: --proteins

The --proteins option is recommended when you have good quality reference genomes and want to ensure gene naming is consistent. Some species use specific terminology which will be often lost if you rely on the default Swiss-Prot database included with Prokka.

If you have Genbank or Protein FASTA file(s) that you want to annotate genes from as the first priority, use the --proteins myfile.gbk. Please make sure it has a recognisable file extension like .gb or .gbk or auto-detect will fail. The use of Genbank is recommended over FASTA, because it will provide /gene and /EC_number annotations that a typical .faa file will not provide, unless you have specially formatted it for Prokka.

Option: --prodigaltf

Instead of letting prodigal train its gene model on the contigs you provide, you can pre-train it on some good closed reference genomes first using the prodigal -t option. Once you've done that, provide prokka the training file using the --prodgialtf option.

Option: --rawproduct

Prokka annotates proteins by using sequence similarity to other proteins in its database, or the databases the user provides via --proteins. By default, Prokka tries to "cleans" the /product names to ensure they are compliant with Genbank/ENA conventions. Some of the main things it does is:

set vague names to hypothetical protein
consistifies terms like possible, probable, predicted, ... to putative
removes EC, COG and locus_tag identifiers

Full details can be found in the cleanup_product() function in the prokka script. If you feel your annotations are being ruined, try using the --rawproduct option, and please file an issue if you find an example of where it is "behaving badly" and I will fix it.

Databases

The Core (BLAST+) Databases

Prokka uses a variety of databases when trying to assign function to the predicted CDS features. It takes a hierarchical approach to make it fast.
A small, core set of well characterized proteins are first searched using BLAST+. This combination of small database and fast search typically completes about 70% of the workload. Then a series of slower but more sensitive HMM databases are searched using HMMER3.

The three core databases, applied in order, are:

ISfinder: Only the tranposase (protein) sequences; the whole transposon is not annotated.
NCBI Bacterial Antimicrobial Resistance Reference Gene Database: Antimicrobial resistance genes curated by NCBI.
UniProtKB (SwissProt): For each --kingdom we include curated proteins with evidence that (i) from Bacteria (or Archaea or Viruses); (ii) not be "Fragment" entries; and (iii) have an evidence level ("PE") of 2 or lower, which corresponds to experimental mRNA or proteomics evidence.

Making a Core Databases

If you want to modify these core databases, the included script prokka-uniprot_to_fasta_db, along with the official uniprot_sprot.dat, can be used to generate a new database to put in /opt/prokka/db/kingdom/. If you add new ones, the command prokka --listdb will show you whether it has been detected properly.

The Genus Databases

⚠ This is no longer recommended. Please use --proteins instead.

If you enable --usegenus and also provide a Genus via --genus then it will first use a BLAST database which is Genus specific. Prokka comes with a set of databases for the most common Bacterial genera; type prokka --listdb to see what they are.

Adding a Genus Databases

If you have a set of Genbank files and want to create a new Genus database, Prokka comes with a tool called prokka-genbank_to_fasta_db to help. For example, if you had four annotated "Coccus" genomes, you could do the following:

% prokka-genbank_to_fasta_db Coccus1.gbk Coccus2.gbk Coccus3.gbk Coccus4.gbk > Coccus.faa
% cd-hit -i Coccus.faa -o Coccus -T 0 -M 0 -g 1 -s 0.8 -c 0.9
% rm -fv Coccus.faa Coccus.bak.clstr Coccus.clstr
% makeblastdb -dbtype prot -in Coccus
% mv Coccus.p* /path/to/prokka/db/genus/

The HMM Databases

Prokka comes with a bunch of HMM libraries for HMMER3. They are mostly Bacteria-specific. They are searched after the core and genus databases. You can add more simply by putting them in /opt/prokka/db/hmm. Type prokka --listdb to confirm they are recognised.

FASTA database format

Prokka understands two annotation tag formats, a plain one and a detailed one.

The plain one is a standard FASTA-like line with the ID after the > sign, and the protein /product after the ID (the "description" part of the line):

>SeqID product

The detailed one consists of a special encoded three-part description line. The parts are the /EC_number, the /gene code, then the /product - and they are separated by a special "~~~" sequence:

>SeqID EC_number~~~gene~~~product~~~COG

Here are some examples. Note that not all parts need to be present, but the "~~~" should still be there:

>YP_492693.1 2.1.1.48~~~ermC~~~rRNA adenine N-6-methyltransferase~~~COG1234
MNEKNIKHSQNFITSKHNIDKIMTNIRLNEHDNIFEIGSGKGHFTLELVQRCNFVTAIEI
DHKLCKTTENKLVDHDNFQVLNKDILQFKFPKNQSYKIFGNIPYNISTDIIRKIVF*
>YP_492697.1 ~~~traB~~~transfer complex protein TraB~~~
MIKKFSLTTVYVAFLSIVLSNITLGAENPGPKIEQGLQQVQTFLTGLIVAVGICAGVWIV
LKKLPGIDDPMVKNEMFRGVGMVLAGVAVGAALVWLVPWVYNLFQ*
>YP_492694.1 ~~~~~~transposase~~~
MNYFRYKQFNKDVITVAVGYYLRYALSYRDISEILRGRGVNVHHSTVYRWVQEYAPILYQ
QSINTAKNTLKGIECIYALYKKNRRSLQIYGFSPCHEISIMLAS*

The same description lines apply to HMM models, except the "NAME" and "DESC" fields are used:

NAME  PRK00001
ACC   PRK00001
DESC  2.1.1.48~~~ermC~~~rRNA adenine N-6-methyltransferase~~~COG1234
LENG  284

FAQ

Where does the name "Prokka" come from?
Prokka is a contraction of "prokaryotic annotation". It's also relatively unique within Google, and also rhymes with a native Australian marsupial called the quokka.
Can I annotate by eukaryote genome with Prokka?
No. Prokka is specifically designed for Bacteria, Archaea and Viruses. It can't handle multi-exon gene models; I would recommend using MAKER 2 for that purpose.
Why does Prokka keeps on crashing when it gets to the "tbl2asn" stage?
It seems that the tbl2asn program from NCBI "expires" after 6-12 months, and refuses to run. Unfortunately you need to install a newer version which you can download from here.
The hmmscan step seems to hang and do nothing?
The problem here is GNU Parallel. It seems the Debian package for hmmer has modified it to require the --gnu option to behave in the 'default' way. There is no clear reason for this. The only way to restore normal behaviour is to edit the prokka script and change parallel to parallel --gnu.
Why does prokka fail when it gets to hmmscan?
Unfortunately HMMER keeps changing its database format, and they aren't upward compatible. If you upgraded HMMER (from 3.0 to 3.1 say) then you need to "re-press" the files. This can be done as follows:

cd /path/to/prokka/db/hmm
mkdir new
for D in *.hmm ; do hmmconvert $D > new/$D ; done
cd new
for D in *.hmm ; do hmmpress $D ; done
mv * ..
rmdir new

Why can't I load Prokka .GBK files into Mauve?
Mauve uses BioJava to parse GenBank files, and it is very picky about Genbank files. It does not like long contig names, like those from Velvet or Spades. One solution is to use --centre XXX in Prokka and it will rename all your contigs to be NCBI (and Mauve) compliant. It does not like the ACCESSION and VERSION strings that Prokka produces via the "tbl2asn" tool. The following Unix command will fix them: egrep -v '^(ACCESSION|VERSION)' prokka.gbk > mauve.gbk
How can I make my GFF not have the contig sequences in it?

sed '/^##FASTA/Q' prokka.gff > nosequence.gff

Dependencies

Mandatory

BioPerl
Used for input/output of various file formats
Stajich et al, The Bioperl toolkit: Perl modules for the life sciences. Genome Res. 2002 Oct;12(10):1611-8.
GNU Parallel
A shell tool for executing jobs in parallel using one or more computers
O. Tange, GNU Parallel - The Command-Line Power Tool, ;login: The USENIX Magazine, Feb 2011:42-47.
BLAST+
Used for similarity searching against protein sequence libraries
Camacho C et al. BLAST+: architecture and applications. BMC Bioinformatics. 2009 Dec 15;10:421.
Prodigal
Finds protein-coding features (CDS)
Hyatt D et al. Prodigal: prokaryotic gene recognition and translation initiation site identification. BMC Bioinformatics. 2010 Mar 8;11:119.
TBL2ASN Prepare sequence records for Genbank submission Tbl2asn home page

Aragorn
Finds transfer RNA features (tRNA)
Laslett D, Canback B. ARAGORN, a program to detect tRNA genes and tmRNA genes in nucleotide sequences. Nucleic Acids Res. 2004 Jan 2;32(1):11-6.
Barrnap
Used to predict ribosomal RNA features (rRNA). My licence-free replacement for RNAmmmer.
Manuscript under preparation.
HMMER3
Used for similarity searching against protein family profiles
Finn RD et al. HMMER web server: interactive sequence similarity searching. Nucleic Acids Res. 2011 Jul;39(Web Server issue):W29-37.

Optional

minced
Finds CRISPR arrays Minced home page
RNAmmer
Finds ribosomal RNA features (rRNA)
Lagesen K et al. RNAmmer: consistent and rapid annotation of ribosomal RNA genes. Nucleic Acids Res. 2007;35(9):3100-8.
SignalP
Finds signal peptide features in CDS (sig_peptide)
Petersen TN et al. SignalP 4.0: discriminating signal peptides from transmembrane regions. Nat Methods. 2011 Sep 29;8(10):785-6.
Infernal
Used for similarity searching against ncRNA family profiles
D. L. Kolbe, S. R. Eddy. Fast Filtering for RNA Homology Search. Bioinformatics, 27:3102-3109, 2011.

Changes

Read the release notes
Look at the Github commits

Feedback

Submit problems or requests to the Issue Tracker.

Citation

Seemann T. Prokka: rapid prokaryotic genome annotation Bioinformatics 2014 Jul 15;30(14):2068-9. PMID:24642063 DOI:10.1093/bioinformatics/btu153 CFF

Licence

GPL v3

Author

Torsten Seemann