Migrate Files from Server to DB

DOKA-NG stores information in the database, but additionally in sequential files in the file system. There are different groups of files:

• Logs (system and change logs)
• Interface files
• Transaction data (display files, TRNDATA, documents)
• Configuration (INI) files

It is possible to store these files with few technically conditioned exceptions as so-called BLOBS in the database. Also, it can be decided separately whether INI files are stored in the file system or in the database, whereby the configuration required for starting DOKA-NG must always be stored in the file system. For interface files it can also be decided programmatically whether they are to be stored in the file system or in the database. The advantage of files in database is that distributed DOKA-NG installations can access a common database and no separate “common” file system has to be managed.

This transaction is used to migrate data from the server file system to the “Files-in-Database” system.

Provision has been made for the migration of

• data (documents and messages)
• ini (configuration)
• template (reports)

partitions. Adjustments at project level are possible.

The purpose this transaction is allow what may be very large amounts of data (several hundred GB) to be transferred to a “File-in-Database” system incrementally over several days or weeks prior to the migration of the actual system.

Doing so is possible as most of this data will no longer be modified during ongoing operations, since it consists of historical data kept for information purposes. As an example, the transaction can be started in production weeks ahead of the activation of the “Files-in-Database” system by running it every night from 20h00 for 10 hours (600 minutes). A final launch after the system is shut down prior to the final changeover ensures that a updated copy of the data is held in the DB file system.

When the transaction is started, a complete list of all files to be copied is compiled internally. The list is sorted according to the timestamp of the files, and the files are then copied in the ascending order of the timestamp.

The transaction can be started either in the foreground or in the background. The length of time during which the transaction migrates data after each launch is stored in the configuration file sysfdb.ini, in the entry 'Runtime' of the [Configuration] section. When the time is up, the transaction terminates the copying process.

The progress of the migration is stored in the configuration file as the time of the last modification timestamp by which all files were migrated.

In an interactive start of SYSFDB, the file ini/sysfdb.ini is supplied by this transaction (e.g, the runtime in minutes is stored there).

The file will only need to be edited (e.g. using SYSIXU) if SYSFDB is to be configured without interactive start, or if the “Timestamp last copied” is be modified for test purposes.

Individual directories (e.g., directories that will remain in the file system, or directories that are the result of previous migrations and which are no longer needed) or individual files (e.g. files that are never used from the database and whose existence could cause confusion) can be excluded.

In this list, directories must be marked with a path delimiter ('/') at the end. The entries are stored in sysfdb.ini, Section [ExcludeList] as a block, e.g.

[ExcludeList]
Path1=/dok2host/
Path2=/host2dok/
Path3=/td2.env

When generating the copying list internally, SYSFDB compares whether the indicated path sections are postfixes of a path name. This means that leading delimiters ('/') in the name do not refer to absolute path names.

Directory content compressed into .zip files using the transaction SYSZIP in .zip are extracted as individual files and transferred to the DB file system under their individual names. It is assumed that the contents of the CLOBs in the SYF table will be saved by the database in appropriately compressed format.

SYSFDB leaves all files (.zip and others) in the file system unchanged (otherwise parallel operation with the production system would not be possible). Only the files temporarily created by SYSFDB as a result of the extraction from the .zip files will be deleted again by SYSFDB.

More information on the subject of “File-in-Database” is provided in the TradeDesign documentation, in the section “Files in Database”.

File Handling