The SFTP transport connector makes it possible to send and receive files to IFS Applications by using the SSH2 File Transfer Protocol; commonly referred to as SFTP and "Secure FTP". SFTP provides secure file transfers by utilizing strong network encryption, server authentication (by the use of "known hosts" file) and user authentication (by the use of password or identity files).
The SFTP Reader gets files from an IN directory in any format, executes them and puts an answer back to the RESPONSE directory. The FTP Sender has the possibility to put files in different xml-formats or plain text to a OUT directory.
The SFTP Reader and SFTP Sender are powered by JSch, an open source (BSD licensed) pure Java library.
SFTP requires a "known hosts" file. The "known hosts" file contains the public key of the SSH2 server, and the purpose of this file is to ensure that another server cannot maliciously impersonate another server (e.g. man-in-the-middle attacks).
The known hosts file may already have been created by an SSH2 client and may be reused by SFTP Reader. If the known hosts file does not exists, or is in an incompatible file format, a new known hosts file can be created with the following command:
java -cp C:\ifs\javaruntime\ifs-connector-sftp.jar;C:\ifs\javaruntime\jsch-0.1.31.jar ifs.fnd.connectserver.sftp.SftpCreateFiles -CreateKnownHosts=KNOWNHOSTS -hostname=SFTPHOST -username=USERNAME -password=PASSWORD
Example: To create the file C:\ifs\knownhosts containing the server sftp.example.com for which the user account sftp with password h4rd_pPpasZzw0oRd is used to logon, issue the following command:
java -cp C:\ifs\javaruntime\ifs-connector-sftp.jar;C:\ifs\javaruntime\jsch-0.1.31.jar ifs.fnd.connectserver.sftp.SftpCreateFiles -CreateKnownHosts=C:\ifs\knownhosts -hostname=sftp.example.com -username=sftp -password=h4rd_pPpasZzw0oRd
Important! You are recommended to specify a fully qualified domain name, FQDN, when specifying hostname. In other words you should specify sftp.example.com . Do not use short hostname like sftp or an IP like 10.1.2.3. This is not strictly necessary, but generally tends to reduce the need for maintenance/reconfiguration. Also note that the hostname in known host file should match the connection settings.
The username and password is not stored in the known hosts file; they are necessary to provide in this step due to JSch internals.
This is an optional step you may ignore if it does not apply to your environment.
Some SFTP servers do not do user authentication with passwords, but with public key cryptography instead. A user then create a "identity file" which contains the private credential needed to authenticate the user against the SFTP server. A second file, a public file, is also created when creating an identity file. The public file needs to be uploaded and installed into the SFTP server's user repository to enable identity based user authentication.
The identity file may already have been created by an SSH2 client and may be reused by SFTP Reader. If the identity file does not exists, or is in an incompatible file format, a new identity file can be created with the following command:
java -cp C:\ifs\javaruntime\ifs-connector-sftp.jar;C:\ifs\javaruntime\jsch-0.1.31.jar ifs.fnd.connectserver.sftp.SftpCreateFiles -CreateIdFile=IDFILE -keytype=KEYTYPE -comment=COMMENT -passphrase=PASSPHRASE -keysize=KEYSIZE
| Setting | Description |
|---|---|
| IDFILE | The full path name of the (private) identity file. The public key will be stored into a second file which begins with the same name but is suffixed with ".pub". |
| KEYTYPE | Key algorithm to use. Valid values are dsa and rsa. |
| COMMENT | An descriptive comment to add to the identity file. Anything goes. |
| PASSPHRASE | A passphrase/password used to encrypt the identity file, for additional protection. |
| KEYSIZE | The bit-length of the encryption key stored in the identity file. |
To create the file C:\ifs\identity with keytype dsa, comment IFSConnect, and passphrase veRy:c0mpl3X_pq$sw0rT and keysize 1024, issue the following command:
java -cp C:\ifs\javaruntime\ifs-connector-sftp.jar;C:\ifs\javaruntime\jsch-0.1.31.jar ifs.fnd.connectserver.sftp.SftpCreateFiles -CreateIdFile=C:\ifs\identity -keytype=dsa -comment=IFSConnect -passphrase=veRy:c0mpl3X_pq$sw0rT -keysize=1024
The following common settings exists for both SFTP Sender and SFTP Reader configuration.
Server connection settings are settings which gives the connector information about which server to connect to..
| Server connection settings | ||
|---|---|---|
| Parameter | Example value | Description |
| HOST | sftp.example.com | The hostname of the host which runs the SFTP service. It is recommended to specify a fully qualified domain name (FQDN). |
| PORT | 22 | Port number the SFTP service use. Default port is 22. |
Important! You are recommended to specify a fully qualified domain name, FQDN, when specifying hostname. In other words you should specify sftp.example.com . Do not use short hostname like sftp or an IP like 10.1.2.3. This is not strictly necessary, but generally tends to reduce the need for maintenance/reconfiguration. Also note that the hostname in known host file should match the connection settings.
Server authentication settings helps the SFTP transport connector to identity spoofing attacks (malicious servers attempting to impersonate the SFTP server). The way it works is compare the servers public key with the one previously recorded into known hosts file.
| Server Authentication settings | ||
|---|---|---|
| Parameter | Example value | Description |
| SERVERAUTH_KNOWNHOSTS | C:\ifs\knownhosts | File name containing known (authorized) public server keys. |
Consult Setting up SFTP authentication files regarding how to create the known hosts file.
User authentication settings helps the SFTP transport connector to identify itself when connecting to the SFTP server. Typically USERAUTH_METHOD is set to Password authentication (0) and a username/password pair is used to authenticate. Optionally, public key identification can be provided by the use of an identity file.
| User Authentication settings | ||
|---|---|---|
| Parameter | Example value | Description |
| USERAUTH_USERNAME | User account name used for login on to the SFTP server | |
| USERAUTH_METHOD | Password authentication | Is either set to Password authentication (0), or Identity file authentication (1). This should be set to Password authentication unless you are using an identity file. If you are not sure what you are using, you are probably using Password authentication. |
| USERAUTH_PASSWORD | The password of the user account. | |
| USERAUTH_IDFILE | C:\ifs\identity | Identity file for public key based authentication. This should be ignored unless you are using an identity file. |
| USERAUTH_IDPASSPHRASE | The passphrase which unlocks the encrypted identity file. This should be ignored unless you are using an identity file. |
|
To utilize Identity file based authentication, please consult Setting up SFTP authentication files.
Directory path name is on an SFTP server is always specified in SFTP syntax; which is similar to UNIX path names. This is true for all SFTP servers standard-compliant servers even if they are hosted on Windows Operating system.
| Syntax | Root directory | Current directory | Path separator |
|---|---|---|---|
| SFTP | / | . | / |
| UNIX | / | . | / |
| Windows | A:\ C:\ etc... |
. | \ |
Examples of SFTP syntax:
The SFTP Sender connector uploads files from Connect to an SFTP server. The filename to be uploaded is configured in the SFTP Sender address.
Most SFTP settings exists in both SFTP Reader and SFTP Sender. Please refer to "Common settings for SFTP Sender and SFTP Reader" regarding how to setup connection and authentication settings.
File transmit settings generally control how often and how many times SFTP Sender should attempt to retransmit file. Tweak values to achieve desired balance between number of retries and
| File transmit settings | ||
|---|---|---|
| Parameter | Example value | Description |
| MAX_RETRIES | 1 | Max number of send retries. SFTP Sender will upon failure retry until MAX_RETRIES has been reached. If MAX_RETRIES is reached, Connect will be informed that the file transmission has failed. |
| RETRY_INTERVAL | 1 | Time in seconds between retransmit trials. A large number will leave more time for the recipient server, network etc more time to resolve problems. However, too large a value will lead to longer recovery times which might be unnecessary to resolve temporary connection/upload problems. Some network services (firewall devices etc) might consider it "hammering" if the value is set too low. |
The directory settings controls where temporary files are to be written.
| Directory settings | ||
|---|---|---|
| Parameter | Example value | Description |
| REMOTE_TMP_DIRECTORY | ./tmp | If the temporary directory is not set, the file will be directly uploaded to the location specified in the Connect
address. By setting a temporary directory you help prevent race conditions; recipients polling an directory will not poll the file until the entire file has been uploaded. The file will first be uploaded to the temporary directory on the SFTP server. Once the upload has completed, the file will be moved from temporary directory to the location specified in the Connect address. Refer to Remote directory or path names on a SFTP server for help with specifying directory name. |
| LOCAL_TMP_DIR | C:\ifs\tmp | The local directory for to write temporary files to. Leave empty for Java's default. Directory path name is specified in operating system specific syntax. |
The SFTP Reader is a connector for reading files from an SFTP file server. The SFTP Reader is a polling device which operates according to a simple state machine, understanding the state machine may help e.g. interpret debug output.
| STATE | Actions performed in this state |
|---|---|
| OPEN | Reader connects to SFTP server. It will also create the necessary directories on server, if not already present. |
| LIST | Reader lists files on SFTP server. If it finds files to download, it will move to the READ state. Otherwise it will sleep for a while and then and try again. |
| READ | Reader downloads a file from SFTP server. |
| EXECUTE | Reader sends file content to IFS Connect and obtains a response. |
| RESPONSE | Reader uploads a response file to SFTP server and deletes the processed file. If any listed file remains unread the state machine changes to READ, otherwise LIST. |
Most SFTP settings exists in both SFTP Reader and SFTP Sender. Please refer to "Common settings for SFTP Sender and SFTP Reader" regarding how to setup connection and authentication settings.
General settings are settings which gives the connector some basic information on how it is expected to operate (jar file to load, amount of debug information to load, etc)
| General settings | ||
|---|---|---|
| Parameter | Example value | Description |
| JAR_FILE | C:\ifs\javaruntime\ifs-connector-sftp.jar | Path to this connector's jar-file. |
| DEBUG | false | Enable/disable debug information generation. |
| DEBUG_LEVEL | MEDIUM | Controls the amount of information outputted by Use LOW if you want to use the log file monitor a connector which is working. Use MEDIUM if you want to do a quick investigation, e.g. determine cause of file transfer errors.. Use HIGH if you want to investigate a hard problem which requires in-depth debug traces. |
| DEBUG_FILE | C:\logs\sftpreader1.log | Specify the file in which the debug information will be written. |
Polling settings are settings which gives the connector information like how often to poll, which directories to poll,
and which file to poll, what to do with a file after it has been processed,
| Polling settings | ||
|---|---|---|
| Parameter | Example value | Description |
| IN_DIRECTORY | ./in | The directory for this SFTP_Reader to read input files from. Refer to Remote directory or path names on a SFTP server for help with specifying directory name. |
| IN_FILE_FILTER | ^.*\.xml$ | Filename filter (regexp syntax). Matching files will be polled, others will be ignored. If the filter is empty, any file will be polled. |
| RESPONSE_DIRECTORY | ./response | The directory for this SFTP_Reader to write response files to. Refer to Remote directory or path names on a SFTP server for help with specifying directory name. |
| REMOTE_TMP_DIRECTORY | ./tmp | If the temporary directory is not set, the response file will be directly uploaded to the response directory. By setting a temporary directory you help prevent race conditions; recipients polling the response directory will not poll the file until the entire file has been uploaded. The file will first be uploaded to the temporary directory on the SFTP server. Once the upload has completed, the file will be moved from temporary directory to the response directory. Refer to Remote directory or path names on a SFTP server for help with specifying directory name. |
| LOCAL_TMP_DIRECTORY | C:\ifs\tmp | The local directory for this SFTP_Reader to write temporary files to. Leave empty for Java's default, C:\WINDOWS\Temp. |
| ARCHIVE_DIRECTORY | ./archive | Directory to move to files read from IN_DIRECTORY when finished processing them. If empty, files are deleted. Refer to Remote directory or path names on a SFTP server for help with specifying directory name. |
| POLL_TIME | 30 | How many seconds SFTP_Reader should wait before scanning the In_Directory. A lower value means files are polled more often, which reduces waiting times but might increase CPU utilization etc. |
Regular expressions (regexp) are used to create file name filters. Regular expressions provides many useful features to match filenames.
A very basic expression only contain "normal" character sequences, for example xml will match any file name containing xml, such as this-is.xml and this-is-not-xml.txt. A more powerful example is \.xml$ which will only match files which ends with .xml.
Some simple expressions are included in the table bellow to help build more advanced expressions:
| Expression | Meaning |
|---|---|
| ^ | Start of string. File name filters usually begin with this character. |
| $ | End of string. File name filters usually end with this character. |
| . | Any character. |
| [a-z] | Any character in the range a to z. |
| \ | The following character will not be considered a regular expression special character. A common example is \. which will match . rather than any character. |
| * | The previous character will be matched 0 or more times. a* will match none, one or many a characters. |
| + | The previous character will be matched 1 or more times. a+ will match one or many a characters. |
| {n} | The previous character will be matched n times. a{3} will match aaa. |
Examples by usage:
^.*\.xml$ matches all .xml files.
^[a-z]{4}-[0-9]{3}\.xml$ matches aaaa-000.xml to zzzz-999.xml.
^invoice[0-9]+\.xml$ matches invoice1.xml, invoice51.xml, invoice953.xml etc, but not invoice.xml.
^invoice[0-9]*\.xml$ matches invoice1.xml, invoice51.xml, invoice953.xml etc, and also invoice.xml.
It's possible to create new instances of the File Reader Transport Connector provided that they should scan different directories, or has in-file filters without overlapping file names.