Intel SFT How To Configure Guide



Intel Secure File Transfer (SFT)

Content Replication

How To Configure Guide

|Document Information |

|Name |Intel Secure File Transfer (SFT) Content Replication How To Configure Guide |

|Version |v1.0 |

|Date Created |April 27, 2011 |

|Date Updated |September 25, 2011 |

Legal Notices and Disclaimers

Disclaimers

INTEL CORPORATION MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. INTEL CORPORATION ASSUMES NO RESPONSIBILITY FOR ANY ERRORS THAT MAY APPEAR IN THIS DOCUMENT. INTEL CORPORATION MAKES NO COMMITMENT TO UPDATE NOR TO KEEP CURRENT THE INFORMATION CONTAINED IN THIS DOCUMENT.

THIS SPECIFICATION IS COPYRIGHTED BY AND SHALL REMAIN THE PROPERTY OF INTEL CORPORATION. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED HEREIN.

INTEL DISCLAIMS ALL LIABILITY, INCLUDING LIABILITY FOR INFRINGEMENT OF ANY PROPRIETARY RIGHTS, RELATING TO IMPLEMENTATION OF INFORMATION IN THIS SPECIFICATION. INTEL DOES NOT WARRANT OR REPRESENT THAT SUCH IMPLEMENTATIONS WILL NOT INFRINGE SUCH RIGHTS.

NO PART OF THIS DOCUMENT MAY BE COPIED OR REPRODUCED IN ANY FORM OR BY ANY MEANS WITHOUT PRIOR WRITTEN CONSENT OF INTEL CORPORATION.

INTEL CORPORATION RETAINS THE RIGHT TO MAKE CHANGES TO THESE SPECIFICATIONS AT ANY TIME, WITHOUT NOTICE.

Legal Notices

Intel software products are copyrighted by and shall remain the property of Intel Corporation. Use, duplication or disclosure is subject to restrictions stated in Intel's Software License Agreement, or in the case of software delivered to the government, in accordance with the software license agreement as defined in FAR 52.227-7013.

The Intel logo is a registered trademark of Intel Corporation.

Other brands and names are the property of their respective owners.

Copyright ( 2009, Intel Corporation, All Rights Reserved

Document Version History

|Version |Date |Document History |

|1.0 |May 3, 2011 |Created by Jeremy Morrissey |

| | | |

| | | |

| | | |

| | | |

| | | |

Table of Contents

Legal Notices and Disclaimers 2

Disclaimers 2

Legal Notices 2

Table of Contents 4

1. Introduction 5

2. SMTP/Email Alert Option 5

3. File Alert Option 8

4. Controlling the Size of Content Replication (maxfilesize and maxfilecount) 10

5. Logging File Names Contained in Replic TAR Files 11

6. Generating a LIST file for files extracted onto Destination Systems 12

7. Using a custom working directory to avoid using the default user’s TMP folder 13

8. Replacing the use of the FTP “mget” command with “get –r” or another command 13

9. Minimizing Command Line Parameters using Configuration File 14

10. Logging to SYSLOG on Unix/Linux systems 15

11. Using SSH Public Keys On Windows Platform 17

12. Unix/Linux System Configuration Test Scripts 20

13. Developing custom alerting/error logging procedures 23

14. Script State Tracking 24

15. Using resendArchive.pl 28

16. Using skipfile.pl 29

17. Using restoreskippedfile.pl 31

18. Changing Your Password Used in SFTP 32

19. Migrating To an New Intel SFT Environment 33

20. Troubleshoot Issues 35

Introduction

The purpose of this guide is to provide detailed installation instructions on how to configure various features of the Intel SFT Content Replication capability on a Source or Destination system.

This guide depends on the Intel SFT Content Replication Install Guides for Unix/Linux or Windows procedures being followed to create an installation of this capability.

SMTP/Email Alert Option

Intel SFT Content Replication has the option that can be configured to send email alerts when an ERROR is logged during the execution of the ReplicateDir.pl or DownloadFiles.pl PERL scripts. All logs written by the script during the course of the script execution are send in the body of the email address.

Prerequisites for Unix/Linux:

← The “sendmail” program must be configured on the system to be able to send email

← You must have a valid email address that can be used in the FROM address of the email alerts.

← You must have one or more valid email addresses to send the alerts to

Prerequisites for Windows:

← Microsoft .NET FW 2.0 must be installed on the system

← A valid SMTP host name that can be used to connect over the network port 25 to send email

← You must have a valid email address that can be used in the FROM address of the email alerts

← You must have one or more valid email addresses to send the alerts to

[pic]IMPORTANT INFORMATION BEFORE ENABLING

• All users should be aware that when --loglevel= command line parameter is set to DEBUG a lot of information can be containing in the log and any error alerts that are triggered. Depending on your system constraints, this may result in a delay in receiving the alert or alert failing to be sent. It is recommended to use default --loglevel= setting when alerting is enabled. Refer to the troubleshooting guide for how --loglevel= parameter is set.

• Windows users that are configuring passwords in their intelsftconfig.ini files should avoid enabling this alerting feature when --loglevel= command line parameter is set to DEBUG. Refer to the troubleshooting guide for how --loglevel= parameter is set. To disable, set the ALERT_SMTPTOADDRESS to empty.

To configure this capability follow these steps:

1. Prior to following the below steps, you must have followed the steps found in the Intel SFT Content Replication Install Guide (for either Windows or Unix/Linux) to configure your system as the Source and/or Destination system.

2. Locate the following files in your installation

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

3. Open each of the above configuration files for editing

You will find the below properties that need to be edited in these files. Edit each property using the instructions below.

|Property |Value |

|ALERT_SCRIPT |With a standard installation, this property should be set to |

| |ALERT_SCRIPT=perl sendAlert.pl |

| | |

| |The value points to a PERL script that is used to send the email |

| |alert and is provided in a standard installation. |

| | |

| |If you do not find this property in your config file, add it with |

| |the value above. |

|ALERT_SMTPHOST |Windows users need to set this to the host name of an SMTP server |

| |that this sytem can connect to and send email |

| | |

| |Unix/Linux users – leave this value blank |

|ALERT_SMTPTOADDRESS |The email addresses to send the alerts to |

| | |

| |Windows users need to use the semicolon (;) as the delimiter for |

| |multiple email addresses |

| | |

| |Unix/Linux users need to use the comma (,) as the delimiter for |

| |multiple email addresses |

|ALERT_SMTPFROMADDRESS |A valid email address to send the alerts from. This should be |

| |recognizable to the recipients of the alert and should not be |

| |treated as spam or junk mail. |

|ALERT_SMTPSUBJECT |The text that shows in the subject line for all email alerts. The |

| |default value is “Intel SFT CR Alert”. The actual subject line |

| |will be appended with a colon (:) and the name of the script that |

| |produced the error, such as “: ReplicateDir” or “: DownloadFiles” |

4. If the scripts are running on the Linux/Unix platform and your sendmail program is not found in the /usr/sbin/sendmail path then update the ALERT_SCRIPT configuration property value to pass an additional command line parameter indicating your sendmail program path

For example, if your sendmail program path was “/usr/bin/sendmail” then use the following:

ALERT_SCRIPT=perl sendAlert.pl --sendmail=/usr/bin/sendmail

5. Verify that alerting was configured correctly and email alerts can be sent

Run the following command to test your alerting configuration. This will not trigger a replication process and is ok to run even if your system is not a Source system.

perl replicatedir.pl --source=test --data=test --archive=test --transfer=test --loglevel=4

The recipients configured should expect to see an alert with the subject similar to “Intel SFT CR Alert : ReplicateDir”

The alert body should be similar to what is shown below:

2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started

2011-05-15 08:14:09 232 DEBUG: init: begin

2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'!

2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped

File Alert Option

Intel SFT Content Replication has the option that can be configured to write a log file to a configurable path on your system when an ERROR is logged during the execution of the ReplicateDir.pl or DownloadFiles.pl PERL scripts. All logs written by the script during the course of the script execution are contained within this log file. The file name is uniquely named using the process id + a date time stamp.

You must have the below versions or later of both of the scripts to configure this option. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110808

DownloadFiles.pl: 20110715

[pic]IMPORTANT INFORMATION BEFORE ENABLING

• Windows users that are configuring passwords in their intelsftconfig.ini files should avoid enabling this alerting feature when --loglevel= command line parameter is set to DEBUG. Refer to the troubleshooting guide for how --loglevel= parameter is set. To disable, set the ALERT_SMTPTOADDRESS to empty.

To configure this capability follow these steps:

1. Locate the following files in your installation

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

2. Open each of the above configuration files for editing

Locate the ALERT_SCRIPT property in the configuration files

Change the ALERT_SCRIPT property to have the following file:

ALERT_SCRIPT= perl sendFileAlert.pl

IMPORTANT REMINDER: Depending on your system or how you are running the perl commands, you may be required to specify the full path to the “sendFileAlert.pl” script in the above property value. If so, an example for you to reference may be:

ALERT_SCRIPT= perl /usr/example/sendFileAlert.pl

Locate the ALERT_DIRECTORY property in the configuration files

If you cannot find the ALERT_DIRECTORY property, then add this property

The value of this property must be where you desire to have the alert log files written. The full path must be specified and the user account running the perl scripts must have write access. For an example property and value within your configuration:

ALERT_DIRECTORY=/usr/alertlogs

3. Verify that alerting was configured correctly and email alerts can be sent

Run the following command to test your alerting configuration. This will not trigger a replication process and is ok to run even if your system is not a Source system.

perl replicatedir.pl --source=test --data=test --archive=test --transfer=test --loglevel=4

A file should be written to the directory that you configured for ALERT_DIRECTORY within the intelsftconfig.ini files.

The file name format is ERROR__YYYY-MM-DD_HH_MM_SS.log

The file contents should be similar to what is shown below:

2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started

2011-05-15 08:14:09 232 DEBUG: init: begin

2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'!

2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped

Controlling the Size of Content Replication (maxfilesize and maxfilecount)

There may be a need to control the size of the content being replicated using the ReplicateDir.pl command. By default the ReplicateDir.pl command will package all files it finds to replicate and transfer them in one package (tar.gz file). The size of this package may need to be controlled on your system. This can be achieved using the following two options:

1. Specifying a maximum size in bytes of files to be added to the package

And/or

2. Specifying a maximum number of files to be added to the package

Regardless of the option used above, the files that are not packaged will queue up and be packaged in a subsequent run of ReplicateDir.pl.

You must have the below versions or later of both of the scripts to configure these options. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110808

DownloadFiles.pl: 20110715

The following command line parameters can be added to the ReplicateDir.pl to use these options:

--maxfilesize=

When the parameter --maxfilesize is present in the command line to ReplicateDir.pl only files totaling up to that file size will be replicated in that run. All others will wait until the next ReplicateDir.pl script run. An exception is when an individual file size is greater than this --maxfilesize value and in this case, this file will be packaged in a tar file by itself and all others will wait until the next time ReplicateDir.pl script is run. This is to avoid skipping these files and never replicating them.

--maxfilecount=

When the parameter --maxfilecount is present in the command line to ReplicateDir.pl only that number of files will be replicated in that run. All others will wait until the next ReplicateDir.pl script run.

Logging File Names Contained in Replic TAR Files

Intel SFT Content Replication has the feature to log the file attributes of files that are replicated between the Source and Destination systems. This feature is something must be enabled and is not enabled by default.

You must have the below versions or later of both of the scripts to configure this option. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110808

DownloadFiles.pl: 20110715

To configure this feature follow these steps:

1. To configure the Source system (that uses ReplicateDir.pl) edit your ReplicateDir.pl command line to add the below additional command line parameter:

--logfilesinarchive=Y

2. To configure the Destination system (that uses DownloadFiles.pl) edit your …/DownloadFiles/intelsftconfig.ini configuration file to enable this feature

Open the ../intelsft/Services/DownloadFiles/intelsftconfig.ini file or your custom configuration file you are using in your command line for downloadFiles.pl

Locate all the properties within this file that have a property name LSP or end with LSP and notice that they call a perl script called “auto_untar_gz.pl”

Edit each of those properties so that an additional command line parameter is added to. The additional command line parameter is “--logfilesinarchive=Y”. For example:

LSP = perl auto_untar_gz.pl --logfilesinarchive=Y --extractdestpath=…

IMPORTANT REMINDER: Depending on your system or how you are running the perl commands, you may be required to specify the full path to the “auto_untar_gz.pl” script in the above property value. If so, an example for you to reference may be:

LSP = perl /usr/example/auto_untar_gz.pl --logfilesinarchive=Y --extractdestpath=…

Generating a LIST file for files extracted onto Destination Systems

Intel SFT Content Replication has the feature to create a verbose list file containing file attributes of files that are replicated between the Source and Destination systems. This feature is something must be enabled and is not enabled by default.

The main difference between this feature and the logging feature described above is that this feature creates a separate file for each package of files extracted on the destination system whereas the logging feature logs the same info into the log that is configured for the downloadFiles.pl script.

You must have the below versions or later of both of the scripts to configure this option. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110916

DownloadFiles.pl: 20110916

This feature specifically uses options on your tar program configured

tar –tvf

To enable:

1) Open the DownloadFiles/intelsftconfig.ini files or your custom configuration file that you may have configured downloadFiles.pl to use

2) Edit all property values for properties that are prefixed with “LSP” to contain the command line parameter of “--extractfilenames=Y”.

As an example:

LSP=perl auto_untar_gz.pl --extractfilename=Y

--extractedfilenameslog=/home/logpath/ --filetoprocess=

Once enabled, for every content replication downloaded by downloadFiles.pl, you will find a file created with the following naming convention in the path that --extractedfilenameslog parameter is configured with.

._.log

Example file names:

[pic]

The below is an example of the contents when ran on SUSE Linux

[pic]

Using a custom working directory to avoid using the default user’s TMP folder

The default folder that the scripts use for temporary files is the user’s Operating System managed TEMP folder.

All command lines support an optional --sftwd= parameter that when specified will instruct the script to use that folder and not your user accounts “temp” folder that the Operating System provides to you.

replicateDir.pl … [--sftwd=] …

downloadFiles.pl … [--sftwd=] …

Replacing the use of the FTP “mget” command with “get –r” or another command

Some SFTP programs do not have support for the FTP “mget” command. These SFTP programs are able to use other FTP commands that provide the same functionality; therefore, can be configured for use in the scripts.

Edit the following file:

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

Review the file for a MGET_COMMAND property. If you do not find one, then add one: “MGET_COMMAND=”

Set the MGET_COMMAND to the FTP command that replaced “mget” for your SFTP program

For example, to replace “mget” with “get –r” (recursive get), then you would have this property set:

MGET_COMMAND=get -r

Minimizing Command Line Parameters using Configuration File

The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110916

DownloadFiles.pl: 20110916

All command line parameters except for specifying a custom configuration file can be specified in the intelsftconfig.ini configuration file or your custom configuration file. This can be helpful to centralize configuration as well as shorten the command line used to execute the PERL scripts. Please note that not all configuration parameters used in intelsftconfig.ini file or your custom configuration file can be used as command line parameters.

When a configuration is specified in BOTH the configuration file and as a command line parameter, the parameter value in the command line parameter is used. This may be helpful in defining a central configuration file with all default values set that command lines can overwrite and use different values.

To define a command line parameter in the intelsftconfig.ini file or your custom configuration file, drop the “--" prefix and use UPPERCASE letters in the property name as an example:

Command line before:

[pic]

All parameters added to intelsftconfig.ini file:

[pic]

Now the command line is:

[pic]

This applies to the following PERL scripts:

1) ReplicateDir.pl

2) downloadFiles.pl

3) auto_untar_gz.pl (--processfiles is a param that must stay on the command line)

Logging to SYSLOG on Unix/Linux systems

The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110916

DownloadFiles.pl: 20110916

This optional feature can be enabled to log to the System Log (SYSLOG) on Unix/Linux systems. This feature is not currently available on the Windows platform.

The severity of the log message will map to your System log severities:

ERROR ( ERR

WARN ( WARN

INFO ( INFO

DEBUG ( DEBUG

VERSOSE ( DEBUG

To enable:

3) Open the intelsftconfig.ini files or your custom configuration file that your PERL scripts are using. The typical location of these are as follows.

/intelsft/Services/DownloadFiles/intelsftconfig.ini

/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

4) Add the following line into the configuration files:

LOGGING = syslog

Contact your system administrator for instructions on how to access and manage your system log.

An example of what you may see on SUSE Linux is shown below:

[pic]

Using SSH Public Keys On Windows Platform

The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110916

DownloadFiles.pl: 20110916

To configure SSH Key Authentication when running the PERL scripts on a Windows platform, complete the following steps:

1) Locate the following input parameters for these steps

You will need a new SFTP host name that allows for public key authentication

Contact your Intel representative for this input parameter if you are unable to locate the above

2) Login with an administrative account that is used to run all scheduled tasks required by the Intel SFT CR scripts.

3) Store the SSH Remote Host Key in your system cache using the following steps:

a. Start a Windows command prompt

b. Run the following command within the new Command Window. Change the below path if your installation is something other than C:\intelsft\. Replace Host with the new host name input parameter.

Run: c:\intelsft\services\downloadfiles\psftp.exe Host

[pic]

c. If prompted, choose “Y” to store the key in cache.

d. Close the command window (ignore the prompt to login)

4) Generate a Private/Public Key pair using the Putty Key Generation tool with the below steps:

a. Start a Windows command prompt

b. Run the following command within the new Command Window. Change the below path if your installation is something other than C:\intelsft\.

Run: c:\intelsft\services\downloadfiles\puttygen.exe

c. A screen similar to the one below display. First, enter the key length in the box at the bottom right corner. This number represents the key length, and the standard length for use with Intel SFT is 2048 bits, so enter “2048” in the box.

[pic]

d. Next, click the “Generate” button. You will be prompted to move your mouse over the blank area as indicated below. The movements from the mouse are collected and used in the process of generating your key. Depending upon your system, this may take a few seconds to a minute to complete.

[pic]

e. After the key has been generated a screen will display similar to the one below with the “Save public key” and “Save private key” buttons enabled. Click the “Save public key” button, and choose a folder to save your public key. This file contains the key that will be provided to Intel.

f. After saving the public key, click the “Save private key” button. This file should be stored in an access controlled secure location on your system. The system or user account running the Intel SFT CR scripts will need access to read this file. The Intel SFT CR scripts You may also want to create a backup of the public and private key pair and store in a secure location.

5) Provide your Intel representative with the Public Key file saved above. Do not send the Private Key.

6) Configure your Intel SFT PERL scripts to use your SSH Keys generated above by following the below steps:

a. Open the following files or your custom configuration file for editing

/intelsft/Services/DownloadFiles/intelsftconfig.ini

/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

b. Make the following changes in the configuration files

i. Update the “host” property to the provided remote host name that will allow connections requiring SSH key authentication

host=ift-keyauth.

ii. Set the value of the “pwd” property to “key”

pwd=key

iii. Add a new property to the configuration file for “KEYPATH” and set the value to the full path and file name of the Private key file generated and saved above (file extension will be .ppk).

KEYPATH=

Example: KEYPATH=d:\keys\mykey.ppk

7) Verify the PERL scripts function as expected

Unix/Linux System Configuration Test Scripts

The steps below apply only to Linux/Unix platforms and typically used during the installation of the Intel provided PERL scripts.

The steps below will help ensure that your configuration is set correctly to avoid the potential for loss of data.

Follow the steps below on the Source system using the ReplicateDir commands to replicate files.

1) Change directory to your ../Utilities/ReplicateDir directory

2) Run the below from your command line. If you are using a different config file with the downloadFiles.pl command, then specific the full page to it using the --configfilepath= command line parameter.

perl systemtest_unix_upload_rc_ms.pl

3) Follow the prompt to enter a Folder that is shown in the directory listing and hit your Enter key. The folder is case sensitive.

4) If your configuration is correct, you should see a message indicating ”No action required...”

5) Otherwise, follow the instructions in the output and rerun the above steps once you completed the action to verify

For your reference, the script output shown below was ran on SUSE LINUX Enterprise Server 9 with an incorrect configuration. The PERL script is expected to indicate what action is required. Your system may have a different value to set.

plxs0101> perl systemtest_unix_mget_rc.pl

ACTION REQUIRED: Set MGET_NO_MATCHES_RETURNCODE=10 in the ../DownloadFiles/intelsftconfig.ini or your custom config file

Again, for your reference, the output shown below again on the same system after the action required was taken. The configuration is confirmed to be correct with the “No Action Required” statement below.

plxs0101> perl systemtest_unix_mget_rc.pl

No Action Required: Current configuration is set correctly (MGET_NO_MATCHES_RETURNCODE=10) in the ../DownloadFiles/intelsftconfig.ini or your custom config file

Follow the steps below on the Destination system using the DownloadFiles commands to download files.

1) Change directory to your ../Services/DownloadFiles directory

2) Run the below from your command line. If you are using a different config file with the downloadFiles.pl command, then specific the full page to it using the --configfilepath= command line parameter.

perl systemtest_unix_mget_rc.pl

3) If your configuration is correct, you should see a message indicating ”No action required...”

4) Otherwise, follow the instructions in the output and rerun the above steps once you completed the action to verify

For your reference, the script output shown below was ran on SUSE LINUX Enterprise Server 9 with an incorrect configuration. The PERL script is expected to indicate what action is required. Your system may have a different value to set.

plxs0101> perl systemtest_unix_upload_rc_ms.pl

File listing script output (using systemtest_ftp_ls.scr):

sftp> pwd

/

sftp> ls

.:

CR_CMPA2BCH/

sftp> quit

Please enter the folder from the above list that you'd like to test with

CR_CMPA2BCH

ACTION REQUIRED: Set PUT_SFTP_CHMOD_RETURNCODE=7 in the ../ReplicateDir/intelsftconfig.ini or your custom config file

plxs0101>

Again, for your reference, the output shown below again on the same system after the action required was taken. The configuration is confirmed to be correct with the “No Action Required” statement below.

plxs0101> perl systemtest_unix_upload_rc_ms.pl

File listing script output (using systemtest_ftp_ls.scr):

sftp> pwd

/

sftp> ls

.:

CR_CMPA2BCH/

sftp> quit

Please enter the folder from the above list that you'd like to test with

CR_CMPA2BCH

No Action Required: Current configuration is set correctly (PUT_SFTP_CHMOD_RETURNCODE=7) in the ../ReplicateDir/intelsftconfig.ini or your custom config file

Developing custom alerting/error logging procedures

Intel SFT Content Replication has the flexibility to call a custom PERL script at the end of each script execution when an ERROR log is written during the course of its execution. This custom PERL script would be passed the location of a temporary file containing all logs written by the Content Replication script and can process that file in a custom way.

To develop a custom/error logging procedure follow these steps:

1. Develop a custom PERL script

Use sendAlert.pl as a reference. A copy of this file can be found in the following folders.

…/intelsft/Services/DownloadFiles/

…/intelsft/Utilities/ReplicateDir/

The custom PERL script must accept the following command line parameters:

|subject |This parameter will provide a string that can be helpful to identify the context in which the error was |

| |generated in, such as “ReplicateDir” or “DownloadFiles”. |

|alertLog |This parameter will contain the absolute path of a file that will contain all logs written by the Content |

| |Replication script during its execution. Below is a sample format. |

| |2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started |

| |2011-05-15 08:14:09 232 DEBUG: init: begin |

| |2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'! |

| |2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped |

Develop your PERL script to process the above command line parameters

Do not delete the alertLog within your PERL script

2. Locate the following files in your installation

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

3. Open each of the above configuration files for editing

You will find the below property that needs to be edited to point to your custom PERL script.

|Property |Value |

|ALERT_SCRIPT |Your PERL script command line and any custom command line |

| |parameters it takes other than those found in above step |

4. Verify the configuration of the custom PERL script

Run the following command to test your alerting configuration. This will not trigger a replication process and is ok to run even if your system is not a Source system.

perl replicatedir.pl --source=test --data=test --archive=test --transfer=test --loglevel=4

Your custom PERL script should be invoked with the subject “ReplicateDir”

The alert log that it was passed the absolute path to should be similar to below:

2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started

2011-05-15 08:14:09 232 DEBUG: init: begin

2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'!

2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped

Script State Tracking

The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110916

DownloadFiles.pl: 20110916

This optional feature can be enabled to call a custom PERL script developed to track the state of the scripts

ReplicateDir.pl can move through the following states:

|State |Description |Comments |

|STARTED |The script started | |

|FILES FOUND TO REPLICATE |Files were found to replicate | |

|FILES PACKAGED |Files were packaged into a TAR file |The name of the LST file that contains all the |

| | |files packaged is tracked with this state |

|FILES COMPRESSED |The TAR file was compressed |The name of the TAR archive file compressed is |

| | |tracked with this state |

|FILE TRANSFER BEGIN |The SFTP upload started |The remote path and file name is tracked with |

| | |this state |

|FILE TRANSFER END SUCCESS |The SFTP upload ended successfully |The remote path and file name is tracked with |

| | |this state |

|FILE TRANSFER END ERROR |The SFT upload ended on error |The remote path and file name that encountered |

| | |an error is tracked with this state |

|STOPPED |The script stopped | |

DownloadFiles.pl can move through the following states:

|State |Description |Comments |

|STARTED |The script started | |

|FILES TO DOWNLOAD FOUND |Files were found to replicate | |

|FILE TRANSFER BEGIN |The SFTP file download started |The remove path and file name of the file |

| | |downloaded is tracked with this state |

|FILE TRANSFER END SUCCESS |The SFTP file download ended successfully |The remove path and file name of the file |

| | |downloaded is tracked with this state |

|FILE TRANSFER END ERROR |The SFTP file download ended on error |The remove path and file name of the file |

| | |downloaded is tracked with this state |

|FILES DECOMPRESSED |Files were packaged into a TAR file |The path and file name of the compressed file |

| | |is tracked with this state |

|FILES EXTRACTED |The TAR file was compressed | |

|STOPPED |The SFTP upload started | |

To develop a custom PERL script to track the state of the scripts, do the following:

1) Use sessionState.pl as a reference. A copy of this file can be found in the following folders.

…/intelsft/Services/DownloadFiles/

…/intelsft/Utilities/ReplicateDir/

The custom PERL script must accept the following command line parameters:

|--event |This parameter will contain the state name as shown in the above tables |

|--pid |This parameter will contain the Process ID of the script to associate states to a particular script |

| |execution. |

|--callingScript |This parameter will contain the PERL script name |

|--data |This parameter will contain additional optional data that may be tracked with the state |

2) Locate the following files in your installation

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

3) Open each of the above configuration files for editing

Add the SESSIONSTATE property to the file if it does not already exist.

|Property |Value |

|SESSIONSTATE |Your PERL script command line and any custom command line |

| |parameters it takes other than those found in above step |

| | |

| |For example: perl sessionState.pl --sessionStateLog=state.log |

4) Verify the configuration of the custom PERL script

Run the PERL script that you have configured this with and confirm the expected results

As an example that is provided with the install of Intel SFT, the demonstration “sessionState.pl” script can be configured and used using the following states:

1) Locate the following files in your installation

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

2) Open each of the above configuration files for editing and add the following line to the file if it does not already exist

SESSIONSTATE = perl sessionState.pl --sessionStateLog=state.log

3) Run your scripts and review the state.log file that will be found in the scripts working directory it is executing in. Below are examples of what you may see in this file:

ReplicateDir.pl successful run (contents of state.log):

[pic]

DownloadFiles.pl successful run (contents of state.log)::

[pic]

Using resendArchive.pl

You may have a need to retransfer content that was previously replicated to the destination. Normally, this request would come from the Destination system or via instructions when troubleshooting an issue.

“resendArchive.pl” will allow you to manually trigger a transfer of a tar archive that is found within your /archive folder that you have configured on the ReplicateDir.pl command line. Once transferred, the Destination system is expected to be able to download the archive.

You must have the below versions or later of both of the scripts to use these instructions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110808

DownloadFiles.pl: 20110715

The “resendarchive.pl” can be found within the ../intelsft/Utilities/ReplicateDir/ folder

Instructions:

1. Start a command line and change directory to ../intelsft/Utilities/ReplicateDir/

2. At your command line shell, type the following (all one line) replacing the following:

a. Replace with the path to your archive folder and the archive file name that needs to be resent. The path to your archive can be found as the value of the --archive parameter in your ReplicateDir.pl command line used normally to replicate files from your source folder.

b. Replace with the --sftint parameter that can be found in your ReplicateDir.pl command line used normally to replicate files from your source folder.

perl resendArchive.pl --archivefile= --sftint=

3. Verify that you see the message “transferFile: file successfully transferred”. You may see some WARN messages that you can ignore if you see the message at the end.

For reference, the following examples show the resendArchive.pl executing on the Windows and Unix platform:

Windows:

[pic]

Unix/Linux:

plxs0100> cd intelsft/Utilities/ReplicateDir/

plxs0100> perl resendArchive.pl --archivefile=/tmp/CR_CMPA/source/archive/replic_2011-08-03_18_59_39.tar.gz --sftint=CR_CMPA2BCH

2011-08-03 22:53:35 21902 INFO: resendArchive.pl 20110801 started

2011-08-03 22:54:12 21902 INFO: transferFile: file successfully transferred: >/tmp/CR_CMPA/source/archive/replic_2011-08-03_18_59_39.tar.gz< to >CR_CMPA2BCH<

2011-08-03 22:54:12 21902 INFO: resendArchive.pl 20110801 stopped

Using skipfile.pl

You may have a need to flag a file that would otherwise be downloaded (or attempted to be downloaded) by your DownloadFiles.pl script to be skipped. Normally, this request would come from instructions when troubleshooting an issue.

“skipfile.pl” will allow you to manually flag a file to be skipped by your DownloadFiles.pl script command. These files can be restored by using the “restoreskippedfile.pl” script.

You must have the below versions or later of both of the scripts to use these instructions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110808

DownloadFiles.pl: 20110715

The “skipfile.pl” can be found within the ../intelsft/Services/DownloadFiles/ folder

Instructions:

1. Start a command line and change directory to ../intelsft/Services/DownloadFiles/

2. At your command line shell, type the following (all one line) replacing the following:

a. Replace with the archive file name that needs to be skipped. This should not include the path to the file.

b. Replace with destination interface code (folder) that the file is downloading from. You may want to refer to the CONTENT_TO_DOWNLOAD property within your DownloadFiles folder

perl skipfile.pl --file= --sftint=

NOTE: File name parameter is case sensitive. Also, additional command line parameters are available for advanced configuration. To see them, run the script without command line parameters

3. Verify that you see the message “successfully skipped”. You may see some WARN messages that you can ignore if you see the message at the end.

Using restoreskippedfile.pl

You may have a need to restore a previously skipped file (see skipfile.pl above). Normally, this request would come from instructions when troubleshooting an issue.

“restoreskippedfile.pl” will allow you to restore a previously skipped file so that it can then be downloaded by your DownloadFiles.pl script command.

You must have the below versions or later of both of the scripts to use these instructions. The version and instructions on how to check the version can be found in the current version release notes found here:

ReplicateDir.pl: 20110808

DownloadFiles.pl: 20110715

The “restoreskippedfile.pl” can be found within the ../intelsft/Services/DownloadFiles/ folder

Instructions:

1. Start a command line and change directory to ../intelsft/Services/DownloadFiles/

2. At your command line shell, type the following (all one line) replacing the following:

a. Replace with the archive file name that needs to be restored. This should not include the path to the file.

b. Replace with destination interface code (folder) that the file is downloading from. You may want to refer to the CONTENT_TO_DOWNLOAD property within your DownloadFiles folder

perl restoreskippedfile.pl --file= --sftint=

NOTE: File name parameter is case sensitive. Also, additional command line parameters are available for advanced configuration. To see them, run the script without command line parameters

3. Verify that you see the message “successfully restored”. You may see some WARN messages that you can ignore if you see the message at the end.

Changing Your Password Used in SFTP

You may want to change the password used in password based SFTP authentication (n/a for public key authentication) under the following scenarios:

• You just received a new username/password and need to change the password so that your team is only aware of it

• The password is expiring or is expired. Passwords expire periodically. Contact your Intel representative to determine when the password expires.

• The password may have been compromised

• The password needs to be changed for other security reasons

To change your password of an SFTP account used in password based authentication, do the following:

1. Browse to the Intel Profile Center web site from within a Web browser

2. Login with the username and current password

3. If your password is currently expired, you will be prompted immediately after login to change your password

4. Select Settings page

5. Select Edit Section next to "Change Password and Security Question".  Warning:  Do not choose to Edit your login id information as your account will not longer work with SFT after doing so.

6. Enter your old password, new password, and several security related questions and click Save.

Important Note on special characters within passwords: Do not use the special characters the ampersand (&), equals sign (=), double quotations marks ("), less than sign () in the new password.

7. You password now has been changed

8. Configure the password in the below files or your custom config file that you may be pointing to in the command line parameters:

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

If you have forgotten your password, browse to the Intel Profile Center and use the Forgot Password tool. Alternatively, you can contact your Intel representative and request a password reset.

Migrating To an New Intel SFT Environment

You may have been requested to point your installation to a new environment, perhaps to from the TEST environment to the PRODUCTION environment. The below steps will help guide you through this process.

1. Locate the following input parameters for these steps

You will need

8) A new SFTP host name

9) A username for your account

If your installation is on Windows, you may get a password for your account in addition to the above. You will not be provided a password if you are expecting to use an SSH Key for authentication.

Contact your Intel representative for these input parameters if you are unable to locate them

2. If you received a password for your account (n/a for Public Key Authentication users), then change your accounts password by following the steps below

a. Browse to the Intel Profile Center web site from within a Web browser

b. Login with the username and current password. Once entered, you will be prompted immediately after login to change your password

c. Enter your old password, new password, and several security related questions and click Save.

Important Note on special characters within passwords: Do not use the special characters ampersand (&), equals sign (=), double quotations marks ("), less than sign () in the new password.

d. You password now has been changed

3. Update any firewalls or Access Control Lists as necessary

This may be required to allow to allow a connection TO the Intel SFT host server (see new host name input parameter above) FROM your system through the network port 22 using SSH.

If necessary, please contact your network engineering team or system administrator for help.

4. Verify connectivity and add the new SSH Host Identity Key to your system cache as necessary

Troubleshooting for the below can be found in the SFT CR Troubleshooting Guide

Unix/Linux users can follow the below instructions

1. Using the user account that the Intel SFT CR scripts are running under, run your SFTP program using the following syntax at the command line with the username and new host name input parameters:

sftp username@newhostname

2. If prompted to accept the SSH Server Host key, please indicate ‘yes’ to ensure it is stored in your ssh key cache.

3. Confirm that the execution of the above command brings you immediately to a prompt ready for you to issue FTP commands. You should not see any prompt asking for a password. Type “pwd” as your first command to confirm that it responds with the current remote working directory. Please note that Intel SFT requires “pwd” as the first command durng an interactive mode session. Any other command prior to “pwd” will error and will abort the SFTP session. This is expected behavior.

4. At the SFTP prompt connected to the remote SFT server, type the following:

pwd

ls

5. Review the directory listing to confirm you see one or move folders with the prefix CR_. When verified, type the following to exit the SFTP session.

quit

Windows users can follow the below instructions

1. Login with an administrative account that is used to run all scheduled tasks required by the Intel SFT CR scripts.

2. Start a Windows command prompt

3. Run the following command within the new Command Window. Change the below path if your installation is something other than C:\intelsft\. Replace Host with the new host name input parameter.

Run: c:\intelsft\services\downloadfiles\psftp.exe Host

[pic]

4. If prompted, choose “Y” to store the key in cache.

5. Ignore the prompt for password and close the command window

5. Update the configuration files with the new parameters

Open for edit the following files:

…/intelsft/Services/DownloadFiles/intelsftconfig.ini

…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini

Locate the “host” property and change the value to the new host name

Locate the “username” property and change the value to the username

If you were provided a password, then locate the “pwd” property and change the value to your new password you changed it to in the above step.

Save each file

6. Validate the change

Monitor the log files on your Source and/or Destination systems when the CRON jobs/ Scheduled Tasks run to verify no errors.

Troubleshoot Issues

A separate guide has been created specifically for troubleshooting. See the SFT CR Troubleshooting Guide [pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download