Intel SFT How To Configure Guide
Intel Secure File Transfer (SFT)
Content Replication
How To Configure Guide
|Document Information |
|Name |Intel Secure File Transfer (SFT) Content Replication How To Configure Guide |
|Version |v1.0 |
|Date Created |April 27, 2011 |
|Date Updated |September 25, 2011 |
Legal Notices and Disclaimers
Disclaimers
INTEL CORPORATION MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. INTEL CORPORATION ASSUMES NO RESPONSIBILITY FOR ANY ERRORS THAT MAY APPEAR IN THIS DOCUMENT. INTEL CORPORATION MAKES NO COMMITMENT TO UPDATE NOR TO KEEP CURRENT THE INFORMATION CONTAINED IN THIS DOCUMENT.
THIS SPECIFICATION IS COPYRIGHTED BY AND SHALL REMAIN THE PROPERTY OF INTEL CORPORATION. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED HEREIN.
INTEL DISCLAIMS ALL LIABILITY, INCLUDING LIABILITY FOR INFRINGEMENT OF ANY PROPRIETARY RIGHTS, RELATING TO IMPLEMENTATION OF INFORMATION IN THIS SPECIFICATION. INTEL DOES NOT WARRANT OR REPRESENT THAT SUCH IMPLEMENTATIONS WILL NOT INFRINGE SUCH RIGHTS.
NO PART OF THIS DOCUMENT MAY BE COPIED OR REPRODUCED IN ANY FORM OR BY ANY MEANS WITHOUT PRIOR WRITTEN CONSENT OF INTEL CORPORATION.
INTEL CORPORATION RETAINS THE RIGHT TO MAKE CHANGES TO THESE SPECIFICATIONS AT ANY TIME, WITHOUT NOTICE.
Legal Notices
Intel software products are copyrighted by and shall remain the property of Intel Corporation. Use, duplication or disclosure is subject to restrictions stated in Intel's Software License Agreement, or in the case of software delivered to the government, in accordance with the software license agreement as defined in FAR 52.227-7013.
The Intel logo is a registered trademark of Intel Corporation.
Other brands and names are the property of their respective owners.
Copyright ( 2009, Intel Corporation, All Rights Reserved
Document Version History
|Version |Date |Document History |
|1.0 |May 3, 2011 |Created by Jeremy Morrissey |
| | | |
| | | |
| | | |
| | | |
| | | |
Table of Contents
Legal Notices and Disclaimers 2
Disclaimers 2
Legal Notices 2
Table of Contents 4
1. Introduction 5
2. SMTP/Email Alert Option 5
3. File Alert Option 8
4. Controlling the Size of Content Replication (maxfilesize and maxfilecount) 10
5. Logging File Names Contained in Replic TAR Files 11
6. Generating a LIST file for files extracted onto Destination Systems 12
7. Using a custom working directory to avoid using the default user’s TMP folder 13
8. Replacing the use of the FTP “mget” command with “get –r” or another command 13
9. Minimizing Command Line Parameters using Configuration File 14
10. Logging to SYSLOG on Unix/Linux systems 15
11. Using SSH Public Keys On Windows Platform 17
12. Unix/Linux System Configuration Test Scripts 20
13. Developing custom alerting/error logging procedures 23
14. Script State Tracking 24
15. Using resendArchive.pl 28
16. Using skipfile.pl 29
17. Using restoreskippedfile.pl 31
18. Changing Your Password Used in SFTP 32
19. Migrating To an New Intel SFT Environment 33
20. Troubleshoot Issues 35
Introduction
The purpose of this guide is to provide detailed installation instructions on how to configure various features of the Intel SFT Content Replication capability on a Source or Destination system.
This guide depends on the Intel SFT Content Replication Install Guides for Unix/Linux or Windows procedures being followed to create an installation of this capability.
SMTP/Email Alert Option
Intel SFT Content Replication has the option that can be configured to send email alerts when an ERROR is logged during the execution of the ReplicateDir.pl or DownloadFiles.pl PERL scripts. All logs written by the script during the course of the script execution are send in the body of the email address.
Prerequisites for Unix/Linux:
← The “sendmail” program must be configured on the system to be able to send email
← You must have a valid email address that can be used in the FROM address of the email alerts.
← You must have one or more valid email addresses to send the alerts to
Prerequisites for Windows:
← Microsoft .NET FW 2.0 must be installed on the system
← A valid SMTP host name that can be used to connect over the network port 25 to send email
← You must have a valid email address that can be used in the FROM address of the email alerts
← You must have one or more valid email addresses to send the alerts to
[pic]IMPORTANT INFORMATION BEFORE ENABLING
• All users should be aware that when --loglevel= command line parameter is set to DEBUG a lot of information can be containing in the log and any error alerts that are triggered. Depending on your system constraints, this may result in a delay in receiving the alert or alert failing to be sent. It is recommended to use default --loglevel= setting when alerting is enabled. Refer to the troubleshooting guide for how --loglevel= parameter is set.
• Windows users that are configuring passwords in their intelsftconfig.ini files should avoid enabling this alerting feature when --loglevel= command line parameter is set to DEBUG. Refer to the troubleshooting guide for how --loglevel= parameter is set. To disable, set the ALERT_SMTPTOADDRESS to empty.
To configure this capability follow these steps:
1. Prior to following the below steps, you must have followed the steps found in the Intel SFT Content Replication Install Guide (for either Windows or Unix/Linux) to configure your system as the Source and/or Destination system.
2. Locate the following files in your installation
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
3. Open each of the above configuration files for editing
You will find the below properties that need to be edited in these files. Edit each property using the instructions below.
|Property |Value |
|ALERT_SCRIPT |With a standard installation, this property should be set to |
| |ALERT_SCRIPT=perl sendAlert.pl |
| | |
| |The value points to a PERL script that is used to send the email |
| |alert and is provided in a standard installation. |
| | |
| |If you do not find this property in your config file, add it with |
| |the value above. |
|ALERT_SMTPHOST |Windows users need to set this to the host name of an SMTP server |
| |that this sytem can connect to and send email |
| | |
| |Unix/Linux users – leave this value blank |
|ALERT_SMTPTOADDRESS |The email addresses to send the alerts to |
| | |
| |Windows users need to use the semicolon (;) as the delimiter for |
| |multiple email addresses |
| | |
| |Unix/Linux users need to use the comma (,) as the delimiter for |
| |multiple email addresses |
|ALERT_SMTPFROMADDRESS |A valid email address to send the alerts from. This should be |
| |recognizable to the recipients of the alert and should not be |
| |treated as spam or junk mail. |
|ALERT_SMTPSUBJECT |The text that shows in the subject line for all email alerts. The |
| |default value is “Intel SFT CR Alert”. The actual subject line |
| |will be appended with a colon (:) and the name of the script that |
| |produced the error, such as “: ReplicateDir” or “: DownloadFiles” |
4. If the scripts are running on the Linux/Unix platform and your sendmail program is not found in the /usr/sbin/sendmail path then update the ALERT_SCRIPT configuration property value to pass an additional command line parameter indicating your sendmail program path
For example, if your sendmail program path was “/usr/bin/sendmail” then use the following:
ALERT_SCRIPT=perl sendAlert.pl --sendmail=/usr/bin/sendmail
5. Verify that alerting was configured correctly and email alerts can be sent
Run the following command to test your alerting configuration. This will not trigger a replication process and is ok to run even if your system is not a Source system.
perl replicatedir.pl --source=test --data=test --archive=test --transfer=test --loglevel=4
The recipients configured should expect to see an alert with the subject similar to “Intel SFT CR Alert : ReplicateDir”
The alert body should be similar to what is shown below:
2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started
2011-05-15 08:14:09 232 DEBUG: init: begin
2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'!
2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped
File Alert Option
Intel SFT Content Replication has the option that can be configured to write a log file to a configurable path on your system when an ERROR is logged during the execution of the ReplicateDir.pl or DownloadFiles.pl PERL scripts. All logs written by the script during the course of the script execution are contained within this log file. The file name is uniquely named using the process id + a date time stamp.
You must have the below versions or later of both of the scripts to configure this option. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110808
DownloadFiles.pl: 20110715
[pic]IMPORTANT INFORMATION BEFORE ENABLING
• Windows users that are configuring passwords in their intelsftconfig.ini files should avoid enabling this alerting feature when --loglevel= command line parameter is set to DEBUG. Refer to the troubleshooting guide for how --loglevel= parameter is set. To disable, set the ALERT_SMTPTOADDRESS to empty.
To configure this capability follow these steps:
1. Locate the following files in your installation
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
2. Open each of the above configuration files for editing
Locate the ALERT_SCRIPT property in the configuration files
Change the ALERT_SCRIPT property to have the following file:
ALERT_SCRIPT= perl sendFileAlert.pl
IMPORTANT REMINDER: Depending on your system or how you are running the perl commands, you may be required to specify the full path to the “sendFileAlert.pl” script in the above property value. If so, an example for you to reference may be:
ALERT_SCRIPT= perl /usr/example/sendFileAlert.pl
Locate the ALERT_DIRECTORY property in the configuration files
If you cannot find the ALERT_DIRECTORY property, then add this property
The value of this property must be where you desire to have the alert log files written. The full path must be specified and the user account running the perl scripts must have write access. For an example property and value within your configuration:
ALERT_DIRECTORY=/usr/alertlogs
3. Verify that alerting was configured correctly and email alerts can be sent
Run the following command to test your alerting configuration. This will not trigger a replication process and is ok to run even if your system is not a Source system.
perl replicatedir.pl --source=test --data=test --archive=test --transfer=test --loglevel=4
A file should be written to the directory that you configured for ALERT_DIRECTORY within the intelsftconfig.ini files.
The file name format is ERROR__YYYY-MM-DD_HH_MM_SS.log
The file contents should be similar to what is shown below:
2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started
2011-05-15 08:14:09 232 DEBUG: init: begin
2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'!
2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped
Controlling the Size of Content Replication (maxfilesize and maxfilecount)
There may be a need to control the size of the content being replicated using the ReplicateDir.pl command. By default the ReplicateDir.pl command will package all files it finds to replicate and transfer them in one package (tar.gz file). The size of this package may need to be controlled on your system. This can be achieved using the following two options:
1. Specifying a maximum size in bytes of files to be added to the package
And/or
2. Specifying a maximum number of files to be added to the package
Regardless of the option used above, the files that are not packaged will queue up and be packaged in a subsequent run of ReplicateDir.pl.
You must have the below versions or later of both of the scripts to configure these options. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110808
DownloadFiles.pl: 20110715
The following command line parameters can be added to the ReplicateDir.pl to use these options:
--maxfilesize=
When the parameter --maxfilesize is present in the command line to ReplicateDir.pl only files totaling up to that file size will be replicated in that run. All others will wait until the next ReplicateDir.pl script run. An exception is when an individual file size is greater than this --maxfilesize value and in this case, this file will be packaged in a tar file by itself and all others will wait until the next time ReplicateDir.pl script is run. This is to avoid skipping these files and never replicating them.
--maxfilecount=
When the parameter --maxfilecount is present in the command line to ReplicateDir.pl only that number of files will be replicated in that run. All others will wait until the next ReplicateDir.pl script run.
Logging File Names Contained in Replic TAR Files
Intel SFT Content Replication has the feature to log the file attributes of files that are replicated between the Source and Destination systems. This feature is something must be enabled and is not enabled by default.
You must have the below versions or later of both of the scripts to configure this option. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110808
DownloadFiles.pl: 20110715
To configure this feature follow these steps:
1. To configure the Source system (that uses ReplicateDir.pl) edit your ReplicateDir.pl command line to add the below additional command line parameter:
--logfilesinarchive=Y
2. To configure the Destination system (that uses DownloadFiles.pl) edit your …/DownloadFiles/intelsftconfig.ini configuration file to enable this feature
Open the ../intelsft/Services/DownloadFiles/intelsftconfig.ini file or your custom configuration file you are using in your command line for downloadFiles.pl
Locate all the properties within this file that have a property name LSP or end with LSP and notice that they call a perl script called “auto_untar_gz.pl”
Edit each of those properties so that an additional command line parameter is added to. The additional command line parameter is “--logfilesinarchive=Y”. For example:
LSP = perl auto_untar_gz.pl --logfilesinarchive=Y --extractdestpath=…
IMPORTANT REMINDER: Depending on your system or how you are running the perl commands, you may be required to specify the full path to the “auto_untar_gz.pl” script in the above property value. If so, an example for you to reference may be:
LSP = perl /usr/example/auto_untar_gz.pl --logfilesinarchive=Y --extractdestpath=…
Generating a LIST file for files extracted onto Destination Systems
Intel SFT Content Replication has the feature to create a verbose list file containing file attributes of files that are replicated between the Source and Destination systems. This feature is something must be enabled and is not enabled by default.
The main difference between this feature and the logging feature described above is that this feature creates a separate file for each package of files extracted on the destination system whereas the logging feature logs the same info into the log that is configured for the downloadFiles.pl script.
You must have the below versions or later of both of the scripts to configure this option. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110916
DownloadFiles.pl: 20110916
This feature specifically uses options on your tar program configured
tar –tvf
To enable:
1) Open the DownloadFiles/intelsftconfig.ini files or your custom configuration file that you may have configured downloadFiles.pl to use
2) Edit all property values for properties that are prefixed with “LSP” to contain the command line parameter of “--extractfilenames=Y”.
As an example:
LSP=perl auto_untar_gz.pl --extractfilename=Y
--extractedfilenameslog=/home/logpath/ --filetoprocess=
Once enabled, for every content replication downloaded by downloadFiles.pl, you will find a file created with the following naming convention in the path that --extractedfilenameslog parameter is configured with.
._.log
Example file names:
[pic]
The below is an example of the contents when ran on SUSE Linux
[pic]
Using a custom working directory to avoid using the default user’s TMP folder
The default folder that the scripts use for temporary files is the user’s Operating System managed TEMP folder.
All command lines support an optional --sftwd= parameter that when specified will instruct the script to use that folder and not your user accounts “temp” folder that the Operating System provides to you.
replicateDir.pl … [--sftwd=] …
downloadFiles.pl … [--sftwd=] …
Replacing the use of the FTP “mget” command with “get –r” or another command
Some SFTP programs do not have support for the FTP “mget” command. These SFTP programs are able to use other FTP commands that provide the same functionality; therefore, can be configured for use in the scripts.
Edit the following file:
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
Review the file for a MGET_COMMAND property. If you do not find one, then add one: “MGET_COMMAND=”
Set the MGET_COMMAND to the FTP command that replaced “mget” for your SFTP program
For example, to replace “mget” with “get –r” (recursive get), then you would have this property set:
MGET_COMMAND=get -r
Minimizing Command Line Parameters using Configuration File
The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110916
DownloadFiles.pl: 20110916
All command line parameters except for specifying a custom configuration file can be specified in the intelsftconfig.ini configuration file or your custom configuration file. This can be helpful to centralize configuration as well as shorten the command line used to execute the PERL scripts. Please note that not all configuration parameters used in intelsftconfig.ini file or your custom configuration file can be used as command line parameters.
When a configuration is specified in BOTH the configuration file and as a command line parameter, the parameter value in the command line parameter is used. This may be helpful in defining a central configuration file with all default values set that command lines can overwrite and use different values.
To define a command line parameter in the intelsftconfig.ini file or your custom configuration file, drop the “--" prefix and use UPPERCASE letters in the property name as an example:
Command line before:
[pic]
All parameters added to intelsftconfig.ini file:
[pic]
Now the command line is:
[pic]
This applies to the following PERL scripts:
1) ReplicateDir.pl
2) downloadFiles.pl
3) auto_untar_gz.pl (--processfiles is a param that must stay on the command line)
Logging to SYSLOG on Unix/Linux systems
The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110916
DownloadFiles.pl: 20110916
This optional feature can be enabled to log to the System Log (SYSLOG) on Unix/Linux systems. This feature is not currently available on the Windows platform.
The severity of the log message will map to your System log severities:
ERROR ( ERR
WARN ( WARN
INFO ( INFO
DEBUG ( DEBUG
VERSOSE ( DEBUG
To enable:
3) Open the intelsftconfig.ini files or your custom configuration file that your PERL scripts are using. The typical location of these are as follows.
/intelsft/Services/DownloadFiles/intelsftconfig.ini
/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
4) Add the following line into the configuration files:
LOGGING = syslog
Contact your system administrator for instructions on how to access and manage your system log.
An example of what you may see on SUSE Linux is shown below:
[pic]
Using SSH Public Keys On Windows Platform
The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110916
DownloadFiles.pl: 20110916
To configure SSH Key Authentication when running the PERL scripts on a Windows platform, complete the following steps:
1) Locate the following input parameters for these steps
You will need a new SFTP host name that allows for public key authentication
Contact your Intel representative for this input parameter if you are unable to locate the above
2) Login with an administrative account that is used to run all scheduled tasks required by the Intel SFT CR scripts.
3) Store the SSH Remote Host Key in your system cache using the following steps:
a. Start a Windows command prompt
b. Run the following command within the new Command Window. Change the below path if your installation is something other than C:\intelsft\. Replace Host with the new host name input parameter.
Run: c:\intelsft\services\downloadfiles\psftp.exe Host
[pic]
c. If prompted, choose “Y” to store the key in cache.
d. Close the command window (ignore the prompt to login)
4) Generate a Private/Public Key pair using the Putty Key Generation tool with the below steps:
a. Start a Windows command prompt
b. Run the following command within the new Command Window. Change the below path if your installation is something other than C:\intelsft\.
Run: c:\intelsft\services\downloadfiles\puttygen.exe
c. A screen similar to the one below display. First, enter the key length in the box at the bottom right corner. This number represents the key length, and the standard length for use with Intel SFT is 2048 bits, so enter “2048” in the box.
[pic]
d. Next, click the “Generate” button. You will be prompted to move your mouse over the blank area as indicated below. The movements from the mouse are collected and used in the process of generating your key. Depending upon your system, this may take a few seconds to a minute to complete.
[pic]
e. After the key has been generated a screen will display similar to the one below with the “Save public key” and “Save private key” buttons enabled. Click the “Save public key” button, and choose a folder to save your public key. This file contains the key that will be provided to Intel.
f. After saving the public key, click the “Save private key” button. This file should be stored in an access controlled secure location on your system. The system or user account running the Intel SFT CR scripts will need access to read this file. The Intel SFT CR scripts You may also want to create a backup of the public and private key pair and store in a secure location.
5) Provide your Intel representative with the Public Key file saved above. Do not send the Private Key.
6) Configure your Intel SFT PERL scripts to use your SSH Keys generated above by following the below steps:
a. Open the following files or your custom configuration file for editing
/intelsft/Services/DownloadFiles/intelsftconfig.ini
/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
b. Make the following changes in the configuration files
i. Update the “host” property to the provided remote host name that will allow connections requiring SSH key authentication
host=ift-keyauth.
ii. Set the value of the “pwd” property to “key”
pwd=key
iii. Add a new property to the configuration file for “KEYPATH” and set the value to the full path and file name of the Private key file generated and saved above (file extension will be .ppk).
KEYPATH=
Example: KEYPATH=d:\keys\mykey.ppk
7) Verify the PERL scripts function as expected
Unix/Linux System Configuration Test Scripts
The steps below apply only to Linux/Unix platforms and typically used during the installation of the Intel provided PERL scripts.
The steps below will help ensure that your configuration is set correctly to avoid the potential for loss of data.
Follow the steps below on the Source system using the ReplicateDir commands to replicate files.
1) Change directory to your ../Utilities/ReplicateDir directory
2) Run the below from your command line. If you are using a different config file with the downloadFiles.pl command, then specific the full page to it using the --configfilepath= command line parameter.
perl systemtest_unix_upload_rc_ms.pl
3) Follow the prompt to enter a Folder that is shown in the directory listing and hit your Enter key. The folder is case sensitive.
4) If your configuration is correct, you should see a message indicating ”No action required...”
5) Otherwise, follow the instructions in the output and rerun the above steps once you completed the action to verify
For your reference, the script output shown below was ran on SUSE LINUX Enterprise Server 9 with an incorrect configuration. The PERL script is expected to indicate what action is required. Your system may have a different value to set.
plxs0101> perl systemtest_unix_mget_rc.pl
ACTION REQUIRED: Set MGET_NO_MATCHES_RETURNCODE=10 in the ../DownloadFiles/intelsftconfig.ini or your custom config file
Again, for your reference, the output shown below again on the same system after the action required was taken. The configuration is confirmed to be correct with the “No Action Required” statement below.
plxs0101> perl systemtest_unix_mget_rc.pl
No Action Required: Current configuration is set correctly (MGET_NO_MATCHES_RETURNCODE=10) in the ../DownloadFiles/intelsftconfig.ini or your custom config file
Follow the steps below on the Destination system using the DownloadFiles commands to download files.
1) Change directory to your ../Services/DownloadFiles directory
2) Run the below from your command line. If you are using a different config file with the downloadFiles.pl command, then specific the full page to it using the --configfilepath= command line parameter.
perl systemtest_unix_mget_rc.pl
3) If your configuration is correct, you should see a message indicating ”No action required...”
4) Otherwise, follow the instructions in the output and rerun the above steps once you completed the action to verify
For your reference, the script output shown below was ran on SUSE LINUX Enterprise Server 9 with an incorrect configuration. The PERL script is expected to indicate what action is required. Your system may have a different value to set.
plxs0101> perl systemtest_unix_upload_rc_ms.pl
File listing script output (using systemtest_ftp_ls.scr):
sftp> pwd
/
sftp> ls
.:
CR_CMPA2BCH/
sftp> quit
Please enter the folder from the above list that you'd like to test with
CR_CMPA2BCH
ACTION REQUIRED: Set PUT_SFTP_CHMOD_RETURNCODE=7 in the ../ReplicateDir/intelsftconfig.ini or your custom config file
plxs0101>
Again, for your reference, the output shown below again on the same system after the action required was taken. The configuration is confirmed to be correct with the “No Action Required” statement below.
plxs0101> perl systemtest_unix_upload_rc_ms.pl
File listing script output (using systemtest_ftp_ls.scr):
sftp> pwd
/
sftp> ls
.:
CR_CMPA2BCH/
sftp> quit
Please enter the folder from the above list that you'd like to test with
CR_CMPA2BCH
No Action Required: Current configuration is set correctly (PUT_SFTP_CHMOD_RETURNCODE=7) in the ../ReplicateDir/intelsftconfig.ini or your custom config file
Developing custom alerting/error logging procedures
Intel SFT Content Replication has the flexibility to call a custom PERL script at the end of each script execution when an ERROR log is written during the course of its execution. This custom PERL script would be passed the location of a temporary file containing all logs written by the Content Replication script and can process that file in a custom way.
To develop a custom/error logging procedure follow these steps:
1. Develop a custom PERL script
Use sendAlert.pl as a reference. A copy of this file can be found in the following folders.
…/intelsft/Services/DownloadFiles/
…/intelsft/Utilities/ReplicateDir/
The custom PERL script must accept the following command line parameters:
|subject |This parameter will provide a string that can be helpful to identify the context in which the error was |
| |generated in, such as “ReplicateDir” or “DownloadFiles”. |
|alertLog |This parameter will contain the absolute path of a file that will contain all logs written by the Content |
| |Replication script during its execution. Below is a sample format. |
| |2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started |
| |2011-05-15 08:14:09 232 DEBUG: init: begin |
| |2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'! |
| |2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped |
Develop your PERL script to process the above command line parameters
Do not delete the alertLog within your PERL script
2. Locate the following files in your installation
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
3. Open each of the above configuration files for editing
You will find the below property that needs to be edited to point to your custom PERL script.
|Property |Value |
|ALERT_SCRIPT |Your PERL script command line and any custom command line |
| |parameters it takes other than those found in above step |
4. Verify the configuration of the custom PERL script
Run the following command to test your alerting configuration. This will not trigger a replication process and is ok to run even if your system is not a Source system.
perl replicatedir.pl --source=test --data=test --archive=test --transfer=test --loglevel=4
Your custom PERL script should be invoked with the subject “ReplicateDir”
The alert log that it was passed the absolute path to should be similar to below:
2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 started
2011-05-15 08:14:09 232 DEBUG: init: begin
2011-05-15 08:14:09 232 ERROR: init: unknown transfer type 'test'!
2011-05-15 08:14:09 232 INFO: replicateDir.pl 20110916 stopped
Script State Tracking
The information in this sections applies only to the version stated below or later versions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110916
DownloadFiles.pl: 20110916
This optional feature can be enabled to call a custom PERL script developed to track the state of the scripts
ReplicateDir.pl can move through the following states:
|State |Description |Comments |
|STARTED |The script started | |
|FILES FOUND TO REPLICATE |Files were found to replicate | |
|FILES PACKAGED |Files were packaged into a TAR file |The name of the LST file that contains all the |
| | |files packaged is tracked with this state |
|FILES COMPRESSED |The TAR file was compressed |The name of the TAR archive file compressed is |
| | |tracked with this state |
|FILE TRANSFER BEGIN |The SFTP upload started |The remote path and file name is tracked with |
| | |this state |
|FILE TRANSFER END SUCCESS |The SFTP upload ended successfully |The remote path and file name is tracked with |
| | |this state |
|FILE TRANSFER END ERROR |The SFT upload ended on error |The remote path and file name that encountered |
| | |an error is tracked with this state |
|STOPPED |The script stopped | |
DownloadFiles.pl can move through the following states:
|State |Description |Comments |
|STARTED |The script started | |
|FILES TO DOWNLOAD FOUND |Files were found to replicate | |
|FILE TRANSFER BEGIN |The SFTP file download started |The remove path and file name of the file |
| | |downloaded is tracked with this state |
|FILE TRANSFER END SUCCESS |The SFTP file download ended successfully |The remove path and file name of the file |
| | |downloaded is tracked with this state |
|FILE TRANSFER END ERROR |The SFTP file download ended on error |The remove path and file name of the file |
| | |downloaded is tracked with this state |
|FILES DECOMPRESSED |Files were packaged into a TAR file |The path and file name of the compressed file |
| | |is tracked with this state |
|FILES EXTRACTED |The TAR file was compressed | |
|STOPPED |The SFTP upload started | |
To develop a custom PERL script to track the state of the scripts, do the following:
1) Use sessionState.pl as a reference. A copy of this file can be found in the following folders.
…/intelsft/Services/DownloadFiles/
…/intelsft/Utilities/ReplicateDir/
The custom PERL script must accept the following command line parameters:
|--event |This parameter will contain the state name as shown in the above tables |
|--pid |This parameter will contain the Process ID of the script to associate states to a particular script |
| |execution. |
|--callingScript |This parameter will contain the PERL script name |
|--data |This parameter will contain additional optional data that may be tracked with the state |
2) Locate the following files in your installation
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
3) Open each of the above configuration files for editing
Add the SESSIONSTATE property to the file if it does not already exist.
|Property |Value |
|SESSIONSTATE |Your PERL script command line and any custom command line |
| |parameters it takes other than those found in above step |
| | |
| |For example: perl sessionState.pl --sessionStateLog=state.log |
4) Verify the configuration of the custom PERL script
Run the PERL script that you have configured this with and confirm the expected results
As an example that is provided with the install of Intel SFT, the demonstration “sessionState.pl” script can be configured and used using the following states:
1) Locate the following files in your installation
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
2) Open each of the above configuration files for editing and add the following line to the file if it does not already exist
SESSIONSTATE = perl sessionState.pl --sessionStateLog=state.log
3) Run your scripts and review the state.log file that will be found in the scripts working directory it is executing in. Below are examples of what you may see in this file:
ReplicateDir.pl successful run (contents of state.log):
[pic]
DownloadFiles.pl successful run (contents of state.log)::
[pic]
Using resendArchive.pl
You may have a need to retransfer content that was previously replicated to the destination. Normally, this request would come from the Destination system or via instructions when troubleshooting an issue.
“resendArchive.pl” will allow you to manually trigger a transfer of a tar archive that is found within your /archive folder that you have configured on the ReplicateDir.pl command line. Once transferred, the Destination system is expected to be able to download the archive.
You must have the below versions or later of both of the scripts to use these instructions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110808
DownloadFiles.pl: 20110715
The “resendarchive.pl” can be found within the ../intelsft/Utilities/ReplicateDir/ folder
Instructions:
1. Start a command line and change directory to ../intelsft/Utilities/ReplicateDir/
2. At your command line shell, type the following (all one line) replacing the following:
a. Replace with the path to your archive folder and the archive file name that needs to be resent. The path to your archive can be found as the value of the --archive parameter in your ReplicateDir.pl command line used normally to replicate files from your source folder.
b. Replace with the --sftint parameter that can be found in your ReplicateDir.pl command line used normally to replicate files from your source folder.
perl resendArchive.pl --archivefile= --sftint=
3. Verify that you see the message “transferFile: file successfully transferred”. You may see some WARN messages that you can ignore if you see the message at the end.
For reference, the following examples show the resendArchive.pl executing on the Windows and Unix platform:
Windows:
[pic]
Unix/Linux:
plxs0100> cd intelsft/Utilities/ReplicateDir/
plxs0100> perl resendArchive.pl --archivefile=/tmp/CR_CMPA/source/archive/replic_2011-08-03_18_59_39.tar.gz --sftint=CR_CMPA2BCH
2011-08-03 22:53:35 21902 INFO: resendArchive.pl 20110801 started
2011-08-03 22:54:12 21902 INFO: transferFile: file successfully transferred: >/tmp/CR_CMPA/source/archive/replic_2011-08-03_18_59_39.tar.gz< to >CR_CMPA2BCH<
2011-08-03 22:54:12 21902 INFO: resendArchive.pl 20110801 stopped
Using skipfile.pl
You may have a need to flag a file that would otherwise be downloaded (or attempted to be downloaded) by your DownloadFiles.pl script to be skipped. Normally, this request would come from instructions when troubleshooting an issue.
“skipfile.pl” will allow you to manually flag a file to be skipped by your DownloadFiles.pl script command. These files can be restored by using the “restoreskippedfile.pl” script.
You must have the below versions or later of both of the scripts to use these instructions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110808
DownloadFiles.pl: 20110715
The “skipfile.pl” can be found within the ../intelsft/Services/DownloadFiles/ folder
Instructions:
1. Start a command line and change directory to ../intelsft/Services/DownloadFiles/
2. At your command line shell, type the following (all one line) replacing the following:
a. Replace with the archive file name that needs to be skipped. This should not include the path to the file.
b. Replace with destination interface code (folder) that the file is downloading from. You may want to refer to the CONTENT_TO_DOWNLOAD property within your DownloadFiles folder
perl skipfile.pl --file= --sftint=
NOTE: File name parameter is case sensitive. Also, additional command line parameters are available for advanced configuration. To see them, run the script without command line parameters
3. Verify that you see the message “successfully skipped”. You may see some WARN messages that you can ignore if you see the message at the end.
Using restoreskippedfile.pl
You may have a need to restore a previously skipped file (see skipfile.pl above). Normally, this request would come from instructions when troubleshooting an issue.
“restoreskippedfile.pl” will allow you to restore a previously skipped file so that it can then be downloaded by your DownloadFiles.pl script command.
You must have the below versions or later of both of the scripts to use these instructions. The version and instructions on how to check the version can be found in the current version release notes found here:
ReplicateDir.pl: 20110808
DownloadFiles.pl: 20110715
The “restoreskippedfile.pl” can be found within the ../intelsft/Services/DownloadFiles/ folder
Instructions:
1. Start a command line and change directory to ../intelsft/Services/DownloadFiles/
2. At your command line shell, type the following (all one line) replacing the following:
a. Replace with the archive file name that needs to be restored. This should not include the path to the file.
b. Replace with destination interface code (folder) that the file is downloading from. You may want to refer to the CONTENT_TO_DOWNLOAD property within your DownloadFiles folder
perl restoreskippedfile.pl --file= --sftint=
NOTE: File name parameter is case sensitive. Also, additional command line parameters are available for advanced configuration. To see them, run the script without command line parameters
3. Verify that you see the message “successfully restored”. You may see some WARN messages that you can ignore if you see the message at the end.
Changing Your Password Used in SFTP
You may want to change the password used in password based SFTP authentication (n/a for public key authentication) under the following scenarios:
• You just received a new username/password and need to change the password so that your team is only aware of it
• The password is expiring or is expired. Passwords expire periodically. Contact your Intel representative to determine when the password expires.
• The password may have been compromised
• The password needs to be changed for other security reasons
To change your password of an SFTP account used in password based authentication, do the following:
1. Browse to the Intel Profile Center web site from within a Web browser
2. Login with the username and current password
3. If your password is currently expired, you will be prompted immediately after login to change your password
4. Select Settings page
5. Select Edit Section next to "Change Password and Security Question". Warning: Do not choose to Edit your login id information as your account will not longer work with SFT after doing so.
6. Enter your old password, new password, and several security related questions and click Save.
Important Note on special characters within passwords: Do not use the special characters the ampersand (&), equals sign (=), double quotations marks ("), less than sign () in the new password.
7. You password now has been changed
8. Configure the password in the below files or your custom config file that you may be pointing to in the command line parameters:
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
If you have forgotten your password, browse to the Intel Profile Center and use the Forgot Password tool. Alternatively, you can contact your Intel representative and request a password reset.
Migrating To an New Intel SFT Environment
You may have been requested to point your installation to a new environment, perhaps to from the TEST environment to the PRODUCTION environment. The below steps will help guide you through this process.
1. Locate the following input parameters for these steps
You will need
8) A new SFTP host name
9) A username for your account
If your installation is on Windows, you may get a password for your account in addition to the above. You will not be provided a password if you are expecting to use an SSH Key for authentication.
Contact your Intel representative for these input parameters if you are unable to locate them
2. If you received a password for your account (n/a for Public Key Authentication users), then change your accounts password by following the steps below
a. Browse to the Intel Profile Center web site from within a Web browser
b. Login with the username and current password. Once entered, you will be prompted immediately after login to change your password
c. Enter your old password, new password, and several security related questions and click Save.
Important Note on special characters within passwords: Do not use the special characters ampersand (&), equals sign (=), double quotations marks ("), less than sign () in the new password.
d. You password now has been changed
3. Update any firewalls or Access Control Lists as necessary
This may be required to allow to allow a connection TO the Intel SFT host server (see new host name input parameter above) FROM your system through the network port 22 using SSH.
If necessary, please contact your network engineering team or system administrator for help.
4. Verify connectivity and add the new SSH Host Identity Key to your system cache as necessary
Troubleshooting for the below can be found in the SFT CR Troubleshooting Guide
Unix/Linux users can follow the below instructions
1. Using the user account that the Intel SFT CR scripts are running under, run your SFTP program using the following syntax at the command line with the username and new host name input parameters:
sftp username@newhostname
2. If prompted to accept the SSH Server Host key, please indicate ‘yes’ to ensure it is stored in your ssh key cache.
3. Confirm that the execution of the above command brings you immediately to a prompt ready for you to issue FTP commands. You should not see any prompt asking for a password. Type “pwd” as your first command to confirm that it responds with the current remote working directory. Please note that Intel SFT requires “pwd” as the first command durng an interactive mode session. Any other command prior to “pwd” will error and will abort the SFTP session. This is expected behavior.
4. At the SFTP prompt connected to the remote SFT server, type the following:
pwd
ls
5. Review the directory listing to confirm you see one or move folders with the prefix CR_. When verified, type the following to exit the SFTP session.
quit
Windows users can follow the below instructions
1. Login with an administrative account that is used to run all scheduled tasks required by the Intel SFT CR scripts.
2. Start a Windows command prompt
3. Run the following command within the new Command Window. Change the below path if your installation is something other than C:\intelsft\. Replace Host with the new host name input parameter.
Run: c:\intelsft\services\downloadfiles\psftp.exe Host
[pic]
4. If prompted, choose “Y” to store the key in cache.
5. Ignore the prompt for password and close the command window
5. Update the configuration files with the new parameters
Open for edit the following files:
…/intelsft/Services/DownloadFiles/intelsftconfig.ini
…/intelsft/Utilities/ReplicateDir/intelsftconfig.ini
Locate the “host” property and change the value to the new host name
Locate the “username” property and change the value to the username
If you were provided a password, then locate the “pwd” property and change the value to your new password you changed it to in the above step.
Save each file
6. Validate the change
Monitor the log files on your Source and/or Destination systems when the CRON jobs/ Scheduled Tasks run to verify no errors.
Troubleshoot Issues
A separate guide has been created specifically for troubleshooting. See the SFT CR Troubleshooting Guide [pic]
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.