Morningstar FTP System



[pic]

Morningstar FTP system – Technical Summary for Clients

System: ftp.

Document Updated: 5/31/2017

Introduction

Morningstar, Inc. hosts various file transfer mechanisms for providing data to clients or collecting data from data providers, clients and other parties.

ftp. is the primary system for provision of data to customers in North America and several other regions. The system is hosted in two datacenters. This document provides a brief technical overview meant to answer the most commonly asked questions and provide best practices.

Technical Description

ftp. is a group of Windows servers in redundancy and load balancing mode. They are running an FTP product named "SERV-U" by RhinoSoft.

It is setup for the following transmission protocols: regular FTP, SSL FTP (FTPS), or SSH FTP (SFTP). Clients can connect to the system with whichever protocol they prefer or Morningstar may designate a transmission protocol at their discretion. SSL FTP or SSH FTP sessions can be forced on a per client basis. The system supports both Passive and Active FTP and most well-known FTP clients work with the system. These systems are collectively known as ftp. via the Internet and internally to the Morningstar network.

Morningstar only allows the active FTP (TCP 20, 21), SSH FTP (TCP 22), SSL FTP (TCP 21 or 990) and PASV FTP (24200-24249) ports into the FTP servers from the Internet.

 

Clients and other parties should note that the system is served from multiple IP addresses potentially requiring firewall changes on their infrastructure.

Accounts are configured by the Morningstar network services team and follow a standard configuration.

 

Morningstar maintains a strict retention policy for data on this system. Files are removed from the server after a specific number of days – please consult your product documentation for the retention policy time frame. 

Recommended FTP practices and things to avoid:

1. Use the file naming conventions (and date stamps included in the file name) to verify that the day’s files have been posted. Please refrain from using commands like mdtm or list as these commands check each file and can cause slow response times for all clients.

2. Implement smarter checking mechanisms in your processes to avoid unnecessary duplicated robot downloads. This means that once a file is found and downloaded, your processes shouldn’t look for the file again until the next scheduled delivery time frame.

3. Please keep in mind that processes that download the same files many times a day are unnecessarily tying up FTP and processing resources. Consider how you can make your process as efficient as possible to save on resource waste.

4. For secure file transfers, FTP over SSH (SFTP) is recommended over FTP over SSL (FTPS). The firewall setup is much simpler for SFTP than for FTPS.

5. Idle-time out settings should be set to 300 seconds.

6. Passive mode FTP is recommended over Active mode.

7. Consider implementing retry mechanisms for reliable downloads. Your process should retry downloading a file only when the previous process fails. The retry interval should be at least 15 minutes after the previous attempt fails.

8. For all connections, firewall rules should be updated to allow access to ftp.’s IP addresses:

216.228.224.25

216.228.237.25

Frequently Asked Questions

1. As a client, I cannot connect to ftp.. Are there any firewall restrictions at Morningstar’s end?

a. No. There are no IP restrictions at ftp.. Please check with your firewall team to make sure the required rules are in place to permit outgoing FTP connections to ftp..

2. What are the required firewall rules that should be put in place to allow access to ftp.?

a. That depends on what kind of FTP protocols you will use. For all connections, firewall rules should be updated to allow access to ftp.’s IP addresses:

216.228.224.25

216.228.237.25

b. Here are additional rules to observe:

i. For SFTP connections: TCP 22

ii. For Regular FTP and FTPS (explicit) connections: TCP21, PASV 24200-24249

iii. For FTPS (implicit) connections: TCP 990, PASV 24200-24249

iv. The IP address of ftp. should be resolved via DNS query. External clients should update their FTP application to use DNS name ftp. instead of the IP address.

How many concurrent connections can we have to the FTP?

3 The number of concurrent sessions is currently unlimited, but ftp. will block users who connect more than 30 times within 30 seconds for 1 minute.

3. What FTP clients does Morningstar support?

a. ftp. supports ftp clients that are written with standard FTP protocols. The following three ftp clients are recommended: FileZilla (Freeware), FTP voyage from RhinoSoft and WS_FTP from Ipswich.

4. I can login but the connection is not stable. What should I do?

a. Please change the FTP client’s idle time out to 300 seconds to match the FTP server’s setup.

5. I tried to use an SFTP connection and the log says it is connected but there is no login prompt. What should I do now?

a. Please make sure that you have specified SFTP as the protocol in the application. Below is an example showing how to specify SFTP when using FileZilla:

[pic]

6. If I am still having issues connecting via an SFTP connection, what other information should I provide to Morningstar?

a. You may need to provide your Morningstar with your public SSH key (.pub key) so that our team can apply it to your FTP account.

b. To generate a SSH key pair on a Windows Machine:

i. Download PuTTYgen.exe and run it.

ii. Select the SSH-2 RSA radio button in the Parameters section near the bottom of the page. [pic]

iii. Click the Generate button.

iv. Move the mouse around in the blank area as instructed, until the PuTTYgen generates the key pair.

[pic]

v. Click the Save public key button and save the public key with the name id-rsa in the folder C:\Documents and Settings\username\.ssh, where username is your Windows user name.

vi. Click the Save private key button and save the private key with the name id-rsa.ppk in the same folder.

vii. Close PuTTY gen.

viii. Send the public key file id-dsa to your Morningstar contact.

c. To generate an SSH key pair on a Macintosh or Linux machine:

i. Open a terminal window.

ii. Enter this command line:

ssh-keygen -t rsa

iii. Select the default values for all options.

This command generates two SSH key files, id_dsa and id_dsa.pub, in the directory home/username/.ssh, where username is your user name.

iv. Send the public key file id_dsa.pub to your Morningstar contact.

7. When reporting issues related to the FTP, what information should I provide to Morningstar?

a. Full FTP path, EX.

b. Full name of file as found on the FTP, EX. DataWarehouse36_FO_USA_20160511.xml.gz

c. FTP Username used to retrieve the file

d. Issue they are experiencing:

i. File is not on FTP and is typically picked up at X time

ii. File is on FTP but client is having issues downloading

1. If attempt to download is not successful, send full log file of last failed transmission

iii. File is on FTP, client downloaded, but cannot open the file

1. Client should send error message text, log, or screen shot of error

e. Name of their Relationship Manager (and that RM should be included in any email messages.

Data issues will need to be addresses by the data team and relationship manager during normal business hours

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download