Purpose of this documentation - Harvard University



User’s Guide to FAS RCTable of Contents TOC \o "1-3" \h \z \u Purpose of this documentation PAGEREF _Toc56694554 \h 1FAS RC Cohort Datasets applied PAGEREF _Toc56694555 \h 1Getting Started PAGEREF _Toc56694556 \h 1Step 1: Request an account PAGEREF _Toc56694557 \h 1Step 2: Request access to the Health Professionals User Group (hpnh files) PAGEREF _Toc56694558 \h 1Step 3: Create a directory for yourself in the hpnh Users folder (/n/hpnh/Users) PAGEREF _Toc56694559 \h 1Step 4: Set up two-factor authentication PAGEREF _Toc56694560 \h 2Step 5: Set up VPN PAGEREF _Toc56694561 \h 2Step 6: Log in to the cluster using a terminal application PAGEREF _Toc56694562 \h 3Step 7: Mounting storage on your desktop or laptop PAGEREF _Toc56694563 \h 3Other Helpful Resources PAGEREF _Toc56694564 \h 4FAS RC websites PAGEREF _Toc56694565 \h 4List of applications you will need to download/ install: PAGEREF _Toc56694566 \h 4Basic code for the terminal application PAGEREF _Toc56694567 \h 5Guide to the virtual desktop site PAGEREF _Toc56694568 \h 6Cohort Docs (Google Site) PAGEREF _Toc56694569 \h 6Analysis Macros (SAS) PAGEREF _Toc56694570 \h 7HPFS Guide in FAS RC PAGEREF _Toc56694571 \h 7Run SAS jobs in FAS RC PAGEREF _Toc56694572 \h 7HPFS FAS RC directories guide PAGEREF _Toc56694573 \h 8HPFS Admin Guide PAGEREF _Toc56694574 \h 9Users in the HPFS groups PAGEREF _Toc56694575 \h 10HPFS directory mapping/flow chart PAGEREF _Toc56694576 \h 11Purpose of this documentationTo get access to the FAS RC clusterTo set computer up to move forward with analysis using FAS cluster and HPFS dataTo provide information once in FAS RC clusterFAS RC Cohort Datasets appliedHealth Professionals Follow-up Study (HPFS)Multiethnic Cohort (MEC)Getting StartedStep 1: Request an accountGo to the FASRC Website and under the “Documentation” tab, select “Request an Account”Read the “How Do I Get a FAS Research Computing Account?” page and click on the Account Request Tool – this should bring you to the RC PortalPick the appropriate User type and submit the request, selecting the appropriate lab group so the PI can give you approvalWait for approval from the PI and set up your username and password when promptedStep 2: Request access to the Health Professionals User Group (hpnh files)Open the RC portal (portal.rc.fas.harvard.edu) and login with your FAS username and passwordClick on “Cluster Access” and then “Add Grants” and search for “HPFS” – select “Health Professionals HPFS User Group”NOTE: If you will also be responsible for updating project files (i.e. cancer files, death files, macros etc.) please also select “Health Professionals Admin Group”Email Ruifeng Li (rli@hsph.harvard.edu) if you have questions about which groups you should have access toStep 3: Create a directory for yourself in the hpnh Users folder (/n/hpnh/Users)Email rchelp@rc.fas.harvard.edu and request a directory under /n/hpnh/UsersYou can only request this once you have received access to the Health Professionals User Group (step 2)Note: FAS gives you a default home directory under your username, but you should do all of your HPFS-related work (and potential future other projects/ cohorts) in this hpnh-specific personal directoryDefault home directory path will look something like this: /n/home##/username (i.e. /n/home04/jharvard)Personal hpnh user directory path will look something like this: /n/hpnh/Users/username (i.e. /n/hpnh/Users/jharvard)Step 4: Set up two-factor authenticationThere are two main ways to do this:Using the google authenticator app (preferred method) – download on your phoneOR using the Java desktop app – download on your computer*note: you do not have to do both*Go to this website: (if the link is broken, go to the FASRC website Documentation Getting Started OpenAuth)The instructions on this site should walk you through this processClick the link under “Requesting Your Token” and login with your username and passwordYou should receive an email with a link to the install page with instructions to get your tokenIf you are using the Google Authenticator App method, scan the QR code If you are using the Java desktop app method, follow the instructions on the OpenAuth documentation pageNow, when you open either app, it should generate a 6-digit code every 30 seconds that you can use when you log inStep 5: Set up VPNDownload and install Cisco AnyConnect Secure Mobility ClientGo to this website for instructions: (if the link is broken, go to the FASRC website Documentation Login & Authentication VPN Setup)Click the link under VPN Software Installation ()If you are using MacOSX 10.15 (Catalina), download the latest version at this linkLog in with your FASRC username and add @fasrc at the end (username@fasrc), your password, and the 6-digit authentication code from the Google Authenticator AppDownload Cisco AnyConnect and install Open Cisco AnyConnect and type in “vpn.rc.fas.harvard.edu” and click connectFor username, type your username@fasrc (i.e. jharvard@fasrc)Your first password is the same password you used to login to the portalFor your second password, open up the Google Authenticator App and type in the 6-digit codeStep 6: Log in to the cluster using a terminal applicationMac OS users: use the built-in Terminal applicationOpen a new terminal window Type the following: ssh username@login.rc.fas.harvard.edu (replace username with your FASRC username)Type in your password when promptedOpen the Google Authenticator App and type in the 6-digit code when you are prompted to type in the verification codeWindows users: there are several options listed below (taken from FASRC website page about terminals – follow the instructions for the application you pick)Git BASHPuTTYMobaXtermSecureCRTXMing (standalone)Step 7: Mounting storage on your desktop or laptopLog in to the VPN using CiscoUsing the Terminal, log in using the ssh code and type in “pwd” to find the directory path for your home directory (this should be something like /n/home##/username – where ## is something like 01 or 02)Mac users:Open Finder and click on “Go” from the top dropdown menu optionsSelect “Connect to Server”To mount your home directory, type smb://rcstore.rc.fas.harvard.edu/homes/home##/username/ and click connectType in your log in credentials when prompted. Check “Remember my credential” so you don’t have to log in every timeTo mount the hpnh directory to your computer, type smb://hpnhfs.rc.fas.harvard.edu Type in your log in credentials when prompted and check “Remember my credential”Click hpnh when you are asked to select the volumes you want to mountWindows users:Open the File Explorer WindowRight click “This PC” to choose “Map network drive” from the pop-up menu Fill in the following: \\hpnhfs.rc.fas.harvard.edu\hpnh - it will ask for the credential again where the username is rc\username (fill in your username) and same password for your FASRC account. Check “Remember my credential” so you don’t have to log in every timeMap to your home directory by doing the same thing as above, just typing in the following: \\rcstore.rc.fas.harvard.edu\homes\home##\usernameYou now should be able to open, edit, create and save files in your personal hpnh user directory using the text editor of your choiceOther Helpful ResourcesFAS RC websitesFAS RC Homepage FAS RC Documentation page has a lot of documentation – it can be difficult to navigate, but if you have a question, it is probably answered somewhere on this siteFAS RC Portal submit a ticket, request access to the cluster, or to look at the jobs you have runningOn this site under modules is the list of applications on the cluster – you can search these and the code to use to load them (and add to your .bashrc directory)Introduction to the Cluster orientation-of-sorts – there is a link to a YouTube video and slides with some helpful hintsFAS RC Virtual Desktop Site: you need to be logged into the VPN using Cisco to access this siteSee below for more informationList of applications you will need to download/ install:Cisco AnyConnect Secure Mobility Client Download needs to be done through the link on the FASRC websiteTo set up a VPN on your computerGoogle authenticator (on your phone)For two-factor authentication codesDownload in the app store on your phone and add the FASRC QR code generated in Step 3)Terminal applicationThis comes built in on Mac computers (search “terminal” in applications)For PC users, there are several options (taken from FASRC website page about terminals – follow the instructions for the application you pick)Git BASHPuTTYMobaXtermSecureCRTXMing (standalone)Text editor of your choosingFor SAS: SAS Universal Viewer (Windows only)Sublime Text (you can install SAS Syntax for easy editing)nano/emacs/vi within FAS RC cluster/terminalFile transfer software FileZillaInstructions and download link found here: (if link is broken, search FileZilla on the FASRC website docs page)PuTTY WinSCPBasic code for the terminal applicationpwd = present working directory (tells you the path of the directory you are currently in – default is your home directory)ls = lists all files and folders in the current directory you are incd = change directory – if you want to move to another directory, type this code and then the name of the directory you want to move to. If you just type cd, it will bring you back to your home directoryrm <file name> = deletes the filerm -i<file name> = deletes the file with prompting for confirmation, equivalent to Channing rmcp <file name> <new directory> = copies the specified file to the specified new directorychmod = changes the permissions for a given file each document has a set of permissions (to read the file, write in the file, and execute the file) for the owner, group and everyone else (-rwx-rwx-rwx) – these can be changed with the “chmod” code to make it more open or more restrictedlook up linux/unix permissions for more information about thisGuide to the virtual desktop siteThe virtual desktop site allows you to submit, run, and view jobs all through your browser (although you do need to be logged into the VPN using Cisco) – it has its own terminal window and text editor window as well as servers that allow you to logon and run code interactivelyOutlined below are the main sections of the site:Interactive Apps: applications you can use on the VDI are found hereThey include: RStudio Server, Stata, Jupyter Lab and notebook, Matlab, remote desktops, and othersUsing the RStudio Server, you can run R code interactively line-by-line and utilize all the tools you need there (file browser, editor, viewer etc.) If you use SAS, you can also use this server to edit SAS code in RStudio and then submit it through the command line in the terminal window. Alternatively, you can try writing SAS code in RStudio with readable markdown annotations (see this link for instructions)Files: with a link to your home directory (/n/home##/username), you can see the files there or search any path to see the files and folders that directory containsYou can also view, edit (for the files that you have ‘w’ permissions), create, download or copy/paste files from this siteThe VDI has a text editor you can open in your browser to edit and create new filesJobs: allows you to see your active jobs and create a new job using the “Job Composer” toolThis tool helps you compose your job either from a default template, a specified path or a selected jobClusters: this brings you to a terminal window in your browser where you can type code as you typically would in a terminal applicationMy Interactive Sessions: shows your personal use of the interactive apps and is another way to open up a new session with one of the apps listed aboveThis site is arguably the biggest trick of FAS RC – it is difficult to find and figure out, but certainly can make your programming life easier!Cohort Docs (Google Site)CohortDocs is a Google Site maintained by the Channing Division of Network Medicine that has information about the Health Professionals Follow Up Study (HPFS), the Nurses Health Studies I, II & III (NHSI/ NHSII/ NHSIII), and the Growing Up Together Study (GUTS)Of note, as of November 2020, GUTS and the Nurses Health Studies are not on FAS RC, just on the Channing Server. This site has detailed information and helpful tips for coding using the HPFS data, and detailed examples for the SAS Analysis Macros that are copied in the Doc folder (see below)To get access to this site, email Kathleen Florentine (kflorentine@hsph.harvard.edu) and ask her to be added to the list of HPFS investigators that needs access to Cohort Docs. You’ll need to send her a gmail email address that you can use for access.Analysis Macros (SAS)Biostatisticians at Harvard/ Channing have created SAS analysis macros to help you perform analyses – the SAS code for each of these is in the “AnalysisMacro” folder in the hpnh directory (/n/hpnh/AnalysisMacro)There is a “Documentation” folder in the AnalysisMacro directory with a PDF that outlines each of the macros – what it does, how to call it in your program, and the parameters it requires.You can incorporate these macros by using the 5 lines of SAS code below. This is also outlined in a file located in the Doc folder (/n/hpnh/HPFS/Doc/hpfs_channing2fas_dir.txt)filename hpstools ‘/n/hpnh/HPFS/ReadMacro’;filename channing '/n/hpnh/AnalysisMacro';libname readfmt '/n/hpnh/HPFS/ReadMacro/formats';options mautosource sasautos=(channing hpstools);options fmtsearch=(readfmt) nofmterr;HPFS Guide in FAS RC FAS RC gives you a default home directory under your username, but you should do all of your HPFS-related work (and potential future other projects/ cohorts) in this hpnh-specific personal directoryDefault home directory path will look something like this: /n/home##/username (i.e. /n/home04/jharvard)Personal hpnh user directory path will look something like this: /n/hpnh/Users/username (i.e. /n/hpnh/Users/jharvard)Run SAS jobs in FAS RCIn your home directory, open .bashrc using any text editor such as nano, please add the following: module load sas/9.4-fasrc01 &> /dev/null alias qs='sbatch -o stdout -p shared --mem=4000 sas' The above need to only do once, and you can type the following to make it work for the first time, but you don’t need to it anymore for next login: source .bashrcTo run SAS, you only need to type: qs *YOUR SAS PROGRAM** (Make sure you are in the directory that has the SAS file in it; otherwise you will have to put the entire path name with the file (i.e. /n/home04/jharvard/folder/test.sas) )You can use squeue to check your job status and scancel to cancel the jobs you just submitted:squeue = after you submit a job, this tells you which jobs are still being processed on the clusterscancel <job number> = cancels the job that you submittedPlease see /n/hpnh/HPFS/Doc/how_to_submit_sas_job.txt for detail. Also use as guide for how to run jobs including SAS ones in FAS RC.HPFS FAS RC directories guideFollow this path to the “Doc” folder: /n/hpnh/HPFS/DocContents of this folder include the following:Directory Structure folder (dir_structure)Each file has a branching structure designed to help you find the particular dataset or program you’re looking forTest SAS programs with corresponding log and output files: Example 1 – shows how to read in formats, Read Macros, and Analysis Macros, and how to call variables from the questionnaires as well as a derived dataset, an endpoint dataset, and the death datasetExample 2 – looks at the association of body weight with all-cause and CVD mortality among survivors of myocardial infarctionInstructions on how to run SAS in FAS (how_to_submit_sas_job.txt)Directory and file mapping between Channing system and FASIn both an Excel spreadsheet and a text file (hpfs_channing2fas_dir)This file has the FAS version of common Channing SAS statements including:Read macrosAnalysis macrosQuestionnaire data pathBiomarker data pathMen’s life validation study (MLVS) data pathEndpoint data pathsHPFS Admin GuideThis section is only for people who have HPFS admin accessIf you aren’t sure if you need this access, please email Ruifeng (rli@hsph.harvard.edu) for clarificationIf you need to create/manipulate/analyze data, it must be done in your directory /n/hpnh/Users/username (instructions for creating this directory are found in step 3)Once the data files are finalized in your directory (/n/hpnh/Users/username), you can then copy the files to an appropriate subdirectory for the disease that you are working on. Before copying the data, please do the following:Contact Ruifeng (rli@hsph.harvard.edu) regarding the correct destination directoryCreate a “readme” file for the first version of the data file, or add comments on the top of the existing “readme” with dates, your name, what changed, and where the original programs (SAS or other) used to create those data files are locatedMake sure the new files won’t be replaced for the new derivationBack-up the data files to be replaced in the archive subdirectoryReminder: there are many other folders under /n/hpnh/HPFS besides the disease that you are working on. Please be very careful and make sure not to change anything in those other folders. For the directory of the disease that you are working on, please also make sure not to change anything except adding (copy and paste) the new files from your directory (under /n/hpnh/Users/username)Users in the HPFS groups Group Name in the FAS RC Portal Grant Page:In FAS RC /n/hpnhWho should be added: Health Professionals User GrouphpnhEveryone with lmucci_lab or Molin_lab or Shuji etc. should belong to this. Health Professionals Air adminhpnh_airShould be for /n/hpnh/ExposureProject/Air, not /n/hpnh/ExposureProjectHealth Professionals HPFS adminhpnh_hpfsOnly a few people who should have ‘w’ for this. For now, we have Bailey and Amparo G. for prostate; Samantha Molsberry for Neural disease; Tomotaka & Haruki for Colorectal tissue; Scott Smith for Colorectal cancer; Yanping Li for Diabetes.Health Professionals Macro adminhpnh_macroOnly a few people who should have ‘w’ for this. For now, only Biling.Health Professionals Nutrition adminhpnh_nutritionOnly a few people who should have ‘w’ for this—Steven Steward controls. For now, we have Lauren D.; Laura K.; Mary F.; Paula T.HPFS directory mapping/flow chart ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download