Lippincott Williams & Wilkins



Supplemental Digital Content. Data Processing Codecleanpagers.sh# This script collects all csv files in the pagers folder, cleans them# with the PagerData script, and appends them to the database file. DATE=$(date +"%Y%m%d")# set variable to be used in folder naming laterfor f in *.csv; do (python PagerData.py "${f}" "${f}".clean); done# iterate csv filescat *.clean > newandclean.csv# concatenate all cleaned filescp /DIR/archive/allpages.csv ./allpages.csvcat allpages.csv newandclean.csv > allnewpages.csv# add new data to copy of dbsort -ud allnewpages.csv -o allnewpages.dedup.csv# sort and dedup dbmkdir /DIR/archive/$DATE# create dir with current datecp allnewpages.dedup.csv /DIR/archive/allpages.csv# replace db with new rm *.clean# clean working filesmv *.csv /DIR/archive/$DATE/# move data downloads to archivePagerData.pyimport csv, sysinputfile = sys.argv[1]outputfile = sys.argv[2]# open csv file hereRawPagerData = open(inputfile, "rU")# create destination csv fileFinalPagerData = outputfile# set-up csv reader, initiate line-counter and variablescsvReader = csv.reader(RawPagerData)n = 0num = "Pager Number"date = "Date"time = "Time"msg = "Text Message Content"#make open listSortedPagerData = []# loop through lines in csv alternating b/w odd and evenfor row in csvReader: n += 1 # if row is odd pull pager number and date stamp if ((n % 2)==1): num = row[0] date = row[4] # if row is even pull time stamp else: time = row[0] msg = row[1] SortedPagerData.append([date, time, num, msg])# print pager data organized by pager number, date and timestampwith open(FinalPagerData, "w") as output: for sublist in SortedPagerData: for item in sublist: output.write(str(item) + ",") output.write("\n") ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download