Sweapinstruments.org



[pic]

The BEAP Process

Instruction Guide

Manual Version 1.6 – 2/17/03

Prepared by: Philip K Ng

Restrictions:

Redistribution and use of this manual in physical and binary formats, with or without modification, are allowed under all conditions. This manual also does not cover the procurement of a valid license for this software. The user of this manual is expected to purchase a valid serial key from the software producer.

Warranties:

This manual is provided by the author "as is" and all expressed or implied warranties are disclaimed. In no event shall the author be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to loss of use, data, profits, or business interruption) arising in any way out of the use of this manual.

Table of Contents

Restrictions …….…………………………………………………………………………Pg 1

Warranties .………………………………………………………………………………Pg 1

Section 1 – Introduction ……………………………………………………………….Pg 3

1.1 – User Expectations and Expected Training ……………………………….Pg 3

1.2 – Applicability ……………………………………………………………….Pg 3

1.3 – Purpose and Usage Description ……………………………………………….Pg 3

1.4 – Conventions ……………………………………………………………….Pg 3

1.5 – Contact ……………………………………………………………………….Pg 3

Section 2 – Instructions and Guide ……………………………………………………….Pg 4

2.1 – Installing X-Win32 ……………………………………………………….Pg 4

2.2 - Using X-Win32 to run GUI programs on a remote UNIX workstation ……….Pg 5

Section 3 – Known Error Messages, Known Problems and Error Recovery ……….Pg 7

3.1 – Cannot Find Display ……………………………………………………….Pg 7

3.2 – X11 Tunneling not enabled ……………………………………………….Pg 8

Section 4 – Bibliography ……...……………………………………………….……….Pg 9

Section 5 – Glossary ……………………………………………………………………….Pg 9

Section 1 - Introduction

1.1 - Experience and Training

This manual was written for the database managers of the BEAP database. This manual assumes that the database managers have knowledge in the following PhilSoft software utilities: BEAP File Converter, BEAP SPSS Automater Utility, BEAP File Mover, and the BEAP MassMailer Utility This manual also assumes that the user is familiar with the use of the Microsoft Windows Operating System. A familiarity with the following software packages is desired but not required: SPSS, Microsoft Outlook, and Internet Explorer.

1.2 - Applicability

This document covers installation and basic use of the software on a computer of the following minimum specifications:

• Intel Pentium 2 MMX running at 200 MHz

• 128 Mb of System RAM memory

• 10 GB Hard Drive

• Windows NT4 or better

In addition, the BEAP process requires facilities to “op-scan” the forms, and return such data as a column delimited data file.

1.3 – Purpose and Usage Description

The PhilSoft software packages were written to assist in the complex data-crunching that was required to process each submission to the BEAP database. This manual was written to guide the user through the conversion of a single submission to the BEAP database. This manual was also written to guide a user through a “rehashing” or reprocessing of the BEAP database.

1.4 - Conventions

Illustrations, with figure numbers and captions immediately below in italics, show the screens before each action. Important text will be highlighted in a bold font. Links, such as email addresses or web addresses will be in dark grey underlined font. Notes and important information will be shown with bullets inset from the text. If a program is used and its name is shortened in the subsequent text, the program will be named and the abbreviation will be placed in capital letters immediately to right of the term.

1.5 – Improvement

The author of this manual welcomes any comments, questions or complaints to further improve this and other documents. You can reach him by sending an email to pkn1587@cs.rit.edu.

Section 2 – Instruction Guide

Section 2.1 – Sending a file to the processing center

This section covers the conversion of one single file through the BEAP process, from start to end.

1) The submission is received in the BEAP processing facility through postal mail.

2) The submission is preprocessed for potential errors. The data on the monthly report (facesheet) is compared to the existing database for errors in reporting. If the program already exists in the database, the program’s contact information is checked first. This includes the program’s contact person and the contact person’s email address. Next, the number of graduates is checked against the existing data. If the number does not correspond with the existing data, the submission should be flagged for data verification.

3) The submission’s forms will be verified against the monthly coversheet. Each form is counted, and checked against the numbers entered on the monthly report. If any of the form counts do not correspond to the monthly coversheet, the submission should be flagged for data verification.

4) The submission is now clear of major processing errors. For each set of forms (consisting of forms of the same month, year and type). The following process must be applied.

a. The forms will be separated if necessary.

b. Place a “post-it” or sticky note on the top of the “front” of the set of forms. On the note the following data will be written: filename, type, form count, and date.

The filename consists of the following:

i. First 4 characters = Proganym (If program is new to BEAP, a unique one needs to be created from its university or college initials. Otherwise use the one previously created.)

ii. Characters 5 & 6 = EN, VN, EX, VX, AL, or EM (depending on the BEAP instrument being submitted) Character 7 = Month indicated on facesheet (1-9=Jan-Sep, R=Oct, S=Nov, T=Dec, A=rescan)

iii. Character 8 = Year indicated on facesheet (0, 1, 2)

iv. Character 9-12 – “.dat”

c. The remainder of the note consists of the ”SCANTOOLS PROGRAM" and "#". These are to be selected from the following choices: (entrance #901, values #902 (Use with either SWVI form), exit #903, alumni #904, employer #905)

See Figure 2 for a sample note.

[pic]

Figure 2 – Sample post-it note. In this sample, RITN is the proganym. This is a submission of Exit data done during May 2003. There were 13 forms in this submission. The “thanks!! –Phil” is not necessary (but is a nice touch).

5) Place in an interoffice envelope, or in a box (8 1/2" x 11" x ?') and address to:

Debi Fitts, ITS Data Center Operations, A-367 Ross

6) Send the envelope or box of submissions via interoffice mail.

7) Enter facesheet data into the BEAP Facesheet Data Excel spreadsheet. For every set of forms submitted in one month (all forms collected in April of 2001 are considered a “set”), a new row must be created in the facesheet spreadsheet. The monthly report data should be entered in the following order:

FULLNAME, PROGANYM, DATE, TODAY’S DATE, PROGID, PROGTYPE, STATE, CSWEREG, BPDREG, MAXGPA, AUSPICES, CANDIDCY, ACCREDIT, NUMGRADS, EN-OUT, EN-BACK, VN-OUT, VN-BACK, EX-OUT, EX-BACK, VX-OUT, VX-BACK, AL-OUT, AL-BACK, EM-OUT, EM-BACK

For all data entered after “NUMGRADS”, the cells which contain data should be highlighted in light green to signify that they are sent to scanning. See Figure 3 for an example.

[pic]

Figure 3 – Facesheet spreadsheet shown with green highlighting. The key is located on the upper left hand corner of the screen, under “color code”.

8) Save Excel spreadsheet and increment version number after an input batch is finished.

9) Repeat steps 2 through 8 to process more data for scanning.

10) Save Excel spreadsheet and increment version number after an input batch is finished.

11) Close the Excel Spreadsheet.

Section 2.2 – Processing of BEAP files returned from data processing center

This section covers the processing of the files received from data processing. After we send the processed and labeled forms to data processing, the data processing center staff will scan the forms and upload the files to the VAXC account. The staff will send us an e-mail, stating that the forms have been scanned, and will return the scanned forms via interoffice mail. After we receive an e-mail from the processing center, we can proceed to proceed to process the files. The process is detailed below.

Part 1 – Fetching scanned data from the data processing center

1) An e-mail from the data processing center is received.

2) Open the telnet program, via Start Menu>Run>>”telnet vaxc.rit.edu”

3) Log onto vaxc.rit.edu as the username “694dept”.

4) Change directory to beap by typing the command “beap”.

5) Create/or access the folder structure in the following structure: year, month, date. This is to keep a backup of the received scanned data. See Figure 4 for an example of this structure.

[pic]

Figure 4 – Naming Structure of the backup folders on the VAX account

6) Enter the command “receive *.*;* *.*;*” to receive all of the files uploaded by the data processing center staff.

7) Then type “ex” command to leave receive mode, and “lo” command to log off the server.

Part 2 – Receiving data from the VAXC server

After receiving the files from the data processing center, they can be transferred using ftp for processing on the BEAP database manager’s machine. The following outlines this process.

1) Run WS-FTP to transfer files to the database processing machine (the machine equipped with the PhilSoft programs listed in the introduction, section 1.1 ).

2) Use ftp to connect to “rit.edu”. The username is 694dept.

3) Navigate to the location which the files were received in section 2.2 part 1 step 5.

4) Using WS-FTP, transfer the scanned files to the incoming scanned data folder of the BEAP database. This is by default “e:\1data\beap\1scanned”.

5) Log off WS-FTP by closing the program.

Part 3 – Data Processing

1) Open the Excel facesheet database.

2) Save the facesheet database as a “facesheetXX.CSV” (where XX is the current version of the facesheet) file so that the PhilSoft conversion utilities can operate with less user input.

3) Close Excel to prevent making changes to the CSV file as opposed to the facesheet database.

4) Reopen the Excel facesheet database (which was opened in step 1). Make sure that the file opened is not the csv file.

5) Open the BEAP Native DAT file converter (CONVERTER). If the “Tell me when there are newer files in the BEAP Facesheet folder” option is selected in the “Start/Facesheet” pane in the “Settings” pane of the CONVERTER, it will inform you if there are newer files in the folder which the CSV is located. See figure 4 for the message that is displayed when there are newer files in the CSV directory.

[pic]Figure 4 – Message displayed when then “Tell me when there are newer files in the BEAP Facesheet folder” option is selected and there are newer files in the CSV’s directory.

6) Change the CSV file location of the converter to match the file that was chosen on step 2 above.

7) Make sure that the settings in the setting panel of the CONVERTER are correct. This includes the correct converted DAT file output file location, database location, scanned file moving location, facesheet CSV file location, etc. See Appendix Section 1 for a detailed explanation of all of the converter’s settings.

8) On the left hand side of the CONVERTER program, locate the DAT files which you wish to convert. After locating the files you wish to convert, click on the desired file to select it. The converter will load the file, and a blue screen will show the expected results from conversion. If the file was named and entered into the facesheet database correctly before sending the file to the data processing center (Section 2.1), the end of each line will be appended with this file’s facesheet data (See figure 5) . If the data that is appended reads “XL DATA HERE >|”, then the CSV file representing the facesheet spreadsheet has not been loaded or the file has either been incorrectly named or entered in the facesheet spreadsheet (See Section 3.2 for possible resolution).

[pic]

Figure 6 – The results of loading a file which has been correctly named and entered in the facesheet spreadsheet. The circle (1) shows the vital statistics of the file, such as the filename and the number of forms it has found in the file. The circle (2) shows the data that it has extracted from the facesheet.(note: the asterisks will become spaces after conversion.)

9) Locate the same entry in the BEAP facesheet. It will be highlighted in green. (See figure 7)

[pic]

Figure 7 – The entry which corresponds with the file to be converted above (with the entry highlighted in green)

10) Press on the “fill box with color” button in Excel, and select yellow. (See figure 8) This indicates that the file has been converted, and is available to you for future reference.

[pic]

Figure 8 – Selection of yellow to mark completion of an entry in the Facesheet database

11) Save the facesheet database.

12) Switch to the converter and press the “Save Conversions” button located near the top center. This will convert the file, append the facesheet data, and perform the functions selected in the “Settings” panel of the BEAP Native DAT File Converter.

13) You may repeat Steps 7-11 to convert more DAT files.

Part 4 – Data Processing

Note: This section only applies if the user has set the converter to automatically create SPSS syntax files. If the user has chosen not to have the SPSS syntax files automatically created, the user must manually create the syntax files before proceeding.

1) Start SPSS.

2) Run the “Automater” (AUTOMATER) program to convert the files to reports and perform the residual data processing tasks. See Appendix Section 2 for a detailed explanation of all of the AUTOMATER’s settings. After running the AUTOMATER program to process the files, there will be several html files in the output directory that was specified in the Automater’s settings.

3) Run the “HTML Purifier, File Mover, and Compressor” (FILEMOVER) program to copy the html files to their respective directories in the BEAP database, move them to the folder structure of the web server, remove HTML tags that cause errors in Netscape browsers, and compress them to save space on the web server. After this process, the output directory that was specified in the AUTOMATER’s settings will contain several directories, each html output file will be filed into a directory named with the first four letters of the filename (the proganym).If the “compress the files to save space on the web server” option was set in the FILEMOVER, the file will be compressed and will have the extension “.htm.gz”.

4) Run WS-FTP to transfer files from the database processing machine to the web server. To upload, first locate the “users” directory in the “www” folder of the BEAP account on the right hand pane. Then in the left hand pane, navigate to the output directory that was set in AUTOMATER, select all of the directories on the left and pane and then press the upload button, or the button labeled “->”.

5) Using WS-FTP, navigate to “cgi-bin” directory in “www” folder of the BEAP account in the right hand pane. Afterwards, navigate to where the CSV file is located that was generated in Section 2.2, Part 3, Step 2, and upload that file to the web server.

6) Run Internet Explorer to navigate to the BEAP website. Login to the BEAP website using the administrator account. On the BEAP administrative site, locate the “Import CSV File as Database Source” section. Enter your administrative password again, and the filename of the CSV file that was generated in Section 2.2, Part 3, Step 2. After entering this data, press the “Press Here to Continue” to load the facesheet’s updated information to the web site. See Figure 5 for an illustration of the BEAP website administration page, and Figure 6 for an illustration of the resulting page generated.

[pic]Figure5 – BEAP administration page showing CSV file importing.

[pic]

Figure 6 – BEAP Administration page showing successful CSV facesheet file loaded on the web server.

7) Run the “Mass Mailer Utility”, and then press the pink “Start” button to send an email message to all of BEAP users who have new files that have just been processed. See Appendix Section 4 for a detailed explanation and illustration of the “Mass Mailer Utility’s” settings.

8) Using WS-FTP, navigate to the “cgi-bin” directory in the “www” directory of the BEAP account, and delete the CSV facesheet file on the web server.

9) Erase all the directories and files in the following directories:

a. The SPSS “ready for processing” folder that was setup in the CONVERTER program (1spssready)

b. The output folder that was set up in the AUTOMATER’s settings (1report)

After completing all of these steps, you have finished processing data.

Section 2.3 – How to Recompile the Database

Because data in the BEAP database is constantly changing, data on the facesheet occasionally changes, and over time, the reports will no longer be consistent with the corresponding entries in the database. Also occasionally, the syntaxes to generate the reports changes and it is appropriate to regenerate the reports. This process of regenerating reports to match the latest changes in the database is known as “recompiling” the database. To perform this operation, please follow the steps below.

1) Start the CONVERTER

2) Make sure that the settings in the setting panel of the CONVERTER are correct. This includes the correct converted DAT file output file location, database location, scanned file moving location, facesheet CSV file location, etc. See Appendix Section 1 for a detailed explanation of all of the converter’s settings.

3) Check to make sure that a valid and current facesheet CSV file is loaded, and that the settings for creating SPSS files is correctly set up. Refer to Appendix Section 2.1, Part 2.3 for detailed instructions and explanations of how to set up the SPSS syntax generation feature.

4) Click on the “Settings / Other” tab of the Converter.

5) Click on the “Recompile” sub-panel of the “Settings/Other” tab.

6) Navigate to the BEAP database folder using the three left hand fields. You only need to go to the higher level folder. See Figure 7 for an illustration.

[pic]

Figure 7 – The converter after navigating to the BEAP database folder. Note that there are no files listed in the lower file list box.

7) Press the “Start Auto-Recompile” button. This will process every subdirectory in the BEAP database and will convert every raw data file according to the settings of the converter. If an error is encountered during the conversion process, it will be ignored and it will continue with the next file.

8) Perform the “Data Processing” steps found in Section 2.2 Part 4.

Section 2.4 – How to Generate Total School and Data files

1) Start the “BEAP Totalizer Utility.” (TOTALIZER)

2) Ensure that the settings are correct. For a detailed explanation and illustration of these settings, please refer to Appendix Section 4

3) Press the pink “Start” button.

4) Perform the “Data Processing” steps found in Section 2.2 Part 4.

Section 3 – Known Error Messages, Known Problems and Error Recovery

Section 3.1 – Data Verification Process

This section describes the process which must occur if there are errors on the monthly coversheet which was provided to us. This process should occur well before processing (splitting and labeling) of the forms described in section 2.1.

A facesheet is considered “needs data verification” or “bugged” if the facesheet is:

• The facesheet is missing entirely.

• Number of graduates reported to CSWE is missing.

• Number of graduates reported to CSWE in that time frame does not correspond to the number reported in previous submissions for that time frame. All submissions between November of one year and November of the following year is considered in the same time frame. If the user submits a facesheet in November 2000 with the number of graduates reported in November of 2000 as being 10, and in December 2000 submits another set of data with the number of graduates reported to CSWE in November 2000 as 15, then the data is requires verification.

• Program data such as maximum GPA attainable, program type, or program auspices is missing and cannot be ascertained from previous submissions in the database.

For each of the situations listed above, the problem can be solved by sending an email or by calling the program in need of data verification, and asking for the data required. If the facesheet it missing, the program should be asked to fill one out and have it faxed or mailed to us.

Section 3.2 – ‘XL DATA HERE > |’ is appended to the end of each line of a data file after selecting a file in the BEAP Native DAT file Converter.

This section describes the process which must occur if the BEAP Native DAT File Converter cannot find the appropriate facesheet data string to append to the end of each file. Such errors would occur in Section 2.2, Part 3, Item 7.

Possible solutions are listed below.

• The facesheet has not been set or loaded. Set the facesheet CSV location in the “Start/Facesheet” pane in the “Settings” pane of the DAT File Converter.

• Facesheet lookup has been disabled in the settings screen. Enable it in the “Start/Facesheet” pane in the “Settings” pane of the DAT File Converter, or manually copy the data from the Facesheet spreadsheet and follow the screen prompts to complete manual conversion.

• The facesheet CSV file is an older version, and this file’s data is not in it. Close the Converter and follow Section 2.2 Part 3 to restart the conversion process.

• The file was not correctly named, or was not correctly entered in the facesheet spreadsheet. Check the spreadsheet and filename against the facesheet that was received and adjust accordingly.

To restart the conversion process on this file, one can follow the steps detailed in Section 2.2 Part 3 or use the “refresh” button located on the lower left corner of the converter, or press the “Apply and Reload Facesheet” button located in the “Start/Facesheet” pane in the “Settings” pane of the Converter.

Section 4 – Appendices

Section 1 – Converter’s Settings

Part 1: - Startup / Welcome / Common Screen

[pic]

1) This selects what drive volume the current file listing (2) will show.

2) This selects what files the current directory (3) will show.

3) This selects what file to open. As soon as one clicks on a file in this directory, the converter will open the file that is selected, and attempt to convert it. If it is successful, then the conversion will be shown in the “Expected Conversion” tab (9), and the original file will be displayed in the “Original File” tab (8).

4) This is the Save Conversions button. If a file is selected, and there is data in the “Expected Conversion” tab (9), and the facesheet data has been looked up, (assuming the option is chosen in the “Settings” tab (Part 2.1, Item 3 - 5), then the file will be converted and saved to the settings in the “Output Settings” tab (Part 2.2 all). If the facesheet is not loaded, or the file does not match the facesheet’s data, or facesheet lookup is disabled, a box will appear, requesting the Excel spreadsheet data. To manually look up the facesheet data, find the line in the spreadsheet that corresponds to the file, and copy the fields “PROGID” through to XX-BACK, where XX is the file type. Place this copied data in the input box, and remove the items which are after the number of graduates and before the last two items separated by the “|” character. The resulting string end with ”XXX|XXX|XXX” where X is a number. See Figure A1 for an illustration of this.

[pic]

Figure A1 – finished Excel spreadsheet data that is correctly entered.

5) This area displays file information such as the number of forms in the file as well as its name. This will always remain on the top after resizing the screen.

6) This button refreshes the file listing box (3).

7) This button renames a file selected in the file listing box. To use this feature,

a. Select a file in the file listing box (3).

b. Press this button.

c. An input box will appear asking for the desired resulting name.

i. If the file cannot be renamed or no file is selected, an error message will appear.

8) This is the panel that the original contents of an unconverted file will be shown. (Note that this is a read-only text panel, and changing it has no effect on the resulting file.)

9) This is the panel shows the expected conversion contents of an unconverted file. This conversion process concatenates forms with the same ID numbers, stripping out the form numbers. This data is then removed of all asterisks, and if the option is set, the excel spreadsheet data is appended on the end of each line. (Note: that this is a read-only text panel, and changing it has no effect on the resulting file.)

10) This panel shows the results of the file conversion. The asterisk stripping operation is performed again, if the Excel spreadsheet data contains asterisks. (Note: that this is a read-only text panel, and changing it has no effect on the resulting file.)

11) This is the settings panel, and is described in detail in part 2.

Part 2.1 – Settings Sub Panel

[pic]

1) Our mascot.

2) If you enable this option, the converter will ignore all of the settings that you have set in the BEAP Facesheet Settings frame (3, 4, 5), and will ask you for the excel spreadsheet string that you would like to append to the end of each line in every file. This setting will is not remembered and will be reset each time the program is loaded.

3) This is the facesheet CSV location. After a CSV file is loaded or reloaded, the read only text field immediately below the filename will display the last modified date of the file. If the CSV file cannot be loaded or the file is not specified, the information field will display “CSV file not found or cannot be loaded.” To select the CSV file using a graphical method, press the “..” button to display a Windows file selection box. Please note that this setting is common among all of the BEAP processing programs and changing it will affect them.

4) This button is used to apply the changes to the CSV filename field, (re)load the CSV file, and save the settings to the registry.

5) This option, if selected, this program, on startup, will inspect the directory where the CSV file specified, and determine if there are any files that have been updated more recently then the timestamp that is on the CSV file. If there are such files, it will display a message for each file. (This setting will be applied and saved as soon as it is changed.)

6) This field displays the directory that the converter will start up in default. This folder will be loaded in the “drive selection”, “folder selection list” and “file display list” (Appendix Section 1, Part 1, items 1, 2, 3.)

Part 2.2 – Output Management Sub Panel

[pic]

1) There are two major output modes that this converter is in. The first mode that the converter will be in is the “output to a folder” mode. The converter is in this mode if the green button that the label (1) is on shows “To Database” and the lower 3 text fields are grayed out.

In “To a Folder” mode, the converter will:

a. Convert the files, append the ending “–c.dat” to the file and save them in the directory that is specified in the “Data Output To.” The original files will be left alone.

i. If the field labeled “Data Output To” is empty, the files will be saved in the directory that this application is stored.

ii. If the folder does not exist, an error message will be displayed.

iii. If you wish to graphically select this directory, you can use the “..” button to display a Windows Folder Select window.

In “To Database” mode, the converter will:

a. Convert the file, append the ending “-c.dat” to the end of the file and save the converted files to the location specified in the “Converted File To:” field. It will also by default, move the original scanned file to the location specified by the “Move Originals To:” field (where @Proganym@ refers to the first four letters of the file).

i. If the locations specified by either of the “Converted File To:” or “Move Originals To:” field do not exist, the respective directories will be created. If they cannot be created, the converter will display an error message and exit without converting the file selected.

a. To setup the “To Database” mode correctly, after pressing the “To Database” button, a wizard will appear to assist you by asking several questions. These folders will be added to the “Folder Creation” section (Appendix Section 1, Part 2.7)

2) If this option is set, the unconverted scanned data file will be copied to the “Move Originals To” field as opposed to moving it. The file will remain after converting it. Note that this setting has no effect unless the converter is in the “To Database” mode. (This setting will be applied and saved as soon as it is changed.)

3) No settings will be applied or saved unless this button is pressed.

Part 2.3 – Syntax Creation Sub Panel

[pic]

1) Our November 2003 Reno Conference mascot.

2) This is the directory where the converter will search for SPSS template files.

a. If this field is empty, the converter will try to look for these files in the directory that the program’s files are located in.

b. The template directory will be in this particular format:

i. There will be a file named “id.ini” with a line for each SPSS syntax file, followed by a comma and then a tag. The tags that are understood are: en4, vn4, ex4, vx4, al4, and em4. The tags en, vn, ex, vx, al, and em4 are read, and will display the following error message: “the file [filename] is not a CSWE 4 digit file.” All other tags are ignored. An example is: entrance4.sps,en4

ii. If the “id.ini” file cannot be found then syntax creation will not occur. An error message will appear indicating this.

iii. For every SPSS syntax file, the converter will convert the following strings into their equivalents:

a. %%@PATH@%% - the location where the converted file is saved. This is specified in Appendix, Section 1, Part 2.2.1.

b. %%@PROGANYM@%% - the first four letters of the original scanned data file.

c. %%@DATE@%% - the last two letters of the original scanned data file.

3) This is the directory where the converter will output SPSS syntaxes to, if it has correctly loaded the template syntax files. Each file converted will have its own SPSS syntax file generated for it.

Part 2.4 – Mouse Macro Sub Panel

[pic]

1) This checkbox shows whether or not mouse macro mode is enabled. See below for instructions on how to use “Mouse Macro Mode” (This setting will be applied and saved as soon as it is changed.)

2) This button sets the location of the “Make Selection Yellow” button of Excel. (This setting will be applied and saved as soon as it is changed.)

3) This button takes control of your mouse and moves it to where the previous setting for “Mouse Macro Mode” is. (This setting will be applied and saved as soon as it is changed.)

The converter has a special macro mode which allows the user to quickly convert files without having to constantly switch between Microsoft Excel and the converter. To set this feature up, the user must follow the following steps.

1) Open the converter and place it near the bottom of the screen.

2) Open the Excel Spreadsheet and move the window to the top portion of the screen. Allow enough room to fit the converter below it. See the Figure A2.4 for an example.

[pic]

Figure A2.4 – Correct Screen layout for Mouse Macro Mode.

3) Enable the mouse macro mode by checking off the “Enable Mouse Macro Mode”(1) checkbox.

4) Press the “Train Mouse Mode” button (2). A message box will appear. After clicking “Ok”, switch over to Excel and hover over the “Make Selection Yellow” ([pic]) button.

5) After 5 seconds, the converter will report that it has remembered the position.

To use mouse macro mode during Data Processing Mode (Section 2, Part3),

0) In the file selection box (Appendix, Section 1, Part 1, item 3), select the first file at the top of the list.

Perform the following steps instead of steps 9 and 10 in Section 2 Part 3:

9) Switch to Excel (which should be in the same arrangement as Figure A2.4), and find the entry which represents the file you have selected.

10) Press the “Shift” Key on your keyboard. The converter will convert the file normally. The converter will perform steps 1-8 for you.

11) The converter will have automatically loaded the next file for conversion.

12) Repeat 9 – 11 as required.

Part 2.5 – Log Sub-Panel

[pic]

1) This shows the log of every action performed by the converter. This log is appended to a file named “log.txt” when the program exits in the directory where this program is located. Note that changes to this log will be saved to the log file.

Part 2.6 – Recompile Sub Panel

[pic]

1) This will start the recompile process. See Section 2.3 for details.

Part 2.7 - Folder Creation Sub Panel

[pic]

The original converter required a companion program called the “Folder Creator” to create the folder structure required for SPSS and the FileMover to function correctly. As of version 5.01 of the converter, the program contains the capability to create the folder structure by itself.

1) This screen shows the folders that will be created when a new proganym is created and the first files from that school have been returned to the BEAP processing center. This includes both of the folders in “Output Management” tab (Appendix 4, Section 1, Part 2.2)

a. Note: If you change the folder names in “output management”, the old folders will be renamed here, and any subsequent conversions will use the new settings. The folder name change will not be applied to the rest of the database until the next recompile.

2) This loads the default folder creation list. This list is composed of the two folders listed in “Output Management,” and the following: “total”, “reports”, “saved-data”, and “schoolmaster-data.” (In the example above, the two folders in the “Output Management” tab were “scanned” and “raw-data.”

3) This button will add a new folder to the list on the left. A message box will appear asking you for the name of the folder to be added.

a. If the folder is already present on the list, an error message will appear and the item will not be added.

4) This button removes an item from the list. First select an item and then press this button.

a. If no item is selected an error message will appear and nothing will happen.

b. If the item is one of the folders listed in the “Output Management” tab, an error message will appear and nothing will happen.

c. If it is not one of the folders listed in the “Output Management” tab, a confirmation box will appear.

i. If yes is selected then the folder will be removed from the list.

ii. This folder will no longer be created when a new proganym’s files are converted for the first time.

iii. This folder will not be erased from the database. Existing folders in the database of this name will be left as is.

5) No changes to this list will be applied to the settings until the “apply” button is pressed.

Section 2 – Automater’s Settings

Part 1 – General Layout

[pic]

1) This button will hide or show the control panel of this program (shown in Part 2.)

2) This meter indicates the current CPU usage. Please note that this bar will only be displayed if the system is in CPU monitoring mode. If it is in timed / legacy mode, then nothing will be displayed here.

3) Pressing this button will start the automation process. After pressing this button, a message box will appear requesting that you start SPSS and then switch to it. It will first check all of the files in the SPSS syntax input directory, and if an output file exists with that name in the output directory, it will not automate that file. It will convert the SPSS files found in the directory specified by the settings panel (Appendix Section 2, Part 2, item 1), and save them in a directory specified in the settings panel (Appendix Section 2, Part 2, item 2). After issuing a set of commands, the automater checks to make sure that there exists a file in the output directory. If it does not exist, a message will be displayed and the process will stop.

Part 2 – Input/Output and Optimization Settings

[pic]

1) This is where this program will start up in and hunt for SPSS files to automate. The “..” button may be used to browse for this folder.

2) This is the directory where the automater will tell SPSS to export the reports. The “..” button may be used to browse for this folder.

Legacy / Speed Constant / Timed Operation Mode

Please note that this section is kept for historical purposes.

[pic]

1) This is the speed that you want to automater to run at to process normal non-total SPSS files. If this is set too high, the time it takes to process a file may be prohibitively high, while if it is set too low, some files may not finish and an error will occur. To calculate the correct timing numbers, use the table below. Please note that if the system is in CPU monitoring mode, this box will be disabled.

2) This is the speed that the automater uses to process total files. Because total files contain more data, this number is generally slightly larger than normal non-total SPSS files. See table below for how this timing data is used. Please note that if the system is in CPU monitoring mode, this box will be disabled.

3) No savings will be applied unless the “apply settings” button is pressed.

How the Automater computes times for processing files:

File Type

------------------------------------------------------

"VN", "VX" wait multiplier = 0.29

"EN" wait multiplier = 0.32

"EM" wait multiplier = 0.20

"AL" wait multiplier = 1.00

"EX" wait multiplier = 0.75

Optimization Coefficient= If processing a non-total file, use the number in field (3), otherwise use (4).

The time between issuing the “open syntax” command and sending the “run all” command is: Optimization Coefficient X Wait multiplier X 3 seconds

The time between issuing the command to “run all” and “save output” is: Optimization Coefficient X Wait multiplier X 60 seconds.

The time between issuing the “save output” command and closing the file is: Optimization Coefficient X Wait multiplier X 15 seconds.

All waits between all other commands is exactly .7 of a second.

All typing will occur at 65 words per minute with .125 second delay between simulated “key presses”.

CPU (Machine) Monitoring Mode

The timed execution model of this automater was time inefficient at best. To ensure that all files were executed and finished, the worst case scenario was used to calculate the coefficients in the process. If something unexpected, such as a virus scan were to occur, the process would stall and would have to be manually restarted. To solve this problem, as of version 1.04, this program was given the ability to measure the CPU’s current processing load and depending on the processor’s load, would issue commands (the theory being that if the CPU was not busy, then the last command was completed.) This section goes into depth about this mode. Please note that when the utility is in “CPU monitoring mode”, all “legacy / timed” settings are ignored and disabled.

[pic]

1) Checking this checkbox will enable CPU monitoring mode. After checking this option, the box indicated by the number two will be enabled, and all settings in the “Speed Constants” section of this program will be disabled. Please note that the CPU meter will appear and will actively monitor and display the current CPU load.

2) This box will specify the number of seconds the program will wait after the CPU load drops below 15% to issue the next command. Testing performed in the summer of 2004 indicates that 2 seconds will be all that required 99% of the time.

3) No savings will be applied unless the “apply settings” button is pressed.

Error Detection and Resumption Settings

Earlier versions of this program only measured one failure scenario, which was whether or not the output file existed after the process was complete. As of version 1.04, the program has an additional error checking ability; that of detecting whether or not the output was complete.

[pic]

1) Checking this box will enable the “Check output file for presence of this text” capability. Enabling this feature will also enable the field indicated by number two.

2) This field indicates what text this program will look for to be present in the output report. If this text is not present, then you will be notified of this at the end of the processing session.

3) As of version 1.04, this feature is not complete. This enables this utility to restart the process by ending the SPSS program under the following conditions:

a. Low memory – SPSS version 11.5 has a bug which consumes 40-50KB of memory for each report processed. Although this is rather insignificant, the summation of 2500 of such reports consumes a lot of system memory. If the system reaches such a point, then the SPSS program will be terminated and the system will attempt to resume from the last operation.

b. “Check output file text” – the text specified in number two is not found in the output file. The program will be restarted and the last output is tried again.

c. No output file was found – if the last file processed did not generate output, then the program will restart SPSS and try again.

If the process fails again, the program will go on to the next file. If this process fails twice, then this program will abort and report failure.

4) This specifies the SPSS process classname to terminate. Please note that this functionality was placed here for testing purposes.

5) This specifies where your SPSS program resides.

6) No changes will be applied unless the “Apply Changes” button is pressed.

Section 3 – FileMover Settings

Part 1 – General Layout

[pic]

1) This starts the filemover process, moving, compressing, and cleaning using the settings in the setting panel (described in part 2)

2) Pressing this button will hide the settings panel if it is currently visible, or show it if it is hidden. This setting will be remembered and applied the next time the file mover is run.

3) This is a status panel that displays what the program is currently doing.

Part 2.1 – Introduction and Startup Folder Sub Panel

[pic]

1) This is where the file mover will search for html files to process.

a. If this field is empty, the program assumes that the directory to hunt for html files is the directory where this program is located.

2) The settings for the startup directory will not be applied unless this button is pressed.

Part 2.2 – SPSS Cleanup Sub Panel

[pic]

1) This checkbox indicates whether or not the file mover will strip out the excess html code that SPSS generates. These codes are not required for proper display of the reports. These codes consist of: resetting the font to “Times New Roman” and background color of the report to “white” on every other line. Clicking this option will result in a 40-50% reduction in the size of the report file. This also fixes a rendering error that occurs in Version 4 of Netscape/Mozilla browsers. (This setting will be applied and saved as soon as it is changed.)

2) By default, SPSS generates reports with filenames in uppercase letters. Checking this option will rename the files to all lowercase letters. (This setting will be applied and saved as soon as it is changed.)

Part 2.3 – File Move Options Sub Panel

[pic]

1) Enabling this option will allow the filemover to make a copy each report found in the startup directory ( Appendix, Section 3, Part 1, item 1), and copy it to the “reports” directory of each proganym in the database. If a file of the same name is found in the output directory, it will be overwritten with the newer file. (This setting will be applied and saved as soon as it is changed.)

2) This field specifies where the BEAP database is located. To graphically select the directory you can use the “..” button. Note: As soon as this field is changed, the test settings field (4) is changed to reflect the changes.

3) This field specifies in the proganym’s folder in the BEAP database, where the reports are located. Note: As soon as this field is changed, the test settings field (4) is changed to reflect the changes.

4) For the file “alvc entrance May 2003”, this is where it would be filed according to the settings in the top two fields.

5) If this checkbox is checked, the reports will not be moved into folders named with the first four letters of the reports (i.e. “alvc entrance May 2003” will not be moved into a folder named “alvc” in the directory where the reports are found. (This setting will be applied and saved as soon as it is changed.)

6) None of the settings on this page will be saved or applied unless this button is pressed.

Part 2.4 – Compression Sub Panel

[pic]

1) Even though the filemover can remove the useless html code in report, many reports still end up being 200KB or more. Even though this is not a large file, thousands of these reports take up hundreds of megabytes of storage. Enabling this option will compress the files with an algorithm called “gzip” (GNU zip.) The BEAP website as of version 2.71 can seamlessly decompress these files, without user interaction. This will reduce most html files from 200KB to around 7kB. Please note that a program called “Winzip” () can open the files compressed by this program. (This setting will be applied and saved as soon as it is changed.)

Section 3 – Mass Mailer Settings

Part 1 – Introduction / Tag Text Replacement

For every text field in this program, special tags may be used. A list of these tags will be displayed below. The mass mailer will search the CSV file for proganyms that match the first four letters of the file, and lines that have an email address will be loaded. The data that is associated with a tag correspond to the field on those loaded lines.

|Tag |Meaning |

|%%SchoolName%% |The mass mailer will replace this with data that was present in the “Fullname” (a) row of the facesheet.|

|%%Proganym%% |The mass mailer will replace this with data that was present in the “proganym” (b) row of the facesheet.|

|%%Contact%% |The mass mailer will replace this with data that was present in the “contact” (d) row of the facesheet. |

|%%UserId%% |The mass mailer will replace this with data that was present in the “userid” (e) row of the facesheet. |

|%%Password%% |The mass mailer will replace this with data that was present in the “p-word” (f) row of the facesheet. |

|%%SchoolPw%% |The mass mailer will replace this with data that was present in the “schoolpw” (g) row of the facesheet.|

|%%Email%% |The mass mailer will replace this with data that was present in the “e-mail” (h) row of the facesheet. |

|%%Date%% |The mass mailer will replace this with today’s date, formatted |

| |DD-MMM-YY |

|%%Files%% |This enumerates all of the files found for a particular proganym. |

Note: This program uses the raw row data from the CSV file as opposed to using the header row data.

Note: If more than one entry was found for a proganym, an email will be sent to the email address of each entry.

Note: The following fields do not use the tag replacement feature for obvious reasons:

“BEAP Facesheet CSV Location” field in the “Settings” tab (Appendix Sect. 4 Part 4, Item 1)

“Mail Server” field in the “Settings” tab (Appendix Sect. 4 Part 4, Item 3)

“Startup Directory” in the “Settings” tab (Appendix Sect. 4 Part 4, Item 4)

“My Email Address” field in the “Settings” Tab (Appendix Sect. 4 Part 4, Item 5)

“Proganym” field in the “Special” tab (Appendix Sect. 4 Part 5, Item 1)

Note: It is possible, but not recommended, to place two tags immediately after each other without anything separating them.

Part 2 – Standard Report Email Settings

[pic]

1) This is a status display. It will display the current operation of the mass mailer.

2) Pressing this button will commence the emailing process. The process is as follows:

a. The massmailer loads the facesheet CSV file, only considering lines that contain the email address.

b. The massmailer will search all the subdirectories of the directory specified in the “Startup Directory” located in the “Settings” tab (Appendix, Section 4, Part 4, Item 4) for html files.

c. It assumes that the first four letters of each file represent the proganym.

d. It groups all of the html files by the assumed proganym

e. For every html file found it

i. It looks through the loaded lines of the facesheet CSV file for lines that match the proganym of the html files.

ii. It sends an email each of the email addresses that match the proganym of the html files, replacing the tags in all of the email messages. It also sends a log email to the specified address.

1. If no contact information can be located for the file an error message will appear.

2. If the massmailer encounters an error sending an email, it will retry twice and then display an error message if it has failed twice.

iii. It resets the massmailer to the original state

iv. Repeats steps 2.e.i through 2.e.iv as needed.

3) This field represents to what email address the mass mailer will mail the report email to. It is by default set to “%%EMAIL%%” (see part 1 for a description of what these tags mean)

4) This field represents the subject that the report email will receive.

5) This field represents the message body that the report email will receive.

6) Pressing this button will save the email message. Note that any changes to any of the fields on this tab are applied immediately, but are not saved for future use, unless this button is pressed.

7) Pressing this button will return the email message to original pre-programmed canned message.

Part 3 – Mass (Broadcast) Email Settings

[pic]

Mass Email is similar to Report Email, except that when one presses the Mass Email button, the following happens:

a. The massmailer loads the facesheet CSV file, only considering lines that contain the email address.

b. For every email address found in the spreadsheet,

i. It sends an email each of the email addresses that match the proganym of the html files, replacing the tags in all of the email messages. Please note that no log email is sent.

1. If the massmailer encounters an error sending an email, it will retry twice and then display an error message if it has failed twice.

ii. It resets the massmailer to the original state

c. Repeats steps a through b as needed.

1) This field represents to what email address the mass mailer will mail the report email to. It is by default set to “%%EMAIL%%” (see part 1 for a description of what these tags mean)

2) This field represents the subject that the report email will receive.

3) This field represents the message body that the report email will receive.

4) Pressing this button will save the email message. Note that any changes to any of the fields on this tab are applied immediately, but are not saved for future use, unless this button is pressed.

5) Pressing this button will return the email message to original pre-programmed canned message.

Part 4 – Settings Panel

[pic]

1) This field represents what facesheet CSV file this program will use. To graphically select the CSV file, you can use the “..” button. Note: This field is shared with the converter. Changing this field will affect both of these programs.

2) If you wish to reload this CSV file, you can press this button.

3) This field is the SMTP mail server that this program will send emails to.

4) This field represents where this program will search for reports. To graphically select the directory, you can press the “..” button. If this field is blank, then the mass mailer will assume that the directory that the mass mailer is located in is the directory to search.

5) This field represents the your email address. Please note the recipients of the emails will display this email address as the sender.

6) This field represents the subject of the email that will be sent as a log for every report email sent.

7) This field represents the message that will be sent as a log.

8) This button saves fields 5, 6, and 7. Note that changes are applied as soon as one of those fields are changed, but are not remembered for the next time the program is run unless this button is saved.

9) This button resets fields 5, 6, and 7 to their original pre-programmed state.

10) This button saves the fields 1, 3, and 4, and applies their changes to the program.

Part 5 – Special/Other Panel

[pic]

1) In the occasion that a BEAP user has lost the email with his password information, this program will send out contact information for all the contact people related to that proganym. To use this feature, follow the steps below.

a. Type the proganym that you wish to have contact information to

b. Press “Send”

i. If the proganym that you have typed does not exist in the facesheet CSV file, an error message will appear.

ii. Otherwise a report e-mail will be sent out, to the settings in the “Report E-Mail” tab, and a log message will be sent.

iii. If an error occurs in sending the email message, the mass mailer will retry twice, and then produce an error message.

2) If this checkbox is checked, no emails will be sent out. This is used to test whether or not the massmailer is correctly generating email messages. Note that this setting will not be saved and is reset to “unchecked” every time the mass mailer is run.

3) This is a list of the tags, for reference. Please refer to Appendix, Section 4, Part 1 for more information regarding tag replacement.

Section 4 – Totalizer Settings

Part 1 – All Settings / General Layout

[pic]

1) Pressing this button will start the totalizer process, which is detailed below.

a. Load the template SPSS files in the directory specified in item (7).

b. Create a folder in the directory where the totalizer is stored named “temp”.

c. Erase all of the files in the “temp” folder

d. Erase all of the files in the directory specified in item (5)

e. Search all the subdirectories of the BEAP database folder (3) for files matching the following pattern: “????????-c.dat.”

f. For each file that matches that pattern,

i. Open the file, read all of the data

ii. Append the data to a file with a filename consisting of the proganym and file type, with a “.dat” appended to the end to the “temp” folder. (i.e. alvcen83-c.dat is appended to alvcen.dat.)

iii. Append the data to a data file named “total_[file type]”.dat in a folder specified in item (5).

g. Search the “temp” folder for files with filenames 10 letters long that end with “.dat.”

h. For each file that matches that pattern,

i. Generate an SPSS syntax file if the option is chosen, and save it to the folder specified in item (7).

ii. Move it to the folder specified (4), filing it in that directory in each proganym’s folder in the BEAP database.

i. After all of this is complete, the program will close itself.

2) Pressing this button will minimize the settings panel if the settings panel is shown, and restore the settings panel if it is minimized. This setting is remembered between sessions (i.e. if the settings panel was hidden, it will be hidden the next time the program is run.)

3) This is the field that represents the BEAP database’s location. This is where the totalizer initially searches for files when the start button is pressed. If this field is empty, the totalizer will assume that the database is located in the directory that the totalizer is located.

4) This is where the totalizer will copy the totalized files to. The first half of this field specifies where the BEAP database is, the second part is what folder in the proganym’s folder the totalizer copies to. If the left field is empty, the totalizer will assume that the database is located in the directory that the totalizer is located. If the right field is empty, the total data files will be saved to the proganym’s directory as opposed to a directory within the proganym’s directory.

5) This field specifies where the total files for the entire database will be stored. In that directory, the following folders will be created: entrance, values@entrance, exit, values@exit, alumni, employer, and the total database’s data files will be stored there. If the field is empty, the totalizer will output to the directory where the totalizer is located.

6) The checkmark to the left enables syntax generation. If the option is not selected, the text field of where to output the SPSS syntaxes is not shown. The text field specifies what directory the totalizer will output syntax files to. If the field is empty, the totalizer will generate syntaxes to the directory where the totalizer is located.

7) This specifies what directory the totalizer will attempt to read template files from. This follows the same conventions as the template syntax directory in the converter. See Appendix, Section 1, Part 2.3, Item 2 for a description of this structure. When the totalizer is run, this directory is read following the guidelines that the converter uses.

8) None of the settings on the converter will be applied or saved unless this button is pressed.

9) Section 5 – BEAP Administrative System Settings

Part 1 – Introduction

The BEAP website is the online portion of the BEAP program. It not only supplies information about the program, it also provides file access for each school in the program. The administrative setup and control is described in this section. To access administration page, log into the BEAP website using your administrative login.

Part 2 – BEAP Administrative System Main Screen

[pic]

After Logging into the BEAP website using the administrative password, the following screen will appear. The following options are available to the administrator.

1) This allows the administrator to view the access log of the server. If any failures or security threats are noticed, this log will contain a transcript of it.

2) This allows the administrator to update the access list on the server. This requires a pre-existing CSV file representing the facesheet on the server already. To update the server, enter the administrative password in the topmost field, and the path to the filename on the lower field. If the password or CSV file is not present, an error message will appear. If the operation was successful, then a list of the current passwords will be generated (See Part 4). Note that this listing of all of the current passwords on the server is generated only once, and cannot be retrieved after the window is closed.

3) This field specifies where the CSV file is located. See item 2.

4) This is the submit button. After pressing it, the website will process the CSV file specified. See item 2.

5) This is the field specifying the news that the administrator wishes to add to the site. HTML code is allowed in the input, but bear in mind that the news will appear in the left-hand navigation bar of every page, and this bar is only 127 pixels wide. If the code “dellast” is inputted, the last entry in the news bar will be removed as soon as the “submit” button is pressed. If the code “clear” is entered, all of the news will be removed as soon as the “submit” button is pressed.

Part 3 – System Log Screen

[pic]

After the link to “view log file” is clicked, this page will appear.

1)This shows the system log. If a crash occurred, a detailed error message will show up here.

Part 4 – BEAP CSV File Read Successful.

[pic]

After a successful CSV file read the following screen will appear. The top of the file will show any warnings that occurred during processing. The first thing after the warnings is the timestamp and name of the CSV file read. For every entry in the facesheet, a short summary will be displayed of the BEAP user. At the end of this list shows any failures that might have occurred. If the password is preceded by an underscore (i.e. “_password”,) the entry of that user will be removed from the database.

Part 5 – BEAP New Changed Screen

[pic]

After news has been submitted, the website will show you a list of the files that it has changed, as well as the latest news on the left navigation bar.

Part 6 – Folder Listing Window Description

[pic]

This shows a standard folder listing view. When a BEAP user logs on to the BEAP website, this window is shown, displaying

Part 7 – File View Popup Window

[pic]

Section 6 – BEAP Database File Checker And Verifier Settings

Part 1 – General Layout and Settings

[pic]

Section 7 – Data File Generator Settings

Part 1 – General Layout / Startup and Output Settings

[pic]

Part 2.1 – Strip Settings - Introduction

[pic]

Part 2.2 – Strip Settings – A Form’s Settings

[pic]

Part 3 – Log Tab

[pic]

Part 4 – Total Tab

[pic]

Section 7 – Default BEAP Folder Structure and Setup

Section 8 – Default Web Site Structure and Setup

Section 5 - Glossary

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download