Batch Processing with SAS®: Beyond Running Programs Overnight

[Pages:2]Batch Processing with SAS?: Beyond Running Programs Overnight Lisa Sanbonmatsu, Consultant, Somerville, MA

ABSTRACT

The usefulness of batch processing goes well beyond being able to submit several SAS? programs all at once. Batch processing allows you to automate steps involving several different software applications. This streamlines the process, documents it, and reduces the chance of errors. This paper discusses the use of DOS batch files, the SAS X command, and the SAS Call System routine. Tips for avoiding quoting problems are provided.

INTRODUCTION

Many of us are aware that SAS programs can be submitted in batch mode and left running over night. However, the usefulness of batch processing goes beyond this convenience. By automating processes that cross several different applications, batching allows you to streamline, document and reduce the likelihood of error.

Consider the following situation. Each month your company needs to create a report from raw data through the following steps: 1) Use SAS to read and summarize the raw data, 2) Use a conversion software application to convert the summary SAS data set to a spreadsheet, 3) Use a zip program to compress and archive the raw data, 4) Use DOS to delete the unzipped raw data after it has been archived.

Although each step could be implemented separately, this has several disadvantages. First, it could be time consuming to monitor when each application is finished so you can start the next. Second, when you need to run the report again, you will have no documentation of what you did. And third, manually executing the different programs increases the likelihood of errors due to skipping a step or specifying a wrong parameter.

Batch processing avoids these problems. Batch processing can be achieved by writing a DOS batch file, using SAS's X Command, or using SAS's Call System routine. (Although this paper focuses on PC SAS, the basic principles of batch processing can be applied to UNIX.)

DOS BATCH FILES

DOS batch files allow you to call and execute different programs sequentially. DOS batch files are easy to write and execute. The steps are: 1. Create a text file with the extension ".bat" 2. In this text file, list each command you want to execute followed by a . Note that commands can be a) DOS prompt commands such as "cd" or "del" or b) calls of other applications. 3. Run the file as you would any program (i.e., from the Run Menu, Explorer, or from the DOS prompt).

To produce the monthly report described in the introduction, we could create a text file called "august.bat" which contains the following code:

echo "Step 1-Call SAS to read in & summarize data" call "c:\Program Files\SAS\sas.exe"

"c:\example\readsum08.sas" echo "Step 2-Call transfer software to convert SAS

summary to a spreadsheet" call st "c:\Example\sum08.sd2"

"c:\Example\report08.xls" echo "Step 3-Call zip program to compress & archive

the raw data" call "c:\Program Files\WinZip80\Wzzip.exe"

"c:\example\My Archive\archive08.zip" "c:\Example\data08.dat"

echo "Step 4-Delete raw data after it has been zipped" del "c:\Example\data08.dat"

Placing "call" before an application command, rather than just typing the command, instructs the program to finish executing the other application before continuing with the DOS batch file. "Echo" displays comments during execution. Also note that "del" should be used with extreme caution and never with wild cards.

The batch file was set up for the month of August. To run the September report we could manually change all of the `08's to `09's in the file. Another solution would be to parameterize the number of the month. DOS parameters are similar to parameters in a macro call. To insert the parameter into your batch file, simply replace the values you want to parameterize with a `%1' for the first parameter, a `%2' for the second, etc.

Below is the revised code ("Report.bat") which parameterizes the month:

call "c:\Program Files\SAS\sas.exe" "c:\example\readsum%1.sas"

call st "c:\Example\sum%1.sd2" "c:\Example\report%1.xls"

call "c:\Program Files\WinZip80\Wzzip.exe" "c:\example\My Archive\archive%1.zip" "c:\Example\data%1.dat"

del "c:\Example\data%1.dat"

To specify the month parameter when we run the batch file, we just add the parameter after the batch file name:

>report.bat 09

If additional parameters had been used, they would just follow the 09:

>report.bat 09 2000

The DOS Batch file can also be used to change the directory location before running a command. For example, we could change the directory and then issue a copy command:

cd \ cd "Example\My Archive" copy archive%1.zip backup%1.zip

X COMMAND

While DOS batch files provide for some parameterization, batch processing from within SAS is even more flexible. The easiest way to batch process from within SAS it to use an X Command. The general format of the X Command is:

"X" + single space + DOS (or UNIX) command + semicolon

The command can be unquoted or enclosed in single quotes. However, if you follow the "X" with a quote, keep in mind that SAS assumes that the command ends at the point where that quote closes. For example, the following X command will not generate the intended results:

X "dir" "c:\example"; /* WRONG RESULT*/;

The above command generates a directory listing of the current subdirectory rather than the "Example" subdirectory because everything after the closing of the first double quote is ignored.

In this case, fixing the code is simple. We can either place single quotes around the entire command or just start the command without any quotation mark:

X dir "c:\example"; /* CORRECT RESULT */ X 'dir "c:\example"'; /* CORRECT RESULT */

However, if we were not using a DOS prompt command or needed to insert a macro variable, these solutions would not have worked:

/* GENERATES ERROR */; X '"c:\Program Files\WinZip80\Wzzip.exe"

"c:\Example\My Archive\archive&month..zip" "c:\Example\data&month..dat"';

/* GENERATES ERROR */; X c:\Program Files\WinZip80\Wzzip.exe

"c:\Example\My Archive\archive&month..zip" "c:\Example\data&month..dat";

In the first statement, the "&month" macro will not resolve because of the single quotes and in the second statement, the subdirectory will be seen as only "c:\Example\My" if the quotes are left off. The easiest solution to these problems is to place a "Call" before the command:

X Call "c:\Program Files\WinZip80\Wzzip.exe" "c:\Example\My Archive\archive&month..zip" "c:\Example\data&month..dat";

Using "Call" we could run the monthly report from within SAS:

%let month=08; X call "c:\Program Files\SAS\sas.exe"

"c:\example\readsum&month..sas"; X call st "c:\Example\sum&month..sd2"

"c:\Example\report&month..xls"; X call "c:\Program Files\WinZip80\Wzzip.exe"

"c:\example\My Archive\archive&month..zip" "c:\Example\data&month..dat"; X del "c:\Example\data&month..dat";

We could also specify how the X command operates with these global options:

XWAIT (default) - Keeps DOS shell open after executing command. The user must close it by typing EXIT or clicking the close box.

XNOWAIT - Closes DOS shell without any action by user.

XSYNC (default) - Finishes execution of the batch command before continuing to run the rest of the SAS program.

XNOSYNC - Starts the batch command but returns to running the SAS program immediately whether or not the batch command has finished.

While the X Command is fairly flexible and can be executed anywhere in your code (it is a global statement), it has two major limitations. First, each X Command is executed separately, this means that unlike with a batch file, it makes little sense to try to alter the DOS environment:

X `cd..'; X cd "Example"; X cd "My Archive"; X copy "Archive&month.zip" "Backup&month.zip";

This will yield an error message because each time an X command is issued DOS is re-invoked. A second limitation of the X Command is that it does not allow conditional execution-it is always executed immediately.

CALL SYSTEM ROUTINE

Call System is similar to the X Command but since it is a routine, it can be executed conditionally. For example, if we only wanted to delete archives that were over 6 months old, we could use:

%let month =09; data _null_;

do i = 1 to 12; if i < (&month - 6) then do; if i < 10 then del_com = 'del "c:\Example\'|| 'My Archive\archive0' ||compress(i)||'.zip"'; else del_com = 'del "c:\Example\'|| 'My Archive\archive' ||compress(i)||'.zip"'; call system(del_com); end;

end; run;

The general syntax of the Call System routine is:

call system();

The command can either be a character variable containing a command (such as del_com in the above example) or a command enclosed in quotes:

call system(`md c:\Example\Report09');

or an expression that evaluates to a command:

call system('rd '||'c:\Example\Report09');

CONCLUSION

Batch processing allows you to automate across SAS and other applications. For simple batch jobs, DOS batch files often provide the best solution. For more complex parameters, X commands are also easy to use. However, for the most flexibility and conditional execution, the Call System is your best option.

CONTACT INFORMATION

Lisa Sanbonmatsu Harvard University Malcolm Wiener Center for Social Policy 79 JFK Street Cambridge, MA 02138 Work Phone: 617-495-5131 Email: Lisa_Sanbonmatsu@

SAS is a registered trademark or trademark of SAS Institute Inc. in the USA and other countries. ? indicates USA registration.

Other brand and product names are registered trademarks or trademarks of their respective companies.

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download