Using Unnamed Pipes to Simplify Access to External Files

Using Unnamed Pipes to Simplify Access to External Files

John Wenston, Pfizer Inc., New York, NY

ABSTRACT

Unnamed pipes allow the SAS? user to run a program or execute an operating system command outside of the SAS system and redirect the output back into SAS. This enables the programmer to get data directly from an external program without having to create an intermediate data set or enter information manually.

the output directed to the screen is:

f011596.trm f011796.trm f012896.trm f020696.trm f021896.trm

This paper presents an example that uses unnamed pipes to redirect the output from the DIR command. The examples presented have been used with SAS 6.11 for both Windows 95 and OS/2?.

By using the FILENAME command with the PIPE option and the DIR command, this output can be directed to a virtual file and read into a SAS dataset. For example:

INTRODUCTION

The example presented here was used to process files of coded terms sent to our Medical Dictionary Management Group on a periodic basis. As each batch of terms was received, it was sent to a directory used for holding unprocessed terms and given a name with a file type of .TRM. At 6-8 week intervals, all of the term files on the directory were read into SAS and used to update the dictionaries. At times, there could be as many as 40-50 files that required processing.

USAGE

Rather than entering each file name in SAS FILENAME statements every time a dictionary update was run, an unnamed pipe redirecting the output from the DOS DIR command was used to process the files. The syntax for using unnamed pipes with operating system commands is:

filename pipe "";

where command is a DOS command. The PIPE option directs the output from the command to a virtual text file, referenced by fileref, that can be accessed in a Data Step.

For example, suppose the directory c:\data contains 5 files of type .TRM. When the following command is executed

DIR C:\DATA *.TRM/B

filename terms pipe 'dir c:\data\*.trm/b';

data fnames; infile terms pad missover; input @1 filename $12.;

proc print data=fnames; run;

OBS FILENAME

1 f011596.trm 2 f011796.trm 3 f012896.trm 4 f020696.trm 5 f021896.trm

Once the information is in a SAS data set, it becomes easy to work with. In the example presented here, the use of pipes was combined with the macro language to run a processing program on each file in sequence. For example, suppose there is a macro called %PROCESS that processes the observations in a file (e.g., adds or updates terms in the dictionary based on the code associated with the term):

%macro process(input=);

data transact; infile &input; input med_code med_term $50.;

proc sort data=transact; by med_code;.

data diction; update diction transact; by med_code;

%mend;

1

The following program will read each of the 5 file names returned by the DIR command into macro variables &file1 to &file5 and run the macro %Process on each file in sequence:

****get all files of type Trm *******;

filename terms pipe 'dir c:\data\*.trm/b';

**put each file name in a macro variable**; **put number of files in variable NFILES**;

data _null_; infile terms pad missover end=eof; input @1 filename $12.; call symput('file' || left(_n_),filename); if eof then call symput('nfiles',left(_n_)); run;

***Run processing program on each file***;

%macro run_em; %do i=1 %to &nfiles;

%process(input=c:\data\&&file&i); %end; %mend run_em; %run_em;

Another similar application of pipes is to test whether a file or files exist on a directory. The following program determines whether any files of type .TRM exist on directory c:\mydata, and sets a macro variable &howmany equal to the number of such files. This variable can then be used, e.g., in conditionals later in the program to direct processing.

CONCLUSION

There are any other number of ways that unnamed pipes can be used. They provide a powerful, flexible tool that can be used to make programs simpler and more efficient;

REFERENCES

SAS Institute Inc. (1996) SAS Companion for the Microsoft Windows Environment, Version 6, Second Edition, Cary, NC: SAS Institute Inc.

ACKNOWLEDMENTS

SAS is a registered trademark or trademark of SAS Institute in the USA and other countries. OS/2 is a registered trademark or trademark of International Business Machines Corporation. ? indicates USA registration.

Other brand and product names are registered trademarks or trademarks of their respective companies.

I would like to thank Fan Zhou for his input and assistance.

AUTHOR'S ADDRESS

John Wenston Pfizer, Inc. 235 E. 42 St. New York, NY Phone: (212) 573-3992 E-Mail: wenstj@

**Read filenames into SAS data set ***;

Filename terms pipe `dir c:\mydata\*.trm/b';

data files; infile terms pad missover end=eof; input @1 filename $12.;

***Set macro var Howmany to n obs **;

data _null_; call symput('howmany',left(n_obs)); if 0 then set files nobs=n_obs;; stop; 2

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download