TrWeb - System Guide



JDN/T#(00)xxx

Issue 1.0: 03/09/2003

UNCONTROLLED WHEN PRINTED

System Guide to the TRANSP MDSPlus Tree Management System: trWEB.

Author: D.G.Muir

Abstract

A detailed system guide to a new web based system, named trWEB, to manage MDSPlus data trees for the TRANSP transport analysis code, is presented.

Keywords

Architectural Design, HTTP, MDSplus, PERL, PPF.

|R.O.: David G Muir |Reviewer: Jim Conboy |Approver: Geoff Cordey |

Distribution

DCM, followed by distribution list for outside of DCM. Note DCM distribution will be by e-mail

Table of Contents

1. introduction 3

2. System components 3

2.1 HTML Presentation Layer 3

2.2 JavaScript Functions 8

2.3 Perl CGI Scripts 11

2.3.1 Configuration File 11

2.3.2 browse.pl 13

2.3.3 mds-idl.pl 15

2.4 IDL Programs 17

2.5 MDSPlus TDI Functions 19

3. Making System changes 20

3.1 Promotion from Development to Production 20

3.2 Changing Tree Names or Adding New Tree Names 23

3.3 Problem Resolution 23

introduction

A project is underway to enable TRANSP to be run remotely at Princeton from Jet. Access to their Fusion Grid servers will use secure GLOBUS client grid tools. Indirect access to Jet data held in the PPF system will be made via the MDSPLUS Server and specific MDSPLUS Data trees. This note describes the system details of one aspect of this project, namely, that component which manages MDSPlus data trees. This component is called trWEB and is run using the following http link:

.

System components

The system comprises a web browser presentation layer running HTML and JavaSript. This layer uses HTTP to communicate via the CGI of an Apache Web server running on a JAC Linux computer. The CGI is written in Perl that automatically builds and executes IDL programs on the same JAC. These IDL programs connect to the MDSPlus server, which in turn accesses Jet data via the PPF data server. HTML web pages are generated dynamically by both the IDL programs and Perl scripts, whilst JavaScript functions and data structures validate data entry and internally manage data. A schematic of this architecture is shown in figure 1.

The trWEB site map, together with associated links through the Perl CGI scripts and IDL programs highlighting the data processing flow, is shown in figure 2.

1 HTML Presentation Layer

All interactions between the user and the system occur using a web browser running HTML. Users have a choice of which browser to execute: Microsoft’s Internet Explorer on a Windows client PC or Mozilla and Netscape on a Unix session. Also, subject to security, the system could be run from outside the firewall.

The HTML scripts use version 4, which should not create problems with the latest Browser versions.

Table 1 lists all the permanent HTML files. Other HTML pages are created dynamically, by Perl scripts and by IDL programs. These will be discussed later.

All HTML files are located in the directory: /home/pshare/trweb/browser. Image files used by these HTML pages are located in the images subdirectory.

Figure 1: Schematic High Level Architecture of TRANSP MDSPlus System.

Figure2: System Site Map and Data Processing Flow.

Table 1: Permanent HTML Pages.

|File Name |Description |

|tree_menu.html |The system Home Page, which provides links to the html files listed in this table. |

|create2.html |A large HTML form allowing the user to specify all requirements for the MDSPlus analysis tree. All fields are |

| |validated on change using JavaScript, and the complete form is validated prior to submission to the Perl CGI |

| |script: mds-idl.pl. Also passed is the name of the IDL program to execute: create2.pro. This program constructs |

| |the MDSPlus tree and returns the status of this task to the CGI script. The script in turn dynamically creates an|

| |HTML page informing the user of the outcome and presenting next step actions. |

|edit.html |A small HTML Form requesting the user to identify the MDSPlus tree to be edited. This information is validated |

| |using JavaScript and the information is passed using a POST method to the Perl CGI script: browse.pl. Also passed|

| |is the name of the IDL program to execute: edit2.pro. This program reads the tree data and dynamically constructs|

| |an HTML Form for data entry to allow the user to modify the contents of the tree. This form is very similar to |

| |the permanent HTML file: create2.html. |

|copy.html |A small HTML Form requesting the user to identify the MDSPlus tree to be copied together with information needed |

| |to specify the tree to be created. This information is validated using JavaScript and the information is passed |

| |using a POST method to the Perl CGI script: browse.pl. Also passed is the name of the IDL program to execute: |

| |copy2.pro. This program reads the tree data and dynamically constructs an HTML Form for data entry to allow the |

| |user to modify the contents of the tree. This form is very similar to the permanent HTML file: create2.html. |

|browse.html |A small HTML Form requesting the user to identify the MDSPlus tree to be browsed. This information is validated |

| |using JavaScript and the information is passed using a POST method to the Perl CGI script: browse.pl. Also passed|

| |is the name of the IDL program to execute: browse.pro. This program reads the tree data and dynamically |

| |constructs an HTML Table displaying the contents of the tree. This table is very similar in layout to the form |

| |presented by the permanent HTML file: create2.html. |

|help.html |A simple user guide to the system is presented. |

|temp.html |A message indication that the link is Under Construction is displayed. |

2 JavaScript Functions

There are two libraries of JavaScript functions. The first contains functions specifically needed to define and instantiate browser data objects, see Table 2. The second contains general data validation functions together with specific functions to validate data entry forms prior to submission to the Perl CGI Scripts. These are listed in Table 3.

JavaScript libraries are located in the subdirectory: /home/pshare/trweb/browser/jscripts.

Table 2: The JavaScript functions associated with Browser Data Objects: startup.js

|Function |Description |

|Nodes1 |Defines a data structure associated with Rank 1 MDSPlus nodes |

|Nodes2 |Defines a data structure associated with Rank 2 MDSPlus nodes |

|Nodes3 |Defines a data structure associated with Rank 3 MDSPlus nodes |

|Update |Updates all data structures with data entered into an HTML form prior to submission of name value pairs |

| |to a Perl CGI script. |

|Instant |Instantiate the Nodes1, Nodes2 and Nodes3 data structures. |

|Defaults1d |Apply Rank 1 default values to the Nodes1 data structure. |

|Defaults1d |Apply Rank 2 default values to the Nodes2 data structure. |

|Defaults1d |Apply Rank 3 default values to the Nodes3 data structure. |

Table 3: JavaScript Validation functions: validation.js

|Function |Form |Description |

|validRequired |- |Checks the data field is populated |

|validLength |- |Checks the Length of the data field against a maximum value |

|validMinimumLength |- |Checks the Length of the data field against a minimum value |

|allDigits |- |Checks the data field only contains digits 0-9 |

|allAlpha |- |Checks the data field only contains characters a-z or A-Z |

|allAlphaNumeric |- |Checks the data field only contains digits 0-9 or characters a-z or A-Z |

|inValidCharSet |- |Checks the data field only contains characters from a specified set. |

|validInt |- |Checks the data field is a valid Integer number |

|validFloat |- |Checks the data field is a valid Floating Point number |

|validAlpha |- |Checks the data field is a valid Alphabetic character |

|validLeadingAlpha |- |Checks the leading character of the data field is a valid Alphabetic character |

|validAlphaNumeric |- |Checks the data field is a valid Alphabetic or Numerical character |

|validDate |- |Checks the data field is a valid Date with format: MM/DD/YYYY (Not Used at present). |

|valid_data |- |Called by function validate to specifically check data MDSPlus data node attributes. |

|validate |Create2.html |Calls the functions validform and valid_data and checks the owner user ID field. |

| |Copy2.pro | |

| |Edit2.pro | |

|validcopy |Copy.html |Validates the information required to identify the tree to be copied and the tree to be|

| | |created. |

|validform |Browse.html |Validates the information required to identify the tree to be browsed. |

|validform2 |Edit.html |Validates the information required to identify the tree to be edited. |

3 Perl CGI Scripts

There are only two Perl CGI scripts used by this application. In addition there is a configuration file which identifies the library where the IDL programs are located. These components are described blow.

There are many common features shared by the two scripts. These are outlined next:

Table 4: Script Variables:

|Variable Name |Description |

|$TMP |Identifies the directory where temporary files will be saved: /tmp/trweb. |

|$CONF |Identifies the name of the script configuration file: mds-idl.cfg |

|$IDL_EXE |Give the full path name to the IDL executable |

|$pname |A unique filename based on the process id, e.g., p12345 |

|$LIB_REF |The IDL program library reference name |

|$PGM_PATH_NAME |The directory where IDL programs are located |

|$IDLPROG |The full path name of the IDL program to be executed |

|$DEBUG |If set to y (default) the temporary files have their permissions changed to universal read/write access. |

| |Otherwise, they are deleted. |

1 Configuration File

The configuration file is identified within the Perl script by the variable: $CONF. This file contains the location of the directory holding the IDL programs executed by the script. When an HTML POSTs name value pairs to the CGI, one of the names identifies the IDL program to run. For example, the format to identify the IDL browse program is as follows: _program=idllib.browse.pro. The directory is referenced by the library name idllib, which precedes the name of the program browse.pro. They are joined together by a . operator. The Perl script needs to know where the library idllib is and that is the function of the configuration file. Within that file the link between the reference and the actual directory location is documented. The format is library name followed by the full path to the directory containing the program source code. An exclamation mark joins the two components.

There is only one configuration file: mds-idl.cfg. This is located in the CGI directory and contains the following: idllib!/home/pshare/trweb/idllib .

There is only one reference library identified within the configuration file: idllib. In principle, many could be named so that different versions of the IDL programs could be accessed by the application depending on the situation: test, development, production etc.

2 browse.pl

The Perl Script browse.pl is used to execute IDL programs that dynamically create HTML Forms to browse, edit or copy MDSPlus data trees. These latter two forms, when submitted, call the Perl script mds-idl.pl to actually create the replacement tree or a new tree.

The script works as follows.

• The processing environment is defined

• A temporary and unique filename is created from the process id

• A temporary work directory, $TMP (/tmp/trweb), is created on the web server (JAC-37)

• The name value pairs POSTED from the HTML form are input as a hash data structure

• An IDL procedure , written to $TMP, is created with the unique filename: $pname.pro

• The name value pairs are read from the list and written into the IDL procedure as assignment statements.

• Names beginning n_ are numerical and those beginning s_ are strings.

• The IDL program pointed to by the _program variable from the HTML form is concatenated to the $pname.pro file.

• An IDL startup file, written to $TMP, is created: $pname.in

• The IDL path is extended to include all the MDSPlus routines required

• A journal file , written to $TMP, is opened to help trace any errors: $pname.jrn

• The IDL procedure is compiled and executed

• No status value is returned to the Perl session.

• The Perl session environment variables are modified

• The path is extended to include all the IDL and MDSPlus libraries

• The IDL environment variable: IDL_STARTUP is defined and points to the IDL startup file.

• The IDL executable is invoked. This runs the set of commands saved in the IDL startup file pointed to by the IDL_STARTUP environment variable.

• This IDL process execute the procedure $pname.pro

• The IDL procedure writes HTML and JavaScript to the redirect file $pname.out also on $TMP.

• The Perl Script return the contents of $pname.out to the web browser.

• If this file is missing, an error message is returned to the browser.

• The permissions of the temporary files are changed to allow universal read/write access.

All temporary files are written to the $TMP Directory, see Table 5, which can be checked should any problems occur. In addition, information about IDL program execution are written into the Web server log, located at: /usr/local/depot/apache-1.3.26/logs/error_log which can also be examined should any problems occur.

Table 5: Temporary Files Written to the $TMP Directory

|Temporary Filename |Contents |

|$pname.pro |The IDL procedure to be executed |

|$pname.in |The IDL startup file containing a set of command to execute |

|$pname.jrn |The IDL Journal file containing a log of the data processing |

|$pname.out |The IDL output file containing HTML and JavaScript to be sent to the browser |

3 mds-idl.pl

The Perl Script mds-idl.pl is used to execute IDL programs that return a status code and to dynamically create an HTML page depending on the return code. The IDL program called is the create2.pro program that is responsible for creating new MDSPlus data trees.

In addition to the process description given above for the browse script, the following occurs:

• Initiates parameters to hold values required by a HTML form

• Writes array declarations to the $pname.pro IDL procedure file to hold all the MDSPlus node data assigned from name value pairs.

• Assigns values to the parameters required by a HTML form

• The IDL Startup file now returns a status value

• The Perl script outputs HTML to the browser that is dependent on the returned status value. This is described in Table 6.

Table 6: HTML Page returned depending on the Returned Status Value from the IDL Program.

|Status |HTML Description |

|0 |The MDSPlus tree has been successfully created, edited or copied. An HTML page allowing the user to either return to the system|

| |home page or to browse the tree just created, edited or copied is presented. |

|1 |A message is displayed informing the user that the MDSPlus Tree could not be opened or does not exist. |

|2 |A message is displayed informing the user that the MDSPlus Tree already exists so a new tree cannot be created or an existing |

| |tree modified. |

|3 |A message is displayed informing the user that they are not the owner of the tree and therefore cannot edit it. |

|other |A message is displayed informing the user that an unknown processing error has occurred and to report it to the TRANSP |

| |Responsible Officer (RO). |

4 IDL Programs

There are three IDL procedures and four partial IDL procedures located in the IDLLIB library. These are as described in Table 7.

Table 7: IDL Programs and their Function.

|File Name |Description |

|security.pro |Checks the ownership of a tree from the contents of the node \TOP:USERID |

| |Three superusers are defined with permission to edit all trees: pshare, dgm and jconboy. |

| |Returns true if the tree can be edited. |

|tree_exist.pro |Connects to the MDSPlus server on the Web Server (w3.jac.jet.uk) and determines whether or not a tree exists. |

| |Returns a status value 1 if the tree exists, otherwise a 0 is returned. |

|end_nodes.pro |Returns information on nodes stored in a tree. Given a path to the node, the information returned is: an array of node |

| |names; the number of nodes; the decompiled contents of the node; the node usage attribute. |

|browse.pro |A partial procedure with the lead part written by the Perl CGI Script. |

| |Connects to the MDSPlus Server w3.jac.jet.uk on the Web Server. |

| |Calls the end_nodes procedure. |

| |Writes HTML and JavaScript to the output stream to display the tree’s contents by the web browser. |

|copy2.pro |A partial procedure with the lead part written by the Perl CGI Script. |

| |Calls the tree_exist procedure. |

| |Connects to the MDSPlus Server w3.jac.jet.uk on the Web Server. |

| |Calls the end_nodes procedure. |

| |Writes a HTML Form and associated validation JavaScript to the output stream to display the tree’s contents by the web |

| |browser and allow the user to make any required modifications. On submission, the Perl script mds-idl.pl is executed |

| |together with the IDL procedure create2.pro to create a new tree. |

|edit2.pro |A partial procedure with the lead part written by the Perl CGI Script. |

| |Connects to the MDSPlus Server w3.jac.jet.uk on the Web Server. |

| |Call the security procedure. |

| |Calls the end_nodes procedure. |

| |Writes an HTML Form and associated validation JavaScript to the output stream to display the tree’s contents by the web |

| |browser and allow the user to make any required modifications. On submission, the Perl script mds-idl.pl is executed |

| |together with the IDL procedure create2.pro to replace the tree. |

Table 7 Continued.

|File Name |Description |

|create2.pro |Creates or replaces MDSPlus trees. |

| |A partial procedure with the lead part written by the Perl CGI Script. |

| |Calls the tree_exist procedure. |

| |Sets a TREE_PATH environment variable depending on the name of the tree. |

| |Builds the model tree using MDSPlus in local mode. |

| |Creates the tree for the specific pulse and analysis run id using the model tree as a template. |

| |Changes the permissions of the tree files to universal read/write access. |

| |Deletes the model tree. |

All primary IDL programs access the MDSPlus server that runs on the same machine as the web server. However, whereas the server is used to read and deconstruct the data trees, it is not used by the create2.pro program to write a new tree or to replace an existing tree.

5 MDSPlus TDI Functions

There are three TDI functions specifically for this application. All three are used to read and return data from the JET PPF data server. These functions are located in the directory: /usr/local/depot/mdsplus-1.3/tdi/jet/dgm that is accessed by MDSPlus servers. Table 8 lists these functions and describes their purpose.

Table 8: MDSPlus TDI Functions to Access PPF data.

|Function |Description |

|ppf_seq.fun |For a given userid, ppf dda name and pulse number, this function return the ppf sequence number. |

|ppf_transform.fun |For a given userid, ppf dda name, pulse number, and ppf sequence number this function will access the PPF server,|

| |read the data, transform the data by scaling both the data values and the x-vector, add an offset to the time |

| |vector, and finally, reorder the data from Data(X,T) to Data(T,X). |

|mry.fun |For a given pulse number and sequence number, the TRANPPF user’s PPF MSCR and datatype MRY is accessed for |

| |boundary moment data. |

Making System changes

1 Promotion from Development to Production

The HTML files contain references to two directories on the web server. These are mappings to the actual directories and can be easily changed by the web server administrator. When testing or developing parts of the system, the reference mappings should be those for development only. On promotion to the production environment, these mappings should be changed. (If there were separate development/test and production web servers then the same names could and should be used so that these changes would be unnecessary).

The IDL directory reference is held in the Perl Script configuration file, which is not part of the web server set-up. It is therefore not managed by the web server administrator. Consequently, it is the responsibility of the system maintainer to ensure that the Perl configuration file points to the correct IDL directory and the Perl scripts open and read the correct configuration file. Also, within the IDL procedures there are hard coded references to the IDL directory. These must also be changed according to the system environment. One simple improvement here would be to change these to directory references read from a session environment variable defined and set from within the Perl script.

Table 9: The name of Web Server Mappings to Production and Development Environments.

|System Environment |HTML Directory |CGI Directory |IDL Directory |

| |Mapping |Mapping | |

|Development and Test |dgmwebtest |dgmtest |idllib |

|Production |trweb |cgi-trweb |idllib |

Table 10: The Full Directory Paths of the Web Server Mappings

|Web Server Directory Mapping |Full Directory |

|dgmwebtest |/home/dgm/dgmwebtest/ |

|dgmtest |/home/dgm/cgi-bin/ |

|trweb |/home/pshare/trweb/browser/ |

|cgi-trweb |/home/pshare/trweb/cgi-bin/ |

|idllib |/home/dgm/cgi-bin/idllib/ (for Development) |

| |/home/pshare/trweb/idllib/ (for Production) |

Table 11: Files Containing Directory References

|File Type |File Name |Contains HTML References |Contains CGI References |Contains IDL References |

|HTML |tree_menu.html |Yes |No |No |

| |browse.html |Yes |Yes |Yes |

| |create2.html |Yes |Yes |Yes |

| |copy.html |Yes |Yes |Yes |

| |edit.html |Yes |Yes |Yes |

| |help.html |No |No |No |

|GIF |efda_3.gif |No |No |No |

|JavaScript |startup.js |No |No |No |

| |validation.js |No |No |No |

|Perl |mds-idl.pl |Yes |Yes |Yes |

| |browse.pl |No |No |No |

| |new-idl-perl.cfg |No |No |Yes |

|IDL |browse.pro |Yes |No |Yes |

| |copy2.pro |Yes |Yes |Yes |

| |edit2.pro |Yes |Yes |Yes |

| |create2.pro |No |No |Yes |

| |end_nodes.pro |No |No |No |

| |security.pro |No |No |No |

| |tree_exist.pro |No |No |No |

|TDI |ppf_seq.fun |No |No |No |

| |ppf_transform.fun |No |No |No |

| |mry.fun |No |No |No |

2 Changing Tree Names or Adding New Tree Names

MDSPlus trees are created directly without using the MDSplus server. This is to allow greater flexibility over ownership. However, the price is that the directory where these trees are written to has to be coded twice. The first time, the environment variable treename_PATH=directory path is defined within the MDSPlus server environment. The second time is within the IDL program create2.pro. Both these locations have to be changed when a tree name is modified or a new tree added.

Currently, there are three trees available for use. Two are for production and one is for testing and development. These are as described in table 12.

Table 12: The location of MDSPlus trees.

|Tree Name |Purpose |Path |

|trdata |Production |/home/pshare/Data/mds/trdata |

|transp |Production |/home/pshare/Data/mds_trin_tree |

|transp1 |Development and Testing |/home/dgm/mdsplus/transp/trees |

3 Problem Resolution

Should problems occur, there are two locations which should be checked for information about the problem. The first is the web server error log. The second is the web server temporary directory containing the IDL programs, journal and output file. These locations are listed in table 13.

Table 13: Locations to Examine if Problems Occur.

|What |Where |

|Web Server Error Log |/usr/local/depot/apache-1.3.26/logs/error_log |

|Run time IDL Programs |On JAC-37: /tmp/trweb |

| |Use the date time stamp to identify the set of files associated with the problem: These files have |

| |extensions .pro .in .jrn and .out. Generally, the journal file is the best source of information on the |

| |problem. In addition, you can invoke IDL and compile or execute the IDL procedure to repeat the problem |

| |and identify/develop a fix if required. |

On possible although highly unlikely problem that could occur is if two users attempt to create a tree with the same name at the same time, regardless of pulse number or analysis run tags. Because MDSPlus creates a model tree using the name of the tree only before the pulse/analysis specific tree is created, both users could attempt to create the model tree simultaneously. The outcome of this contention is unknown.

Trdata tree

Each tree has two main branches. The first branch is called META and holds all metadata that describes the PPF data to be accessed from the second tree branch. The second tree branch is called INPUTS and holds references to TDI functions that read and prepare the PPF data. The internal organisation of nodes linked to these two main branches is shown in Table 4.1.1.

| |Tree Node Name |Description | |

|/TOP |:NAME_OF_CODE |Name of the Analysis Code using these trees: TRANSP | |

| |:DEVICE |Name of the Tokamak Device: JET | |

| |:COMMENT |Description of the TRANSP Analysis | |

| |:SOURCE_SHOT |The JET Pulse Number being analysed | |

| |:RUNID |The Analysis Run Identifier: Character + Two Digit Integer | |

| |:DATE |The Date of the Jet Pulse (Not Used) | |

| |:USERID |The Tree Owner User ID | |

| |.META.SCALES |.DATA:node |Holds data re-scaling multiplication factors |

| | |.TIME: node |Holds time vector offset value |

| | |.X_SCALE: node |Holds x-vector re-scaling multiplication factors |

| | |.REVERSE: node |Holds a Boolean value (1=TRUE; 0=FALSE) to indicate whether or not to |

| | | |re-arrange the data organisation within a Rank-2 PPF data array. |

| |.META.PPF |.UID: node |Holds the userids of the PPFs |

| | |.PPF: node |Holds the PPF names |

| | |.TYP: node |Holds the Data Type names |

| | |.SEQ: node |Holds the PPF Sequence Numbers |

| | |.RANK: node |Indicates the Rank of the PPF data array |

| | |.DESC: node |Holds descriptions of the data |

| |.INPUTS: node | |Holds TDI function calls to the PPF data. |

| | | | |

| | | |Ranks 1 and 2 PPF data use PPF_TRANSFORM, whilst Rank 3 PPF data are |

| | | |individual data structures perhaps requiring specially written TDI |

| | | |functions. |

Table 4.1.1: TRANSP MDSplus Tree Organisation

Currently, there is only one defined TDI function for Rank 3 data. This is associated with Equilibrium Boundary Moments and is accessed using the MRY TDI function. If another Rank 3 node is added to the tree, the PPF_TRANSFORM TDI function is called. It is then the responsibility of the owner of this tree to understand the data organisation within the PPF and to deconstruct it within the program that uses it. Otherwise, special TDI functions have to be written and the IDL program CREATEn.PRO modified.

1 Tree disgnostics

Signals in the tree can be displayed with JETdsp, as described in the User Guide. The tree contents may be directly inspected with traverser.

Figure 4.1.1 shows the top level of the trdata tree–

Figure 4.1.1 trdata tree, from traverser

The node INPUTS stores the Transp input signals ( figure 4.1.2 ) and META stores signal transformation parameters, The remaining nodes are unused ; the data is stored in the trjob tree – see JDN ????? ( Fusion grid access .. ) for details.

Figure 4.1.2 Input signals, from traverser

By right clicking on a node, and selecting Display Data from the drop down menu, the stored function reference may be displayed ( figure 4.1.3 ) -

Figure 4.1.3 Function reference stored at node .INPUTS:CUR

1 META data

The META data structure is shown in figure 4.1.4

META data contains two nodes –

PPF which describes how the data is obtained from the PPF system, and

SCALES which describes how the data is to be transformed.

Figure 4.1.4 META data structure

Figure 4.1.5 shows how the PPF description is stored for input CUR, at META.PPF.DESCR:CUR

Figure 4.1.5 Content of PPF.DESC:CUR

Figure 4.1.6 shows the x scaling entry for QPR

Figure 4.1.6 x scaling entry

-----------------------

[pic]

[pic]

-----------------------

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download