University of Washington



Web service shell architecture2012/11/19Purpose:This document outlines the function of the Web Service Shell (WSS) that communicates with both an HTTP client and an internal handler (e.g. command line utility) responsible for generating the content returned to the client. This document also includes a technical description of the communication protocol between the shell and the handler program. This shell architecture is primarily targeted at International FDSN web services.General architecture:??WSS ServletWeb service client?HTTP????InternalJavaInterface???Unix pipes?Exit status??ExecuteExternalWorker???HandlerData CenterSpecific???ConfigurationFiles?Logging??OR?Web Service Container??Container ManagedAuthentication???WSS ServletWeb service client?HTTP????InternalJavaInterface???Unix pipes?Exit status??ExecuteExternalWorker???HandlerData CenterSpecific???ConfigurationFiles?Logging??OR?Web Service Container??Container ManagedAuthentication?The WSS translates web service parameters into options for a localized data content handler that returns data in the format expected by the client. The Shell performs the following core functions:Manage HTTP connection with clientFirst order validation of request parameters; see “Parameter Validation” below.Manage connection with localized handler according to configuration parameters.Log service requests, including time series shipment summariesHandles (via the servlet container) HTTP authentication (basic or digest).The web service shell is written in Java and designed to run in a Tomcat container, other common containers (e.g. Glassfish) will also work.Parameter ValidationThe WSS will check for expected parameters as defined in the configuration file and will validate their type only (date, number, text). Unexpected parameters or parameter values that do not match the expected type will result in an error returned to the client. The WSS will not check the values for valid ranges or for the appropriate grouping of parameters. The content handler should perform all remaining validation of the parameters.Data Content HandlersThe handler program is responsible for extracting data from the appropriate data center repository, and then returning it in the format expected by the client. There are two options for creating a localized handler: a) an external command-line program that is executed by the shell and b) a Java “stub” that returns data via an internal interface.External program handlerThe shell executes an external program handler using inputs that are supplied through command line arguments or via the standard input stream. The handler transmits data content via the standard output stream. The overall status of the request is communicated using the handler’s exit status. An optional message may be transmitted from the handler to the shell via the standard error stream. Only *nix systems are supported.InputsRequest parameters are passed to the handler using command line options roughly equivalent to parameters received via a GET request. In the case of POST requests that may include numerous selections, the POST body is passed to the handler via the standard input stream.Environment variablesThe shell will set the following environment variables for use by the handler:VariableDescriptionREQUESTURLFull request URL submitted to the shellUSERAGENTHTTP UserAgent submitted to the shellIPADDRESSIP address of client submitting requestAPPNAMEApplication name as set in the configuration fileVERSIONWeb service version as set in the configuration fileData contentThe handler returns the results via the standard output stream. The data content should be limited to the data only (e.g., returned data should not include any HTTP headers). This data is streamed directly to the client after the shell has added the appropriate HTTP headers.Request statusThe handler’s exit status indicates the overall status of the request and is translated to HTTP status codes as follows:Exit statusHTTP statusDescription0200Successfully processed request, data returned via stdout.1204No data. Request was successful but results in no data.2400Invalid or unsupported argument/parameter.3413Too much data requested.4500General error. An error description may be provided on stderr.Optional messageThe handler may return an optional message via the standard error stream. Normally this is a mechanism for the handler to report a description of an error. The shell may include this description in an error message sent back to the client.Internal Java interfaceAs an alternative to calling an external program, data content may be returned via an internal Java interface. This requires developing data center specific code in Java. In some cases this may allow data extraction to be optimized. For example, an intermediate cache could be used instead always retrieving from the repository.Unsupported optionsIf a handler does not support a parameter-argument that is supported by the shell, then it should exit with an exit status of 2 to indicate an error and return the following text on stderr: “Unsupported option: XX”.Configuration FilesThere are 2 configuration files associated with the WSS. The first is the application configuration file that allows control over basic operational parameters. The second controls the list of allowable query parameters and their basic types (for simple validation by the WSS).Configuration files are located in the WSS servlet’s /webserviceshell/WEB-INF/classes/META-INF directory. The configuration file format is simply a list of name=value pairs. Names and values are case-sensitive. Whitespace is ignored.Application Configuration file (service.cfg)ParameterDescriptionrootServicePathURI base of the service interface, e.g. service.iris.edu/fdsn/eventrootServiceDocURI of document to be served to the client for the service root page. The following strings, if present, will be translated in the document by the shell:BASEURL – replace with the value of rootServicePathVERSION – replace with service version HOST – replaced with host name* see Documentation section belowappNameName of application, e.g. “fdsn-station”. Used in error messages and logging.versionWeb service version, e.g. “1.0.0”. Used in error messages, logging, and potentially content.outputType Default output type from set [text, xml, seed, json]. Determines the default output MIME type, defaults to text.handlerProgramPath to the content handler program that returns content, e.g.: “/usr/local/bin/extractminiseed”handlerTimeoutA timeout value in seconds. If the handler does not respond within the timeout duration the process is killed and an HTTP status code of 503 is returned to the caller.handlerWorkingDirectoryPath to the directory on the host where the handlerProgram will nominally run, i.e. where data files may be read, created, etc.?? This directory is referenced to the main directory of the web service shell servlet on the application server.? This directory will be erased during standard deployment of the web application unless it is part of the project, i.e. the? WAR.? Attempts to use a working directory outside of the application’s context on the server will result in an exception.? Defaults to “/”.loggingTypeLogging type from set [LOG4J, JMS]. Defaults to LOG4J.jndiUrl(JMS specific) URL to the JNDI configuration data for JMS loggingconnectionFactory(JMS specific) connectionFactoryName for establishing publishing connectiontopicDestination(JMS specific) Topic to which to publishsingletonClassName(Java interface specific) Qualified name of Java class to load at application startstreamingOutputClassName(Java interface specific) Qualified name of Java class to use for data access.Query Parameter Configuration File (param.cfg)This configuration file controls the list of allowed query parameters along with their basic types. Allowable types are listed in the table below.Parameter TypeDescription / Validation RequirementDATEValid FDSN data format: YYYY-MM-DDTHH:MM:SSNUMBERNumber (value must be parseable as a number)TEXTText string (not validated)Example param.cfg file:starttime=DATEendtime=DATEnetwork=TEXTstation=TEXTlocation=TEXTchannel=TEXTmaxlatitude=NUMBERminlatitude=NUMBERMapping between HTTP parameters and handler argumentsThe WSS will pass each parameter to the handler by simply translating each parameter to a command line argument and value.Service parameter: key=valueCommand line equivalent: --key valueThe handler is expected to use the same formats and conventions for the parameter values as specified for the service parameters. For example, if the service accepts a time value as YYYY-MM-DDTHH:MM:SS the handler should also accept the equivalent time value in that format. Another example, if service accepts a depth as a value in kilometers, the equivalent option in the handler should take the value as kilometers.DocumentationThe base URL of the web service is expected to return documentation in HTML format. The page should generally contain an overview of the service operation with links to further documentation as needed. To maximize flexibility the template for this page is not embedded into the service. The service will fetch and return the HTML document indicated by rootServiceDoc and the strings BASEURL, VERSION and HOST will be replaced as indicated in the configuration parameter table. The value of rootServicePath (BASEURL) should be the URL used by clients to access the service and can be used to create links in documentation to example queries and services methods (e.g. WADL or a version method).Error conditionsOn handler errors or a handler timeout, the shell will translate the errors to an appropriate HTTP status and error message and terminate the handler if it is still running. On client errors the shell will terminate the handler.Terminating the local handler is done in the following stages:Send the process the termination signal (SIGTERM).If the process remains after 10 seconds, send the kill signal (SIGKILL) and disconnect all streams from the process (stdin, stdout, stderr).Usage LoggingThe WSS can log each request using either Log4j or via an IRIS-designed scheme utilizing JMS (Java Message Service, a publish / subscribe system). The Log4j mechanism will be used by default and is configured to write the log entries to files. Standard Log4j configuration is used which allows the use of a variety of existing Log4j ‘appenders’. The default logging configuration is a daily rolling log.The JMS mechanism is far more complex and is designed to utilize a centralized (possibly clustered) logging server to which messages are submitted by multiple services. Multiple subscribers can then listen for these messages and log them to a file, database, etc.In addition to an entry for each request, the WSS will log a summary of each data segment for time series requests.The usage log message content is designed to be consistent between service types to allow for aggregation and uniform storage. WSS LoggingThe WSS servlet itself utilizes Log4j to log internal log messages indicating the internal state of the WSS program. These are written to a separate log file from the Usage Logging described above.The locations of the usage and WSS log files are configurable via the Log4j properties configuration file.Authentication (for fdsn-dataselect only)For service methods that require authentication for access to restricted data (e.g. fdsn-dataselect/queryauth) the web application container should be configured to negotiate the authentication and provide the WSS servlet with the authenticated user name.If authentication is successful, the user name will be supplied to the data content handler with a command line argument of “--username <user>” or an internal variable. The implication is that this user has been authenticated and data content should be allowed based on the user name. Except for the user name no other authentication credentials are provided to the data content handler.Example mapping for the fdsn-station serviceHTTP ParameterHandler argumentDescription (from the specification)starttime--starttimeStart time (YYYY-MM-DDTHH:MM:SS)endtime--endtimeEnd time (YYYY-MM-DDTHH:MM:SS)startbefore--startbeforeStart beforestartafter--startafterStart afterendbefore--endbeforeEnd beforeendafter--endafterEnd afternetwork--networkSEED network codestation--stationSEED station codelocation--locationSEED location IDchannel--channelSEED channel codesminlatitude--minlatitudeMinimum latitude (degrees)maxlatitude--maxlatitudeMaximum latitude (degrees)minlongitude--minlongitudeMinimum longitude (degrees)maxlongitude--maxlongitudeMaximum longitude (degrees)latitude--latitudeLatitude for radius (degrees)longitude--longitudeLongitude for radius (degrees)minradius--minradiusMinimum radius, default 0 (degrees)maxradius--maxradiusMaximum radius (degrees)level--levelResult level: net,sta,chan,respincluderestricted--includerestrictedInclude metadata if access is restrictedincludeavailability--includeavailabilityInclude availability updatedafter--updatedafter Limit to metadata updated after timeA request for:fdsn/station/query?network=IU&starttime=2012-01-01T12:13:14might result in handler execution similar to:/usr/local/bin/MetadataHandler --network IU --starttime 2012-01-01T12:13:14Ideas for consideration:* Include an administrative “status” page that shows the configuration of the server. This page could potentially allow changing of log levels. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download