Web Service for Event Metadata

Foundations:

ColManWS

ColManWSClient

AIDA TLD

Applications:

API (access from code)

WWW (access from browser)

GUI (access from studio)

Distributed Event Metadata Architecture

J.Hrivnac/LAL for Atlas SW WS, February 2005 in CERN

Work is based on AIDA interface

to AttributeList implemented by

SQLTuple and ColMan packages.

Foundation: ColManWS

ColManWS is a Web Service wrapper over ColMan, it contains two services:

ColMan.war to be used via XML-RPC (for example by ColManWSClient),

ColMan-GUI.war to be used directly from the Web Browser.

ColManWS is build using JWSDP (Java Web Service Development Package) 1.5.

JWSDP is one of the best Web Service Frameworks, it is already installed within Atlas release.

Other WebService frameworks could be used too, e.g. Axis.

ColManWS WAR files (ColMan.war and ColMan-GUI.war) are deployed in Tomcat 5.0 Web Container.

Tomcat is the most widespread Web Container.

Other standard Web Containers could be used too to deploy the same war files.

Could we have Atlas Tomcat installed in CERN ?

ColMan

JWSDP

ColManWS

Tomcat

deploy

build

ColManWS.war

ColManWS-GUI.war

network

connection

Foundation: ColManWSClient

ColManWSClient is an example of client to ColManWS.

Other clients (in many languages) can be easily build from WSDL specification using JWSDP or other Web Service Framework.

Tomcat

ColManWS

ColManWS interface ?

WSDL

WSDL

JWSDP

ColManWSClient

build

ColManWSClient.exe.jar

use

ColManWS.wsdl

Foundation: AIDATLD (AIDA Tag Library)

AIDA Tag Library allows access to standard AIDA objects from JSP (Java Server pages). If you know AIDA, you can immediateoly use AIDATLD.

In JSP, one can mix HTML, XML defined by Tag Libraries and pure Java code.

All processing is performed on the server, steered from the client.

<%@taglib prefix="c"

uri="http://java.sun.com/jsp/jstl/core" %>

<%@taglib prefix="aida"

uri="http://java.freehep.org/jsp/aida" %>

<html>

<head>

<title>AIDA Tree from XRootd</title>

</head>

<body>

<c:set var="rootDataURI"

value="root://demo.cern.ch/demo.root"/>

<c:set var="histoPath"

value="/h110"/>

<aida:plotter>

<aida:region>

<aida:plot dataSourceURI="${rootDataURI}"

plotObjectPath="${histoPath}"/>

</aida:region>

</aida:plotter>

</body>

</html>

Application: API

ColManWS (ColMan.jar), deployed in Tomcat, exports EventSelector WSDL.

ColManWSClient uses that WSDL to create everything needed to access ColManWS EventSelector via XML-RPC protocol.

Similar clients could be created in other languages.

Processing currently ends with Pool Tokens; once LCG provides a Web Service for their mapping to files, further processing will be possible. (Maybe Clarens Pool-Light catalog Web Service could be used.)

Stub stub = (new EventSelector_Impl()).getEventSelectorWSPort();

stub._setProperty(javax.xml.rpc.Stub.ENDPOINT_ADDRESS_PROPERTY, webServiceUrl);

EventSelectorWS selector = (EventSelectorWS)stub;

Object[] tokens = selector.select(collectionUrl,

queryString,

user,

password);

for (Object token : tokens) {

System.out.println(token);

}

created automatically from ColManWS WSDL

Application: WWW

ColManWS (ColMan-GUI.jar), deployed in Tomcat, can be accessed directly from the Web Browser.

Almost all ColMan and AIDA operations can be made available.

get all AttributeList NTuples

from an SQL database

(either one database or

cluster of databases

via C-JDBC virtual server)

get Tokens for events satisfying a query

plot selected attributes

Application: GUI

Most SQLTuple/ColMan functionality is available remotely from JAS3 via ColMan plugin using JDBC protocol.

All operations can be performed from GUI or supported (scripting) languages (Java, Python, PNuts,...).

Open

Analyse

Select

Distributed Event Metadata

database

WWW

JAS3

ColMan

WebService

database

database

database

code

JDBC

JDBC

JDBC

JDBC

JDBC

XML RPC

JDBC

JDBC

XML RPC

Event Metadata

(MySQL, PostgreSQL,

Oracle, McKoi,...)

database

database

database

database

database

Collection Metadata

Catalogs

Data

AMI Web Service

Grid Web Service

LCG/Pool Web Service ?

ColMan

Plugin

database

database

Http, Xrootd

Http, Xrootd

Event Metadata

(AIDA XML, Root files)

Clients

Service Discovery Service ?

?

Distributed Event Metadata

Distributed architecture is sometimes claimed to be slow. That is almost always because the wrong technology is used (for example SOAP to access bulk data).

There are many standard protocols for distributed operations: SOAP, XML-RPC, JDBC, RMI, HTTP, Xrootd, etc. The right one should be chosen with respect to its

functionality,

performance,

interoperability.

All functionality should be available using non-proprietary standard protocol.

GAE Clarens (from CMS) already has a lot of that functionality, could we reuse it ? Or maybe ARDA ?

Summary, Links

Various ways of distributed access to Event Metadata are ready to be deployed; new clients can be quickly implemented.

Standard solutions are heavily (re)used, there is very little of new code.

All that works on any Linux, MacOSX, Windows etc. and it doesn't care what gcc & glibc happens to be installed. (Most of it actualy works even on PDA and Mobile Phones.)

More:

ColManWS:

http://home.cern.ch/hrivnac/Activities/Packages/ColManWS

ColManWSClient:

http://home.cern.ch/hrivnac/Activities/Packages/ColManWSClient

AIDATLD:

http://aidatld.freehep.org

JAS3:

http://jas.freehep.org/jas3

SQLTuple, ColMan, AIDA, C-JDBC,... in Dec'04 SW WS presentation.

Presented Web Service is based on ColMan/SQLTuple packages. They have been presented during December'04 Atlas SW WS.

SQLTuple: AIDA interface for SQL databases, compatible with Pool AttributeList. It is based on FreeHEP implementation of AIDA.

http://home.cern.ch/hrivnac/Activities/Packages/SQLTuple

ColMan: Collection Management utilities on top of SQLTuple. It has been used during DC2 Tier0 excercise.

http://home.cern.ch/hrivnac/Activities/Packages/ColMan

*.war files are build using JWSDP. Those files can then be deployed on any standard Web Service Container. Tomcat is currently used.

There is apparantly no public Tomcat server in CERN. Should we install our Atlas Tomcart server ?

It is trivial to create a Web Service client from WSDL interface specification.

JSP is very powerfull system for building dynamic web pages. Any (Java) code can be embedded inside HTML.

http://java.sun.com/products/jsp

AIDATLD is used, for example, by GLAST:

http://web007.slac.stanford.edu:8888/SystemTests/

Currently, only ColMan EventSelector is exported as a Web Service, but other utilities can be included almost immediately.

Pool-Light catalog WS:

http://hepgrid1.caltech.edu/GAE/services/poolCatalog/

...

...

C-JDBC acts as a virtual SQL server hiding the complex network of physical servers behind. It can dynamicaly re-configure databases to optimise access. C-JDBC has been presented during Dec'04 Atlas SW WS.

http://c-jdbc.objectweb.org/

GAE Clarens: Web Services:

http://hepgrid1.caltech.edu/GAE/services/

All presented packages are based on proven solid technologies. There is no reason to re-invent them, they already exist.

Julius Hrivnac

2005-02-18T01:11:24