Java RootIO

Implementation:

Reading

Writing

Xrootd

Tools:

Root Object Browser

AIDA Tools

Web Interface

ColMan

Made by:

SLAC: Tony Johnson, Mark Donszelmann, Victor Serbo, Max Turri

LAL: Julius Hrivnac

Prague: Petr Vokac

J.Hrivnac/LAL for SGSD WS in Lyon, Jan'05

in memory

Implementation

Interfaces and Proxies are generated dynamically at run-time using BCEL (ByteCode Engineering Library). They can be saved if needed.

Streamers use java.nio (high-performance IO) to do actual reading.

No external knowledge about Root file content is needed.

No C++ is needed.

Some foundation Root classes are not described in TStreamInfo, or are described incorrectly. Their Interfaces and Proxies have been pre-implemented.

12k lines of code (Rio: 15k, Root: 200k).

Root File

TStreamInfo

Data

Generator

Interface

Accessor methods for each data member

Proxy

Custom version based on how object is stored in file

Streamer

in files

generates

reads

implements

writes

(optional)

Problems

Root file format is completely undocumented so a lot of archaeological work is needed. Every new Root release brings surprises.

System is quite robust against changes.

Each problem is included as a test file. Tests currently exist for Root versions 3.00.06, 3.01.06, 3.02.07, 3.03.07, 3.05.07, 3.10.02, 4.00.08.

LCG Pool has “invented” another private format of streaming information (apparently based on Seal dictionary). This has not yet been decrypted so Pool files are unreadable, except for trivial cases (like AttributeList, etc.).

Example of Use

Benchmarks

Task: Loop over all events, loop over all tracks, sum px for all tracks. 5000 events files created with “event” test program distributed with Root.

Java 1.4 on MS Windows used.

S = Split level = 0,1,2

C= Compression = 0,1

Each test run 5 times in succession, lines show range in results

in later runs file is typically cached in memory == faster if task IO limited.

Red = Root

Green = Root (coded for split)

Black = Java

Writing

Standalone application (written by student P.Vokac).

It will be merged into FreeHEP.

Limitations:

Can't create new StreamInfo entry – can only write objects of classes which has been read from another file.

Even if it could create new StreamInfo entries – those may not be acceptable by Root.

Has problems with TTrees.

Good enough for packages like AIDA.

Root File

TStreamInfo

Root File

TStreamInfo

Writer

Data

reads

to get

TStreamInfo

writes

with

new Data

Xrootd

Ability to talk directly to rootd/xrootd server.

Authenticator to authenticate to rootd (not all authentication methods implemented yet).

Extended RootFileReader to accept as URL:

file:* to read directly files

root:* to access rootd/xrootd (random access)

http:* to access http server (use “ranges” if supported by server)

Root Object Browser

Allows to visit data as well as StreamInfo.

AIDA on Root Files

JAIDA (FreeHEP implementation of AIDA) provides AIDA interface for Root NTuple/Histogram/... files.

Can be used directly, in Root Histogram Browser or within JAS3.

AIDATLD

AIDA Tag Library for use in JSP (Java Server Pages) to put AIDA on the Web.

ColMan

Allows management of (LCG Pool) AttributeLists (Tag databases – Event Metadata) between SQL tables (using SQLTuple), AIDA XML files and Pool Root files.

Can be used as JAS3 Plugin.

Has been used in Atlas DC2.

Athena Job

Metadata

(Pool Root File)

per job

Events

(Pool Root File)

ColMan

Metadata

(Oracle/MySQL)

per physics stream

Metadata

(Oracle/MySQL)

per physics stream

Metadata

(Oracle/MySQL)

per physics stream

Athena Job

Athena Job

Events

(Pool Root File)

Events

(Pool Root File)

Metadata

(Pool Root File)

per job

Metadata

(Pool Root File)

per job

split

and

merge

ColMan

analyse

More Info

FreeHEP Java Library:

http://java.freehep.org

FreeHEP Root IO:

http://java.freehep.org/lib/freehep/doc/root

JAIDA:

http://java.freehep.orgt/jaida

AIDATLD:

http://aidatld.freehep.org

JAS3:

http://jas.freehep.org/jas3

ColMan:

http://home.cern.ch/hrivnac/Activities/Packages/ColMan

Julius Hrivnac

2005-01-30T12:36:42