SQL Replication

Four utilities are under development/evaluation:

ColManReplicator – part of ColMan, ready to replicate Geometry database.

Generic Replicator – standalone, ready to replicate Geometry database.

JDBC Importer – SourceForge, under evaluation.

Enhydra Octopus – ObjectWeb, under evaluation.

All these utilities support practicaly any Relational Database (as they use standard JDBC API).

J.Hrivnac (LAL/Orsay) for Atlas SW WS, Sep'04

http://home.cern.ch/hrivnac/Activities/2004/September/Replication

ColManReplicator

ColManReplicator is a simple utility (class) based on SQLTuple.

SQLTuple can interpret SQL tables as AIDA NTuples (read/write) as long as tables use only types supported by AIDA (which is true for Geometry database). Default type mapping can be customized. Complex conversions are not supported.

SQLTuple currently supports MySQL, PostgreSQL, McKoi and Oracle. Other backends can be easily added. SQLTuple works over the network.

SQLTuple allows also conversion from/to AIDA XML files. This can be useful for input/modification of data.

SQLTuple introduces certain overhead due to supported AIDA functionality (not needed for replication).

java -jar <AtlasExternal>/Colman/ColMan-0.9.6/lib/ColManReplicator.exe.java

jdbc:oracle:thin:@oradev9.cern.ch:1521:D9

jdbc:mysql://<mysql server>/<database>

<oracle id> <oracle passwd> <mysql id> <mysql passwd>

tables.txt

Fallback

Generic Replicator

sql2xml

replicator

SQL DB

SQL DB

SQL Schema (XML)

SQL Schema (XML)

Replication Configuration (XML)

sql2xml

configurator

Conversion Scripts (Groovy)

<database url=”...”>

<table name=”...”>

<column name=”...”>

<technology name=”MySQL” type=”float”/>

<technology name=”Oracle” type=”real”/>

<technology name=”McKoi”>

<from>

<!-- script to convert column from Mckoi -->

</from>

<to>

<!-- script to convert column to McKoi -->

</to>

</column>

<table name=”...”>

<technology name=”...”>

<from>

<!-- script to convert table from ... -->

</from>

<to>

<!-- script to convert table to ... -->

</to>

</technology>

...

</table>

...

</database>

All scripts use Groovy for its direct support of SQL and XML.

Scripts sql2xml, configurator and replicator exist.

Conversion scripts are not yet supported.

Already functional enough for Geometry database replication between Oracle and MySQL.

Default

script

database

configuration/schema file

JDBC Importer

SourceForge utility for flexible importing/exporting of SQL databases.

Importing/exporting rules are described by XML configuration files.

Format of intermediate file can be customised (XML, CSV and TXT are used on default).

JDBC Importer supports not only direct column-by-column copy of databases, but also sophisticated conversions involving manipulating several tables. Simple conversions can be described directly by XML configuration files, more complex conversions can be described by ExportEngine subclasses.

http://jdbcimporter.sourceforge.net

Evaluation

Enhydra Octopus

ObjectWeb utility for flexible database replication.

It can copy, synchronise, backup and modify SQL databases.

Configuration and mapping is customisable via XML files.

Runs from GUI, command-line, Ant and Java.

Replicates in two steps using intermediate files (in different formats): Generation and Loading.

Looks like exactly what we need !

http://octopus.objectweb.org

Evaluation

Enhydra Octopus

Evaluation

Julius Hrivnac

2004-09-17T12:11:44

Julius Hrivnac