Table of Contents
Table A.1. Properties used for the replication
Property Name | Where | Type | Default | Comment |
---|---|---|---|---|
db.user | String | null | This is mandatory for both Master and Slave. Specifies the database user name | |
db.password | String | null | This is mandatory for both Master and Slave. It is the password used to login to the database | |
db.url | String | null | This is mandatory for both Master and Slave. It is the url of the database to use | |
dbWatcher.maxCollectedStatements | int | 0 |
Specifying a value here will allow you to send more than one transaction in the same message. This
is useful for cases where performance is a problem. This will speed up replication but can not be used
if transactions occurring on the master have to be reproduced one to one on the slave. If you choose
this option you may choose in your properties (for example in the
xmlBlaster.properties file) the following line:
MimePublishPlugin[ReplManagerPlugin][1.0]=\ org.xmlBlaster.contrib.replication.impl.ReplManagerPluginwhich will allow you to see each transaction in the counters and in the queues. When you choose this plugin you will not see the messages really queued in the counters of your replication monitor. You will see a fake representing each transaction. So for example if you have three messages in the queue each of which containing 3 transactions, you will see 9 entries in the holdback messages column. If you do not choose this plugin you will see the real messages, loosing this way control over the number of real transactions generated on the source. | |
dbWriter.caseSensitive | boolean | false | Is used on the DbWriter to specify if the comparisons made on the properties have to be done case sensitive or not. You may want to set it to false if your databases are casesensitive and you have objects having the same case-insensitive name. In most cases leave this untouched. | |
dbWriter.prePostStatement.class | String | (empty) | This is the name of the class to use as an Implementation of the interface I_PrePostStatement. You can write your own implementation and activate it with this property by specifying the name name of your class if you want business specific code to be executed before and after an INSERT, UPDATE or DELETE statement are executed on the Slave. | |
dbWriter.queryImportedKeys | boolean | true | Setting this value to false will avoid filling the imported key (forein key) information on the writer. Since this info is currently not really used (only for logging and comparison reasons) and since retrieving this information on some databases may be very time consuming it can be switched off. | |
dbWriter.quoteColumnNames | boolean | false | For cases where the name of a column is a reserved keyword an sql statement could fail. Setting this flag to true will wrap/protect the columnname by a queue. | |
dbWriter.sqlPrePostStatement.sql. ${PREPOST}.${OP}.${SCHEMA}.${TABLE} | String | (empty) | This properties take effect only if dbWriter.prePostStatement.class was set to org.xmlBlaster.contrib.replication.SqlPrePostStatement. ${PREPOST} can either be pre or post, ${OP} can either be insert, update or delete. ${SCHEMA} is the schema and ${TABLE} is the table name for which the property must take effect. For each defined property the name of a stored procedure is specified as the value. dbWriter.sqlPostStatement.sql.post.insert.XBL.TEST = DUMMY_PROC would for example mean that for every replicated INSERT operation on the table TEST belonging to the schema XBL the stored procedure DUMMY_PROC (which does not take any arguments) will be executed after (since post) the INSERT operation has taken place but still in the same transaction. This allows to execute business specific code directly invoked by the replication. | |
dbWriter.xslTransformerName | String | (empty) | The name of an optional xsl transformation file. If you specify a file, an xsl transformation of the content of the replication message will be done before any processing done on the slave. Not specifying anything will let the message unchanged. | |
mom.* | String | null | These properties are normally inherited from the DbWatcher. In this context only the major ones are described. For further details please consult the reference documentation for the DbWatcher and the DbWriter . | |
mom.loginName | String | DbWriter/1 | Is used on several plugins. For the DbWriterPlugin we can define a comma separated list of values. For each value found in this list, an own instance of a DbWriter is created. For example DbWriter/1, Dummy/2 will create two instances of a DbWriter. The associated mom.password property can in such a case also be expressed as a list of values as for example secretOne, secretTwo. If you only specify one single password, this will be used for both DbWriter instances. The idea of using a list of values permits us to define a single plugin (with one single configuration) for several slaves. Also there is the possibility to add such slaves via jmx in an hotpluggable fashion. | |
mom.password | String | secret | Is used on several plugins. For the DbWriterPlugin we can define a comma separated list of values. Every entry of the list is mapped to the entries in the list specified in mom.loginName, if less entries are found in the password list, the last value is used for all undefined values. If the number of entries in password is bigger than the number of entries in for the login names, the exceeding passwords are ignored. | |
mom.statusTopicName | String | null | Name of the topic used on the slave to publish status messages. This is currently unused. | |
mom.topicName | String | replication. + replication.prefix | Name of the topic on which to publish the replication messages. This is used on the Master. | |
replication.alters | boolean | true | If set to false entry messages will be ignored and therefore will not be written. This is a slave property | |
replication.bootstrapWarnings | boolean | false | If set to true, warnings are displayed when bootstrapping. Used on the Master. This flag makes it quite noisy, can be used for debugging purposes. | |
replication.charWidth | String | 50 | The number of characters used for the normal columns in the tables | |
replication.charWidthSmall | String | 10 | Number of characters used for columns containing short texts. | |
replication.dbSpecific.class | String | org. xmlBlaster. contrib. replication. impl. SpecificOracle | The name of the implemenetation class for I_DbSpecific to be used. This class contains the code which is specific to a particular vendor. | |
replication.createDropAlterDetection | boolean | true | Used on the master to detect CREATE TABLE, ALTER TABLE and DROP TABLE operations. if this flag is set to 'false' such detections do not occur. | |
replication.creates | boolean | true | Used on the slave to tell if CREATE replication messages have to be executed (true) or ignored (false) | |
replication.drops | boolean | true | Used on the slave to tell if DROP replication messages have to be executed (true) or ignored (false) | |
replication.forceSending | boolean | false |
Normally a check is made for messages to be sent to the slave. If such messages are outdated or if the replication key identifying such messages is not monoton incresing it is normally considered an error and messages are not dispatched. For Multimaster approaches this is not true since the replication keys can come from several Masters. For such cases you can disable the check by passing true here. This will mean that messages which are believed to be wrong under a single master perspective are sent anyway. | |
replication.keepDumpFiles | boolean | false |
If it is set to true, then the file used for the
initial update is not erased after having been processed. This can be useful for debugging purposes.
This flag has only effect if you have a replication.initialCmd string set.
| |
replication.importLocation | String | ${java.io.tmpdir} |
This is used on the Slave. It specifies where the file used for the initial update is temporary
stored. This flag has only effect if you have a replication.initialCmd string set.
| |
replication.initialFilesLocation | String | ${user.home}/tmp | This is the location where the data of an initial update is stored per default in case you choose to transfer this data manually per files. The same value is used also to retrieve the data on the slave side. This is an optional field used on both master and slave. On the master it has to be defined on the ReplManagerPlugin. | |
replication.initial.stringToCheck | String | rows exported | The string to be used to see if the export command in the initial update was
successful. If the string is found, it is assumed to be successful. This is used for external
processes executed within our Master java process does which do not return their error code correcty.
This flag has only effect if you have a replicatin on.initialCmd string set.
| |
replication.initialCmd | String | null | If set the code is executed as a child process. This is normally a batch job. For examle it could be an export (or dump) of a database. Used on both Master and Slave. | |
replication.initialCmd.sleepDelay | long | 10L | The sleep delay in milliseconds to wait when checking for new entries in the log buffer. This has only effect if an initialCmd will be used. Tests show that not using a delay may use unnecessary CPU. This is a tuning parameter. | |
replication.initialDataTopic | String | repl.initialData | If set, the same topic will be used to send the initial data. This slightly saves resources (since the same topic will be reused) and keeps logs cleaner. | |
replication.initialDumpAsXml | boolean | false | If set to true, it will send the initial dump as a (big) xml. This has an additional overhead since the content must be base64 encoded, increasing the size of the initial update by 33 %. It should only be used if you need xml compatibility not only for updates but also for the initial dump. Choosing this flag automatically reduces the real payload of the chunk by 33 %, i.e. the replication.initialDumpMaxSize of the raw content (before base64 encoding and after base64 decoding) will be maximally 66 % of what specified in order to keep the chunk size once encoded more or less the same. | |
replication.initialDumpMaxSize | int | 1048576 | This is the maximum size of the chunks of the initial update dump. If the content of the database export is bigger than this size, it will be sent as multiple chunks, each chunk being maximum the size specified here in bytes. | |
replication.mapper.class | String | org. xmlBlaster. contrib. replication. impl. DefaultMapper | The name of the class to use to make the mapping between source database and destination database. This class can map the name of a schema to another, the name of a table to another or the name of a column to another. | |
replication.mapper.column.* | String | null |
Used to map columns. This is a prefix, not a complete name. You can define several such mapping entries, one for each column you have to map. The syntax for the name of such properties is the prefix replication.mapper.column. added with ${schema}.${table}.${column}. As value you specify the name to be used for that column on the Writer side: only the bare column name without any schema or table name. Used on the Slave. Has only effect if you use the DefautMapper. | |
replication.mapper.schema.* | String | null |
Used to map schemas. This is a prefix, not a complete name. You can define several such mapping entries, one for each column you have to map. The syntax for the name of such properties is the prefix replication.mapper.schema. added with ${schema}. As value you specify the name to be used for that schema on the Writer side.Used on the Slave. Has only effect if you use the DefautMapper. | |
replication.mapper.table.* | String | null |
Used to map tables. This is a prefix, not a complete name. You can define several such mapping entries, one for each table you have to map. The syntax for the name of such properties is the prefix replication.mapper.table. added with ${schema}.${table}. As value you specify the name to be used for that table on the Writer side: only the bare table name without any schema name. Used on the Slave. Has only effect if you use the DefautMapper. | |
replication.maxRowsOnCreate | int | 250 |
The maximum number of rows (entries) to be sent in one single message when a CREATE operation has been
detected. This has only effect if | |
replication.monitor.statusPollerInterval | long | 5000 |
The poller interval for updating the data exposed to the JMX Monitor in ms. Note that this is not the refresh rate of the monitor itself. The refresh rate is specified in the customize.xsl file as the property refresh.rate. The property statusPollerInterval controls a background thread which updates the data exposed. This approach has been choosen because there are properties which synchronize objects when invoking their getter method. This would result in the monitor being blocked while shuch operations are ongoing and in some case it could result in lock up. Worth to know is that there is only one single thread for all slaves keeping this way resource consumption low even when choosing lower values for this property. This is a property which has to be defined on the ReplManagerPlugin. | |
replication.oracle.wipeoutFunctions | boolean | true |
Used only on oracle. On initial updates, before importing the data the Functions of this schema will be deleted if true. | |
replication.oracle.wipeoutExIfConnected | boolean | false |
Used only on oracle. On initial updates, before cleaning up all resources belonging to the schema to wipe out an exception will be thrown if you specify true and there still are users logged in to that schema. If you specify false only a warning will be logged. | |
replication.oracle.wipeoutIndexes | boolean | true |
Used only on oracle. On initial updates, before importing the data the Indexes of this schema will be deleted if true. | |
replication.oracle.wipeoutPackages | boolean | true |
Used only on oracle. On initial updates, before importing the data the Packages of this schema will be deleted if true. | |
replication.oracle.wipeoutProcedures | boolean | true |
Used only on oracle. On initial updates, before importing the data the Procedures of this schema will be deleted if true. | |
replication.oracle.wipeoutSequences | boolean | true |
Used only on oracle. On initial updates, before importing the data the Sequences of this schema will be deleted if true. | |
replication.oracle.wipeoutSynonyms | boolean | true |
Used only on oracle. On initial updates, before importing the data the Synonyms of this schema will be deleted if true. | |
replication.oracle.wipeoutTables | boolean | true |
Used only on oracle. On initial updates, before importing the data the Tables of this schema will be deleted if true. | |
replication.oracle.wipeoutTriggers | boolean | true |
Used only on oracle. On initial updates, before importing the data the Triggers of this schema will be deleted if true. | |
replication.oracle.wipeoutViews | boolean | true |
Used only on oracle. On initial updates, before importing the data the Views of this schema will be deleted if true. | |
replication.overwriteTables | boolean | false | If set to true it will overwrite a table even if a table exists at the time a replication message wants to create the same table again. Per default however it will get an error and the disactivate the delivery of messages to the involved Slave. It is then the responsability of the administrator to remove the error and activate the delivery again. | |
replication.path | String | ${user.home}/tmp |
Tells where to temporary store the data to be sent to the slave during an initial update. This has only
effect if replication.initialCmd was used.
| |
replication.plsql.debug | boolean | false | used to debug the PL/SQL Code in triggers and functions. Used on the Master. | |
replication.plsql.debugFunction | String | null | used to debug the PL/SQL Code in triggers and functions. Used on the Master. | |
replication.prefix | String | repl_ |
Has mainly two purposes: the first is to identify the source of the replication (the Master).
The second purpose is to be used as a prefix for all resources used in the database for the replication.
For example if you specify here DEMO , then the ITEMS table will
be called DEMOITEMS .
| |
replication.prefixGroup | String | ${replication.prefix} | It has the purpose to identify a group of replication sources. This way if you choose to have an own implementation of the monitor you can group the replications the way you want and monitor such groups in different pages. Used on the Master. | |
replication.recreateTables | boolean | false | Setting this property to true will overwrite (i.e. make a drop table) the table when a create operation is replicated. When doing so the complete old content of the table is lost. This flag has only effect if replication.creates is set to true. | |
replication.searchable.* | String | null |
Used to overwrite the default search mechanism used on the slave when updating or deleting entries. If nothing is specified here, when an update or a delete operation is replicated, the entry to update or delete is searched in the destination database. If the table has primary keys defined, these primary keys are the only columns used for the search. If the table has no primary keys, all searchable columns are used in a SELECT .. WHERE ... AND statement. This can be timeconsuming and it does not allow the user to change such columns on the destinations. Here you can define which columns have to be used to search. An example: replication.searchable.XMLBLASTER.TABLE1=COL1,COL2 would update the entry which meets the SELECT XMLBLASTER.TABLE1 WHERE COL1='aaa' AND COL2='bbb' statement (here aaa and bbb are the values to be searched). In the example XMLBLASTER is the schema and TABLE1 is the table name. If such a configuration is specified for a given table, then it is used even if the table has primary keys defined, however a warning is output on its first usage. If you specify a column which is not searchable, a warning is written and the inherent column is not used for the search. If you specify a column which does not exist, a warning is output but the entry is still considered in case the column is added later. The key of the property has the syntax replication.searchable.${schema}.${table} and the value is a comma separated list of column names used for the search. | |
replication.sendInitialTableContent | boolean | true | If true, on a CREATE event, all contents of the table are sent. | |
replication.sendUnchangedUpdates | boolean | true | If false, on an UPDATE event the old content is compared with the new one. If is then only sent if the two differe from each other. | |
replication.sqlMaxEntries | long | 10 | This is the maximum number of Entries in the responses of the SQL Statements. Has to be defined in the ReplManagerPlugin. | |
replication.sqlTopic | String | null | currently unused. | |
replication.statements | boolean | true | If set to true statements are executed on the slave. | |
replication.writer.schemaToWipeout | String | null |
The name of the schema to be wiped out during an initial update. Wiping out the schema is
necessary in cases an import operation takes place in the command specified in
replication.initialCmd . The wipeout is executed just before executing
the initial command. Used on the Slave.
| |
table.* | String | null |
The syntax for the key is table.${catalogName}.${schemaName}.${tableName} where the catalog and the schema are optional while the table name is mandatory. The syntax of the values is actions=${actions},trigger=${triggerName},sequence=${seq} where all arguments are optional. The ${actions} can be a combination of the three characters IDU where I stands for Insert, D for Delete and U for update. These are the actions which have to be replicated. The ${triggerName} is an optional name for the trigger to be added, if you don't specify anything the application will try to assign a valid name for it. Be aware that the length of the trigger name may be limited. Consult the documentation of your DB for this. The ${seq} is an integer telling at which sequence this table will be replicated on initial updates. Low numbers will come first. All tables which have no specified sequence will come in a random sequence (a simple check is made against foreign keys to avoid incosistencies) after the tables for which a sequence number has been explicitly assigned. | |
transformer.class | String | (empty) | An optional class used to make additional transformation to the message to be delivered. It can for example add attributes to the content of the message on a description- or on a row level. Transformations done by this class will affect all destinations of that replication. | |
wipeout.schema | String | null |
Used in the main method of the SpecificDefault class. So it can be run as a standalone application.
If you invoke
java -Dwipeout.schema=XMLBLASTER org.xmlBlaster.contrib.replication.impl.SpecificDefaultit will delete all objects belonging to the XMLBASTER schema. |
: on master
: on slave