Infinispan HotRod Translator

The Infinispan HotRod Translator, known by the type ispn-hotrod, can read the java objects from a remote JDG Cache via the Hot Rod client using the Google Protobuf for serialization. This will enable Teiid to query the remote cache using JDG DSL. This translator extends the Object Translator and uses it for the core processing of reading and writing objects.

If you are using JDG in library mode, see the Infinispan Library Mode Translator for this type of configuration.

Usage

  • Retrieve objects from a cache and transform into rows and columns.

  • Supports performing writes to the cache

  • Use for external materialization to improve query performance

Supported Capabilities

The following are the connector capabilities:

  • Compare Criteria - EQ

  • Compare Criteria Ordered - LT, GT, LE, GE - support for SupportsComapareCriteriaOrdered will be controlled by the version of JDG being accessed. Any JDG version 6.5 and prior will have this set to false due to an issue with JDG.

  • And/Or Criteria

  • In Criteria

  • Like Criteria

  • Order By

  • INSERT, UPDATE, DELETE (non-transactional)

The following will not be pushed down to JDG for processing, but will be done within Teiid:

  • Not (NE, <>)

  • IsNull

Limitations:

  • support for 'Not' has been disabled, because JDG considers <column when null> <> 1 to be true and SQL does not.

  • boolean data type: JDG will throw an exception if no value is specified on the insert or when no default value is defined in the protobuf definition file.

  • char data type: is not a supported type in theProtobuf data types (https://developers.google.com/protocol-buffers/docs/proto#scalar). Would either have to handle conversion in the protobuf marshaller or create a Teiid view with the data type as char.

  • 1-to-Many, currently only supports Collection or Array, not Map’s'

  • XA transactions are not supported by JDG when using Hot Rod client.

To be done:

  • support deleting containing class(s) (i.e., person -→ phones, delete a phone)

  • support updating containing class(s)

Installation

The ispn-hotrod translator is not configured, out-of-the-box, in the standalone-teiid.xml configuration. To configure the translator, run the add-infinispan-hotrod-translator.cli script. This script can also be found in the teiid-jboss-dist.zip kit, under docs/teiid/datasources/infinispan. See the jdg-remote-cache quick start for an example. Note, this assumes you will be installing a separate JDG server, which will be accessed via the Hot Rod Client installed into the Teiid server.

Metadata

Options for Defining

There are several options to defining the metadata representing your object in the cache.

  • "Recommended" Use the Teiid Connection Importer in Teiid Designer to create the physical source model based on your object cache. The table columns will be created from the google protobuf definition, that corresponds to a registered class.

  • Use Teiid Designer to manually create the physical source model based on your object cache using the below Definition Requirements.

  • A simple VDB that only defines the data source to use. Example:

    <model name="People" type="Physical">
        <property name="importer.useFullSchemaName" value="false"/>

        <source name="infinispan-hotrod-connector" translator-name="ispn-hotrod" connection-jndi-name="java:/infinispanRemoteDSL" />
    </model>

The metadata will be resolved by reverse engineering the defined object in the cache. This can be useful when using the Teiid Designer Teiid Connection Importer for building the physical source model(s).

  • You can also define the metadata using DDL. See Object Translator for an example.

Definition Requirements

  • see Object Translator Metadata section for base definition requirements.

  • Columns will be identified as SEARCHABLE if either the protobuf definition for a column indicates its indexed or the pojo class has the attribute/method annotated.

  • A 1-to-* relationship class must have a foreign key to map to the root class/table, where the name in source for the foreign key is the name of the root class method to access those child objects. Note, this is the class method, not a reference in the google protobuf definition.

  • A container/child class will have attributes where the NIS contain a period. Example: phone.number. This is because this maps to to google protobuf definition and what is expected by the DSL query to reference attributes of a inner class.

External Materialization

This translator supports using the cache for external materialization. However, there are specific configuration settings that are required on the JDG HotRod Resource Adapter and at the translator.

Note

Creating or selecting a view to materialize:

Unfortunately for protobufs, complex data types don’t convert to C / C++. So when you are creating a view to use for materialization, it is recommended to stick with primitive data types because protobuffer doesn’t support bigdecimal directly. So there are 3 options:

  1. use all primitive data types

  2. implement a marshaller that will handle the conversion, which means the .proto file will also need to be created (see JDG for creation of files)

  3. create a view that will convert the bigdecimal to a string, then materialize that view.

Native Queries

External materialization is enabled by the use of native queries in the BEFORE_LOAD_SCRIPT and AFTER_LOAD_SCRIPT. A translator override will need to be set to enable native queries: SupportsNativeQueries=true

The following materialization properties must be defined:

Script Native query Description

teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT

truncate cache

To truncate the cache identified as the staging cache

teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT

swap cache names

To swap the aliases for the caches, so that the primary cache points to the recently loaded cache

The following is an example of for defining the load scripts in DDL:

..
"teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'execute StockMatCache.native(''truncate cache'');',
"teiid_rel:MATVIEW_LOAD_SCRIPT" 'insert into StockMatCache.Stock (productId, symbol, price, companyName) SELECT  A.ID, S.symbol, S.price, A.COMPANY_NAME FROM Stocks.StockPrices AS S, Accounts.PRODUCT AS A WHERE S.symbol = A.SYMBOL',
"teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT"  'execute StockMatCache.native(''swap cache names'');',

Native queries are used to simulate how its done using RDBMS and renaming tables, because Infinispan doesn’t currently support renaming a cache. So the native queries will trigger the clearing of the "staging" cache, and the swapping of the cache aliases.

Direct Query Procedure

Additionally, the execution of native queries is done thru the support of direct query procedures. The procedure to be executed is called native.

Warning
 This feature is turned off by default because of the security risk this exposes to execute any command against the source. To enable this feature, override the execution property [Override Execution Properties] called SupportsDirectQueryProcedure to true.

Metadata Requirements

If you manually model the cache table in Teiid Designer, then you will need to add the property extension for defining the property "primary_table". The following is a DDL example:

SET NAMESPACE 'http://www.teiid.org/translator/object/2016' AS n0;

CREATE FOREIGN TABLE Trade (
         ....
	CONSTRAINT PK_TRADEID PRIMARY KEY(tradeId)
) OPTIONS (UPDATABLE TRUE);

CREATE FOREIGN TABLE ST_Trade (
        ....
) OPTIONS (NAMEINSOURCE 'Trade', UPDATABLE TRUE, "n0:primary_table" 'ObjectSchema.Trade');

JCA Resource Adapter

See JDG HotRod Data Sources resource adapter for this translator. It can be configured to lookup the cache container via JNDI, server list, or hot rod properties.

results matching ""

    No results matching ""