This page gives the instructions for deploying the Kafka Event Subscription plugin. Content contains: plugin compilation, Kafka deployment and event subscription examples.

Recommended Configuration

• CPU/ RAM: 16Core / 32G
• DISK: 500G+
• System: Ubuntu / CentOS 64

1. Compile Kafka event plugin

git clone https://github.com/tronprotocol/event-plugin.git
cd event-plugin
./gradlew build

You can find plugin-kafka-1.0.0.zip in the event-plugin/build/plugins/ directory after compilation.

2. Deploy Kafka

Install Kafka

Linux:

cd /usr/local
wget https://downloads.apache.org/kafka/2.8.0/kafka_2.13-2.8.0.tgz
$ tar -xzf kafka_2.13-2.8.0.tgz

Start Kafka

Linux:

cd /usr/local/kafka_2.13-2.8.0
// start the ZooKeeper service
$ bin/zookeeper-server-start.sh config/zookeeper.properties &
// start the Kafka broker service
$ bin/kafka-server-start.sh config/server.properties &

3. Event Subscription

3.1 event subscribe configuration

Add items of 'event.plugin' to the configuration file of your fullnode to support event subscription with Kafka.

3.1.1 Plugin Configurations

event.subscribe = {
  native = {
    useNativeQueue = false// if true, use native message queue, else use event plugin.
    bindport = 5555 // bind port
    sendqueuelength = 1000 //max length of send queue
  }

  path = "" // absolute path of plugin
  server = "" // target server address to receive event triggers
  dbconfig = "" // dbname|username|password
  contractParse = true,
  …………
  …………

}

• The item 'native' contains configurations for TRON's built-in message queue. Please always keep 'useNativeQueue' as 'false' to subscribe events with Kafka.
• 'path' is the local path for 'plugin-kafka-1.0.0.zip'. Please ensure the path is correct.
• 'server' is Kafka's server address in the format of IP:port, default port is 9092. Please use the correct port number and ensure that the Kafka service is accessible.
• 'dbconfig' is for MongoDB. Ignore it.

3.1.2 Event Subscription Configuration Items

TRON Event Subscription supports seven types of events. They are: 'block', 'transaction', 'contractevent', 'contractlog', 'solidity', 'solidityevent' and 'soliditylog'.

Subscription of too many topics will cause performance degradation. Please subscribe 1-2 types of events according to your needs.

Take the block event subscription as an example:

topics = [
    {
      triggerName = "block" // block trigger, the value can't be modified
      enable = true
      topic = "block" // plugin topic, the value could be modified
    }
]

• triggerName: Built-in field. Can't be changed.
• enable: Set as 'true' to subscribe the event of 'block'
• topic: This field corresponds to the pre-created topic in Kafka. The topic name can either be default or customised.

The item of 'filter' works as the filter for subscribed events. For accurate subscription, according to your needs, you can customise the block range(fromblock ~ toblock), contract address(contractAddress) and specific contract topic(contractTopic) .

filter = {
    fromblock = "" // the value could be "", "earliest" or a specified block number as the beginning of the queried range
    toblock = "" // the value could be "", "latest" or a specified block number as end of the queried range
    contractAddress = [
      "" // contract address you want to subscribe, if it's set to "", you will receive contract logs/events with any contract address.
    ]

    contractTopic = [
      "" // contract topic you want to subscribe, if it's set to "", you will receive contract logs/events with any contract topic.
    ]
}

3.2 Create Subscription Topics in Kafka

The name of the Kafka subscription topic should be consistent with the configuration item in 3.1.2. E.g., To subscribe event 'block', please set 'topic' as 'block' in 'block trigger' and create topic 'block' in Kafka to receive 'block' events.

Linux:

bin/kafka-topics.sh --create --topic block --bootstrap-server localhost:9092

3.3 Start an Event Subscription Node

After completing the above configuration, please start your fullnode with '--es' to turn on the event subscription.

java -jar FullNode.jar -c config.conf --es

Check tron.log to see whether the Kafka Event Plugin has been successfully loaded.

grep -i eventplugin logs/tron.log

The loading is successful when this reminder displayed:

[o.t.c.l.EventPluginLoader] 'your plugin path/plugin-kafka-1.0.0.zip' loaded

3.4 Event Subscription Queries

Execute kafka-console-consumer.sh script to get messages of topic 'block' in Kafka.

Linux:

bin/kafka-console-consumer.sh --topic block --from-beginning --bootstrap-server localhost:9092

You have successfully subscribed to the events when the following appears:

{
	"timeStamp": 1539973125000,
	"triggerName": "blockTrigger",
	"blockNumber": 3341315,
	"blockHash": "000000000032fc03440362c3d42eb05e79e8a1aef77fe31c7879d23a750f2a31",
	"transactionSize": 16,
	"latestSolidifiedBlockNumber": 3341297,
	"transactionList": ["8757f846e541b51b5692a2370327f4b8031125f4557f8ad4b1037d4452616d39", "f6adab7814b34e5e756170f93a31a0c3393c5d99eff11e30271916375adc7467", ..., "89bcbcd063a48ef4a5678a033acf5edbb6b17419a3c91eb0479a3c8598774b43"]
}
…………