Monday, March 16, 2015

How to Publish Your own Events (Data) to WSO2 Analytics Platform (BAM, CEP)

We collect data via a Sensor API (a.k.a. agents), send them to servers: WSO2 CEP and WSO2 BAM, process them, and do something with the results. You can find more information about the big picture from the slide deck http://www.slideshare.net/hemapani/introduction-to-large-scale-data-analysis-with-wso2-analytics-platform

This post describes how you can collect data.

We provide a one Sensor API to publish events for both batch and realtime pipelines. The Sensor API is available as Java clients (Thrift, JMS, Kafka), java script clients* ( Web Socket and REST) and 100s of connectors via WSO2 ESB. 

Lets see how we can use the java thrift client to publish events. 

First of all, you need CEP or BAM running. Download, unzip, and run WSO2 CEP or WSO2 BAM (via bin/wso2server.sh). 

Now, lets write a client. Add the jars given in Appendix A or add POM Dependancies given in Appendix B to your Maven POM file to setup the classpath. 

The Java client would look like following. 




Just like you create tables before you put data into a database, first you define streams before sending events to WSO2 Analytic Platfrom. Streams are a description of how your data look like (a.k.a. Schema). Then you can publish events. In the code, the "Event Data" is an array of objects, and it must match the types and parameters given in the event stream definition.

You can find an example client from /samples/producers/pizza-shop from WSO2 CEP distribution. 


Appendix A: Dependancy Jars

You can find the jars from the location ${cep.home}/repository/components/plugins/ of CEP or BAM pack.

  1. org.wso2.carbon.logging_4.2.0.jar
  2. commons-pool_1.5.6.*.jar
  3. httpclient_4.2.5.*.jar
  4. httpcore_4.3.0.*.jar
  5. commons-httpclient_3.1.0.*.jar
  6. commons-codec_1.4.0.*.jar
  7. slf4j.log4j*.jar
  8. slf4j.api_*.jar
  9. axis2_1.6.1.*.jar
  10. axiom_1.2.11.*.jar
  11. wsdl4j_1.6.2.*.jar
  12. XmlSchema_1.4.7.*.jar
  13. neethi_*.jar
  14. org.wso2.securevault_*.jar
  15. org.wso2.carbon.databridge.agent.thrift_*.jar
  16. org.wso2.carbon.databridge.commons.thrift_*.jar
  17. org.wso2.carbon.databridge.commons_*.jar
  18. com.google.gson_*.jar
  19. libthrift_*.jar

Appendix B: Maven POM Dependancies 

Add the following WSO2 nexus repo and dependancies to pom.xml at corresponding sections. 




daily
ignore
wso2-nexus
http://maven.wso2.org/nexus/content/groups/wso2-public/

org.wso2.carbon
org.wso2.carbon.databridge.agent.thrift
4.2
org.wso2.carbon
org.wso2.carbon.databridge.commons
4.2



  

Post a Comment