knowledge For All: inbound

Showing posts with label inbound. Show all posts

WSO2 ESB Feed Inbound Endpoint

Atom 1.0

Atom is the name of an XML-based Web content and metadata syndication format, and an application-level protocol for publishing and editing Web resources belonging to periodically updated websites. All Atom feeds must be well-formed XML documents, and are identified with the application/atom+xml media type.

General considerations:

All elements described in Atom must be in the http://www.w3.org/2005/Atom namespace.
All timestamps in Atom must conform to RFC 3339.
Unless otherwise specified, all values must be plain text (i.e., no entity-encoded html).

Sample Atom Feed

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
        <title>Example Feed</title>
        <subtitle>Insert witty or insightful remark here</subtitle>
        <link href="http://example.org/"/>
        <updated>2003-12-13T18:30:02Z</updated>
        <author>
                <name>WSO@ Inc</name>
                <email>WSO@@wso2.com</email>
        </author>
        <id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>
        <entry>
                <title>Atom-Powered Robots Run Amok</title>
                <link href="http://example.org/2003/12/13/atom03"/>
                <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
                <updated>2003-12-13T18:30:02Z</updated>
                <summary>Some text.</summary>
        </entry>
  
</feed>

RSS 2.0

RSS is a Web content syndication format. Its name is an acronym for Really Simple Syndication. RSS is dialect of XML. All RSS files must conform to the XML 1.0 specification, as published on the World Wide Web Consortium (W3C) website.At the top level, a RSS document is a <rss> element, with a mandatory attribute called version, that specifies the version of RSS that the document conforms to. If it conforms to this specification, the version attribute must be 2.0. Subordinate to the <rss> element is a single <channel> element, which contains information about the channel (metadata) and its contents.

Sample RSS Feed

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
        <channel>
                <title>Example Feed</title>
                <description>Insert witty or insightful remark here</description>
                <link>http://example.org/</link>
                <lastBuildDate>Sat, 02 AUG 2015 18:30:02 GMT</lastBuildDate>
                <managingEditor>WSO2@wso2.com (WSO@ Inc)</managingEditor>
                <item>
                        <title>Atom-Powered Robots Run Amok</title>
                        <link>http://example.org/2003/12/13/atom03</link>
                        <guid isPermaLink="false">urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</guid>
                        <pubDate>Sat, 02 AUG 2015 18:30:02 GMT</pubDate>
                        <description>Some text.</description>
                </item>
        </channel>
</rss>

What is Feed Inbound

WSo2 Feed Inbound Developed Based on Apache Abdera to Consume the Feeds. and inbound filter the feed and its allow only selected Elements of Feed into ESB. We can Configure the time interval to check to new feed updates after it will update to ESB if any new feed Update in given site. for the input URL Atom or RSS but out output will be in Atom format because Atom is the well structured format.

Feed Inbound Sample Configuration

Feed Inbound Configuration

<inboundEndpoint class="org.wso2.carbon.inbound.feedep.FeedEP"
     name="feed" onError="Fault" sequence="TestIn" suspend="false">
     <parameters>
         <parameter name="feed.type">Atom</parameter>
         <parameter name="interval">10000</parameter>
         <parameter name="feed.url">http://news.google.co.in/news?cf=all&amp;hl=en&amp;pz=1&amp;ned=in&amp;output=atom</parameter>
     </parameters>
 </inboundEndpoint>

Feed Inbound Sample Output

Feed Inbound Output

<feed xmlns="http://www.w3.org/2005/Atom">
    <entry>
        <title type="text">Wso2 ESB Feed Inbound Test Title</title>
        <updated>2015-08-31T16:29:00.000Z</updated>
        <content type="text">Feed Inbound Testing
        <link rel="self" href="/feed" />
            <author>
                <name>WSO Inc</name>
            </author>
        <id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>
        </content>
    </entry>
</feed>

Elements of <feed>

A Feed consists of some meta data, followed by any number of entries.

Required feed elements

Here’s a list of the required feed elements, each with a brief description, and an example.

Element

Description

Identifies the feed using a universally unique and permanent URI. If you have a long-term, renewable lease on your Internet domain name, then you can feel free to use your website’s address.

<id>http://example.com/</id>

title

Contains a human readable title for the feed. Often the same as the title of the associated website. This value should not be blank.

<title>Example, Inc.</title>

updated

Indicates the last time the feed was modified in a significant way.

<updated>2003-12-13T18:30:02Z</updated>

Recommended feed elements

Atom makes a number of additional requirements and recommendations for feed elements that you should to be aware of. They are as follows:

content : Contains or links to the complete content of the entry. Content must be provided if there is no alternate link, and should be provided if there is no summary. More info here.

<content>complete story here</content>

author : Names one author of the entry. An entry may have multiple authors. An entry must contain at least one author element unless there is an author element in the enclosing feed, or there is an author element in the enclosed source element. More info here.

            <author>
               <name>WSO2 Inc</name>
            </author>

link : Identifies a related Web page. The type of relation is defined by the rel attribute. An entry is limited to one alternate per type and hreflang. An entry must contain an alternate link if there is no content element. More info here.

<link rel="alternate" href="/feed/1234"/>

Getting Started with Simple WSO2 ESB Custom Inbound Endpoint

WSO2 ESB supports several inbound endpoints, but there can be scenarios that require functionality not provided by the existing inbound endpoints. For example, you might need an inbound endpoint to connect to a certain back-end server or vendor specific protocol.

To support such scenarios, you can write your own custom inbound endpoint by further extending inbound polling or inbound listening. let's Start to build a simple sample Custom Inbound endpoint .

you can download sample source code from here

below one is simple custom inbound skeleton we can implement our own inbound top of this code.

package org.wso2.carbon.inbound.custom.poll;

import java.util.Properties;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.synapse.core.SynapseEnvironment;
import org.wso2.carbon.inbound.endpoint.protocol.generic.GenericPollingConsumer;

public class SamplePollingClient extends GenericPollingConsumer{

 private static final Log log = LogFactory.getLog(SamplePollingClient.class);
 
 /**
    * @param properties
    * @param name
    * @param synapseEnvironment
    * @param scanInterval
    * @param injectingSeq
    * @param onErrorSeq
    * @param coordination
    * @param sequential
    */
   public SamplePollingClient(Properties properties, String name,
                              SynapseEnvironment synapseEnvironment, long scanInterval,
                              String injectingSeq, String onErrorSeq, boolean coordination,
                              boolean sequential) {
    super(properties, name, synapseEnvironment, scanInterval, injectingSeq, onErrorSeq,

                 coordination, sequential);
    log.info("Initialized the custom polling consumer.");
   }

   @Override
   public Object poll() {
    //TODO need to implement the logic here
    log.info("Inside the execute method.");
    return null;
   }

    /**
     * Stopping the inbound endpoint
     */
    public void destroy() {
        //TODO need to implement the logic here
        log.info("Inside the destroy method, destroying the polling inbound ...");
    }
}

after download the source code Now, Build the code you will get the .Jar file copy file and paste it into dropins folder(/../../wso2esb-4.9.0/repository/components/dropins). then Restart the ESB

SamplePollingClient constructor will initialize and setup the running environment and this is the method to set pre-requirements for our custom inbound.

poll is the method we will implement our logic and requirement we have to do. inbound will call this method is a given time interval.

destroy is the method will contain the logic’s and functions to clean and free the resources which inbound used while its in active (clean the Registry, etc)

Sample Configuration through Proxy

<inboundEndpoint name="class" sequence="request" onError="fault"
                            class="org.wso2.carbon.inbound.custom.poll.SamplePollingClient" suspend="false">
   <parameters>
      <parameter name="sequential">true</parameter>
      <parameter name="interval">2000</parameter>
      <parameter name="coordination">true</parameter>
   </parameters>
</inboundEndpoint>

Output

===================

Initialized the custom polling consumer.

Inside the execute method.

Inside the execute method.

Inside the execute method.

Inside the execute method.

Inside the execute method.

.................

.....

...

when destroying the inbound

Inside the destroy method, destroying the polling inbound ...

References

WSO2 ESB Inbound Endpoint

Inbound Endpoint is an endpoint which resides in server side of the ESB. Those are introduced to achieve some of the main features like multi tenancy support for some protocols like JMS, by passing axis2 layer in Inbound side,deploying and undeploying inbound endpoints without server restarting.

Basically there are two behaviours introduced. Selection of the behaviour is done according to protocol.

Polling behaviour
Listening behaviour

Polling behaviour

This behaviour is in JMS, File and Generic Inbound Endpoints. Basically it polls periodically for data with some Conditions and if Conditions is ok then injects that data to given sequence. for example in JMS inbound Endpoint it checks for JMS Queue periodically whether the data is available if so then take that message and inject it to given sequence.So this kind of inbound endpoints have one way operation. It does not require a result so in nature these are asynchronous.

Listening behaviour

This behaviour is in HTTP and CXF-RS inbound endpoints. Basically listener is started in given port and it listens for requests coming to that port. When request is available it injects that request to the given sequence.Here those have two way operation and in nature those are synchronous.

Inbound Endpoint Configuration

<inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"

name="feedep"

sequence="TestIn"

onError="fault"

protocol="http"

suspend="false">

</parameters>

</inboundEndpoint>

According to the above configuration generic parameters for inbound endpoints are added as attributes in inboundEndpoint element and protocol specific parameters are given in the parameters element.

Thread Pools in InboundEndpoint

We need threadpool to execute message injectors to the sequence. Message Injectors are handle by thread pool which avoids blocking in the message consumers from the source or listeners for the source.

There are three options that can be taken.

Per inbound thread pool.
Protocol specific thread pool.
One thread pool for all inbound endpoints.

currently we are having thread pool per inbound and we need to make it configurable.

Multi Tenancy in InboundEndpoint

Multi Tenancy means more tenants are operated in single physical location with logically separated manner. So each tenant can have more inbound endpoints. so when introducing this there are major problems that we need to solved. Basically one way protocols like JMS, File and Generic are run top of the task since tasks are handling multi tenancy we don't need to separately handle it.

But in Listening endpoints like Http and CXF-RS need to handle multi tenancy. So basic problem that we faced is port sharing and tenant unloading. we need to do port sharing between tenants because number of ports are limited.When tenant loading and unloading synapse environment also get loaded and unloaded. So we keep common handler and Listener always running and in runtime we get the synapse environment and get inbound endpoints through it and do the relevant operations.

Clustering

This is achieved by using hazelcast framework and distributed locks.

Reference