Quartz Scheduler advanced features

This is the second part of Quartz Scheduler framework.In the first part we saw the basic features.
In this blog we are going to focus on some advanced feature required features like (Exception management, Clustering, Transaction Management) which may be used by the enterprise products.
  Terracotta acquired Quartz company .Previously they have acquired Ehcache.




TerracottaJobStore is used to store scheduling information (job, triggers and calendars) within a Terracotta server.

TerracottaJobStore is much more performant than utilizing a database for storing scheduling data (via JDBC-JobStore), and yet offers clustering features such as load-balancing and fail-over.
 

Configuration for TerracottaJobStore

org.quartz.jobStore.class = org.terracotta.quartz.TerracottaJobStore
org.quartz.jobStore.tcConfigUrl = localhost:9510

Job Exception Handling :

 In case of exception occurs during Job Execution we can have various options like firing

the Job again or we may choose for deleting the scheduling the job all the  

  JobExecutionException e2 = 
                new JobExecutionException(e);

    Unscheduling:    e.setUnscheduleAllTriggers(true);

    Refirring : e.refireImmediately();

   gracefull shutdown : sched.shutdown(true);

Clustering Support:

•  High Availability - We need to be able to restart your application without losing scheduled jobs
•     Scale Out - Your application now needs more than one node to handle the load it receives and you want your scheduled jobs to distribute across nodes.
•     You are using a database to persist and or scale your scheduler and you are seeing DB load/locking issues and or you find it two difficult to setup or maintain. 
• 
  

Transaction Support:
Jobs can be configuured in JTA Transaction. For this following property needs to be set true.
org.quartz.scheduler.wrapJobExecutionInUserTransaction" -=true
This ensures that whenever a Job execute method starts 
Transaction begin()  method will be called by the scheduler and on completing the execute method commit() will be called.

Pricing:
Quartz provides these enterprise features like clustering, JTA support free under apache licensing.

Why do we need Hadoop?

In this Blog we are going to talk about Hadoop. Question which comes in our mind are :

1.  Why do we need Hadoop?
2.  Do we need this when we already have tools for data processing and Query languages SQL, Integtation platform,BI Tools like Informatica?
   

Motivation:
Compnies like Facebook , Google, Yahoo are datacentric company. According to  the statistics, 1800 Hexabytes data get stored everyday universally.And out of these 20% only are stuctured data where as 80% of them are non structured in other word they are unorganized.

There is no software to process unstructured data. Every day the data is increasing and we require an inexpensive and relaible storage mechanism to store bulk amount of data and to process this huge amount of data. 
The Ideal solution for the above requirement is to use Hadoop.

Use Cases:
The requirements are nt only for Big Companies but this will be required by the Small and Medium sized Compnies as well.For example consider some use cases like we want to

• Analyse daly logs generated for over a period of a month or year
• process various image data uploaded by diffenet user 
• Process unstructured  data
• Image processing.
• Search opeartion on  text, video, log data 
• Analyse all the log in and log out operation happened over a year

Hadoop is 

• An open source MapReduce implementation
• Works on cluster
• Have fault tolerancecapacity:Even if one out of 1000 nodes fails, It will make a note of this and will give to abvailable nodes.
• Very much scalable:You can add any number of nodes at any point of time
• Very flexible to adpat any changes

 Please note that Hadoop undesrtsnds only two things:
 -HDFS
-Map Reduce 

Will discuss nore in details about how does It work the next blog Part II on Hadoop .

Scheduler framework in Enterprise application

Many times Enterprise product has scheduling requirements where same type of tasks/Job   need to be executed again and again. For this we rely on some third party framework. There are different scheduling framework available for use and few of them are below

• Spring Task Scheduler
• Quartz
• jcrontab
•  Fulcrum Scheduler
• Essiembre J2EE Scheduler
• Gos4j
• Oddjob
• Job Scheduler
• JDRing
• jBatchEngine
• MyBatchFramework
• Super
• cron4j

 In this blog we are going to first compare with the Spring Task Scheduler then discuss Quartz Scheduler which is a Enterprise Job Scheduler Framework from project from Opensymphony .

When to use Spring Task Scheduler?  -when our requirements are 'Simple' for eg.

    -Run tasks once after specified time
    -Schedule tasks to run repeatedly (fixed-rate as well as fixed delay)
    -Scheduling tasks based on simple cron expressions, etc.

On the other hand, when the requirements  are complex say the framework supports enterprise-class features like Persistent triggers, Clustering, JTA Transactions, etc. – Quartz scheduler is what you need

 Why to go for Quartz?  

Quartz is very light-weight, and requires very little setup/configuration - it can actually be used 'out-of-the-box' if your needs are relatively basic.Quartz is fault-tolerant and can persist our  scheduled jobs between system restarts.

Quartz is free for use silenced under Apache Licensing.

 Types of  Quartz Jobs:

• Simple Quartz Job
• Simple triggers
• Cron triggers

Job stores:

• RAMJobStore: This has volatile storage of the quartz job details.
• JDBCJobStore: It  maintains the quartz job details over a database via the JDBC. Here the information on the quartz jobs, triggers, calendars etc are available any time in case the system has a downtime and then can be rescheduled once the system is up. We can reschedule the Job. 

Cron expressions consist of the following seven fields:

• Seconds
• Minutes
• Hours
• Day-of-month
• Month
• Day-of-week
• Year (optional field)

The backslash (/) character denotes value increments. For example, "5/15" in the seconds field means every 15 seconds starting at the fifth second.

The question (?) character and the letter-L (L) character are permitted only in the day-of-month and day-of-week fields.  

• The letter-W (W) character in the day-of-month field schedules execution on the weekday nearest to the value specified. Placing "1W" in the day-of month field schedules execution for the weekday nearest the first of the month.
• The pound (#) character specifies a particular instance of a weekday for a given month. Placing "MON#2" in the day-of-week field schedules a task on the second Monday of the month.
• The asterisk (*) character is a wildcard character and indicates that every possible value can be taken for that specific field.

  Scheduling a Cron Job:

// Initiate a Schedule Factory       SchedulerFactory sf = new StdSchedulerFactory("quartz.properties");        // Retrieve a scheduler from schedule factory         Scheduler scheduler = schedulerFactory.getScheduler();          // current time         long ctime = System.currentTimeMillis();                 // Initiate JobDetail with job name, job group, and executable job class         JobDetail jobDetail =             new JobDetail("jobDetail_1", "jobDetailGroup_1", SimpleQuartzJob.class);         // Initiate CronTrigger with its name and group name         CronTrigger cronTrigger = new CronTrigger("cronTrigger_1", "triggerGroup_1");            try {             // setup CronExpression             CronExpression cexp = new CronExpression("0/6 * * * * ?");          // Assign the CronExpression to CronTrigger             cronTrigger.setCronExpression(cexp);         } catch (Exception e) {             e.printStackTrace();         }      cronTrigger.getJobDataMap().put("var1", "Value1");      //Adding job listner to the scheduler      scheduler.addJobListener(new MyJobListener());            // schedule a job with JobDetail and Trigger //The name of a job or trigger must be unique within its group.      scheduler.scheduleJob(jobDetail, cronTrigger);     // start the scheduler     scheduler.start();

 Rescheduling / Deleting Jobs :
 // Initiate a Schedule Factory

  SchedulerFactory sf = new StdSchedulerFactory("quartz.properties");  
 // Retrieve a scheduler from schedule factory
 Scheduler scheduler = sf .getScheduler();
 String[] triggerGroups;
 String[] triggers;
 triggerGroups = scheduler.getTriggerGroupNames();
 for (int i = 0; i < triggerGroups.length; i++) {
    triggers = scheduler.getTriggerNames(triggerGroups[i]);
    for (int j = 0; j < triggers.length; j++) {
    Trigger tg = scheduler.getTrigger(triggers[j], triggerGroups[i]);
    
    if (tg instanceof SimpleTrigger && tg.getName().equals("cronTrigger_1")) {
     ((SimpleTrigger)tg).setRepeatCount(100);
     // reschedule the job
     scheduler.rescheduleJob(triggers[j], triggerGroups[i], tg);
     // unschedule the job
     //scheduler.unscheduleJob(triggersInGroup[j], triggerGroups[i]);
    }
    }
 }        

 // start the scheduler
 scheduler.start();

 Fore more refer to http://www.quartz-scheduler.org/

CXF as Rest Client

In this blog I am going to talk about Java client for Rest Services. Here we will take CXF implementation and go through various ways of invoking Rest Services.

Following are various options for creating CXF client 

•  Injecting proxies
• Proxy-based API
• Web Clients

Injecting proxies

 CXF has deepest integration with Spring.Here the client Id will work exactly like

 Bean and when you use the bean definition form  bean factory It will work as It 

is getting called from the same location.
 

<jaxrs:client id="restClient"

         address="http://localhost:${bookstore.restlocation.url}/services/bookservice/get"

         serviceClass="com.bookstore.client.BookStoreInterface"

         inheritHeaders="true">

         <jaxrs:headers>

             <entry key="Accept" value="text/xml"/>

         </jaxrs:headers>

  </jaxrs:client> 

  Proxy-based API

BookStore store = JAXRSClientFactory.create("http://bookstore.com", BookStore.class);

//Customization

 WebClient.client(proxy).accept("text/xml");

 // (1) remote GET call to http://bookstore.com/bookstore

Books books = store.getAllBooks();

// (2) no remote call

BookResource subresource = store.getBookSubresource(1);

// {3} remote GET call to http://bookstore.com/bookstore/1

Book b = subresource.getBook();

proxies to Web Clients 

BookStore proxy = JAXRSClientFactory.create("http://books", BookStore.class);

Client client = WebClient.client(proxy);

WebClient httpClient = WebClient.fromClient(client);

Web Clients

WebClient client = WebClient.create("http://books");

Book book = client.path("bookstore/books").accept("text/xml").get(Book.class);

WebClient client = WebClient.create("http://books");

client.path("bookstore/books");

client.type("text/xml").accept("text/xml")

Response r = client.post(new Book());

Book b = r.readEntity(Book.class);

 Configuring HTTP clients in Spring

<bean id="myWebClient" class="org.apache.cxf.jaxrs.client.WebClient" 

factory-method="create"> 

        <constructor-arg type="java.lang.String" 

value="http://some.base.url.that.responds/" /> 

        <constructor-arg ref="webClientProviders" /> 

</bean> 

 In the next Blog for CXF will discuss the advanced topics like asynchronous call, thread safety and other things.

For more detailed Information refer to  CXF project documentation on http://cxf.apache.org/docs/jax-rs-client-api.html

Auto complete functionality using Hibernate-Search Module

Search functionality on web should provide results faster otherwise It will create a bad impression to the end user.For this  in case we are using persistent framework like hibernate then we have below two options to make a full text search query

• Use Criteria API
• Use Hibernate Search API

If we are operating over large sets of non indexed data, there might be a performance issue. some of the where clause term may not have been indexed in the database. Or say the criteria is like a regular expression for eg name staring with jon*..if we plan to use such queries we should start thinking of a full text search solution like Hibernate Search/Lucene/Solr .

Apache Lucene is a powerful full text search engine. However It has some issue with the object model or say when Object has a parent child relationship or there is a tree like structure in the object to be indexed.

Hibernate Search is combination of Hibernate and Apache Lucene.It handles the shortcoming of apache lucene gracefully.

The ideal scenario for using hibernate search is say when we need auto-complete/search  functionality. where a User will enter some keyword and the application should displaying matching  element in the dropdown
Following are the steps to search a full text search query using Hibernate search API.

• Creating a search session
• Creating a Lucene query
• executing the query.

Configuration
in hiberanate property file  specify index path

<property name="hibernate.search.default.directory_provider"
          value="filesystem"/>

<property name="hibernate.search.default.indexBase"
          value="/var/lucene/indexes"/>
         

Entity configuration          
 package example;
...
@Entity
@Indexed
public class Book {

  @Id
  @GeneratedValue
  private Integer id;
 
  @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
  private String title;
 
  @Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
  private String subtitle;
  @Field(index = Index.YES, analyze=Analyze.NO, store = Store.YES)
  @DateBridge(resolution = Resolution.DAY)
  private Date publicationDate;

  @IndexedEmbedded
  @ManyToMany
  private Set<Author> authors = new HashSet<Author>();

  public Book() {
  }
 
  // standard getters/setters follow here
  ...
}

• @Indexed marks Book as indexable
• Index.YES will ensure that the text will be indexed,
• analyze.YES ensures that the text will be analyzed using the default Lucene analyzer analyzing means chunking a sentence into individual words and potentially excluding common words like 'a' or 'the'.
•   store=Store.NO, ensures that the actual data will not be stored in the index
•   The benefit of storing it is the ability to retrieve it via projections

  //Search operation
 Session session = SessionFactoryUtil.getFactory().getCurrentSession();<br>
 //    create a full text session
 FullTextSession fSession = Search.getFullTextSession(session);
 fSession.beginTransaction();
 //    create a luceneQuery with a parser
 QueryParser parser = new QueryParser("title", new StandardAnalyzer());
 Query lucenceQuery = null;
 try {       
 lucenceQuery = parser.parse("content:hibernate");
 } catch (ParseException e) {       
 throw new RuntimeException("Cannot search with query string",e);        }
 //    execute the query
articles = fSession.createFullTextQuery(lucenceQuery, Article.class).list();
        for (Article article : articles) {
        System.out.println(article);        }
        fSession.getTransaction().commit();

        If the field is not tokenized use another approach
        List<Article> articles = fSession.createFullTextQuery(
     new TermQuery(new Term(“title”, “About Hibernate”)), Article.class).list();
        for (Article article : articles) {
            System.out.println(article);
        }

JDK1.7 Features

• JVM support for dynamic languages, following the prototyping work currently done on the Multi Language Virtual Machine
• Compressed 64-bit pointers Available in Java 6 with -XX:+UseCompressedOops
• Small language changes (grouped under a project named Coin)

• Strings in switch
• Automatic resource management in try-statement
• Improved type inference for generic instance creation
• Simplified varargs method declaration
• Binary integer literals
• Allowing underscores in numeric literals
• Catching multiple exception types and rethrowing exceptions with improved type checking

• Concurrency utilities 
• New file I/O library to enhance platform independence and add support for metadata and symbolic links. The new packages are java.nio.file and java.nio.file.attribute
• Library-level support for Elliptic curve cryptography algorithms
• An XRender pipeline for Java 2D, which improves handling of features specific to modern GPUs
• New platform APIs for the graphics features originally planned for release in Java version 6u10
• Enhanced library-level support for new network protocols, including SCTP and Sockets Direct Protocol
• Upstream updates to XML and Unicode

1. The ability to use a try-with-resources statement to automatically close resources of type Connection, ResultSet, and Statement
2. RowSet 1.1: The introduction of the RowSetFactory interface and the RowSetProvider class, which enable you to create all types of row sets supported by your JDBC driver.

 ^{Refer to http://www.oracle.com/technetwork/java/javase/jdk7-relnotes-418459.html}

Design Pattern in JDK

                                                               Design Pattern in JDK

Structural
Adapter:
This is used to convert the programming interface/class into that of another.

-java.util.Arrays#asList()
-javax.swing.JTable(TableModel)
-java.io.InputStreamReader(InputStream)
-java.io.OutputStreamWriter(OutputStream)
-javax.xml.bind.annotation.adapters.XmlAdapter#marshal()
-javax.xml.bind.annotation.adapters.XmlAdapter#unmarshal()

Bridge:
This decouples an abstraction from the implementation of its abstract operations, so that the abstraction and its implementation can vary independently.

-AWT (It provides an abstraction layer which maps onto the native OS the windowing support.)
-JDBC

Composite:
Lets clients treat individual objects and compositions of objects uniformly. So in other words methods on a type accepting the same type.

-javax.swing.JComponent#add(Component)
-java.awt.Container#add(Component)
-java.util.Map#putAll(Map)
-java.util.List#addAll(Collection)
-java.util.Set#addAll(Collection)


Decorator:
Attach additional responsibilities to an object dynamically and therefore it is also an alternative to subclassing. Can be seen when creating a type passes in the same type. This is actually used all over the JDK, the more you look the more you find, so the list below is definitely not complete.

-java.io.BufferedInputStream(InputStream)
-java.io.DataInputStream(InputStream)
-java.io.BufferedOutputStream(OutputStream)
-java.util.zip.ZipOutputStream(OutputStream)
-java.util.Collections#checked[List|Map|Set|SortedSet|SortedMap]()

Facade:
To provide a simplified interface to a group of components, interfaces, abstractions or subsystems.

-java.lang.Class
-javax.faces.webapp.FacesServlet


Flyweight:
Caching to support large numbers of smaller objects efficiently. I stumbled apon this a couple months back.

-java.lang.Integer#valueOf(int)
-java.lang.Boolean#valueOf(boolean)
-java.lang.Byte#valueOf(byte)
-java.lang.Character#valueOf(char)

Proxy:
The Proxy pattern is used to represent with a simpler object an object that is complex or time consuming to create.

-java.lang.reflect.Proxy
-RMI


Creational
Abstract factory:
To provide a contract for creating families of related or dependent objects without having to specify their concrete classes. It enables one to decouple an application from the concrete implementation of an entire framework one is using. This is also found all over the JDK and a lot of frameworks like Spring. They are simple to spot, any method that is used to create an object but still returns a interface or abstract class.

-java.util.Calendar#getInstance()
-java.util.Arrays#asList()
-java.util.ResourceBundle#getBundle()
-java.sql.DriverManager#getConnection()
-java.sql.Connection#createStatement()
-java.sql.Statement#executeQuery()
-java.text.NumberFormat#getInstance()
-javax.xml.transform.TransformerFactory#newInstance()


Builder:
Used simplify complex object creation by defining a class whose purpose is to build instances of another class. The builder pattern also allows for the implementation of a Fluent Interface.

-java.lang.StringBuilder#append()
-java.lang.StringBuffer#append()
-java.sql.PreparedStatement
-javax.swing.GroupLayout.Group#addComponent()


Factory method:
Simply a method that returns an actual type.

-java.lang.Proxy#newProxyInstance()
-java.lang.Object#toString()
-java.lang.Class#newInstance()
-java.lang.reflect.Array#newInstance()
-java.lang.reflect.Constructor#newInstance()
-java.lang.Boolean#valueOf(String)
-java.lang.Class#forName()

Prototype:
Allows for classes whose instances can create duplicates of themselves. This can be used when creating an instance of a class is very time-consuming or complex in some way, rather than creating new instances, you can make copies of the original instance and modify it.

-java.lang.Object#clone()
-java.lang.Cloneable

Singleton:
This tries to ensure that there is only a single instance of a class. I didn't find an example but another solution would be to use an Enum as Joshua Bloch suggests in Effective Java.

-java.lang.Runtime#getRuntime()
-java.awt.Toolkit#getDefaultToolkit()
-java.awt.GraphicsEnvironment#getLocalGraphicsEnvironment()
-java.awt.Desktop#getDesktop()

Behavioral

Chain of responsibility:
Allows for the decoupling between objects by passing a request from one object to the next in a chain until the request is recognized. The objects in the chain are different implementations of the same interface or abstract class.

-java.util.logging.Logger#log()
-javax.servlet.Filter#doFilter()

Command:
To wrap a command in an object so that it can be stored, passed into methods, and returned like any other object.

-java.lang.Runnable
-javax.swing.Action

Interpreter:
This pattern generally describes defining a grammar for that language and using that grammar to interpret statements in that format.

-java.util.Pattern
-java.text.Normalizer
-java.text.Format

Iterator:
To provide a consistent way to sequentially access items in a collection that is independent of and separate from the underlying collection.

-java.util.Iterator
-java.util.Enumeration

Mediator:
Used to reduce the number of direct dependencies between classes by introducing a single object that manages message distribution.

-java.util.Timer
-java.util.concurrent.Executor#execute()
-java.util.concurrent.ExecutorService#submit()
-java.lang.reflect.Method#invoke()

Memento:
This is a snapshot of an object’s state, so that the object can return to its original state without having to reveal it's content. Date does this by actually having a long value internally.

-java.util.Date
-java.io.Serializable

Null Object:
This can be used encapsulate the absence of an object by providing an alternative 'do nothing' behavior. It allows you to abstract the handling of null objects.

-java.util.Collections#emptyList()
-java.util.Collections#emptyMap()
-java.util.Collections#emptySet()

Observer:
Used to provide a way for a component to flexibly broadcast messages to interested receivers.

-java.util.EventListener
-javax.servlet.http.HttpSessionBindingListener
-javax.servlet.http.HttpSessionAttributeListener
-javax.faces.event.PhaseListener

State:
This allows you easily change an object’s behavior at runtime based on internal state.

-java.util.Iterator
-javax.faces.lifecycle.LifeCycle#execute()

Strategy:
Is intended to provide a means to define a family of algorithms, encapsulate each one as an object. These can then be flexibly passed in to change the functionality.

-java.util.Comparator#compare()
-javax.servlet.http.HttpServlet
-javax.servlet.Filter#doFilter()

Template method:
Allows subclasses to override parts of the method without rewriting it, also allows you to control which operations subclasses are required to override.

-java.util.Collections#sort()
-java.io.InputStream#skip()
-java.io.InputStream#read()
-java.util.AbstractList#indexOf()


Visitor:
To provide a maintainable, easy way to perform actions for a family of classes. Visitor centralizes the behaviors and allows them to be modified or extended without changing the classes they operate on.

-javax.lang.model.element.Element and javax.lang.model.element.ElementVisitor
-javax.lang.model.type.TypeMirror and javax.lang.model.type.TypeVisitor

Sample Application

There has been lots of development over Web and programming in last 15 years. If we need solution for a problem It may not be a bad Idea of looking the solution and discussion on the web.
 So that we do not invest our time on things which are already been investigated.


Here are the location we can find working solution/ sample examples on teh web

• Git hub https://github.com/
• Source forge sourceforge.net
• code project http://www.codeproject.com/

Sample Code collection:

• vogella: https://github.com/vogella/vogella.git 
• http://www.java2s.com 
• stackoverflow  http://stackoverflow.com/ 

  Web mobile:

• http://coenraets.org/blog/

For concepts:

• http://www.dzone.com
• http://www.cs.gordon.edu/courses/cs