Sunday, April 26, 2015

embedded db h2 storing java object

I intend to develop a small (Java) application. I believe there is no need to have separate database server as I have limited usage, so I have explored on embedded databases found h2 is more suitable for my requirements.
h2 is fast light weight jdbc api which can run in both embedded and server mode, and found quite easy to start with.

we can use jdbc api to get the connection


      String uri = "jdbc:h2:~/test"; //~/ directs to user home folder we can give full path as well "jdbc:h2:c:/myapp/data/test  
      JdbcConnectionPool connectionPool = JdbcConnectionPool.create(uri, "test", "test@123");  
      Connection connection = connectionPool.getConnection();  

or

      Class.forName("org.h2.Driver");  
      Connection connection = DriverManager.getConnection("jdbc:h2:~/test");  

for me every time getting connection was taking around 2 ms, its already quite efficient even though decided to use connection pool due to frequent db excess in batches.


to store objects we should create column of type "other",  java object need to put into db should be serializable. now we can store objects,

 String query = "create table properties (key varchar(255) primary key, value other)";  

     PreparedStatement pstmt = null;  
      if(connection != null) {  
           try {  
                String insertQuery = "insert into properties(key, value) values(?, ?)";  
                pstmt = connection.prepareStatement(insertQuery);  
                pstmt.setString(1, key);  
                pstmt.setObject(2, value, Types.OTHER);  
                pstmt.executeUpdate();  
           } catch (SQLException e) {  
                throw e;  
           } finally {  
                pstmt.close();  
                connection.close();  
           }  
      }  

no need to specify type (OTHER or JAVA_OBJECT) while accessing, desealization is taken care care in api.
 Object value = resultSet.getObject("value");  

source code for example is available here

Sunday, November 23, 2014

Applet Security warning popup issue, removing live connect calls.

removing live-connect dependency from web pages containing applets.

I was using jre7u21, for some reason had to move to on update (jre7u45), it has introduced some serious security restriction on applet side, one of  the impact is live-connect(javascript to applet) calls. As per documentation here 
  • The JavaScript to Java (LiveConnect) security dialog prompt is shown once per Applet classLoader instance.
So what does it mean for us? why do we care first of all.  we don't have to worry until ..
  1. we are fine with older version of jre (older than jre7u45). 
  2. we can afford to our customer giving interface with annoying security warning popup.
  3. we can configure java security verification (default is "show warning if needed)(control panel -> java -> advance tab)
In short we may have to get rid of live connect (javascript to applet) calls if we want to use latest jre, I personally do not prefer applets as its insecure, but  for legacy code base we have to come up with some short of alternative to avoid security warning popup. Here I am going to explain the approach I have followed.

The idea is that security warning popup appears when there is a javascript to java(applet) call, but there is no issue if applet calls to javascript, so we can safely call from applet to javascript. So my approach is to have one thread per applet which will keep on looking if there is any call from javascript, in javascript we can maintain one method call queue and the applet-thread will keep on polling for any method call in queue, if there is method call then applet-thread will invoke the method on itself through reflection.

Monday, October 27, 2014

Shareboard: Shareboard for Tom

Shareboard: Shareboard for Tom: Tom is a university student. He wants to share his recordings and snaps with friends. Usually he shares pictures and small size videos usin...

Sunday, April 28, 2013

Exporting smartgwt grid data to csv

I needed the feature to export the ListGrid Data to Excel file, but this feature is available only for smargwt PRO and EE version. I explored if there is any way can download through js, since the data is already present with client. but couldn't find because Javascript by design do not have access to local disk. and some browsers has limited support for data URI.

so I decided to send the content back to server (I feel its not correct way), and servlet write back data.

here is example for adding context menu to grid with export options.


import com.google.gwt.user.client.ui.FormPanel;
import com.google.gwt.user.client.ui.Hidden;
import com.google.gwt.user.client.ui.RootPanel;
import com.google.gwt.user.client.ui.VerticalPanel;
import com.smartgwt.client.types.Autofit;
import com.smartgwt.client.types.ListGridFieldType;
import com.smartgwt.client.types.SelectionStyle;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.grid.ListGridField;
import com.smartgwt.client.widgets.grid.ListGridRecord;
import com.smartgwt.client.widgets.menu.Menu;
import com.smartgwt.client.widgets.menu.MenuItem;
import com.smartgwt.client.widgets.menu.events.ClickHandler;
import com.smartgwt.client.widgets.menu.events.MenuItemClickEvent;

public class ExportableGrid extends ListGrid{
    public ListGridRecord[] records;

    public ExportableGrid() {
        setHeight100();
        setWidth100();
       
        addContextMenu();
       
    }
   
    private StringBuilder exportCSV(boolean all) {
        StringBuilder stringBuilder = new StringBuilder(); // csv data in here
        ListGridField[] fields = getFields();
        for (int i = 0; i < fields.length; i++) {
            ListGridField listGridField = fields[i];
            if(!listGridField.getType().equals(ListGridFieldType.IMAGE) && !listGridField.getName().equalsIgnoreCase("$72V")) {
                stringBuilder.append("\"");
                stringBuilder.append(listGridField.getName().toUpperCase());
                stringBuilder.append("\",");
            }
        }
        stringBuilder.deleteCharAt(stringBuilder.length() - 1); // remove last
        stringBuilder.append("\n");

        ListGridRecord[] recs = null;
        if (all) {
            recs = getRecords();
        } else {
            recs = getSelection();
        }
        for (int i = 0; i < recs.length; i++) {
            ListGridRecord listGridRecord = recs[i];
            ListGridField[] listGridFields = getFields();
            for (int j = 0; j < listGridFields.length; j++) {
                ListGridField listGridField = listGridFields[j];
                if(!listGridField.getType().equals(ListGridFieldType.IMAGE) && !listGridField.getName().equalsIgnoreCase("$72V")) {
                    stringBuilder.append("\"");
                    String val = listGridRecord.getAttribute(listGridField.getName());
                    stringBuilder.append(val);
                    stringBuilder.append("\",");
                }
            }
            stringBuilder.deleteCharAt(stringBuilder.length() - 1); // remove last ","
            stringBuilder.append("\n");
        }
        return stringBuilder;
    }
   
    public void export(StringBuilder stringBuilder, String fileExt) {   
        final FormPanel formPanel = new FormPanel();
        formPanel.setAction(getExportFileUrl());
        formPanel.setMethod(FormPanel.METHOD_POST);
       
        VerticalPanel verticalPanel = new VerticalPanel();
   
        Hidden data = new Hidden();
        data.setName("exprtData");
        data.setValue(stringBuilder.toString());
        verticalPanel.add(data);
       
        Hidden exportType = new Hidden();
        exportType.setName("exportType");
        exportType.setValue("clientExport");
        verticalPanel.add(exportType);
       
        Hidden fileType = new Hidden();
        fileType.setName("fileType");
        fileType.setValue(fileExt);
        verticalPanel.add(fileType);
   
        formPanel.add(verticalPanel);
        formPanel.setVisible(false);
        RootPanel.get().add(formPanel);   
        formPanel.submit();
       
    }
   
    public void addContextMenu() {
        setCanDragSelect(true); 
        setSelectionType(SelectionStyle.MULTIPLE);
        setAutoFitData(Autofit.VERTICAL); 
        setShowRollOver(false);
       
        MenuItem csvAll = new MenuItem("Export all as CSV");
        csvAll.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(MenuItemClickEvent event) {
                export(exportCSV(true), "csv");
            }
        });
       
        MenuItem csvSelected = new MenuItem("Export selected as CSV");
        csvSelected.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(MenuItemClickEvent event) {
                export(exportCSV(false), "csv");
            }
        });
       
        Menu menu = new Menu(); 
        menu.setWidth(150); 
        menu.addItem(csvAll); 
        menu.addItem(csvSelected); 
       
        setContextMenu(menu);
    }
   
    private String getExportFileUrl() {
        StringBuilder urlBuilder = new StringBuilder("exportFiles/exportFiles.htm?");
        return urlBuilder.toString();
    }
   
    public void updateData() {
        records = new ListGridRecord[2];
        for (int i=0; i            records[i] = new ListGridRecord();
            records[i].setAttribute("Test", "Test"+i);
        }
    }
}


My server side controller looks like below...

import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.web.servlet.ModelAndView;
import org.springframework.web.servlet.mvc.AbstractController;

public class Exporter extends AbstractController {

    @Override
    protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception {
        String exportType = request.getParameter("exportType");
        if(exportType != null && exportType.equalsIgnoreCase("clientExport")){
            String fileType = request.getParameter("fileType");
            String exportData = request.getParameter("exprtData");
            byte[] bytes = exportData.getBytes();
            response.setContentLength((int) bytes.length);
            response.setHeader("Content-type" ,"application/"+fileType);
            String fileNameHeader = "attachment; filename=download."+fileType ;
            response.setHeader("Content-disposition" , fileNameHeader); 
            ServletOutputStream  stream = response.getOutputStream();   
            stream.write(bytes);
        }
        return null;
    }
}

Saturday, April 27, 2013

Water mark plugin for flot

recently i needed to add some marker to all graphs in my project, since many people are using our tool and taking print-screen of the graphs for mail chain, without giving any credit :P.
so we decided to add marker text to all graphs, I found there is no straight forward way or plug-in available to add water mark or text to flot graphs. then I explored the flot API, found its well designed and easy to add plug-ins.
i created jquery.flot.marker.js
(function ($) {
    var options = {
        marker: {
            mode: null, // thinking to use for image based , text base markers
            value: "My Graph", // default value
            color: "rgba(100, 100, 100, 0.80)",
            font: "14px Arial",
            position: 'top-right' //top-left, bottom-right, bottom-left
        }
    };
    function init(plot) {
        plot.hooks.draw.push(function (plot, ctx) {
            var c = plot.getOptions().marker;
            if (!c.mode)
                return;
            //alert(plot.offset().left);
            ctx.save();
            ctx.font=c.font;
            var w;
            var h;
            if((c.position=='top-right') || (c.position=='bottom-right')) {
                w = plot.getPlaceholder().width()*0.88;
            } else {
                w = plot.offset().left-40;
            }
            if((c.position=='top-left') || (c.position=='top-right')) {
                h = 45;
            } else {
                h = plot.height();
            }
            //alert(h);
            ctx.fillStyle=c.color;
            ctx.fillText(c.value, w, h);

        });
    }
    $.plot.plugins.push({
        init: init,
        options: options,
        name: marker,
        version: '1.0'
    });
})(jQuery);
Then including plug-in file helped me to get marker on all graphs.
<script language="javascript" type="text/javascript" src="../jquery.flot.marker.js"></script>
  plot = $.plot($("#placeholder"), 
                      ${data}, 
                          marker: {
                                mode: "text",
                                value:"My Graph",
                                color:"rgba(100, 100, 100, 0.60)",
                                font: "10px Lucida Sans Unicode",
                                position: 'top-right'                           
                            },

                            grid: { hoverable: true, autoHighlight: false },
                            yaxis: { min: -1.2, max: 1.2 }
                        });

Saturday, February 16, 2013

Generating Unique Ids across multiple JVM and over restart using long data type only

/*
* Long.MAX_VALUE : 9 2 2 3 3 7 2 0 3 6 8 5 4 7 7 5 8 0 7  Total 19 digits
*
* lets use fist 2 digit for peer id
* now we are left with 17 digits
* if we restrict peer id between 10 - 91
*
* then rest all 17 could be all 9's to hold in long
*
* lets create 10 UIDGenerator Objects each with unique id [0-9]
*
* so now we are left with 16 digits
*
* so if we start at some time
* System.currentTimeMillis() : 1360443174566 (13 digit)
*
* ID : XXY1360443174566TTT
*                         XX : peer id
*                          Y : object instance ID
*     1360443174566 : System.currentMillis(); //seed will change on every restart
*                       TTT : 000     Allowing to generate 999 message max per millisecond

* so we can generate max 999 unique ids per millisecond
* means safe to generate approx: 1000 per millisecond, to overcome uniqueness over jvm restart
*
* we can generate 9999999999999999-1360443174566000 = 863955682543999 * 10 unique ids
* (Aprox:8000 Trillion )before getting duplicate if we start on 1360443174566
*
* Possible Failure
* 1. system time is reset
* 2. generating more than 10000 per millisecond then does not gives guarantee over jvm restart.
*/

public class UIDUtil {
    private static UIDUtil instance;
    private UIDGenerator[] uidGenerators;
    private static short MAX_ALLOWED_PEER_ID = 91;
    private static short MIN_ALLOWED_PEER_ID = 10;
    private long totalIdsGenerated;
    private long startTime;
    private UIDUtil() {
        short peerId = loadPeerId();
        uidGenerators = new UIDGenerator[10];
        for(byte i=0; i<10; i++) {
            uidGenerators[i] = new UIDGenerator(peerId, i);
        }
        startTime = System.currentTimeMillis();
    }
    public static UIDUtil getInstance() {
        if(instance == null)
            instance = new UIDUtil();
        return instance;
    }
    public long getNextId() {
        return uidGenerators[(int) ((totalIdsGenerated++)%10)].getNextID();
    }
    private short loadPeerId() {
        //TODO:    load peerId from properties, db, remote etc
        short peerId = 90;
        if(peerId > MAX_ALLOWED_PEER_ID || peerId < MIN_ALLOWED_PEER_ID) {
            peerId = (short) (MIN_ALLOWED_PEER_ID + (short)(Math.random() * ((MAX_ALLOWED_PEER_ID - MIN_ALLOWED_PEER_ID) + 1)));
            try {
                throw new Exception("Peer ID must be in the range : "+MIN_ALLOWED_PEER_ID+"-"+MAX_ALLOWED_PEER_ID+" inclusive");
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        return peerId;
    }
    public long getTotalIdsGenerated() {
        return totalIdsGenerated;
    }
    public double getIdProducingRate() {
        long elapsedTime = System.currentTimeMillis()-startTime;
        if(elapsedTime == 0)
            elapsedTime = 1;
        return (double) (totalIdsGenerated/elapsedTime)*1000;
    }
}

 

public class UIDGenerator {
    private long variableId ;
    private long fixedId;

    private static long MAX_ALLOWED_VARIABLE_ID = 9999999999999999l;
    public UIDGenerator(short peerId, byte index) {
        //place peer on proper decimal location
        fixedId = (long) (peerId*Math.pow(10, 17));//exp : 9100000000000000000
        fixedId += (long) (index*Math.pow(10, 16));//exp : 9130000000000000000
        //initialise with 16 digit seed
        variableId = loadVariableId();
    }

    public synchronized long getNextID() {
        long newId = variableId++;
        if(newId == MAX_ALLOWED_VARIABLE_ID) {
            variableId = loadVariableId();
        }
        return (fixedId + newId);
    }

    private long loadVariableId() {
        long currentTime = System.currentTimeMillis();
        long temp = currentTime;
        byte digits = 0;
        while(temp > 0) {
            temp /= 10;
            digits++;
        }
        return (long) (currentTime*Math.pow(10, 16-digits));
    }
}

Sunday, April 8, 2012

Smart Gwt TreeGrid filter Example ..



import java.util.ArrayList;
import java.util.List;

import com.smartgwt.client.widgets.grid.events.FilterEditorSubmitEvent;
import com.smartgwt.client.widgets.grid.events.FilterEditorSubmitHandler;
import com.smartgwt.client.widgets.tree.Tree;
import com.smartgwt.client.widgets.tree.TreeGrid;
import com.smartgwt.client.widgets.tree.TreeGridField;
import com.smartgwt.client.widgets.tree.TreeNode;

public class TreeGridExperiment extends TreeGrid{
    private Tree tree;
    private TreeNodeRecord[] records;
   
    public TreeGridExperiment() {
        tree = new Tree();
        updateData();
        setShowFilterEditor(true);
        setFilterOnKeypress(false);

        enableFiltering();
        TreeGridField field = new TreeGridField(TreeNodeRecord.NODE_NAME);
        field.setCanFilter(true);
        setFields(field);
    }
   
    private void enableFiltering() {
        addFilterEditorSubmitHandler(new FilterEditorSubmitHandler() {
            @Override
            public void onFilterEditorSubmit(FilterEditorSubmitEvent event) {
                String inputValue = event.getCriteria().getAttribute(TreeNodeRecord.NODE_NAME);
                if(inputValue == null){
                    setData(tree);
                    return;
                }
                setData(new Tree());
                List matchingNodes = new ArrayList();
                for (TreeNodeRecord record : records) {
                    if(record.contains(inputValue.toLowerCase(), tree)){
                        matchingNodes.add(record);
                    }
                }
                Tree newTree = new Tree();
                newTree.setData(matchingNodes.toArray(new TreeNodeRecord[0]));
                setData(newTree);
                for (TreeNodeRecord treeNodeRecord : matchingNodes) {
                    newTree.openAll(treeNodeRecord);
                }
            }
        });
    }

    public void updateData() {
        records = new TreeNodeRecord[2];

        for (int i = 0; i < 2; i++) {
            records[i] = new TreeNodeRecord("Display " + i);
            TreeNodeRecord[] leafs = new TreeNodeRecord[2];
            for (int j = 0; j < 1; j++) {
                leafs[j] = new TreeNodeRecord("Leaf" + j+i);
            }
            records[i].setChildren(leafs);
        }
        tree.setData(records);
        this.setData(tree);
    }
}

class TreeNodeRecord extends TreeNode {
    public static String NODE_NAME = "nodeName";
   
    public TreeNodeRecord(String attribute) {
        setAttribute(NODE_NAME, attribute);
    }

    public boolean contains(String inputValue, Tree tree) {
        if(getAttributeAsString(NODE_NAME).toLowerCase().contains(inputValue)) {
            return true;
        } else {
            TreeNode[] childes = tree.getChildren(this);
            for (TreeNode treeNode : childes) {
                TreeNodeRecord record = (TreeNodeRecord)treeNode;
                if(record.contains(inputValue, tree)){
                    return true;
                }
            }
        }
        return false;
    }
}

    

Saturday, July 2, 2011

Extending OpenNMS

The OpenNMS installation directory contains etc/ directory which contains all kind of configuration files which require to run openNMS, it has various configuration files for each daemon module, for example for capability daemon there is capsd.xml for collector daemon it has capsd.xml and for dataCollection definition it has jmx-dataCollection.xml for Mbeans definition, similarity for snmp there is dataCollection.xml for defining what OID’s need to be collected for SNMP services.

Now this etc directory also contains service-configuration.xml which has all module definition plus which method to be invoked for loading the module. These modules can be daemon or non daemon depending upon the requirement. all module follows standard MBean criteria. These modules are started in the sequence they are mention in the file. I mean top most will be executed first and the bottom most ( which is embedded jetty-web server) will be the last.

 

<service-configuration>
  <service>
    <name>:Name=XSLTProcessor</name>
    <class-name>mx4j.tools.adaptor.http.XSLTProcessor</class-name>
  </service>
  <service>
    <name>:Name=HttpAdaptor</name>
    <class-name>mx4j.tools.adaptor.http.HttpAdaptor</class-name>
    <attribute>
      <name>Port</name>
      <value type="java.lang.Integer">8180</value>
    </attribute>
    <attribute>
      <name>Host</name>
…………………………………….

If we add new service definition here, the OpenNMS will load our service ..

for example let say we want to delete all logs or previous data collected on starting up openNMS, we have to add our service in this xml then only openNMS could start our service.

<service-configuration>

<service>
  <name>OurService:Name=CleanupService</name>
  <class-name>org.ourservice.Clean</class-name>
  <invoke at="start" pass="0" method="start"/>

</service>
  <service>
    <name>:Name=XSLTProcessor</name>
    <class-name>mx4j.tools.adaptor.http.XSLTProcessor</class-name>
  </service>
  <service>
    <name>:Name=HttpAdaptor</name>
    <class-name>mx4j.tools.adaptor.http.HttpAdaptor</class-name>
    <attribute>
      <name>Port</name>
      <value type="java.lang.Integer">8180</value>
    </attribute>
    <attribute>
      <name>Host</name>

now CleanupService will be triggered first. for starting this service start method will be invoked of Clean class. The clean Class must implement MBean.
create CleanMBean interface and declare start method signature.

package org.ourservice;

public interface CleanMBean {
    public void start();
}

now create Clean class and implement CleanMBean interface, also implement logic to clean the data.

package org.ourservice;

public class Clean implements CleanMBean {
    public void start(){
       System.out.println(“Cleaned all”);

     }

}

 

 

After compiling the Clean class and other dependant classes. create classes directory inside openNMS installation directory and put all our classes inside. then start openNMS.

Java5 Instumentation API

The instrumentation API is for run time byte manupulation, or you can say bytecode engineering. 

we have to define our instrumentation agent which consist starting point same as main method of any java application with different arguments.

String : arguments to this agent..

Instrumentation object, instrumentation agent is a class with a special method, with a predefined signature, that the JVM will invoke before the rest of the application for you to set up any instrumentation code. Generally, instrumentation is the act of transforming classes for profiling purposes: for example, an interesting feature provided by the instrumentation framework is the ability to measure the memory usage of an object. 
 

import java.lang.instrument.Instrumentation;

public class MyAgent {
    public static void premain(String agentArguments,
            Instrumentation instrumentation) {

        MyTransFormer transformer = new MyTransFormer();
        instrumentation.addTransformer(transformer);
        Node node = new Node();
        long size = instrumentation.getObjectSize(node);
        System.out.println("size of "+ node.getClass().getSimpleName() + size);
    }
}

we have compiled the agent class and  we need to package it into a jar , we also need to create a manifest file

Premain-Class: MyAgent 
java -javaagent:myagent.jar TestMain
 
creating transformer
import java.lang.instrument.ClassFileTransformer; 
import java.security.ProtectionDomain;

public class MyTransFormer implements ClassFileTransformer {


    public MyTransFormer() {
        super();
    }


public byte[] transform(ClassLoader loader, String className,
            Class redefiningClass, ProtectionDomain domain, byte[] bytes) {
        // some changes in byte code // there are some standard API for byte
        // code modification exp: BCEL, ASM etc…
        return bytes;
    }
}


There are some links i went while reading .



  1. http://www.javalobby.org/java/forums/t19309.html#92037417

  2. http://www.cs.nuim.ie/~jpower/Research/instrument/#examples

  3. http://www.javamex.com/tutorials/memory/instrumentation.shtml

Monday, June 13, 2011

Handling more than one exception, Java SE 7 new feature.

In Java SE 7 and later, a single catch block can handle 
more than one type of exception. This feature can reduce code 
duplication and lessen the temptation to catch an overly broad 
exception. 
 
consider the following example... 
try{
.............

}
catch (IOException ex) {
     logger.log(ex);
     throw ex;
catch (SQLException ex) {
     logger.log(ex);
     throw ex;
}
 
 
The following example, which is valid in Java SE 7 and later, 
eliminates the duplicated code:
 
catch (IOException|SQLException ex) {
    logger.log(ex);
    throw ex;
} 

Search Ranjeet's Blog