Java: Basic Dropwizard Project

This entry is part 1 of 5 in the series Dropwizard

This tutorial will guide you through how to create a bare bones Dropwizard app. Ensure you have Eclipse installed or whatever IDE you are deciding to use. You can use their documentation as a guide. This is the first tutorial in the dropwizard series.

Setup Eclipse Archetype:

Select new Maven Project then in the select Archetype if dropwizard isn’t there you can add it by using the below settings.

Filter for Dropwizard Archetype:

Set Project Settings:

POM:

Ensure the pom has the correct “dropwizard.version” and that the dependency “dropwizard-core” is there.

Our Configuration:

Now that the project is created let’s have a look at what our configuration looks like. Pretty basic but that’s all we need right now.

package ca.gaudreault.mydropwizardapp;

import io.dropwizard.Configuration;
import com.fasterxml.jackson.annotation.JsonProperty;
import org.hibernate.validator.constraints.*;
import javax.validation.constraints.*;

public class MyDropwizardAppConfiguration extends Configuration {
    
}

Our Application:

This is what our application class looks like. It’s empty nothing yet.

package ca.gaudreault.mydropwizardapp;

import io.dropwizard.Application;
import io.dropwizard.setup.Bootstrap;
import io.dropwizard.setup.Environment;

public class MyDropwizardAppApplication extends Application {

    public static void main(final String[] args) throws Exception {
        new MyDropwizardAppApplication().run(args);
    }

    @Override
    public String getName() {
        return "MyDropwizardApp";
    }

    @Override
    public void initialize(final Bootstrap bootstrap) {

    }

    @Override
    public void run(final MyDropwizardAppConfiguration configuration,
                    final Environment environment) {

    }
}

Config.yml:

This is what our config.yml file looks like at the start.

logging:
  level: INFO
  loggers:
    ca.gaudreault: DEBUG

Setup Debug Configuration:

Setup your debug configuration like the below setup.

Running:

Once you run it you will be running two sites.

  1. http://localhost:8080
    1. Your main site
  2. http://localhost:8081
    1. Your operational site (health, etc)

Hadoop & Java: Connect Remote Unsecured HDFS

In this tutorial I will show you how to connect to remote unsecured HDFS cluster using Java. If you haven’t install hdfs yet follow the tutorial.

POM.xml:

<dependency>
	<groupId>org.apache.hadoop</groupId>
	<artifactId>hadoop-client</artifactId>
	<version>2.9.1</version>
</dependency>

Imports:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import java.net.URI;

Connect:

//Setup the configuration object.
final Configuration config = new Configuration();

//If you want you can add any properties you want here.

//Setup the hdfs file system object.
final FileSystem fs = FileSystem.get(new URI("hdfs://localhost:50070"), config);

//Do whatever you need to.

HBASE & Java: Search for Data

This tutorial will give you a quick overview of how to search for data using HBASE. If you have not done so yet. Follow the following two tutorials on HBASE: Connecting and HBASE: Create a Table.

Search for Data:

Basically we have to scan the table for data. So we must first setup a scan object then search for the data.

import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.client.Table;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.util.Bytes;

//Lets setup our scan object.
final Scan scan = new Scan();
//Search a particular column
scan.addColumn(Bytes.toBytes("columnFamily"), Bytes.toBytes("columnName"));
//Check the row key prefix
scan.setRowPrefixFilter(Bytes.toBytes("rowkey"));

final TableName table = TableName.valueOf(yourTableName);

//Get the table you want to work with. using the connection from the tutorial above.
final Table table = conn.getTable(table);
//Create our scanner based on the scan object above.
final ResultScanner scanner = table.getScanner(scan);

//Now we will loop through our results
for (Result result = scanner.next(); result != null; result = scanner.next()) {
      //Lets get our row key
      final String rowIdentifier = Bytes.toString(result.getRow());

      //Now based on each record found we will loop through the available cells for that record.
      for (final Cell cell : result.listCells()) {
        //now we can do whatever we need to with the data.
        log.info("column {} value {}", Bytes.toString(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength()), Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
      }
}

HBASE & Java: Create a Table

This tutorial will guide you through how to create a HBASE table using Java 8. Make sure you first follow this tutorial on connecting to HBASE.

Table Exists:

This checks if the table already exists in HBASE.

import org.apache.hadoop.hbase.TableName;

final TableName table = TableName.valueOf(yourTableName);

//Use the connection object to getAdmin from the connection tutorial above.
conn.getAdmin().tableExists(table);

Create Table:

In the most basic example of creating a HBASE table you need to know the name and the column families. A column family is columns grouped together. The data is related in some way and stored together on disk. Notice how we don’t define columns in the table design. Columns are added as we put data. Which I will give example below.

import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;

final TableName table = TableName.valueOf(yourTableName);

final HTableDescriptor hTableBuilder = new HTableDescriptor(table);
final HColumnDescriptor column = new HColumnDescriptor(family);
hTableBuilder.addFamily(column);

//Use the connection object to getAdmin from the connection tutorial above.
conn.getAdmin().createTable(hTableBuilder);

Get a Table:

This will retrieve a table from HBASE so you can use it to put data, etc.

import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Table;

final TableName table = TableName.valueOf(yourTableName);

//Use the connection object from the connection tutorial above.
final Table table = conn.getTable(table);

Put Data:

Now we will put data into the table we have reference to above. Notice how the columns are referenced.

import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;

final byte[] rowKey = Bytes.toBytes("some row identifier");
final byte[] columnFamily = Bytes.toBytes("myFamily");
final byte[] columnName = Bytes.toBytes("columnName");
final byte[] data = Bytes.toBytes(myData);

final Put put = new Put(rowKey);
put.addColumn(columnFamily, columnName, data);

//Insert the data.
table.put(put);
//Close the table.
table.close();

HBASE: Connecting Unsecure

In this tutorial I will show you how to connect to an Unsecure HBASE using Java. It’s rather straight forward. This tutorial assumes no security. There are so many different options you can set we will just take the bare minimum so you can connect.

POM:

<dependency>
	<groupId>org.apache.hbase</groupId>
	<artifactId>hbase-client</artifactId>
	<version>1.4.1</version>
</dependency>
<dependency>
	<groupId>org.apache.hbase</groupId>
	<artifactId>hbase</artifactId>
	<version>1.4.1</version>
	<type>pom</type>
</dependency>

Imports:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;

Config:

We will use the basic configuration here. You should secure the cluster and use appropriate settings for that.

final Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", "myurl.com"); //Can be comma seperated if you have more than 1
config.set("hbase.zookeeper.property.clientPort", "2181");
config.set("zookeeper.znode.parent", "/hbase-unsecure");

Connect:

Now we create the connection.

Connection conn = ConnectionFactory.createConnection(config);

//Later when we are done we will want to close the connection.
conn.close();

Hbase Admin:

Retrieve an Admin implementation to administer an HBase cluster. If you need it.

Admin admin = conn.getAdmin();
//Later when we are done we will want to close the connection.
admin.close();

Java: String.format

To format a string with parameters use String.format. There are different ways to use it depending on boolean, number, etc.

Format with Boolean

String.format("Boolean: %b", true)

Format with Number

String.format("Number: %n", 32)

Format with String

String.format("String: %s", "Information")

AWS: Send Simple Email Service

This entry is part 5 of 5 in the series AWS & Java

If you want to send an email using AWS’ Simple Mail then you need to do the following. This is a very basic example.

Import the following:

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailService;
import com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClientBuilder;
import com.amazonaws.services.simpleemail.model.Body;
import com.amazonaws.services.simpleemail.model.Content;
import com.amazonaws.services.simpleemail.model.Destination;
import com.amazonaws.services.simpleemail.model.Message;
import com.amazonaws.services.simpleemail.model.SendEmailRequest;

Setup Connection to AWS Simple Email Service

final AmazonSimpleEmailService simpleEmailService = AmazonSimpleEmailServiceClientBuilder.standard().withRegion(myRegion)
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKeyId, secretKey)))
.build();

Setup Email:

final SendEmailRequest request = new SendEmailRequest().withDestination(new Destination().withToAddresses(TO)).withSource(FROM)
.withMessage(new Message().withSubject(new Content().withCharset("UTF-8").withData(SUBJECT))
.withBody(new Body().withText(new Content().withCharset("UTF-8").withData(BODY))));

Send Email:

simpleEmailService.sendEmail(request);

AWS: Java Post to Kinesis Queue

This entry is part 4 of 5 in the series AWS & Java

Posting to an AWS Kinesis Queue is rather simple and straight forward. As always you should refer to AWS Documentation.

Put Multiple Records On Queue

Import the following

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.kinesis.AmazonKinesis;
import com.amazonaws.services.kinesis.AmazonKinesisClientBuilder;
import com.amazonaws.services.kinesis.model.PutRecordsRequest;
import com.amazonaws.services.kinesis.model.PutRecordsRequestEntry;
import com.amazonaws.services.kinesis.model.Record;

Put Records

AmazonKinesisClientBuilder clientBuilder = AmazonKinesisClientBuilder.standard().withRegion(myRegion).withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(myAccessKeyId, mySecretKey)));
AmazonKinesis kinesisClient = clientBuilder.build();
PutRecordsRequest putRecordsRequest = new PutRecordsRequest();
putRecordsRequest.setStreamName(myQueue);
List putRecordsRequestEntryList  = new ArrayList<>(); 


//You can put multiple entries at once if you wanted to
PutRecordsRequestEntry putRecordsRequestEntry  = new PutRecordsRequestEntry();
putRecordsRequestEntry.setData(ByteBuffer.wrap(myData));
putRecordsRequestEntry.setPartitionKey(myKey);
putRecordsRequestEntryList.add(putRecordsRequestEntry);


putRecordsRequest.setRecords(putRecordsRequestEntryList);
PutRecordsResult putResult = kinesisClient.putRecords(putRecordsRequest);

Put Single Record On Queue

Import the following

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.kinesis.AmazonKinesis;
import com.amazonaws.services.kinesis.AmazonKinesisClientBuilder;
import com.amazonaws.services.kinesis.model.PutRecordRequest;
import com.amazonaws.services.kinesis.model.Record;

Put Record

AmazonKinesisClientBuilder clientBuilder = AmazonKinesisClientBuilder.standard().withRegion(myRegion).withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(myAccessKeyId, mySecretKey)));
AmazonKinesis kinesisClient = clientBuilder.build();
PutRecordRequest putRecordRequest = new PutRecordRequest();
putRecordRequest.setStreamName(myQueue);

putRecordRequest.setData(ByteBuffer.wrap(data.getBytes("UTF-8")));
putRecordRequest.setPartitionKey(myKey);

PutRecordResult putResult = kinesisClient.putRecord(putRecordRequest);

You now have put a record(s) onto the queue congratulations!

AWS: Java S3 Upload

This entry is part 3 of 5 in the series AWS & Java

If you want to push data to AWS S3 there are a few different ways of doing this. I will show you two ways I have used.

Option 1: putObject

import com.amazonaws.AmazonClientException;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

ClientConfiguration config = new ClientConfiguration();
config.setSocketTimeout(SOCKET_TIMEOUT);
config.setMaxErrorRetry(RETRY_COUNT);
config.setClientExecutionTimeout(CLIENT_EXECUTION_TIMEOUT);
config.setRequestTimeout(REQUEST_TIMEOUT);
config.setConnectionTimeout(CONNECTION_TIMEOUT);

AWSCredentialsProvider credProvider = ...;
String region = ...;

AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(credProvider).withRegion(region).withClientConfiguration(config).build();

InputStream stream = ...;
String bucketName = .....;
String keyName = ...;
String mimeType = ...;

//You use metadata to describe the data.
final ObjectMetadata metaData = new ObjectMetadata();
metaData.setContentType(mimeType);

//There are overrides available. Find the one that suites what you need.
try {
	s3Client.putObject(bucketName, keyName, stream, metaData);
} catch (final AmazonClientException ex) {
	//Log the exception
}

Option 2: MultiPart Upload

import com.amazonaws.AmazonClientException;
import com.amazonaws.event.ProgressEvent;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.event.ProgressEventType;
import com.amazonaws.event.ProgressListener;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.TransferManagerBuilder;
import com.amazonaws.services.s3.transfer.Upload;

ClientConfiguration config = new ClientConfiguration();
config.setSocketTimeout(SOCKET_TIMEOUT);
config.setMaxErrorRetry(RETRY_COUNT);
config.setClientExecutionTimeout(CLIENT_EXECUTION_TIMEOUT);
config.setRequestTimeout(REQUEST_TIMEOUT);
config.setConnectionTimeout(CONNECTION_TIMEOUT);

AWSCredentialsProvider credProvider = ...;
String region = ...;

AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(credProvider).withRegion(region).withClientConfiguration(config).build();

InputStream stream = ...;
String bucketName = .....;
String keyName = ...;
long contentLength = ...;
String mimeType = ...;

//You use metadata to describe the data. You need the content length so the multi part upload knows how big it is
final ObjectMetadata metaData = new ObjectMetadata();
metaData.setContentLength(contentLength);
metaData.setContentType(mimeType);

TransferManager tf = TransferManagerBuilder.standard().withS3Client(s3Client).build();
tf.getConfiguration().setMinimumUploadPartSize(UPLOAD_PART_SIZE);
tf.getConfiguration().setMultipartUploadThreshold(UPLOAD_THRESHOLD);
Upload xfer = tf.upload(bucketName, keyName, stream, metaData);

ProgressListener progressListener = new ProgressListener() {
	public void progressChanged(ProgressEvent progressEvent) {
		if (xfer == null)
			return;
		
		if (progressEvent.getEventType() == ProgressEventType.TRANSFER_FAILED_EVENT || progressEvent.getEventType() == ProgressEventType.TRANSFER_PART_FAILED_EVENT) {
			//Log the message
		}
	}
};

xfer.addProgressListener(progressListener);
xfer.waitForCompletion();

Java: Enums

Below is an example of how to create a enum in Java.

public enum MyEnum {
	VAL(0), NEXTVAL(1);

	private int value;

	private MyEnum(int value) {
		setValue(value);
	}

	public int getValue() {
		return value;
	}

	public void setValue(int value) {
		this.value = value;
	}
}

Java: Input/Output Streams

This page is to give some basic examples of how to convert OutputStreams to InputStreams and vise versa.

Maven:

<dependency>
	<groupId>org.logback-extensions</groupId>
	<artifactId>logback-ext-loggly</artifactId>
	<version>0.1.4</version>
</dependency>

To convert an InputStream to OutputStream we can do it using IoUtils.copy as demonstrated below.

import ch.qos.logback.ext.loggly.io.IoUtils;
import java.io.InputStream;

InputStream input = ##INPUTSTREAM##;

//Convert InputStream to OutputStream

try (FileOutputStream out = new FileOutputStream(file)) {
	IoUtils.copy(input, out);
} catch (final IOException e) {
}

To convert a ByteArrayOutputStream to a ByteArrayInputStream we can do it as demonstrated below.

final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
//Convert to ByteArrayInputStream
final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());

AWS: Java Kinesis Lambda Handler

This entry is part 2 of 5 in the series AWS & Java

If you want to write a Lambda for AWS in Java that connects to a Kinesis Stream. You need to have the handler.

Maven:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.109</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.7.1</version>
</dependency>

This is the method that AWS Lambda will call. It will look similar to the one below.

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.events.KinesisEvent;
import com.amazonaws.services.lambda.runtime.events.KinesisEvent.KinesisEventRecord;
import com.amazonaws.services.kinesis.model.Record;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.JsonNode;

public void kinesisRecordHandler(KinesisEvent kinesisEvent, Context context) {
	final String awsRequestId = context.getAwsRequestId();
	final int memoryLimitMb = context.getMemoryLimitInMB();
	final int remainingTimeInMillis = context.getRemainingTimeInMillis();

	for (final KinesisEventRecord kinesisRec : kinesisEvent.getRecords()) {
		final Record record = kinesisRec.getKinesis();

		//We get the kinesis data information
		final JsonNode recData = new ObjectMapper().readValue(record.getData().array(), JsonNode.class);

		final String bucketName = recData.get("bucket").asText();
		final String key = recData.get("key").asText();
	}
}

The thing to note when you setup you Lambda is how to setup the “Handler” field in the “Configuration” section on AWS. It is in the format “##PACKAGE##.##CLASS##::##METHOD##”.

AWS: Java S3 Lambda Handler

This entry is part 1 of 5 in the series AWS & Java

If you want to write a Lambda for AWS in Java that connects to S3. You need to have the handler.

Maven:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.109</version>
</dependency>

This is the method that AWS Lambda will call. It will look similar to the one below.

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.event.S3EventNotification.S3Entity;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;

public void S3Handler(S3Event s3e, Context context) {
	final String awsRequestId =  context.getAwsRequestId();
	final int memoryLimitMb = context.getMemoryLimitInMB();
	final int remainingTimeInMillis = context.getRemainingTimeInMillis();

	for (final S3EventNotificationRecord s3Rec : s3e.getRecords()) {
		final S3Entity record = s3Rec.getS3();
		
		final String bucketName = record.getBucket().getName()
		final String key = record.getObject().getKey();
	}
}

The thing to note when you setup you Lambda is how to setup the “Handler” field in the “Configuration” section on AWS. It is in the format “##PACKAGE##.##CLASS##::##METHOD##”.

Java: Jackson Json

When you want to work with JSON the package of choice is Jackson. I find it so useful and easy to use.

Maven:

<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-core</artifactId>
	<version>2.7.1</version>
</dependency>
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-annotations</artifactId>
	<version>2.7.1</version>
</dependency>
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-databind</artifactId>
	<version>2.7.1</version>
</dependency>

There are so many things that we can do with Jackson. It’s actually very exciting.

Let’s start with converting a json string to ObjectNode.

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;

//We need an object mapper to help with the conversion
final ObjectMapper mapper = new ObjectMapper();
//Next we read the json string into ObjectNode
final ObjectNode node = (ObjectNode)mapper.readTree(jsonString);

//You can check if the objectnode has a key by
node.has("my_key")

//You can get the value by
node.get("my_key")

//To get the value as a string using 
.asText() at the end of the get

//To get as int
.asInt()

//To get as boolean
.asBoolean()

//To get as double
.asDouble()

//To get as Long
.asLong()

Next let’s convert a json string to JsonNode

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

//You need an object mapper to help with the conversion
final ObjectMapper mapper = new ObjectMapper();
//Next we read the json string into a jsonnode
final JsonNode node = mapper.readTree(jsonString);
//We can if you want to switch it to an object node fairly easily.
final ObjectNode objNode = (ObjectNode)node;

//Put value in the object node
objNode.put("my_key", 1);

//You can also set json node
objNode.set("my_key", myJsonNode);

You can also create a new object node by using JsonNodeFactory. See below.

import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;

//If you want to create an instance of ObjectNode
final ObjectNode objNode = JsonNodeFactory.instance.objectNode();

If you want to work with json array’s it’s also pretty straight forward. Again we work with JsonNodeFactory to declare it out.

import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;

private ArrayNode arrNode = JsonNodeFactory.instance.arrayNode();

//We can then add to it
arrNode.add();

Jackson also has generators which allow us to create json on the fly.

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;

//It throws a "JsonProcessingException'

final JsonFactory jfactory = new JsonFactory();
ByteArrayOutputStream out=new ByteArrayOutputStream();

try (final JsonGenerator jg = jfactory.createGenerator(out, JsonEncoding.UTF8)) {
	jg.setCodec(objectMapper);

	jg.writeStartObject();

	jg.writeFieldName("my_key");
	jg.writeString("hi");

	jg.writeFieldName("next_key");
	jg.writeObject(my_obj);

	jg.writeFieldName("third_key");
	jg.writeTree(my_obj_node);

	jg.writeEndObject();
}

Jackson JsonViews are really cool in my opinion. We can have an object that has multiple properties and we can set a view to each one of these properties. Which means when we serialize the object we will only get those fields that are associated to that view and vise versa on the deserialize.

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonView;

//You can however many views we want to work with in our objects. They must all be classes. 
//If you extend a class then it will contain all properties of that view and the view it extends.
public class MyClassViews {
	public static class ViewA {}
	public static class ViewB extends ViewA {}
}

//Let's create an object for our serialization and deserialization
public class MyClass {
	private final int id;
	private final String value;

	//This constructor is what the deserializer would use.
	@JsonCreator
	public MyClass(@JsonProperty("id") int id, @JsonProperty("value") String value) {
	}

	@JsonView(MyClassViews.ViewA.class)
	@JsonProperty("id") //You don't have to put the name you could leave it as @JsonProperty
	public int getId() {
		return id;
	}

	@JsonView(MyClassViews.ViewB.class)
	@JsonProperty("value")
	public String getValue() {
		return value;
	}
}

I didn’t come up with this next part but it’s so awesome that I wanted to include it. Let’s say you had an object that was populated not using any views but the object does contain views that will eventually pass up to a UI but you only want to pass using a specific view. Then you can do the following.

import javax.ws.rs.core.StreamingOutput;
import com.fasterxml.jackson.databind.ObjectMapper;

final StreamingOutput jsonStream = new StreamingOutput() {
	@Override
	public void write(OutputStream out) throws IOException {
		final ObjectMapper mapper = new ObjectMapper();
		mapper.writerWithView(MyClassViews.ViewB.class).writeValue(out, MyClass);
	}
};

If you want to write the object to string using a view. All you need to do is

final String jsonString = new ObjectMapper().writerWithView(ViewB).writeValueAsString(MyClass);

If you then want to convert our new jsonString to the object using the view do the following:

new ObjectMapper().readerWithView(ViewB.class).forType(MyClass.class).readValue(jsonString);

If you want to convert a json string to object.

new ObjectMapper().readValue(myJsonString, MyClass.class)

Java: Connect to Postgres

Below are the steps to setup a connection to a Postgres DB and some other options that you can use.

Pom.xml:

<dependency>
	<groupId>org.postgresql</groupId>
	<artifactId>postgresql</artifactId>
	<version>9.4.1208.jre7</version>
</dependency>

Import:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;

Build The Connection:

Class.forName("org.postgresql.Driver");
Connection connection = DriverManager.getConnection(##URL##, ##USER##, ##PASS##);

Preparing the Query:
We utilise “PreparedStatement” to setup the connection. Which will allow us to use parameters in the query.

PreparedStatement ps = connection.prepareStatement("select id from table where column=?")

If your query had parameters (IE: ?) then you will need to pass in the value for each parameter. If you notice there is ##POSITION## and ##VALUE##. Position is the location of where the parameter appears in the query. You can set various types of data for example integer, json, etc.

ps.setString(##POSITION##, ##VALUE##);

After we perform a query we need to retrieve the data. We use “ResultSet” for that. Depending on how we return the data depends on how we get the data after the query succeeds. Below are examples of one line returned vs multiple rows returned.

ResultSet rs = ps.executeQuery();

if (rs.next()) {
	int variable = rs.getInt("id");
}

while (rs.next()) {
	//Do Something
}

Insert:
If you want to perform an insert you don’t need to do “executeQuery” you can call just “execute”.

ps = connection.prepareStatement("insert into mytable (column) values (?)");
ps.setInt(##POSITION##, ##VALUE##);
ps.execute();

Batching:
Sometimes we have a lot of updates or inserts to perform. We should batch that.

//You setup the preparedStatement and then add to the batch.
ps.addBatch();

//You can set a max batch size to send the batch once it hits that amount
if (++count % batchSize == 0) {
	ps.executeBatch();
}

Passing Json as a Parameter:
We create the postgres object. Set the type to json and the value as a string in json format. Next we set the preparedStatement parameter as the postgres object that we just created.

final PGobject jsonObject = new PGobject();
jsonObject.setType("json");
jsonObject.setValue(value.toString());

//Now set the json object into the preparedStatement
ps.setObject(##POSITION##, jsonObject);

Java: ExecutorService / Future

If you want to spin off a bunch of threads and manage them and their responses effectively you can do it this way.

final ExecutorService executor = Executors.newFixedThreadPool(numThreads);
final Collection<Future<JsonNode>> futures = new LinkedList<Future<JsonNode>>();

//Write a Loop if you want
final Callable<TYPE> callable = new MyClass();
futures.add(executor.submit(callable));
executor.shutdown();

// We need to monitor the queries for their data being returned.
for (final Future<?> future : futures) {
	try {
		final TYPE val = (TYPE) future.get();
	} catch (final InterruptedException e) {
	} catch (final ExecutionException e) {
	}
}

 

The callable works with a class so you will need that.

package mypackage;

import java.util.concurrent.Callable;
import org.apache.log4j.Logger;

public class MyClass implements Callable<JsonNode> {

    static final Logger logger = Logger.getLogger(MyClass.class);

    MyClass() {
    }

    /**
     * This is how each caller queries for the data. It can be called many times and runs on threads from the calling class
     * So data is returned as it gets it.
     */
    @Override
    public TYPE call() throws Exception {
        try {
                  return null;
        } catch (final Exception e) {
            return null;
        }
    }
}

Java IDE Installation for Eclipse

This tutorial will guide you through configuring Eclipse for Java. Ensure you have followed the tutorial on installing Eclipse first.

You should open java and debug perspectives. To do that just go to “Window”–>”Open Perspective”–>”Java”. This opens Java perspective. To open “Debug” you need to go to “Window”–>”Open Perspective”–>”Other”. Select “Debug” and hit “Ok”.

You should also install Maven 2. Go to “Help”–>”Install New Software”. In the “Work With” type “http://m2eclipse.sonatype.org/sites/m2e” and hit “Add”. Then add the name “Maven2” or whatever name you want and hit “Ok”. Then check “Maven Integration for Eclipse” and hit “Next”. Hit “Next” again for “Install Details” and accept the license agreement. and hit “Finish”. You will need to restart.

If you want you can also open “Project Explorer”, “Markers” and “Problems” views from “Window”–>”Show View”–>”Other”.

FindBugs is also a nice to have and I recommend having it :). Go to “Help”–>”Install New Software”. In the “Work With” type “http://findbugs.cs.umd.edu/eclipse” and hit “Add”. Then add the name “FindBugs” or whatever name you want and hit “Ok”. Then check “FindBugs” and hit “Next”. Hit “Next” again for “Install Details” and accept the license agreement. and hit “Finish”. You will need to restart.

You should open the “FindBugs” perspective as well. To do that just go to “Window”–>”Open Perspective”–>”Other”. Select “FindBugs” and hit “Ok”.

Don’t forget to lock Eclipse to launcher if you want.

Optional:

TypeScript IDE:

Used for React. It’s really handy. Open Eclipse. Click Help–>Install New Software. In the “work with” put “http://oss.opensagres.fr/typescript.ide/1.1.0/” and click Add. Follow the prompts and you are done for now. make sure you select “Embed Node.js” and “TypeScript IDE”.

HTML Editor:

Used for HTML files. Click Help –> Eclipse Marketplace. Search for “HTML Editor”. Then click install. After it is installed and Eclipse is restarted Click Window –> Preferences –> Web. Under “HTML Files” change encoding to “ISO 10646/Unicode(UTF-8). Under “Editor” add “div”. You can get more info and configuration from here.

Java: Embed Python

Let’s say you want to embed Python code in your application. You will use Jython.

pom.xml:

<dependency>
      <groupId>org.python</groupId>
      <artifactId>jython-standalone</artifactId>
      <version>2.7.0</version>
</dependency>

*.Java

import org.python.util.PythonInterpreter;
import org.python.core.*;

private static PythonInterpreter interpreter = new PythonInterpreter();

interpreter = new PythonInterpreter(null, new PySystemState());

//You put the key to register in JavaScript and pass the variable in
interpreter.set(KEY, VARIABLE);

//If you wanted to use this in the mapper as an example you would pass the key, value and context to the JavaScript function. That way when you write to the context in Python it writes it to the applications context.

interpreter.exec(PYTHONCODE);

Java: Embed JavaScript

Let’s say you want to embed straight JavaScript code in your application.

pom.xml:

 <dependency>
      <groupId>org.mozilla</groupId>
      <artifactId>rhino</artifactId>
      <version>1.7.7.1</version>
</dependency>

*.Java

 import org.mozilla.javascript.*;

private static org.mozilla.javascript.Context cx = org.mozilla.javascript.Context.enter();

private static ScriptableObject scope = cx.initStandardObjects();

private static Function fct;

//You put the key to register in JavaScript and pass the variable in
scope.put(KEY, scope, VARIABLE);

cx.evaluateString(scope, JAVASCRIPTCODE, "script", 1, null);

fct = (Function)scope.get(METHOD_IN_CODE, scope);

Scriptable mapper = cx.newObject(scope);

//If you wanted to use this in the mapper as an example you would pass the key, value and context to the JavaScript function. That way when you write to the context in JavaScript it writes it to the applications context.
fct.call(cx, scope, mapper, new Object[] {key, value.toString(), context});