'Spring'ing the new app

'Spring'ing the new app

What to expect from this post?

Building something new, something from scratch, something worth is an opportunity most yearn for, if not all. I feel blessed and lucky to have had opportunities of developing not one but a few of such projects. And yet again, I have an opportunity - a bigger one, a better one, a bit more challenging one.. to build a product .... to architect, design, develop the product and take it to the cloud!

This post intends to provide insights into 'Spring'ing this new app - how our team is approaching building this product, using 'Spring'.

Spring - 'Soul' of the app

Choosing the framework for a J2EE application is really not difficult these days. Spring is such comprehensive a framework, that more often than not, it is the obvious selection as a framework for a J2EE application. What made Spring work for us, besides being a solid, high quality implementation, is the wide availability of features that expedite the development process such as:

- support for restful web services

- spring-hibernate integration

- support for transaction management

- spring-jdbc integration

- spring-quartz integration

- support for asynchronous execution

- support for cross-cutting concerns, through aspect-oriented-programming

- support for exception handling

- support for validations, conversions and custom annotations

- support for unit testing

- support for authentication, through spring-security

.... continued .... the list is big.

REST and Hibernate : hand-in-hand development

The first steps to building an app are setting up the infrastucture - building the controller and integration layers. We chose REST and Hibernate, respectively, as the foundation for our app.

REST-ifying the App

The motivation to go the 'RESTful' way strong considering the inclination to architect our application as microservices. The Spring MVC's support for RESTful Web services and it's easy integration with JSP's or the javascript MVC/MVVM frameworks made Spring MVC a perfect fit for our app. Spring's message converters added the cherry to the cake - simply use libraries like Jackson, JAX-B, XStream, etc and we are done.

Hibernate-ing the App

After evaluating few NoSQL and RDBMS databases, we had zeroed down on Oracle, simply because it fitted our use-case, oracle is considered the best and team was already working on oracle. While there was a direction to go the simple JDBC route since we do not need the 'database-agnostic' feature of hibernate, we went against the direction and implemented hibernate as the database integration layer - to account for the uncertainties in the world of software development. This decision also meant we embrace the learning curve involved for hibernate and we did put our best foot forward. 

Guess what, the situation changed, we had not other option and circumstances demanded we move to 'MySQL' from 'Oracle'. The team's decision to implement hibernate, as the data integration layer, paid off since we needed just a week to migrate our app from Oracle to MySQL. If you have a question 'Why did we need a week to migrate from Oracle to MySQL, if we were using Hibernate', I will address it in another blog post later.

Transaction Management, and integrating JDBC

JDBC Integration

While we used hibernate as our data integration layer API, we also needed JDBC integration for our 10% use cases of batch inserts and updates. Spring proved to be great supporting both integration APIs with its consistent programming model.

Transaction Management

Spring has comprehensive support for transaction management - declarative as well as programmatic, for different transaction APIs like JTA, JDBC, Hibernate, JPA and JDO. 

We adopted the declarative approach, annotation-based with no or little programming required, which is also the recommended approach.

We have two transaction managers, one for hibernate and one for JDBC, working in conjunction with each other.

Other aspects considered in transaction management were 'transaction boundaries', 'read-only' transactions and 'propagation type'.

- Major decision around 'transaction boundaries' was to introduce the facade controller and place the 'transaction boundary' of the transaction over this facade controller.

- The getter APIs are marked 'read-only' so that hibernate optimizes around dirty-checks and manual flush.

- Two types of propagation type were used - 'PROPAGATION_REQUIRED' and 'PROPAGATION_REQUIRES_NEW' - as both use cases were required to be supported.

Centralized Exception handling

One of the good practices of REST APIs is to respond with correct exception details:

- HTTP Status code

- Error Code

- Error Message

Spring provides perfect support to achieve this RESTification practice. With '@ControllerAdvice' annotation, the centralized exception handling can be setup to handle different exceptions and respond correctly with the exception details.

To be continued..

Spring-Quartz integration

Asynchronous execution

Cross-cutting concerns, through aspect-oriented-programming

Support for validations, conversions and custom annotations

Support for unit testing

Support for authentication, through spring-security

... to add more features as encountered and utilized.

Architecting Microservices

Architecting Microservices

What to expect from this post?

You have heard the buzzword .. Yes, go the 'microservices' way!
We heard this term too and started to go down this route..

I intend this post to walk you through what follows there-after.. the path of evaluating, discussing the fundamental aspects, pros and cons of microservices, deciding if we are ready to take 'microservices' on !

Starting 'Microservices'

What's such a big deal with microservices? Microservices is a whole new ball game, in terms of  operations, DevOps(CI/CD), development and architecting.

Below are few interesting discussions we had when we went out exploring Microservices:
- Microservices is a good thing but nobody has done Microservices here. You will not get any help if you get stuck.
- We will need massive investment in terms of infrastructure, monitoring, deployment pipeline. Do we want to go this route, especially if it is only our team and not the organizational guideline?
- We must avoid distributed transactions. Let's not over-complicate the system.
- We should go the Microservices route. You should justify microservices so that management decides on Microservices.
- We are not convinced either ways - To Microservice or not to microservice.

Conclusion : With all the background discussions, we took the first step and decided we will not go the mcroservices way big bang but will keep our architecture in line with Microservices.

Continuing with 'Microservices'

We went understanding the microservices more.. jotting down few more interesting discussions around microservices aspects:

Granularity : 

- I don't think we should have so many micro-services. This should not be a micro-service...

- Breaking into microservice makes sense if we need scalability for this module. Why break application unnecessarily into microservice?

Conclusion: We decided to divide the system in independent logical/functional stand-alone units and let these units serve the purpose of micro-services.

Indepenence :

- Why do we need separate database for each microservice?

- Why cannot 'Module-1' just invoke 'Module-2 database'?

Conclusion: Each microservice has its own database which makes the service completely independent and a replaceable unit.

Inter-service communication :

- Inter-service communication is a big overhead .. system becomes slow. It's a concern with microservice architecture. Do we need to have microservices?

- Why shouldn't we have synchronous inter-servicecommunication.

Conclusion: Inter-service communcation is an overhead but probably an affordable one with the advantages of microservices. The communication can be synchronous or asynchronous. However, with network partitioning coming into picture for distributed computing, asynchronous communication takes care of distributed transaction via eventual consistency.

Service Discovery :

- Why can't we just invoke the REST end-point of Module-2 from Module-1?

- Why do we need any service registry?

Conclusion: Service discovery helps with dynamically allocated resources and more so with dynamically changing resources (like with auto-scaling, failures, etc). NGinx, Eureka are few examples for service discovery.

API Gateway :

- What is an API Gateway?

- Why do we need an API Gateway?

Conclusion: We have decided to do away with an API Gateway, as becoming part of an already existing (chosen/decided) architecture.. By the way, an API Gateway is a server that is the single entry point into the system. It is similar to the Facade pattern from object-oriented design. The API Gateway encapsulates the internal system architecture and provides an API that is tailored to each client.

To be continued...

Implementing 'Microservices'

Handling Distributed Transactions

Dockerizing Microservices

... and will add as things become clearer..

Authoring your REST APIs

Authoring your REST APIs

What to expect from this post?

REST APIs is the trend in the town these days and having worked on REST APIs for  a few years, recently going through setting up our new application on REST, this post intends to add some practical ingredients to the REST application development context.

Below are few pointers to author your REST APIs:

Design REST APIs not to one user interface

The REST APIs should be designed generic and must not tie with the user interface. There could be many versions of user interface that different UX designers can come up with. Also, what holds 'true' today might not be the same tomorrow. Infact, you can have customers using your APIs and have completely different user interfaces. 

How do we deal with this? 

Pretty simple? Not really.. Design REST APIs that can serve different user interfaces. Yes, this is more work. But, if you consider different user interfaces, you will design your system in a manner that is optimal for different user interfaces. Remember, design is something which is not very easy to change once you have your APIs being consumed. Do it right the first time.. at least, try to !

An example of how this can impact your design :
We decided to provide a macro level API which operate on the complete object graph and granular APIs which operate on one level entities. To achieve this, we had to design our hibernate entities differently - we implemented 'CascadeType.DELETE' to delete the complete object graph on One-To-Many relations and used 'session.merge' for inserting/updating these relations.

On-demand REST APIs

Your REST APIs can be consumed by variety of clients - servers, browsers, thick clients. Do not design your REST APIs for a particular client.

Communication

Browsers can handle JSONs well, some clients handle XML better. Your REST APIs should support different media types .. communicate in language that the client demands !

Selective fetches

You do not need all the data all the time. Some client just need a few attributes or just the meta. Provide a way to select the data you need from the API. For example, mobile interface shows just the product name and price. No need to fetch the product description, image .. provide just what the client demands!

Graceful, precise, meaningful exceptions

Your APIs should precisely tell what went wrong! 

Do not respond with complete verbiage of your application stack ! 

REST APIs in case of an exception should respond with:

1. HTTP status code

2. Error code

3. Error message

Diligent HTTP Methods(GET, POST, PUT, PATCH, DELETE)

There are few guidelines around Resource URIs and methods:

- Use GET to retrieve, POST to create, PUT to update, DELETE to delete

- Use 'plural' resources as resource identifiers

- Do not use method names in URIs like 'CreateXXX', 'GetXXX, 'DeleteXXX'

- Use Path parameters for required parameters and query parameters for optional ones

So far so good. 

Any exceptions to the rule(s)? 

Yes, there a few tricky ones where you can choose to go otherwise..

- Use a 'POST' for bulk delete of resources. Servers like Tomcat do support a body in 'DELETE', but not all elements in the pipeline might support the same. 

- Use a 'PATCH' for partial update of resource. Saves the bandwidth since you won't pass along the full object representation.

- Use a 'POST' for APIs like advanced search, sorting, pagination. These APIs might make the GET payload cross the 64K limit. Also, not all elements of the network might be supporting the GET payload.

To continue .....

Search, Sort and Paginate your REST APIs

Version your REST APIs

Document your REST APIs well

Support compression