If There's A God In The Sky Looking Down What Can He Think Of What We've Done To The World That He Created (Is This The World We Created - Queen)

Hello, this week I have reached 25K visits. Not long ago, I wrote that I have reached 10K visits during Japan earthquake. Now thankful  Japan nuclear crisis seems that has passed.

I would like to say thank you to all people that have read my blog, specially people from theserverside.com and springsource for publishing my posts on their site, and also people that have become followers of my blog.

For now that all, I wish I reach 100K as soon as possible, thank you very much all of you for your support.

Alex.
Music: http://www.youtube.com/watch?v=pUOrAengPH8

Mornië Utúlië, Believe And You Will Find Your Way (May It Be - Enya)

A CAPTCHA is a program that can generate and grade tests that humans can pass but computer programs "cannot". One of strategies followed are showing an image to user with distorted text, and user should write text in input area. If showed text is the same as input by user, then we can "assure" that a human is on computer. A captcha example:

Captchas have several applications for practical security, for example:

• Preventing Spam in comment fields.
• Protecting from Massive User Registration.
• Preventing Dictionary Attacks.
• ...

These distorted texts are acquired as follows:

1. Digitizing physical books/newspaper. 
2. Pages are photographically scanned, and then transformed into text using "Optical Character Recognition" (OCR). 
3. OCR is not perfect, each word that cannot be read correctly by OCR is placed on an image and used as a CAPTCHA.
4. Word that cannot be read correctly by OCR is given to a user with another word for which the answer is already known. Then is asked to read both words, if user solves the one for which the answer is known, the system assumes their answer is correct for the new one. The system then gives the new image to a number of other people to determine, with higher confidence, whether the original answer was correct.

Now you know how captcha works, the problem is that if you want to use captchas in your website, you should implement yourself process described above, and of course this is not easy and tedious work is required digitalizing works. For this reason there are some "captcha providers" that have done this work for us. One of these providers is reCaptcha http://www.google.com/recaptcha. reCaptcha is a free captcha service that provides us these captchas ready to be used in our site. As developers we only have to embedded a piece of code in client side for showing captcha image and text area, and in server side, calling a function for resolving input data. reCaptcha provides plugins for dealing with lot of programming languages like Java, PHP, Perl, ...

This post will guide you on how to use reCaptcha in Spring MVC web application. The application consists in a form to register a new user. This form contains a captcha for avoiding a bot starts a massive registration attack.

First step is open an account to reCaptcha site (you can use your google account or create a new one). 

Once you have entered go to My Account - Add New Site.

Then at domain box you should write the domain which will contain captcha validation. For this example I have entered localhost and I have checked Enable this key on all domains (global key). Of course information provided here is for testing porpoise and in production environment should be different. After you have registered your site, two keys are provided, private key (XXXX) and a public key (YYYY).

Before coding, let me show basic life-cycle of a reCAPTCHA challenge. Diagram is from reCaptcha web:

Second step is create a Spring MVC application, no secret here, I am going to explain only parts that are implied in reCaptcha integration. Apart from SpringMVC dependencies, recaptcha4j API should be added:

recaptcha4j.jar is an API that provides a simple way to place a captcha on your Java-based website. The library wraps the reCAPTCHA API.

Integrating reCaptcha into a form, requires two modifications:

• One in client side, for connecting to reCaptcha server and get the challenge.
• Second one in server-side for connecting to reCaptcha server to send the user's answer, and give back a response.

Client side:

For client side a tagfile has been created to encapsulate all logic of reCaptcha API in a single point, so can be reused in all JSP forms.

reCaptcha class requires the private key (XXXX) and the public key (YYYY) provided by reCaptcha in step one. The method createRecaptchaHtml(...) creates a piece of html code to show the challenge. In fact it generates something like:

And finally a JSP page with a form and captcha information:

See that form is generated as usual using Spring MVC taglib, but also we are using created tagfile (<tags:captcha>) for embedding captcha into form.

Server Side:

Server side is even simpler than client side. When a captcha is created using createRecaptchaHtml, two form element fields are created, recaptcha_challenge_field that contains information about the challenge presented to user, and  recaptcha_response_field that contains the user answer to the challenge.

Apart from these two parameters, recaptcha4j requires remote address too. ServletRequest interface has a method (getRemoteAddr()) for this porpoise.

reCaptcha object is injected using Spring. It is important to note that UserInfo (data entered by user in form) does not contain any information about captcha, it only contains "business" data. Using @RequestParam reCaptcha information is retrieved by Spring and can be used directly into reCaptcha object.

The other important part is isValid() method. This method simply checks if response of reCaptcha site is that user has been passed the challenge or not. Depending on result you should act consequently, if challenge is not passed returning to previous page is a good practice.

This bean definition is simply for instantiating reCaptcha class with your private key. Using @Autowire bean is injected into controller.

Step Three:

Last step is watch that created form shows the captcha image and controller redirects you to page depending on what you have entered into captcha text area.

Extra Step:

Now you have a basic notion of how to work with reCaptcha, next step (out of scope of this post) is instead of showing again form without any error message, you could use BindingResult in Controller for notifying to user an error message:

result variable is an attribute passed to submitForm of type BindingResult. Of course JSP should be changed with <form:errors path="captcha"/> for showing the error message.

Another improvement is creating  a HandlerInterceptor for validating forms with captchas. For example  ReCaptchaHandlerInterceptorAdapter would contain reCaptcha management. preHandle method would return true if captcha challenge is resolved correctly by user (allowing defined controller do its work), or false and redirecting to an error page.

With previous handler configuration all forms would have captcha validation.

Hope you find useful this post, and now you can start protecting your web forms from spam or bots.

Download Eclipse Project.

Music: http://www.youtube.com/watch?v=XOykCYDMKBs

This Is a Flash Of Pure Inspiration, Més I Més I Messi, Però Més Però Molt Més (The Feet Continue To Dance - The Wizard Of Ox)

Git is a distributed revision control system, where every working directory is a full-fledged repository with complete history and full revision tracking capabilities. 

Git is categorized as DVCS (Distributed Version Control System), because is not dependant on a central server. So the academic way for working with Git is pushing/pulling data from/to each developer repository. This works in small teams or in a highly distributed development (open source projects that people are working around the world), but in mid-size teams or business companies, that require a central repository because of infrastructure/workflow process like Continuous Integration System, QA Checks before delivering, Environment Backups, External Manual Audits... seem that a traditional SCM should be desired. But this claim is far from reality, Git is still your VCS; how about creating a theoretical central repository? I say theoretical because in Git there is no central repository at a technical level. This repository will act as central because of convention. I call, and in many other posts also call this repository origin.

A Git remote repository is a repository without working directory. Only composed by  .git project directory and nothing else.

Nvie has created a nice schema of this topology:

See that each developer pulls and pushes to origin, but also may exchange data with other peers. For example, if two or more developers are working on a new feature, they can push changes between them before pushing stable version to origin repository.

Git is not tied to any particular transmission protocol, it supports transmitting changes via USB stick, email, ..., or traditional way like HTTP, FTP, SSH, ...

So although Git has broken the typical SCM hub architecture to peer-to-peer structure, we can still create (by convention) a central repository for uploading stable code. And let me write again, "This central repo is just another node in the peer not THE REPOSITORY".

What I am going to explain is how to install and configure this "central repo" in an Ubuntu Server.

We can say that Git only takes care of repository management and leaves transport operations to lower layers. A typical transport configuration for these central repos is using SSH protocol. So let's install and configure a SSH server. (if you have already installed skip to next step).

Install SSH Server:

$ sudo apt-get install openssh-server

after installed try:

$ ssh <username>@<servername>

Configure SSH Server:

In /etc/ssh/sshd_config configure to only use SSH Protocol 2: 

Protocol 2

Next step is to install Git: (You can skip this step if you have already installed).

Install Git (not git-core package):

$ sudo apt-get install git

Then execute Git command to check that has been installed correctly.

Next step is creating a bare repository for the project. By convention, bare repository directories end with .git. So first thing to do is create a .git directory of project. 

Creating a bare repository from existing repository:

$ git clone --bare my_project my_project.git

This command transforms the /my_project/.git to my_project.git.

Creating a new bare repository:

If you are starting a new project you can initialize it directly as bare repository using:

$ mkdir my_project.git

$ cd my_project.git

$ git --bare init

Now all structure is created and ready to be transferred. Case that initial project was started on developer computer you should copy this directory (using scp for example) to origin.

Then execute next command:

$ git init --bare --shared

This command will add propertly group read/write permissions.

And now it is time to clone created repository to developer computer, I assume that developer has already an account in server (for connecting using ssh). So go to developer computer (or open another terminal) and type next command:

$ git clone <username>@<servername>:/<directories>/my_project.git

If user has read permissions to my_project.git directory, repository will be downloaded to local computer. Write permissions are required for checking in changes.

And now I suppose you are thinking that it was so easy creating a remote repository, but now another problem arises. If your company is small you can manually create a new user into your server for each developer, it should be easy to manage, but if your company is bigger, then management of all users is hard. You must create an account for each one, and more important, they will have access to server shell using ssh (not only for uploading code) or ftp, ..., and this fact implies a problem with security, you should take care of what a user can do and what cannot do in his shell.

So arrived at this point, one can setup accounts for everyone, which is straightforward but can be cumbersome. Another way is using an LDAP or any other centralized system, but this is alien topic for this post.

A second method is to create an account called "git" on the server, and ask every user who will have  access, to send its SSH public key, and add that key to the .ssh/authorized_keys file of "git" user. I am sure that this approach sounds you familiar (github way?). So let's explain this way:

First of all each user should send you its public key, (they can find in .ssh directory *.pub file), or simply create new, using ssh-keygen command. See this tutorial for learning how to generate both keys http://github.com/guides/providing-your-ssh-key.

Setting up Git server with user public keys:

First step is create a git user with .ssh directory.

#from server
$ sudo adduser git
$ su git
$ cd
$ mkdir .ssh

Next step is create authorized_keys file where all public keys will be stored:

For example:

#from server
$ cat id_dsa.user1.pub >> ~/.ssh/authorized_keys
$ cat id_dsa.user2.pub >> ~/.ssh/authorized_keys

And now each developer, with public key published in authorized_keys and private key in his own .ssh directory, has access to repository. Let's try, open another terminal (would be developer machine in real scenario) and try to clone existing repo from server:

#from developer computer
$ git clone git@<servername>:<directories>/my_project.git

After repository is cloned to developer computer, modifications can be made and pushed them.

And now you can say, "Ok, I don't have to create one account for each developer but I am still having a problem with security", each developer still has access to shell. Yes it is true, but you can easily restrict the "git" user to only doing Git activities with a limited shell called git-shell. Next step is specifying git-shell instead of bash for Git user, in /etc/passwd.

$ sudo vim /etc/passwd

and change

git:x:1000:1000::/home/git:/bin/sh

to

git:x:1000:1000::/home/git:/usr/bin/git-shell

Now your server is secured, only Git operations are allowed using "git" account with users that have sent their SSH public key.

You have your central remote repository configured and ready to be used; at this point you may consider install Git tools like gitweb, gitosis or gitolite, but in this post are off topic.

I hope you have found this post useful.

Music: http://www.youtube.com/watch?v=q2AemC0cwy0

To Seek Out New Life And New Civilizations, To Boldly Go Where No Man Has Gone Before (TNG Soundtrack - Star Trek)

From Wikipedia: OAuth (Open Authentication) is an open standard for authentication. It allows users to share their private resources (e.g. photos, videos, contact lists) stored on one site with another site without having to hand out their credentials, typically username and password.

There are a lot of posts talking about OAuth from Client Side, for example how to connect to service providers like Twitter or Facebook, but there are less posts about OAuth but from Server Side, more specificaly how to implement an authentication mechanism using OAuth for protecting resources, and not for accessing them (Client Side Part).

In this post I will talk about how to protect your resources, using Spring Security (Spring Security OAuth). The example will be simple enough to understand the basics for implementing an OAuth service provider.

I have found this post that explains with a simple example, what OAuth is and how it works. I think it is a good starting point with OAuth http://hueniverse.com/2007/10/beginners-guide-to-oauth-part-ii-protocol-workflow/

Now it is time to start writing our service provider. First of all I will explain what our Service Provider will offer.

Imagine you are developing a website (called CV) where users will register and after that they will be able to upload their Curriculum Vitae. Now we are going to transform this website to a Service Provider where OAuth will be used for protecting resources (Curriculm Vitae of registered users). Imagine again that some companies have agreed with CV people that when they publish job vacances, users will have the possibility of uploading their curriculum directly from CV site to HR department instead of sending by email or copy & paste from document. As you can see here is where OAuth starts managing security between CV website and Company RH site.

In summary we have a Curriculum Vitae Service Provider (CV) with protected resource (document itself). Companies that offer users the possibility of acquiring directly their Curriculum Vitae from CV are the Consumers. So when a user visits company job vacancies (in our example called fooCompany) and wants to apply for a job, he only has to authorize FooCompany "Job Vacancies" website with permissions to download its Curriculum Vitae from CV site.

Because we will use Spring Security for OAuth authentication, first of all we are going to configure Spring Security into SpringMVC CV application. Nothing special here:

In web.xml file we define Security Filter:

And in root-context.xml we define protected resources and authentication manager. In this case In memory apporoach is used:

Next step, create an Spring Controller that returns the Curriculum Vitae of logged user:

This controller returns directly a String, instead a ModelView object. This String is sent directly as HttpServletResponse.

Now we have got a simple website that returns the Curriculum Vitae of logged user. If you try to access to /cvs resource, if you are not authenticated, Spring Security will show you a login page, and if you are already logged, your job experience will be returned. Works as any other website that are using Spring Security.

Next step is modifing this project for allowing external sites can access to protected resources using OAuth 2 authentication protocol.

In root-context.xml:

First bean, is an OAuth2ProviderTokenServices interface implementation with id tokenServices. The OAuth2ProviderTokenServices interface defines operations that are necessary to manage OAuth 2.0 tokens. These tokens should be stored for subsequent access token can reference it. For this example InMemory store is enough.

Next bean is <oauth:provider>. This tag is used to configure the OAuth 2.0 provider mechanism. And in this case three parameters are configured; the first one is a reference to a bean that defines the client details service, explained in next paragraph. The second one is token service for providing tokens, explained in previous paragraph, and the last one is the URL at which a request for authorization token will be serviced. This is the typically Authorize/Denny page where service provider asks to user if it permits the Consumer (in our case fooCompany) accessing to protected resources (its Curriculum Vitae).

Last bean is <oauth:client-details-service>. In this tag you define which clients you authorize to access to protected resources with previous authentication. In this case because CV company has agreed with foo company that they can connect to its Curriculum Vitae Service, a client is defined with id foo.

Now we have our application configured with OAuth. Last step is creating a controller for taking requests from /oauth/confirm_access URL.

This controller returns a ModelAndView object with client information and which page should be shown for granting permission to protected resources. This JSP page is called access_confirmation.jsp and the most important part is:

As you can see Spring Security OAuth provides helper classes for creating confirmation form and deny form. When the result is submitted, URL /cv/oauth/user/authorize (internally managed) is called, there OAuth decides if returns protected resource (String returned by loadCV() method) to caller or not depending on what option user has chosen.

And that's all about creating an OAuth 2 system using Spring Security OAuth. But I suppose you are wondering how to test it, so for the same price I will explain how to write the client part (Consumer) using Spring Security OAuth too.

Client application (called fooCompany) is also a SpringMVC web application with Spring Security.

 Spring Security part will be ignored here.

The client application contains a home page (home.jsp) that has a link to Spring Controller that will be responsible to download Curriculum Vitae from CV site, and redirecting content to a view (show.jsp).

As you can see is a simple Controller that calls a Curriculum Vitae service. This service will be responsible to connect to CV website, and download required Curriculum Vitae. Of course it deals with OAuth communication protocol too.

Service looks:

The suggested method for accessing those resources is by using Rest. For this porpose Spring Security OAuth provides an extension of RestTemplate for dealing with OAuth protocol. This class (OAuth2RestTemplate) manages connection to required resources and also manages tokens, OAuth authorization protocol, ...

OAuth2RestTemplate is injected into CVService, and it is configured into root-context.xml:

See that OAuth2RestTemplate is created using an OAuth resource that contains all information about where to connect for authorizing access to protected resource, and in this case is CV website, see that we are referencing an external website, although in this example we are using localhost. Also service provider URL (http://localhost:8080/cvs/cv) is set, so RestTemplate can establish a connection to content provider, and in case that authorization process ends successful, retrieving  requested information.

<oauth:resource> defines OAuth resources, in this case, the name of the client (remember that this value was configured in server side client details tag for granting access to OAuth protocol). Also userAuthorizationUri is defined. This is the URI to which the user will be redirected if the user is ever needed to authorize access to the resource (this is an internal URI managed by Spring Security OAuth). And finally accessTokenUri, the URI OAuth provider endpoint that provides the access token (internal URI too).

Also creating a consumer using Spring Security OAuth is simple enough.

Now I will explain the sequence of events that happens when a user wants to give access to foo company for retrieving its Curriculum Vitae.

First of all user connects to foo website, and click on post curriculum vitae link. Then getCV method from controller is called. This method calls cvService, that at the same time creates a connection to resource URI (CV) using OAuth2RestTemplate. And this class acts as a black box, from client side, you don't know exactly what this class will do but it returns your Curriculum Vitae stored in CV website. As you can imagine this class manages all workflow related to OAuth, like managing tokens, executing required URL redirections to get permissions, ... and if all steps are performed successful, stored Curriculum Vitae in CV site will be sent to foo company site.

And that's all steps required to allow your site to act as Service Provider using OAuth2 authorization protocol. Thanks of Spring Security folks, it is much easier that you may think at first.

Hope you find it useful.

Download ServerSide (CV)
Download ClientSide (fooCompany)