Executing Commands and Scripts Remotely with ssh

Often it’s required to execute on a remote server a command or a whole bash script.
Not everyone knows that through ssh it’s possible to execute this task.

Here’s the ssh syntax:

usage: ssh [-1246AaCfgKkMNnqsTtVvXxY] [-b bind_address] [-c cipher_spec]
           [-D [bind_address:]port] [-e escape_char] [-F configfile]
           [-i identity_file] [-L [bind_address:]port:host:hostport]
           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]
           [-R [bind_address:]port:host:hostport] [-S ctl_path]
           [-w local_tun[:remote_tun]] [user@]hostname [command]

In the following example , the ls command is run on the remote server.

ssh remoteuser@remoteserver.com ls

To run a local script on the remote server, it’s required to upload it (through scp), set it as executable and finally run it.

The related example:

scp myscript.sh remoteuser@remoteserver.com:/remotedir/myscript.sh
ssh remoteuser@remoteserver.com "chmod +x /remotedir/myscript.sh"
ssh remoteuser@remoteserver.com /remotedir/myscript.sh

Of course it’s required to type the password after each command (or to use a identity key file)

Looking through the ssh options it’s possible to find many other feature offered by this common command.

Copying Files between Clients and Servers over ssh using scp

It’s quite common to need to upload or download file between one or more servers and the local computer.

If it’s available a ssh access on the servers, using scp to transfer file from and to the server could be a very good option.

Here’s its syntax:

usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2

A very simple example:

scp localfilename.txt remoteuser@www.remotehost.com:remotefilename.txt

The previous example copies localfilename.txt from the local directory to the server www.remotehost.com using remoteuser as the ssh account to authenticate on the remote server. On the remote server the transferred file will be stored as remotefilename.txt in the default login directory of remoteuser.

Copying file from and to specific directories:

scp /localdir/localfilename.txt remoteuser@www.remotehost.com:/remotedir/remotefilename.txt

Compared to the previous example, in this case, the file is taken from /localdir/localfilename.txt and stored remotely on /remotedir/remotefilename.txt.
Obviously remoteuser should have write permission on the remote directory where the file is going to be written.

In the next case, the authentication is made through a keyfile, this is the syntax:

scp -i keyfile /localdir/localfilename.txt remoteuser@www.remotehost.com:/remotedir/remotefilename.txt

In this case to login as remoteuser there will not be a prompt for password, but keyfile is used as identity file.

It’s even possible to copy directly files from one server to another

scp firstremoteuser@www.firstserver.com:/filename.txt anotherremoteuser@www.anotherserver.com:/remotedir/remotefilename.txt

Finally one of the best features is to copy recursively directory trees to the remote server:

scp -r /localdirectory remoteuser@www.remoteserver.com:/remotedirectory

In this case, the whole content of localdirectory is recursively copied into remotedirectory. This can be very useful for moving quickly website structures.
I hope you’ve found some useful information in this tutorial.

Migrate MySQL database from latin1 to utf8

Unluckily it’s very common not to change the default charset of your MySQL server and, since the default is latin1, when someone wishes to store cyrillic or chinese character there are many problems.

The first step is to fix the MySQL installation in order to store internationalized information., so locate your my.cnf configuration file on Linux, or the my.ini on Windows boxes.

Search in the configuration file the [mysqld] section when there is the configuration of the MySQL server.

Insert the following lines and eventually remove any existing configuration option with the same name.

[mysqld]
character-set-server=utf8
default-collation=utf8_unicode_ci

The option character-set-server=utf8 tells to the server that, if not otherwise specified, the character set of the created databases, tables, column will be utf8.

utf8 columns will be able to store cyrillic or simplified chinese character, just to give you two examples.

The collation defines how alphabetical ordering will happen, in few words which is the order of the letters that we expect on ORDER BY columnName clauses.

The suffix _ci means that ordering and comparison will be case insensitive and this is the common behavior used in databases.

Be very careful, because usually programming languages (i.e. Java) have case sensitive .equals(String string) method on String class, so it’s quite common to have some mistakes caused by this incongruency.

Then look for the [client] section of your configuration file, and write this line below it.

[client]
default-character-set=utf8

This is very important because it defines the character set used by the MySQL command-line client, and that’s what will be used to migrate the data from latin1 to utf8.

Now everything is setup, restart MySQL to make sure it’s using the updated configuration, and shut-down any application that is using the database that’s going to be migrated.

First, mysqldump will create a .sql file containing all the data:

mysqldump --skip-set-charset --no-create-db –no-create-info -h hostname --protocol=TCP -P 3306 -u username -p old_database > dump.sql

The option --skip-set-charset prevents that in the dump file will be any reference to the old (and wrong) character sets. The options --no-create-db and --no-create-info are used because the new database name will be defined later.

Now the new database is going to be created: mysql -u username -p and the following SQL should be executed in the terminal:

create schema new_database;
quit

Finally the last step is to populate the brand new database with the dumped data:

mysql -u username -p new_database < dump.sql

In this way all the previous data from old_database is now stored in utf8 format in new_database.

I hope this tutorial can be useful, please ask any question or give your feedback.
Thank You.

REST web application with Struts2.1 Rest and Convention plugins

The sample application developed in this tutorial will handle a simple message system, where it’s possible to read, add and remove messages.
So, the basic bean will be defined in the Message class:

package com.zulutown.struts2.rest;

public class Message {
	private String id;
	private String text;
	private String author;

	public Message() {
		super();
	}

	public Message(String id, String text, String author) {
		super();
		this.id = id;
		this.text = text;
		this.author = author;
	}

	public String getId() {
		return id;
	}

	public void setId(String id) {
		this.id = id;
	}

	public String getText() {
		return text;
	}

	public void setText(String text) {
		this.text = text;
	}

	public String getAuthor() {
		return author;
	}
	public void setAuthor(String author) {
		this.author = author;
	}

	public int hashCode() {
		final int prime = 31;
		int result = 1;
		result = prime * result + ((id == null) ? 0 : id.hashCode());
		return result;
	}

	public boolean equals(Object obj) {
		if (this == obj)
			return true;
		if (obj == null)
			return false;
		if (getClass() != obj.getClass())
			return false;
		Message other = (Message) obj;
		if (id == null) {
			if (other.id != null)
				return false;
		} else if (!id.equals(other.id))
			return false;
		return true;
	}
}

The message will be identified by the id property and will contain its text and its author in the other two class properties.

The next step is to setup a basic business service that will handle the basic operations on the messages: find, save, remove and so on. In a real world application this service would be a singleton that interacts with a database (through JDBC or JPA) but in this little demo the data model is just a very basic (and ugly) Map kept in a static property of the Message class.

package com.zulutown.struts2.rest;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class MessageService {

	public static Map messages = new HashMap();
	private static int nextMessageId = 4;

	static {
		Message message1 = new Message("1", "hello", "john");
		Message message2 = new Message("2", "world", "ted");
		Message message3 = new Message("3", "rest", "sam");
		messages.put("1", message1);
		messages.put("2", message2);
		messages.put("3", message3);
	}

	public static List findAll() {
		return new ArrayList(messages.values());
	}

	public static Message find(String id) {
		return messages.get(id);
	}

	public static void save(Message message) {
		if (message.getId() == null) {
			String id = String.valueOf(nextMessageId);
			message.setId(id);
			nextMessageId++;
		}
		messages.put(message.getId(), message);
	}

	public static void remove(String id) {
		messages.remove(id);
	}
}

In messages it is stored the data and the static initializer of the MessageService class will populate the data model with three messages.
Then, some utility methods are provided: findAll() to retrieve all the messages, find(String id) to retrieve a message with a specific id, save(Message message) to add (or update) a message in the data model and, finally, a remove method to delete a specific message.

Until now, this tutorial is not related at all with Struts 2.1, but now it is required to add a “web layer” to the application, and the REST capabilities provided by Struts2 will be used.

The first task is to add to the Web application the .jar libraries of Struts2. Those files must be placed into /WEB-INF/lib (in WebContent) that is the standard location where a Java Web Application expects to find its dependencies.

Download from the Struts website the archives with the libraries for Struts 2.1.x (currently the GA release is 2.1.6), unpack it, and copy & paste the following jars in /WEB-INF/lib/ of your project.

  • commons-beanutils-1.7.0.jar
  • commons-collections-3.2.jar
  • commons-fileupload-1.2.1.jar
  • commons-io-1.3.2.jar
  • commons-lang-2.3.jar
  • commons-logging-1.0.4.jar
  • ezmorph-1.0.3.jar
  • freemarker-2.3.13.jar
  • json-lib-2.1.jar
  • ognl-2.6.11.jar
  • struts2-convention-plugin-2.1.6.jar
  • struts2-core-2.1.6.jar
  • struts2-rest-plugin-2.1.6.jar
  • xpp3_min-1.1.3.4.O.jar
  • xstream-1.2.2.jar
  • xwork-2.1.2.jar

Two Struts2 plugins are used:

  • Rest Plugin
  • Convention Plugin

First of all create at the root of the src directory the struts.xml file:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE struts PUBLIC
    "-//Apache Software Foundation//DTD Struts Configuration 2.1//EN"
    "http://struts.apache.org/dtds/struts-2.1.dtd">

<struts>
    <constant name="struts.convention.package.locators" value="rest"/>  
    <constant name="struts.convention.action.suffix" value="Controller"/>
    <constant name="struts.convention.action.mapAllMatches" value="true"/>
    <constant name="struts.convention.default.parent.package" value="rest-default"/>
</struts>

Convention plugin makes possible to map classes and methods on automatically generated URLs. It is not required the manual “wiring” usually done in the struts.xml configuration file that, in this case, contains just a few lines of configuration. The property struts.convention.package.locators defines the package name where the Convention plugin will look for Struts2 Actions and Controllers, then with struts.convention.action.suffix it is specified that just the classes with the Controller suffix will be automatically mapped. The REST controller that’s going to be defined in the next step, will have the name MessagesController and will be contained in the package com.zulutown.struts2.rest, then it will match both the configurations described above. Given this configuration, the list of the messages in XML format will be accessible through an HTTP GET method on http://localhost:8080/Struts2-Rest/messages.xml, and to obtain the JSON messages list, it’s enough to call the HTTP GET method on http://localhost:8080/Struts2-Rest/messages.json. Simple, isn’t? Now, the simple code of the Controller class that will handle the REST requests:

package com.zulutown.struts2.rest;

import java.util.Collection;
import org.apache.struts2.rest.DefaultHttpHeaders;
import org.apache.struts2.rest.HttpHeaders;
import com.opensymphony.xwork2.ModelDriven;

public class MessagesController implements ModelDriven<Object> {

	private static final long serialVersionUID = 89268916175477696L;
	private Message model = new Message();
	private String id;
	private Collection<Message> list;

	public HttpHeaders create() {
		MessageService.save(model);
		return new DefaultHttpHeaders("create");
	}

	public HttpHeaders destroy() {
		return new DefaultHttpHeaders("destroy");
	}

	public HttpHeaders show() {
		return new DefaultHttpHeaders("show").disableCaching();
	}

	public HttpHeaders update() {
		MessageService.save(model);
		return new DefaultHttpHeaders("update");
	}

	public HttpHeaders index() {
		list = MessageService.findAll();
		return new DefaultHttpHeaders("index").disableCaching();;
	}

	public Object getModel() {
		return (list != null ? list : model);
	}

	public String getId() {
		return id;
	}

	public void setId(String id) {
		if (id != null) {
			this.model = MessageService.find(id);
		}
		this.id = id;
	}
}

In the Struts2 REST plugin, the method name (in the the Controller class) identifies which kind of operation should be executed on specific HTTP requests.

HTTP GET method on http://localhost:8080/Struts2-Rest/messages.xml calls index() and provides the list of all the messages (in the model property of the controller).

The result is:

<list>
  <com.zulutown.struts2.rest.Message>
    <id>3</id>
    <text>rest</text>
    <author>sam</author>
  </com.zulutown.struts2.rest.Message>
  <com.zulutown.struts2.rest.Message>
    <id>2</id>
    <text>world</text>
    <author>ted</author>
  </com.zulutown.struts2.rest.Message>
  <com.zulutown.struts2.rest.Message>
    <id>1</id>
    <text>hello</text>
    <author>john</author>
  </com.zulutown.struts2.rest.Message>
</list>

HTTP GET method on http://localhost:8080/Struts2-Rest/messages/2.xml calls setId("2") (that loads in the model property the Message identified by the provided id) then show().

The result is:

<com.zulutown.struts2.rest.Message>
  <id>2</id>
  <text>world</text>
  <author>ted</author>
</com.zulutown.struts2.rest.Message>

The previous two methods are easy to call with the browser but, to test the other HTTP methods, it’s better to use the Firefox plugin Poster.

HTTP POST method on http://localhost:8080/Struts2-Rest/messages.xml injects into model a new message, then calls create() that persists it.

In the poster dialog box, the URL must be http://localhost:8080/Struts2-Rest/messages.xml with POST action. In Content to Send it’s required to insert the XML entity for the new message (without specifying its id, because it will be automatically generated by MessageService).

  <com.zulutown.struts2.rest.Message>
    <text>new text</text>
    <author>new author</author>
  </com.zulutown.struts2.rest.Message>
Rest - HTTP Post

Rest - HTTP Post

HTTP PUT method on http://localhost:8080/Struts2-Rest/messages/2.xml calls setId("2") (that causes the loading of the existing message with id=2 in the model property) then, depending on which fields are specified in the XML, those message properties are modified and finally a call to update() saves the updated message.

In the poster dialog box, the URL must be http://localhost:8080/Struts2-Rest/messages/2.xml with PUT action. In Content to Send it’s required to insert the XML entity for the fields to edit (the unspecified fields will keep the previous values).

  <com.zulutown.struts2.rest.Message>
    <text>updated text</text>
  </com.zulutown.struts2.rest.Message>
Rest - HTTP Put

Rest - HTTP Put

After the POST and PUT calls, the data structure will look like this:

<list>
  <com.zulutown.struts2.rest.Message>
    <id>3</id>
    <text>rest</text>
    <author>sam</author>
  </com.zulutown.struts2.rest.Message>
  <com.zulutown.struts2.rest.Message>
    <id>2</id>
    <text>updated text</text>
    <author>ted</author>
  </com.zulutown.struts2.rest.Message>
  <com.zulutown.struts2.rest.Message>
    <id>1</id>
    <text>hello</text>
    <author>john</author>
  </com.zulutown.struts2.rest.Message>
  <com.zulutown.struts2.rest.Message>
    <id>4</id>
    <text>new text</text>
    <author>new author</author>
  </com.zulutown.struts2.rest.Message>
</list>

In this tutorial all the URLs refer to xml content type, but it’s possible to use json just changing URLs from messages.xml to messages.json or messages/2.xml to messages/2.json.

I hope this tutorial has given a simple and easy introduction to the REST capabilities of Struts2.1. Please leave your comments and feedback.

Java EE Web Application with Eclipse Ganymede

First of all setup Eclipse Ganymede for Java EE and add to it a Tomcat6 server, as described in my previous tutorial.

Choose, from the File menu New, Dynamic Web Project in the option panel, give a name to the project (i.e. Struts2-Rest), choose a Target Runtime (that you should have already defined, in my case, Apache Tomcat v.6.0) and click Next.

New Dynamic Web Project

New Dynamic Web Project

Here you can define the web application context root, that’s the relative path where it will be accessibile, if you choose Struts2-Rest as context root, your application will be accessible on http://www.yourserver.com/Struts2-Rest or http://localhost:8080/Struts2-Rest if you use the default Tomcat configuration provided by Eclipse (with the Tomcat HTTP Connector on the port 8080)

The Content Directory parameters defines the directory inside your project where is located the “root” of the .war you’re going to generate, use the default WebContent directory.

And, obviously, Java Source Directory defines the directory where your Java sources will be stored. Just for your information, after compilation, the generated .class files are automatically moved by Eclipse into WebContent/WEB-INF/classes.

New Dynamic Web Project - Web Module

New Dynamic Web Project - Web Module

Just click finish. Your brand new project will appear in the Project Explorer

Project Explorer - New Web Project Created

Project Explorer - New Web Project Created

The Deployment Descriptor is obviously associated with the web.xml file. The Java Resources (src) contains your source code, and WebContent contains all the .jsp files, the static contents (images, css, javascripts) and the WEB-INF directory of your .war that will contain the compiled classes (in /WEB-INF/classes) and the libraries (in /WEB-INF/lib).

Accessing current item properties in a Struts2 iterator

Often you can experience some troubles accessing correctly the objects (and their properties) on which you’re iterating through a iterator tag.

Let’s consider this action that simply returns an Item list obtained from our business service.

 
public class MyAction extends ActionSupport {
  private List myList;
  private MyService myService;

  public String myMethod() {
    myList = myService.getItemList();
    return ActionSupport.SUCCESS;
 }
  public List getMyList() {
    return myList;
  }
}

The MyItem bean class:

 
public class MyItem {
  private String name;
  public String getName() {
    return name;
  }
  public void setName(String name) {
    this.name = name;
  }
}

In the jsp we iterate on myList (that results in the execution of getMyList() in the action) and we try to display the name property of each item (defined in the id attribute of the iterator tag) that calls the getName() method of the Item bean.

These are the results:

 
 
 Doesn't work 
 Works 
 Doesn't work 
 Works 
 Doesn't work 
 Works 
 Works 
 

I hope this little test can be useful.

Setup Tomcat6 on Eclipse

Download Eclipse IDE for Java EE Developers from http://www.eclipse.org/downloads/ and tomcat6 from http://tomcat.apache.org/download-60.cgi

Extract both of them where you prefer. I extracted Eclipse in /opt/eclipse and Tomcat in /opt/tomcat6), then run Eclipse.

When you’ll run eclipse it asks you about creating a new workspace (that will be used to store all of your projects), so create a workspace, usually somewhere in your user home directory.

When Eclipse is up and running, choose preferences from the window menu. Choose from the bar on the left: Server, Runtime Environments.

Preferences window in Eclipse

Preferences

Click the button Add, choose Apache Tomcat 6.

New Server Runtime Environment

New Server Runtime Environment

In the next page, browse on your disk and choose the directory where you previously extracted Tomcat. In my case it’s /opt/tomcat6. Finally click Finish.

New Server Runtime Environment - Server Path

New Server Runtime Environment - Server Path

Well, until now we have just told to our workspace where Tomcat “installation” is located on our disk.

But, if we wish to run Java Web Applications within Eclipse, we should setup a Server and eventually assign to it a specific configuration.
Go in the Servers view, right click and choose New, then Server.

Select “Tomcat v6.0 Server” as server type (or probably it will be automatically pre-selected), then, in the Server Runtime environment select box you’ll have to choose “Apache Tomcat v6.0” (that’s probably the only available option.

New Tomcat Server in Eclipse

New Tomcat Server in Eclipse

Click on Next, Eclipse will prompt you to eventually add (or remove) web projects from this Server, in this case, if your workspace is empty you’ll have not any project to add. So, click Finish.

eclipse-add-remove-projects-from-tomcat

Add remove Eclipse projects from Tomcat

In the Servers panel, you will see the Tomcat you just added, and in the Project Explorer view, a new Server configuration will magical appear.

So you can edit the configuration file server.xml as you prefer, change AJP or HTTP connector ports and so on, start/stop/debug the server and obviously add and remove projects from it.

Editing Tomcat configuration in Eclipse

Editing Tomcat configuration in Eclipse

It is also possible to add other server “instances”, just right click again in the “Servers” view, and follow the procedure described before.

In this way, you will just a single “Tomcat” binaries location (that you defined in the first step of this tutorial), but you’ll have the chance to add many instances of that server, each of them with its specific configuration (imagine the /conf directory of Tomcat) and its specific web application (imagine the /webapps directory).

Multiple Tomcat Instances in Eclipse

Multiple Tomcat Instances in Eclipse

For the more expert ones, it is similar to have more instances on the same tomcat binaries defined on different CATALINA_BASE paths.

I hope this tutorial has been useful, please post any question or comment.