Sunday, July 20, 2008

Unit Testing T4 Text Templates

Text Template Transformation Toolkit, aka T4, is a template-based code generation engine included in Visual Studio 2008 and available for Visual Studio 2005 through the Domain Specific Language (DSL) Tools and the Guidance Automation Extensions (GAX) and Guidance Automation Toolkit (GAT). Text template transformation works with a very simple formula:

TextTemplateFile + Metadata = OutputFile

A text template is a file that contains a mixture of text blocks and control logic. Metadata is provided to the text template as properties defined at its header section. When you transform a text template, the control logic combines the text blocks with the data in a model to produce an output file. Text templates can be used to create any kind of text files, such as source code files and HTML reports. This is similar to some commercial tools such as CodeSmith.

This post from Oleg Sych provides more details about T4 and it includes many links to other resources.

How to Unit Test T4 Templates?

One approach to unit test text template files is to use the Visual Studio Text Templating engine to supply the required data and run the transformation of the template. The Visual Studio Text Templating engine will allow you to see all the errors you might get when processing it.

In order to do this, you will need to install GAX/GAT and add the following references to your project:
  • Microsoft.VisualStudio.TextTemplating
  • Microsoft.Practices.RecipeFramework.VisualStudio.Library
The following sections will describe a sample text template file and a NUnit test for it using the Visual Studio Text Template engine.

The Text Template

The following code represents the template file that we will be testing:

<#@ Template Language="C#" #>
<#@ Assembly Name="System.dll" #>
<#@ Assembly name="TextTemplateTesting.dll" #>
<#@ Import Namespace="System" #>
<#@ Import Namespace="TextTemplateTesting.Data" #>
<#@ Property Processor="PropertyProcessor" Name="EntityName"#>
<#@ Property Processor="PropertyProcessor" Name="Fields" #>
using System;
using System.Collections.Generic;
using System.Text;

namespace BusinessEntities
{
public partial class <#= EntityName #>
{
public <#= EntityName #>()
{
}

<# foreach (Field field in Fields) { #>
private <#= field.Type.Name #> _<#= field.Name #>
public <#= field.Type.Name #> <#= field.Name #>
{
get { return _<#= field.Name #>; }
set { _<#= field.Name #> = value; }
}
<# } #>
}
}

This text template generates a class representing a business entity. Remember that our simple formula is TextTemplateFile + Metadata = OutputFile. The data that is necessary to generate the code is defined by the two processor properties in our template above, and they are:

<#@ Property Processor="PropertyProcessor" Name="EntityName"#>
<#@ Property Processor="PropertyProcessor" Name="Fields" #>

The EntityName processor property is a string that will represent the name of the class in the generated code. The Fields processor property is an array of Field objects representing the properties of the class in the generated code. The Field class is not shown here, but it has two properties: the name of the field and its type. You can download the entire source code from the link at the end of this article.

The Text Template NUnit Test

I am using NUnit to test the text template, but you can use any other compatible test framework. The testing code will be:

using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.IO;
using System.Text;
using Microsoft.Practices.RecipeFramework.VisualStudio.Library.Templates;
using Microsoft.VisualStudio.TextTemplating;
using NUnit.Framework;
using TextTemplateTesting.Data;

namespace TextTemplateTesting.Tests
{
[TestFixture]
public class TextTemplateFixture
{
[Test]
public void TestTextTemplate()
{
// Determine the root dir of the template files.
// Since the template files are located in
// SolutionDir\Templates and the unit tests
// run inside SolutionDir\bin\debug,
// the root dir is ".\..\..\Templates"
string rootDir = Path.Combine(Directory.GetCurrentDirectory(),
@"..\..\Templates");

// Our template file expects data to be passed.
// The following dictionary will contain
// the properties to be passed to the template file.
IDictionary<string, PropertyData> arguments =
new Dictionary<string, PropertyData>();

// The first property is a string with the business entity name.
arguments["EntityName"] = new PropertyData("Customer", typeof(string));

// The second property is an array of Field objects.
Field[] fields = { new Field("CustomerName", typeof(string)),
new Field("CustomerId", typeof(int)) };
arguments["Fields"] = new PropertyData(fields, typeof(Field[]));

// Create an instance of the template host
TemplateHost host = new TemplateHost(rootDir, arguments);

// Set the template file
host.TemplateFile = Path.Combine(rootDir, "BusinessEntity.tt");
Assert.IsTrue(File.Exists(host.TemplateFile),
"Cannot find " + host.TemplateFile);

// Create an instance of the template engine.
Engine engine = new Engine();

// Transform the text template.
engine.ProcessTemplate(File.ReadAllText(host.TemplateFile), host);

// Check if there are errors.
if (host.Errors.HasErrors)
{
// Build a string with all errors
StringBuilder sb = new StringBuilder();
foreach (CompilerError error in host.Errors)
{
if (!error.IsWarning)
sb.AppendLine(error.ErrorText);
}
Assert.IsFalse(host.Errors.HasErrors,
"There are compiling errors:\n\n" + sb);
}
}
}
}

The transformation is very simple. You need basically two main objects: the host and the engine.

The first main object, the host, will contain information such as your text template file name, the data used for the transformation, and the root path where the transformation will happen. Note that text templates can also include other text templates using relative path, and they are relative to this root path defined on the host constructor.

The data is passed to the host through the arguments variable. This variable is a dictionary type where the keys are the processor property name defined in the template file, and the value is a PropertyData object specifying the value and type of the processor property.

The engine is the second main object which provides the ProcessTemplate method. This method takes a string representing the text template file, and a host object as parameters. The Errors collection of the host object will contain all errors and warnings from the transformation. The test code above only fails if there are errors, it ignores all warnings.

I hope this sample provides an useful way for unit testing your t4 files. Testing manually text templates is a time consuming activity which can be impossible to accomplish depending on the total number of templates you are using. By using automated tests you can ensure that changes in the code are not breaking your templates, increasing the confidence of your developers in their code and the confidence of your customers in your product.

Enjoy it!

Sample Code Download

You will need Visual Studio 2005 (or later), GAX/GAT and NUnit installed in order to run this sample code.

Friday, July 04, 2008

WCSF Architecture

Simon Ince has published a series of articles about the Web Client Software Factory architecture. It is a must read if you are using WCSF. The latest article can be found here, which has links to the previous ones.

Wednesday, July 02, 2008

Free data grid control for Silverlight: DevExpress AgDataGrid Suite

DevExpress announced today the availability of their new data grid control for Silverlight. They are providing it to the Silverlight developer community free of charge. Yes, it is free!! They also provide its full source code.


If you never heard about DevExpress, they develop Windows Forms and ASP.NET components, plus IDE productivity tools (CodeRush and Refactor!). I used their ASP.NET components, especially the ASPxGridView which provides superior functionality than the standard ASP.NET GridView. Now they are releasing their first Silverlight control, but providing it for free.

See the link below for more information about this free control, including registration and download. Also, check out the online demo and the video tutorials:

Friday, June 27, 2008

Aspect Oriented Programming (AOP) using AspectJ

You probably have already heard about Aspect Oriented Programming (AOP). AOP is a technique for addressing issues that cross cut objects. A cross-cutting concern is a concern across the scope of an application, it may be a constraint or simply a behavior that every class must perform.

A typical example of cross-cutting concern is logging. The logging aspect of an application is common to many classes across its multiple layers. Besides logging, other examples of cross-cutting concerns are security, caching and transaction. The following diagram, shows the multiple layers of an application (data access, business logic, services interfaces, UI), and some cross-cutting concerns (logging, security, transaction). An aspect is just another term for cross-cutting concern.



What is AspectJ?

AspectJ is one of the most popular aspect-oriented development tools for Java. It is a simple extension to the Java programming language that adds to Java the aspect-oriented programming capabilities.

Downloading and Installing AspectJ

You can download AspectJ from here. Select the latest stable release. At the time of this post, the latest stable release was AspectJ 1.6.1rc1. Just run the downloaded jar file and follow the instructions to install it. It is assumed that you have already Java installed on your machine.

Do not forget to change your PATH and CLASSPATH as suggested at the last screen of the installer application.

AspectJ Concepts and Terms

Before using AspectJ, lets define some important concepts. In AspectJ, cross-cutting concerns can be represented as a special type called aspect. This special type contains two special elements: a pointcut and an advise.

The advice is the code that is executed when an aspect is invoked. For example, if the aspect is logging, the advice will be the code that actually writes some information to a log file.

A pointcut specifies where an aspect will provide an advice, e.g. where the advice code will be executed. The places where this code can be executed are known as joint points, e.g. points within an application that may or may not invoke an advice. Thus, a pointcut binds an aspect to specific join points of the classes. Examples of join points are: when a method is called, when the object is initialized, when an exception is thrown, when a class field is assigned, etc.

Logging Example

The following code snippet represents an aspect to log all exceptions caught by the application.

import java.io.*;

public aspect ExceptionLoggingAspect {

// Specifies the calling advice when any exception is caught.
pointcut exceptionHandlerPointcut(Exception exception) :
handler (Exception) && args(exception);

// Advice declaration.
before(Exception exception) : exceptionHandlerPointcut(exception) {
// Exception logging.
String filename = "log.txt";
try {
String text = "Error: " + exception.toString() + "\n";

FileWriter out = new FileWriter(filename, true);
out.write(text);
out.flush();
out.close();
} catch (IOException ex) {
System.out.println("Failed when logging exception to " + filename);
}
}
}

In this code, the pointcut handler(Exception) binds to join points where an exception is caught, e.g. to code containing catch (Exception ...) statements. Note that this pointcut is for all types of exceptions since we are using the java.lang.Exception class. You could use an specific exception if you want to limit this aspect to be applied to certain exception types. The pointcut
args(exception) allow us to retrieve the exception object and pass it as a parameter to the advice method.

The advice method receives the reference to the exception caught and use the FileWriter class to log its details to a log file called log.txt. The advice method could also throw an exception, for example if the log file cannot be created in the file system. In this situation, an exception thrown inside the advice method will not be logged, since the advice method is not a join point. So, there is no risk of having an infinite loop.

The code above is the only code necessary for exception logging mechanism.
In a traditional approach, we would need to change all the many places where we are catching an exception and add a call to a method to log the exception. This could be a very time consuming task depending on how many places we have to change, and also resulting in duplicating code across the whole application. Using AOP, the joincut will make all the hard work to bind the joint point (catch statements) with the exception logging aspect.

Security Example

This second example is an aspect to detect malicious strings passed to method calls. Web applications are vulnerable to Cross-Site Scripting (XSS) attacks that happen when a user executes a Web page containing script code that has been injected from other sources. A malicious user can take advantage of the lack of proper data validation in data entry page. Fields could be filled with Javascript code that will be persisted in the database when the information is submitted. When this data is rendered in some Web page, the script code will be executed, causing a security flaw in the Web application.

One way to avoid XSS attacks is to validate all strings entered by users and reject those containing malicious scripts. Usually data types use standard method signature to set string fields, for example setSomeString (String text). The following code represents an aspect to validate the string parameter of all setter methods of our data types.

import java.util.regex.*;

public aspect XSSDetectionAspect {

// Specifies calling advice whenever a method from any class
// returns void and receives a string parameter.
pointcut callPointCut (String text) :
call (void *.*(String)) && args(text);

// Advice declaration.
before(String text) : callPointCut (text) {
// Using a regular expression to match valid strings, e.g.,
// containing alphanumeric characters, spaces and the following characters ,;:@().
Pattern pattern = Pattern.compile("^[0-9a-zA-Z\\s.,;:@()]*$");

Matcher matcher = pattern.matcher(text);
if (!matcher.find()) {
throw new IllegalArgumentException("Illegal string parameter: " + text);
}
}
}


In the code above, the pointcut will pickup calls from any method that returns void and accepts a string as a parameter. When a method call satisfies this condition, the advice code is executed. Note that we could restrict to pickup calls from methods belonging to classes of a specific package. For example, if we want to include only classes from com.myapp.datatypes package, we could use the following call pointcut:

call (void com.myapp.datatypes.*.*(String))

The advice method receives the string to be validated. If the string passed does not match the pattern for valid strings (e.g. alphanumeric, space, punctuation, and other characters), then an exception is thrown.

As in the previous example, the code above is everything we need for the string validation mechanism. In a traditional way, we would add validation code to all methods that set fields of string type on the data objects, resulting in duplicated code and maintenance issues.

Sample Code Download

In order to compile the sample code, you will need to install AspectJ (see section above) and then just run ajc *.java at the command prompt window.



For More Information

Tuesday, June 17, 2008

MCAD Certification and Next

I finally got the confirmation from Microsoft of my MCAD certification. I had some troubles with Prometric, the test provider, which erroneously reported that I did not show up for my 70-320 test. Ouch!
After a few emails/phone calls, having to fax my exam score report , and wait for two weeks, Microsoft confirmed my certification. Well, it is finally done!

My original plan was to get the MCAD certification done back in 2005. I passed on the first two exams (70-315 and 70-316) during that year. At that time, I started my graduation program MSc. in Information Systems at Athabasca University, and also moved from Edmonton to Mountain View - California for a new position at Intuit Inc. Well, I got pretty busy and had no time to study for the final exam, putting the MCAD on hold. In the meanwhile the new generation of MS certifications were released (MCTS and MCPD). My goal, of course, is to be certified in the new generation of certifications (MCPD: Enterprise Applications Developer 3.5), but I still want to finish what I started back in 2005, otherwise all the effort for the 2 first exams would be wasted.

What is next?

As I mentioned, I want to get the MCPD certification (MCPD: Enterprise Applications Developer 3.5). I have many choices:
  • 2 upgrade exams to MCSD, then 2 upgrade exams to MCPD Ent. App. Dev. 2.0, and finally 2 upgrade exams to MCPD Ent. App. Dev. 3.5
  • 1 upgrade exam to MCTS Web or Windows Developer 2.0, then 3 exams for .NET 2.0, and finally 2 upgrade exams to MCPD Ent. App. Dev. 3.5
  • start from the scratch and take 70-536 exam (Application Development Foundation) plus all the 5 .NET 3.5 required exams.
No matter way I decide to go, I will have to take 6 exams. So, I decided to start from the scratch. I don't like upgrade exams because they are just subset of normal exams. For example, the exam 70-559 to upgrade to MCTS Web Development .NET 2.0, includes most of the content of the 70-536 plus the content of 70-528. In this case, the content of one upgrade exam is equivalent to almost two regular exams. Also, if you are using the training kits (books) from MS Press, there is no specific book for an upgrade exam, you will need to get the book of each regular exam that it covers. And the final reason to start from the scratch is that it is much more fun to study the latest technology (.NET 3.5) then going to the previous one before.

The following figure shows the possible paths to get to MCPD Enterprise App. Dev. 3.5. The path I am taking (.NET 3.5 from the scratch) starts from 70-536 and go to 70-562 and so on (right hand side).


For more information, see:

Saturday, June 14, 2008

Moving subversion repository from one server to another

Recently I had to move one subversion repository from one server to another and it is a very simple procedure.

The first step is to dump all the information of your current repository into a file. The following command will dump the repository named reponame (located at /svn_old_path) into the file named svndump.

sudo svnadmin dump /svn_old_path/reponame > svndump

Then, copy the svndump file to the new server. I am assuming that subversion is already installed on the new server, if not, this is the guide that I followed:

https://help.ubuntu.com/community/Subversion

I setup subversion to access via WebDAV protocol with SSL encryption (https).

On the new server, I will place all subversions repositories under /home/svn. First, we need to create a new repository:

cd /home/svn
sudo svnadmin create reponame

Now, use the load command to restore all information from the dump file into the new repository:

sudo svnadmin load reponame < svndump

Since I am using subversion access through https, I need to set the proper permissions on the files. In my case, the apache user is www-data and I created a group called subversion.

sudo chown www-data reponame -R
sudo chgrp subversion reponame -R

Finally, change the subversion configuration in Apache and add the new repository. In my case, the configuration file is /etc/apache2/mods-available/dav_svn.conf, and the changes are:

<Location /svn/reponame>
DAV svn
SVNPath /home/svn/reponame/
AuthType Basic
AuthName "Subversion repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
</Location>


Then restart apache:

sudo /etc/init.d/apache2 restart

Sunday, May 04, 2008

VMware Server on Ubuntu 8.04 - Hardy Heron

In my original post, I described how to install VMware client/server on Ubuntu 7.10. Everything was working fine until I updated my Ubuntu to Hardy Heron 8.04. It turned out that there is a patch available it. The following two posts helped me on getting VMware Server working on Hardy Heron:

Spring Boot Configuration Properties Localization

Spring Boot allows to externalize application configuration by using properties files or YAML files. Spring Profiles  provide a way to segr...