As you have probably noticed, this blog has not been updated for a while. It doesn’t really fit my needs anymore so I created a new general blog to take over. The new address is:
Feel free to update your links.
I just watched a very interesting video on YouTube about software development leadership which is on the Google Tech Talks Channel (here), it’s pretty long but worth a watch if that kind of thing is interesting to you. At the very end of the talk (at about 1:26:45) is an interesting story.
The story is about an Italian philosopher that was walking through a quarry a long time ago and came across a stone cutter. He asked the stone cutter “What are you doing?” to which the stone cutter replied “I’m cutting stone. I don’t like it but it’s the only job I could get”. So the philosopher moves on and asks the next guy he comes across the same question: “What are you doing?”. The second stone cutter replies “I’m earning a living. I’m putting a roof over my head and food on the table”. So the philosopher continues and asks a third stone cutter the same question “What are you doing?”. The last stone cutter said “I’m building a cathedral”.
In a software development team you want your developers to be Cathedral Builders not Stone Cutters. How do you know if a developer is a Cathedral Builder? The way to tell is what your developers do when there is something about the job that annoys them. If they complain about it they are stone cutters, if they ignore it they are earning a living but if they fix it then they are Cathedral Builders.
According to the presentation, you get Cathedral Builders by moving responsibility and decision-making to the lowest possible level which allows people to reach their full potential and become Cathedral Builders. It’s the individual that counts and allows them to use there own initiative. The leader’s job is to have the overall vision of the project and to guide the developers through it. If there is something that is annoying the developers, a good leader will help them fix it, not get in the way. You need to make your developers feel valued, help them to feel proud of the product they are making and you need to listen to their ideas. You need to create a culture of constant improvement.
I’m paraphrasing the presentation of course and propbably not doing it justice. So take a look.
OK, I’m currently in the middle of refactoring the current architecture. I’ve been reading about Messages, DTOs (Data Transport Objects) and SOA (Service Oriented Architecture). What I’ve been reading makes sense but I’m finding it hard to locate some really good examples of how it should be done. So I’m going to outline my current understanding and initial idea of what I’m going to do. Hopefully somebody with some experience in this field can give me a few pointers.
Service methods should be course grained which minimize round trips from the client. To my way of thinking a client could be remote and communicates with our services via web-services or a local web/windows application that calls services directly. So in our application, the client is our MonoRail web application where the controllers call the injected services directly. In a Service Oriented Architecture our services should model real world business processes rather than domain entities. I must admit that I sometimes find it hard to name services correctly so they aren’t named after domain entities, but I guess this will come with more experience with SOA.
When I first encountered the idea of DTOs I didn’t really understand the point of them. I thought, why can’t I just pass up the domain objects through the layers up to the front-end? After all, I was always taught that I should reuse code and use the DRY principle (Don’t Repeat Yourself). DTO’s by their very nature introduce code duplication, which seemed all wrong.
I have now seen the light and understand their benefits which out way the code duplication issue. Their benefits include:
Ben has a more detailed post about the advantages of using DTOs which is well worth a read.
Mappers are pretty simple classes that the service uses to map from a domain entity to a DTO and back again. The mappers that I’m using will map from a particular domain entity to a particular DTO or collection of DTOs. Currently I’m doing this “manually”, but I’m thinking about ways to do this automatically using reflection and matching on properties that match by name and type. More on this in a later post if I still think it is a possible solution.
Depending on what you read, the terms Messages and DTOs seem to be interchangeable. However I think there is a difference. I’m currently thinking (which may be wrong) that a service call should accept a request message and should return a response message. Both should be unique to the service call as they are course grained. These messages can contain domain DTOs which are tailored to the particular usage. The DTO’s can be reused in multiple service calls.
I feel an example coming on to help demonstrate my thoughts.
Imagine we have a User domain entity which comprises an id, first name, last name, email, user name, password, status and permissions list. We also have a UserStatus domain entity which includes an id, name and code. These domain objects would be mapped using an ORM such as NHibernate or ActiveRecord which we want to isolate from the front-end.
Now imagine in our front-end we want to be able to show a list of users and be able to filter by their status. We might create a UserDTO which is stripped down to only the fields we want to show in the list:
We might also create a UserStatusDTO that contains the data required to show a filter with all possible status options:
Next we need a single service call that will return a list of users and all possible statuses.
Finally we need two messages which will be unique for the service call, one request and one response:
Obviously we need an implementation for our service interface which I won’t go into now but the service should basically validate the request, call the required repositories, map the domain entities to DTOs and construct the response which is returned. The service now has clear responsibilities.
The benefits here are pretty clear. We have a single service call to return two object lists rather than a separate call for each list. We also stripped down the returned data to only what is required. These benefits are especially useful when calling the service over the network. When called from our MonoRail application (our client) this helps to keep the controllers as thin as possible and allow them to do their job cleanly.
We could extend this example by providing service calls for retrieving and saving a user (for editing). We’d need a new user DTO (for example UserDetailsDTO) which includes the data that a user is allowed to edit and new request and response messages. But I think you get the idea now (and this post is pretty big now).
I’ll be refactoring the current PixelBugs architecture to match this pattern which you can take a look at on Google Code for more examples.
So what do you think? I’d be interested in any thoughts or comments even as a confirmation that I’m on the right lines. Of course I may have completely misunderstood the concepts, if so, please let me know.
I’m going to start taking this blog in a slightly different direction from now on. Up to now I’ve been giving a step-by-step account of development of the PixelBugs project. Since my last post, I’ve made quite a few changes and to describe them all in detail here would take up too much of my very limited time. So from now on I’ll be highlighting important changes, decisions and interesting snippets of code rather that the full commentary. Of course the source is still all available on Google Code as before so you can always check it out. I’ll still be developing in the same TDD way:
This is know as red-green-refactor. So on that note, what’s changed since last time?
The first major change is the move to cards rather than issues. The reason for this is that we have just started to introduce agile development at work at I’m really happy with the way that is going. So I have decided to turn this project into an agile project management tool rather than a traditional issue tracker. I’m not sure what that means for the name, but we’ll see what happens.
So to introduce cards I had to refactor the existing tests that used issues then refactored the issue domain/service with their new names. That was a pretty straight forward and all tests passed after this was done.
Next I extended the domain to include things like card status, priority, type and comments. I also added some other properties for things like story points:
Once these extra properties were added I had to refactor the new card view to have status, priority and type selects and a new story points input box. In addition to this, I refactored the card index view to display cards in lanes by status just like a card wall in real life might be (not styled yet):
I’ve added a show (details) page for a card when it is clicked on the wall and also a new card editing view. I found one interesting bug when editing an existing card. There is a select for the card owner that has an option added to the start of the list so that you can select “No Owner”. To do this I used the firstoption parameter of the $FormHelper.Select method. This worked fine if you are setting it to a user, but if you set a card owner to “No Owner” from a user, it never updated. I tracked this down to the fact that I’m using Guid’s as my primary keys and the default value for the firstoption is 0 (zero). Now because the databinder doesn’t know what to do in this case it doesn’t change the owner value even though I had AutoLoadBehavior.NullIfInvalidKey set. The fix was pretty easy, just add a firstoptionvalue parameter and set it to empty string. This equates to null when databinding and so fixes the bug. Here is the NVelocity:
jQuery in, prototypejs out
So how does this work? Well, it’s actually very straight forward. The first block simply initialises the cards to be draggable. This is done with the CSS selector “div.card”, we then define what should happen when dragging: we’d like to drag a clone rather that the actual card div, we’d like it to be 50% transparent and we’d like to only drag from a handle (not the whole div). The reason for the handle is to fix the conflict between clicking a card to see its details and clicking a card to start the drag.
The second block deals with where the cards can be dropped. So we use another CSS selector to define the lanes a droppable. We specify that they will only accept cards and the CSS class to add when hovering over a lane. Then, when we drop a card we show the wait animation and fire an Ajax call to an action in the controller to update the card status. If this succeeds we move the card in the UI, if it fails we leave the card where it is and show an error message.
The next change was to refactor the new and edit views to remove some duplication. The main part of the form is the same for both views, so I’ve brought that out into it’s own partial view and then use this in both the new and edit views.
Now that we have removed this duplication I wanted to add a WYSIWYG editor instead of the standard textarea control. I looked into TinyMCE but this seemed to be really heavyweight compared to my current needs. After some more searching I found NicEdit which has all the features I currently need and is very lightweight. To add NicEdit to our existing textarea we just use the following:
This code allows us to specify where the icons are located and we can pick which buttons display, very easy to setup and configure.
You may have noticed that I’m not using screenshots for code in this post. That’s because I’ve switched to using Microsoft Live Writer which has a plug-in for formatting code snippets. This has really helped and has reduced the amount of time I spend writing posts which means I can spend more time coding, wahoo!
That’s for for today, except to say that we now have 53 passing tests and 100% code coverage in our web and core assemblies. PixelDragons.Commons has no test coverage at the moment and that’s because this was created before I started TTD. As I touch Commons, I’ll add tests to rectify this. As usual the full source code is available on Google Code:
Svn Revision: r19
Hello again and welcome to the latest edition of Developing PixelBugs. This time around we’re not adding any new features, but instead we’ll be adding a service layer and some refactoring. During this process I found a reason to upgrade to the Castle trunk from RC3, but more on that later.
Adding a service layer
So far we haven’t anywhere for our business logic to sit. It shouldn’t sit in the controllers and it shouldn’t sit in the repositories. So we need the service layer. To cover the functionality so far we need to add two services, one for security and one for managing issues.
I first created a new interface called ISecurityService which has two methods. One for authenticating a user from their username and password and returns a security token. The second method validates a security token and returns the authenticated user:
Next I added some unit tests for this service and refactored the existing security controller tests so that the controller now only takes an ISecurityService as a constructor argument. I modified the tests as required to work with this new interface (going through the normal process) and registered the service with Windsor (via a new services.config file) to allow the acceptance tests to pass. I also refactored the authentication filter tests to use ISecurityService and the authentication filter implementation.
All the tests are now passing. I really see the value in tests now and have full confidence that it’s all working after the significant change I’ve just made.
Next I followed the same process to add another service called IIssueService for use by the issue controller:
After completing the process, all tests are passing
Once the service layer was in I decided to carry out some refactoring. Here’s what I did:
I want to stop using the session to store stuff, so I now store the security token in a cookie rather than the session. This avoids the timeout problem and keeps the user signed in until they sign out. This was great except that one of my tests (Authenticate_Success) failed with a NotSupportedException. This was due to Castle RC3 test support not implementing CreateCookie. So I now have the perfect reason to upgrade to the trunk. This is something I always planned to do, but now I have a reason to do it now (see details below).
Here are some other refactorings:
Upgrading to Castle trunk
Here are the tasks I performed to upgrade from Castle RC3 to the trunk. I found some of the information I needed here:
public bool Perform(ExecuteEnum exec, IRailsEngineContext context, Controller controller)to
public bool Perform(ExecuteWhen exec, IEngineContext context, IController controller, IControllerContext controllerContext)
Castle.MonoRail.Framework.EngineContextModulefrom the web.config
Once all of this was done and the tests were all passing again. I thought I better check my test coverage, so I downloaded the last free version of NCover and NCoverExplorer and gave it a run. I found a few areas that weren’t covered by my tests, so I added more tests. We now have 40 passing tests:
and pretty good coverage (apart from PixelDragons.Commons):
So that’s it for this time. Not much of a post I know but hey. As usual the code is committed to Google Code, so feel free to take a look.
Svn Revision: r10
Here we are again ready for another installment. This time around I’m going to be looking at the security of the application. What do I mean by that? Really I mean authentication and authorization. This is another one of those things that is far easier to add at the beginning (like the resources from last time) rather than once there is a large code base. For this edition I’m not going to go into the full detail as I have before as I think you know the process I go through now, but I will be highlighting the main areas of interest. Don’t forget that all the source is available from Google Code. To recap, this is my normal process:
Without further ado, here is the new class diagram which contains the changes for this iteration which we’ll talk about below:
I’m going to start off with some pretty simple authentication to get the ball rolling. Of course I’m always thinking about improvements and future refactorings, I’m sure we will revisit this area again. I’m currently thinking about storing a user name and password for each user. A user should be able to enter their credentials on a sign in page which submits to an action that authenticates them and stores their user id in the session. Then I’ll create a filter to check the session and load up the user before each action. If the session expires or the loading of the user fails we will redirect to a nice error page and cancel execution of the requested action.
So here is a controller unit test that tests the authentication of correct credentials:
As you can see here I’m calling the repository directly, but I think I’ll be introducing a service layer pretty soon which would be a nicer interface. I will probably increase the security too by only storing the password hash and matching on that instead, but that’s for another day. The next unit test is for our authentication filter:
And here is the implementation of this filter, notice that we add the current user to the controller property bag for convenience:
Finally for authentication, I’ve created a number of acceptance tests that test the filter and security controller. The problem with these tests is that they rely on some test data that needs to be in your database before they pass. So inside the Core assembly is a folder for sql scripts where I’ve added a script for populating the database with test data. Make sure you run this script before running the tests. To make acceptance tests easier to write I’ve created a SignIn method that encapsulates the sign in process that can be called from other tests as all other pages are now protected:
And here is an example of using this method in a simple test:
As I mentioned above, I’ve created a number of acceptance tests so take a look at the source for more examples.
For authorization I’d like to define a number of fine grained permissions that control each feature of the application such as CreateIssues, ViewIssues, EditIssues, DeleteIssues etc. Then these permissions could be grouped together into roles. There might be some built-in roles in the system (such as Administrator that have all permissions), but a user should be able to modify the permissions in a role and even create their own roles. These roles could then be granted to a user. The user’s roles (and therefore their set of permissions) should then control which parts of the application they can access. I’d like this implemented in the easiest way possible for the developer to use and should control which controller actions can be accessed and whether or not to show a particular UI element in the views.
How are we going to do this? After some research and experimenting here is my current solution:
The User domain class should implement the .NET IPrinciple interface. This enables us to store the user object anywhere that supports IPrinciple (like the monorail context). I’ve looked into the built in security support in .NET but I had some issues. The main problem was that it made unit testing much harder. I’d prefer to add attributes that do not impact on testing. It required passing literal strings around which isn’t great. The attributes were also a bit hard to read. So I’ll create a custom attribute to mark the actions with the permission that is required. Then I’ll create an authorization filter to check the permissions on an action and for the current user and only allow execution to continue if the user has the authority to.
Looking at the class diagram above, notice that our User class now has a collection of Roles and a Role has a collection of Permissions. A permission is a system entity that we want to code against. So it makes sense for it to be an enum. We will therefore create our own custom mapping to render the permissions list for a role into a comma separated list which can be stored in one field in the Role table (mainly to reduce the sql joins when loading a user). There is also a new method in the User class called HasPermission which will take a Permission (enum) and check whether that user has the given permission.
Here are the User.HasPermission tests that now pass (after going through the normal process outlined above):
I’ve created a number of unit tests for the new AuhorizationFilter. Here is one that tests the normal success case:
I guess the more interesting parts of this is the actual implementation of the custom attribute, how it’s used and the AuhorizationFilter:
You mark a controller action like this:
Here is the AuthenticationFilter that checks the permissions:
As you can see from the filter, if the current action requires a permission we get the current user and check to make sure they have the authority to execute it. If they do execution continues as normal but if they don’t we redirect them to a nice error page. If the action doesn’t require any permissions the filter allows execution to continue. Sweet!
One final part of authorization is hiding UI elements based on permissions. For example, on the issue list page is a “New Issue” link that takes you to the new issue form. Obviously we want to hide this link to users who do not have the permission to create issues. If you remember, as part of our authentication filter we store the current user in the property bag. This allows us to do this:
I’m not a fan of passing literal strings around but I didn’t know of another way in NVelocity. So I’ve overriden the HasPermission user method to accept a string which is converted to a Permission enum. If this conversion fails an exception is thrown which is ok.
So after all that we now have 24 passing tests:
Well I think a service layer will be required plus some improvements to the repository calls we are making. We’ll also need to improve the issues list with paging etc and add some more properties to an issue. Once we’ve done that I’d like to spend some time on the UI and also on how issues are organized. Possibly via Client – Project – Version – Cycle. But more on this later.
As always the full source is available via Google Code. Thanks to BenL for pointing out the missing files etc that I forgot to commit, oops!
Svn Revision: r9
In this installment I’ll be extending our rather basic Issue domain class to include some extra properties. These properties won’t be the complete set I have in mind, but in the interest of incremental changes (and an amount to work to I can write about in one post), I’ll limit it. We’ll be adding three new properties, CreatedDate, AssignedTo and ReportedBy. The later two will require a new domain class: User. Here is the class diagram of what we intend to add:
First of all we’ll change our New_Success unit test to mock out a new repository called IUserRepository and check that the property bag has a list of users that we get from the new repository that can be used to set the issue’s ReportedBy and AssignedTo properties. You might also notice that I have changed the way we set repositories for controllers. We no longer use public property setters, but now we use the constructor. The cool thing about using Windsor is that we didn’t need to change any configuration. Anyway, here is our new test:
Again, this doesn’t build as User and IUserRepository do not exist. So lets add them in the same way we added the Issue and IIssueRepository. We also need to add the new constructor parameter to the controller and the other unit tests.
The solution builds again now but running the tests, 3 fail. The New_Success unit test fails because we haven’t added the code to the controller that populates the property bag. The two acceptance tests fail because we haven’t injected the new IUserRepository and so the controller cannot be created. Adding these few things now fixes all the tests.
Next on the list is a change to the New_RenderNewIssuePage acceptance test to ensure that a select control exists on the page for the “Assigned To” and “Reported By” user for the issue being created. Running this test fails as expected, so we add controls to the Issue/New.vm view as follows:
All the tests now pass again but we haven’t added the new properties to the Issue class yet. So we change the New_CompleteFormAndSubmit acceptance test to check for these properties. We haven’t stored any users yet (need to solve the test data problem) but for now we can test against the issue being set as unassigned and reported anonymously.
As expected the test fails, so we add the properties to the Issue class and modify the list view to display them:
Running the tests again fail, why? We are getting the following error message:
SqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 PM
This is because we are not setting the new CreatedDate property in the Create action. So lets add that and run the tests again. We get passed that error, but now the test is failing because the “Unassigned” and “Anonymous” text is not present, instead we have: $issue.AssignedTo and $issue.ReportedBy, this is because these properties are Users and they are set to null.
What we need now is a Helper. Helpers are a nice place to encapsulate your view UI formatting code and other such helper methods. Can you guess what’s coming? Yes a new unit test for our non-existent UI Helper:
Of course as we’ve seen before, this doesn’t compile. So we just add the new UIHelper class with an empty method called FormatUser. The solution builds now and the tests fail as expected. Once we add the implementation, these new tests now pass. Of course the New_CompleteFormAndSubmit acceptance test still fails, to fix it we need to register the helper with the IssueController and then use it in the view like so:
As you may have noticed above, I’ve also added a helper method for formatting dates. I won’t bore you with the details here but I followed the same process as the other helper methods above. We now have 8 tests that pass!
Test data for acceptance tests
From the above tests I can see that I haven’t tested the case of selected a user for either the “Assigned To” or “Reported By” fields in my acceptance tests. To do that I need some test data (i.e. a user to select) which I don’t currently have. I’ll have a think about this and see if I can find a good solution. I decided to add a new test for this case and for now, I populated the database with a test user and then created a new acceptance test which selects the first user in the select list (index 1) and then confirms that the name of the user selected is on the following page:
Of course this is only a reliable test if there isn’t already an issue in the list with the same user selected. So we need a better solution for test data.
To get this test to pass I had to make a few minor changes. The first was to change the select list names to include .id so they now should be issue.assignedTo.id in the tests and the views. I also needed to change the base class of the controller to ARSmartDispatcherController so I could use the ARDataBind attribute in the Create action:
Using ARDataBind allows MonoRail to automatically load up the nested objects (the users) from the database via ActiveRecord based on the id’s posted from the form. This loading only works if the select list control name ends in the primary key field, in this case Id.
We now have 9 tests passing (as long as there is a user in the database for the last test). So far so good.
One other job I’ve done in this round is replacing all the hard coded text in the views with resource strings. This is very easy to do and from experience best to put in from day one. To do this I just added a new resource file (.resx) to the web application for this controller and then added a reference to it in the controller class:
Then in the views for this controller, just use the syntax $strings.RESOURCE_NAME. Simple really.
Just a few minor housekeeping changes this time. I noticed that the unit test asserts had the expected and actual parameters the wrong way round, so I’ve swapped these. I’ve also added a FullName property to the User class. This property isn’t under the control of ActiveRecord, it just concatenates the users first name and last name for convenience.
That’s it for now, you can get the full source code from Google Code:
Svn Revision: r7