Posterous theme by Cory Watilo

thegrubbsian

thoughts on software development, .NET, Ruby, JavaScript, Agility, etc

Filed under: .NET

NoRMatic: a gooey sugar layer for NoRM/MongoDB

It's been out on the Githubs for a few weeks now, but I wanted to do a quick writeup on NoRMatic.  If you're using MongoDB with .NET or Mono, it's pretty likely that you've come across NoRM, the great driver and LINQ provider for MongoDB.  While NoRM is awesome, it can be a little low-level at times and for Crew we needed a library to abstract a few common bits.

Some of the features that NoRMatic provides:

  • ActiveRecord style interface (Save, Delete, GetById methods, etc)
  • SoftDelete
  • Versioning
  • Validation (via DataAnnotations attributes)
  • Save and Delete Callbacks (Before/After)
  • Global Query Filters
  • Basic Auditing (DateUpdated, UpdatedBy)
  • Simple Log Listener
  • Connection String Provider (to connect models to different databases)

Check out the documentation at the Github page for more information.

LINQ-to-SQL Sugar

Yes, yes, LINQ-to-SQL is probably, one day, going to be consumed or deprecated by it's big brother the Entity Framework. However, for the current .NET platform (3.5) and likely for the next (4.0) it is still a viable ORM option in the Visual Studio box. There are lots of reasons to use LINQ-to-SQL and lots of reasons not to use it...that is for another post. What I do want to share is a set of extension methods that (for me) made using LINQ-to-SQL just a little bit nicer. The title is indeed descriptive...these are "sugar" added to LINQ-to-SQL and don't fundamentally improve how it works, just how the programmer works with the library. I'm sure developers using LINQ-to-SQL have created scads of other such saccarine extensions to the framework...if you have some I'd be interested in hearing about them. So here's my LinqToSqlExtensions class: 
public static class LinqToSqlExtensions {

    public static TEntity GetById(this Table @this, int id)
        where TEntity : class {
        return @this.SingleOrDefault(x => ((IIdentified)x).ID == id);
    }

    public static TEntity Upsert(this Table @this, int id)
        where TEntity : class, new() {

        var entity = @this.SingleOrDefault(x => ((IIdentified)x).ID == id);

        if (entity == null) {
            entity = new TEntity();
            @this.InsertOnSubmit(entity);
        }

        if (entity is IAuditable)
            ((IAuditable) entity).ModifiedDate = DateTime.Now;
        return entity;
    }

    public static TEntity Upsert(this Table @this, Func predicate)
        where TEntity : class, new() {

        var entity = @this.SingleOrDefault(predicate);

        if (entity == null) {
            entity = new TEntity();
            @this.InsertOnSubmit(entity);
        }

        if (entity is IAuditable)
            ((IAuditable) entity).ModifiedDate = DateTime.Now;

        return entity;
    }

    public static void Delete(this Table @this, TEntity entity)
        where TEntity : class {
        @this.DeleteOnSubmit(entity);
    }

    public static void DeleteById(this Table @this, int id)
        where TEntity : class {
        var entity = @this.GetById(id);
        @this.Delete(entity);
    }

    public static void DeleteAll(this Table @this, 
        IEnumerable entities) where TEntity : class {
        entities.ToList().ForEach(e => @this.Delete(e));
    }

    public static void CommitChanges(this DataContext @this, RepositoryContext context) {

        var changeSet = @this.GetChangeSet();

        for (var i = 0; i (TEntity entity, RepositoryContext context, 
        bool isInsert) where TEntity : IAuditable {
        
        entity.ModifiedDate = DateTime.Now;
        entity.ModifiedBy = context.UserID;

        if (isInsert) {
            entity.CreatedDate = DateTime.Now;
            entity.CreatedBy = context.UserID;
        }
    }
}
Here's a rundown of the methods it creates and how to use them: GetById Simple enough, allows you to avoid using the LINQ syntax to find an element by it's identifier. 
var thing = DataContext.Things.GetById(3);
Upsert This is probably the most convenient addition. It simply checks to see whether the object exists (by it's ID) if so it returns it if not it creates a new one and returns that. It just avoids a lot of query-for-an-object-and-if-its-null-create-a-new-one-and-insert-it code. 
var thing = DataContext.Things.Upsert(2);
var anotherThing = DataContext.Things.Upsert(x => x.Name == "That Thing");
Delete and DeleteById Pretty obvious... 
DataContext.Things.Delete(thing);
DataContext.Things.DeleteById(3);
CommitChanges This is just a replacement for SubmitChanges that allows us to update some audit columns automagically. Some of this magic is provided via a set of partial classes on those that LINQ-to-SQL generates that applies some additional interfaces to those classes. So, I have a Partials.cs file that contains the following: 
public interface IAuditable {
    public DateTime CreatedDate { get; set; }
    public DateTime UpdatedDate { get; set; }
    public int CreatedBy { get; set; }
    public int UpdatedBy { get; set; }
}

public interface IIdentified {
    public int ID { get; set; }
}

partial class Thing : IIdentified, IAuditable { }
public class OtherThing : IIdentified { }
public class LastThing : IAuditable { }
As long as your generated class (and it's related table) have properties of the same names as the IAuditable and IIdentified interfaces the LINQ-to-SQL extension methods can update them appropriately and in the background. I could have accomplished much of this by just inheriting a new DataContext and adding some additional functionality, but doing it this way you can add these methods to even a closed assembly.

Scriptability via the DLR and PostSharp

Making an application scriptable (particularly in a static language) has historically been difficult. With the advent of the DLR (Dynamic Language Runtime) on the .NET platform it becomes almost trivial to add scripting support to any application. For a recent project I needed the ability to add scripting hooks throughout the application and coupling the DLR with PostSharp AOP attributes made this effort pretty straightforward. Here's how it was done:

Bring in the DLR
The DLR is implemented as a set of plain old DLL's on top of the .NET Framework version 3.5...contrary to popular belief, you do not have to be running VS2010 or .NET 4 to use it.  Simply go to the CodePlex page for the DLR (http://dlr.codeplex.com), grab the source, and compile it.  There are solution files for VS2008 and VS2010. For my purposes I used IronPython as the scripting language but IronRuby is available in the download as well.  To use the DLR your project will need to reference the following assemblies:

  • Microsoft.Dynamic.dll
  • Microsoft.Scripting.Core.dll
  • Microsoft.Scripting.dll

In order to use IronPython, you will also need to reference these:

  • IronPython.dll
  • IronPython.Modules.dll
  • Microsoft.Scripting.ExtensionAttribute.dll

Register the Languages
Now that your project contains all the references it needs it's time to set up the little bit of code needed to host the DLR.  First you'll need to add a few things to your configuration file (App.config or Web.config depending on your project type...this works equally well with either).  Just add the following to the node of your config file:

<section name="microsoft.scripting" 
        type="Microsoft.Scripting.Hosting.Configuration.Section, Microsoft.Scripting, 
                Version=0.9.6.20, Culture=neutral, PublicKeyToken=null" 
        requirePermission="false" />

And add the following somewhere in the root node:

<microsoft.scripting>
    <languages>
        <language names="IronPython;Python;py" 
        extensions=".py" displayName="IronPython 2.6" 
        type="IronPython.Runtime.PythonContext, IronPython, 
        Version=2.6.10920.0, Culture=neutral, PublicKeyToken=null" />
    </languages>
</microsoft.scripting>

This basically just tells the runtime what languages to register, what versions of those languages, and some info about file extensions, etc.

Bring in PostSharp
Now I'm going to skip over to PostSharp for a minute. If you're not familiar with PostSharp you should run over to the website (http://www.postsharp.org) and check it out before continuing. In short, it's an AOP (Aspect Oriented Programming) library for .NET that allows easy injection of cross-cutting concerns through an application primarily via attributes. The 10sec. version of how it works is this...it injects code in a post-compilation step. This sounds scary, but it's not. In fact, this allows it to be extremely fast at runtime even if a small penalty is paid at compile-time. In order to get PostSharp setup, it's a simple download and install. If you don't want to install it on your machine, you can simply bin deploy it but it does require a tweak to your .proj file to get the post-compilation to work. If you want to go that route, you can find instructions here. At this point you will either follow those instructions or if you've installed directly, just include the following assemblies in your project:

  • PostSharp.Laos.dll
  • PostSharp.Public.dll

The ScriptHook Attribute
PostSharp's primary way of injecting an aspect is through attributes. In our case we're going to inherit from the OnMethodBoundaryAspect attribute in the PostSharp.Laos namespace. This attribute exposes four virtual methods that allow injection of code at different points in the execution of a method:

  • OnEntry
  • OnExit
  • OnException
  • OnSuccess

Each of these methods accept a MethodExecutionEventArgs as an argument that provides access (including arguments) to the method on which the attribute is applied. Here's the ScriptHook attribute:

[Serializable]
public class ScriptHookAttribute : OnMethodBoundaryAspect {

    private readonly ScriptEventTypes _scriptEventType;

    public ScriptEventAttribute(ScriptEventTypes scriptEventType) {
        _scriptEventType = scriptEventType;
    }

    public override void OnSuccess(MethodExecutionEventArgs eventArgs) {
        var args = eventArgs.GetReadOnlyArgumentArray();
        ScriptRunner.ExecuteEvent(_scriptEventType, args);
    }
}

For now, ignore the ScriptRunner reference, but notice that I've created an enum (ScriptEventTypes) that distinguishes different "types" of scripts to be run, by types, I mean different usages. At some point you can switch on the type of event that's occurring and change which scripts are loaded or how they are handled, etc. You would use the attribute like this:

[ScriptHook(ScriptEventTypes.ThingSaved)]
public void SaveThing(Thing t) { }

[ScriptHook(ScriptEventTypes.ThingDeleted)]
public void DeleteThing(Thing t) { }

The ScriptRunner
At this point we've got the DLR available and we have the PostSharp attribute that's injecting our hooks. Now we need something to execute our scripts. Enter the ScriptRunner:

public static class ScriptRunner {

    public static void ExecuteEvent(ScriptEventTypes scriptEventType, object[] eventContext) {

    IScriptEvent scriptEvent = null;

    switch (scriptEventType) {
        case ScriptEventTypes.ThingSaved:
            scriptEvent = new ThingSavedScriptEvent();
            break;
        case ScriptEventTypes.ThingDeleted:
            scriptEvent = new ThingDeletedScriptEvent;
            break;
        }

        if (scriptEvent != null) {
            try {
                scriptEvent.ExecuteScripts(eventContext);
            } catch {
                throw new ScriptExecutionExecption();
            }
        }
    }
}

The runner is just a simple static class with a single ExecuteEvent method...it's so simple in fact that it speaks for itself. The one thing to note is the eventContext argument to the ExecuteEvent method. If you look back at the ScriptHook attribute you can see that is' passing an object array from the MethodExecutionEventArgs instance. This object array contains all the arguments to the executing method. So for SaveThing() or DeleteThing() for example the eventContext array contains the instance of Thing that was passed.

The IScriptEvent Classes
Lastly, we need something to actually grab the scripts and execute them in something like the context of the executing method. So that's where our example ThingSavedScriptEvent and ThingDeletedScriptEvent come in to play. Here's an example:

public class ThingSavedScriptEvent : IScriptEvent {

    private readonly ScriptRuntime _scriptRuntime;
    private readonly IThingRepository _thingRepo;

    public ProjectCreatedScriptEvent() {
        _thingRepo = new ThingRepository();
        _scriptRuntime = ScriptRuntime.CreateFromConfiguration();
    }

    public void ExecuteScripts(object[] contextValues) {

        var thing = (Thing)contextValues.SingleOrDefault(x => x is Thing);
        
                var scriptScope = _scriptRuntime.CreateScope();
        scriptScope.SetVariable("thingRepo", _thingRepo);
        scriptScope.SetVariable("thing", thing);

        var script = GetScript();

        var scriptEngine = _scriptRuntime.GetEngine("IronPython");
        scriptEngine.Execute(script, scriptScope);
    }

    private static string GetScript() {

        var script = string.Empty;
        try {
            code = File.ReadAllText(HttpContext.Current.Server.MapPath(
                ConfigSettings.ProjectTemplateStorePath + "ThingSaved.py"));
        } catch (FileNotFoundException) { }
        return script;
    }
}

The ScriptRuntime class is where the DLR magic happens. We create an instance of it and then ask it for an engine that can execute IronPython scripts. ScriptScope is a class used for constructing the scope that the script will have available to it. Here for example we're "passing" the 'thingRepo' and 'thing' variables into the script via the ScriptScope. Lastly, we get the engine from the ScriptRuntime, pass it both the ScriptScope and the literal text of the script. And that's that...pretty easy eh?

The Reusable Test Scope Pattern

I’m almost positive someone has come up with this pattern before, but it was a big help on a recent project so I thought I’d share it. The basic goal of the pattern is to make the setup and teardown work of test classes reusable. We found this to cause particular pain for integration tests written against a database where multiple dependent records needed to be created before the actual test could be performed. The second goal was that these externalized bits of code would be composable so that tests requiring multiple setup resources could create just the right ones. Before really describing the workflow of the pattern, here are the basic components of our implementation: The ITestScope Interface
public interface ITestScope {
    void SetupScope();
    void TeardownScope();
}
These two methods (SetupScope and TeardownScope) are where we move all of the test setup and cleanup logic. Any state that needs to be maintained between setup and teardown (i.e. needs to be available to the test itself) can be stored in public properties or fields on the class implementing ITestScope. The TestBase Class 
public abstract class TestBase {
    public abstract void ScopeSetup();
    public abstract void ScopeTeardown();
}
Requiring that all test classes inherit from TestBase insures that at least developers will be considering the setup and cleanup of their tests. The TestBase.ScopeSetup method would include instantiating and calling the TestScope.SetupScope methods for any scopes that are needed in the test class...likewise for teardown. The TestFactory Class 
public static class TestFactory {
    public static TRepository GetRepository() where TRepository : class {}
    public static TController GetController() where TController : Controller {}
}
This isn't directly related to the scope pattern, but it's invaluable as a point where you can use a DI container in the test project. All repository or controllers (or other resources) can be obtained from here. An Example So here's an example test class that might use this pattern. 
public class CustomerTestScope : ITestScope {
        
        private ICustomerRepository _customerRepo;
        
        public Customer TestCustomer { get; private set; }
        
        public CustomerTestScope(ICustomerRepository customerRepo) {
                _customerRepo = customerRepo;
        }
        
        public void SetupScope() {
                TestCustomer = _customerRepo.Create();
                // Other complex test setup logic
        }
        
        public void TeardownScope() {
                _customerRepo.Delete(TestCustomer);
                // The reverse of the complex setup logic
        }
}

[TestClass]
public class CustomerRepositoryTests : TestBase {

        private CustomerTestScope _customerScope;
        private ICustomerRepository _customerRepo;

        [TestInitialize]
        public override void ScopeSetup() {
        
                _customerRepo = TestFactory.GetRepository();
        
                _customerScope = new CustomerTestScope(_customerRepo);
                _customerScope.SetupScope();
        }
        
        [TestMethod]
        public void SaveCustomer_ShouldRenameCustomer_WhenNewNameGiven() {
        
                _customerScope.TestCustomer.Name = "New Name";
                _customerRepo.Save(_customerScope.TestCustomer);
                
                var customer = _customerRepo.GetById(_customerScope.TestCustomer.ID);
                
                Assert.IsTrue(customer.Name == "New Name", 
                        "SaveCustomer failed to update customer name.");
        }
        
        [TestCleanup]
        public override void ScopeTeardown() {
                _customerScope.TeardownScope();
        }
}
If additional scopes were needed then they could simple be added to the test class. For example, if a ProductTestScope was also needed then it would be initialized just like the CustomerTestScope and torn down appropriately. Dependencies between scopes would be supplied via constructor parameters. This does imply that the SetupScope methods be called in sequence and TeardownScope be called in reverse order. That's the pattern, pretty simple but now the logic for creating these setup conditions can be offloaded and reused in a simple way and we've made it easy to clean up that test data after the tests complete.