Building a Pipeline (the Template Pattern)

Building a Simple Pipeline

A lot of my development experience has been building back-end systems. There's been plenty of times where something I've built needed to process a request which was composed of a number of tasks. These tasks had to be performed in the correct sequence forming an algorithm. I've heard these things called many different names including Pipelines.

Unfortunately, many times this need leads to a class which contains both the steps of the algorithm, and the logic to complete the steps. These classes quickly turn into huge monsters. They also tend to become a real pain to test. It turns out, there's a GoF pattern for them: Template Method Design Pattern.

In other words, I wanted to express these simple algorithms like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

public class FileCreator : IRequestHandler<CreatePackage> { private readonly IPipeline<CreatePackage> pipeline; public FileCreator(IPipeline<CreatePackage> pipeline) { this.pipeline = pipeline; } public void Handle(CreatePackage request) { pipeline.Subject(request); pipelline.Do<BoxTheItems>(); pipelline.Do<CreateTheShippingLabel>(); pipeline.Do<ShipTheBox>(); pipeline.Do<SendShippingNotification>(); } }

PS - MediatR is awesome. That's where IRequestHandler<T> comes from.

Enter LittlePipeline

LittlePipeline is a small library, or set of example code on how one can build a template that is easy to read, easy to test, and can work with an IoC container. It's designed to be a starting point, not an end-all solution to the problem. The gist of using the library (or classes if you want to copy/paste the code) is simple...

Start with a subject class. This class is the guy who holds the data necessary to process the request. It can be as dumb (or smart) as you need. The only requirement is the subject be a reference object (a class).

1 2 3 4 5 6

public class CreatePackage { public int OrderId { get; set; } public Address ShipTo { get; set; } public string TrackingNumber { get; set; } }

Create any number of tasks. These tasks should implement the ITask<T> interface where T is the subject type you created earlier.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

public class SendShippingNotification : ITask<CreatePackage> { private readonly IBus bus; public SendShippingNotification(IBus bus) { this.bus = bus; } public void Run(CreatePackage subject) { var notification = new ShippingNotification { TrackingNumber = subject.TrackingNumber }; bus.Publish(notification); } }

Use the baked-in pipeline creator, or your favorite IoC container to register the tasks, and create the pipeline. Then, use it like in the class above. Here's the built-in, no frills, no guaranties example.

1 2 3

var pipeline = MakePipeline.ForSubject<FirstTestSubject>() .With<Increment>(() => new Increment()) .Build();

Oh, and this is what testing a pipeline looks like (with FakeItEasy):

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

public class PipelineTestExample { [Test] public void ThePipelineCanBeTested() { var pipeline = A.Fake<IPipeline<FirstTestSubject>>(); var example = new ThingThatUsesThePipeline(pipeline); var subject = new FirstTestSubject(); example.Run(subject); A.CallTo(() => pipeline.Subject(subject)).MustHaveHappened() .Then(A.CallTo(() => pipeline.Do<Increment>()).MustHaveHappened()) .Then(A.CallTo(() => pipeline.Do<Square>()).MustHaveHappened()); } } public class ThingThatUsesThePipeline { private readonly IPipeline<FirstTestSubject> pipeline; public ThingThatUsesThePipeline(IPipeline<FirstTestSubject> pipeline) { this.pipeline = pipeline; } public void Run(FirstTestSubject subject) { pipeline.Subject(subject); pipeline.Do<Increment>(); pipeline.Do<Square>(); } }

That's It

There you have it, a simple template class that (hopefully) helps clean up some code by separating the algorithm steps from their implementations. The code is on github. The ReadMe.md file gives some information about how it might be tested.

MediatR, FluentValidation, and Ninject using Decorators (Pt. 2)

Validate all the Things?

The last post showed how to combine MediatR, FluentValidation, and Ninject to handle the validation of requests handled by MediatR. The drawback in the previous post is that all requests must have a corresponding validator implemented. Here, we'll look at using the null object pattern to bypass validation when a validator isn't present.

Null Object What?

The null object pattern is a way to pass an object which represents nothing, but does not throw a runtime exception when used. This pattern can be used to add default behavior to a process when there is nothing for the process to use.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

public class ValidatingHandler<TRequest, TResponse> : IRequestHandler<TRequest, TResponse> where TRequest : IRequest<TResponse> { private readonly IRequestHandler<TRequest, TResponse> handler; private readonly IValidator<TRequest> validator; public ValidatingHandler(IRequestHandler<TRequest, TResponse> handler, IValidator<TRequest> validator) { this.handler = handler; this.validator = validator; } [DebuggerStepThrough] public TResponse Handle(TRequest message) { var validationResult = validator.Validate(message); if (validationResult.IsValid) return handler.Handle(message); throw new ValidationException(validationResult.Errors); } }

The handler above is taken from the last post. It accepts a handler and validator for a given request. It validates the request, then either passes the request to the next handler, or throws an exception. Ninject will throw an exception when it cannot find an appropriate validator for the request.

1 2 3 4

public class NullValidator<TType> : AbstractValidator<TType> { }

Enter a null validator. This validator has no rules. When used, it will return no errors, because there are no rules defined in it. Because it's defined as a generic, it can be used to validate any request. When the handler processes a request, the validation passes by default, and the request is passed down the chain.

The cool thing is there's no extra steps needed. Just add the null validator to the solution. It will be picked up by Ninject in the normal registration process. When Ninject can't find the appropriate validator for a request, it will fall back to the generic implementation.

PS...

That's all there is to adding a class which will provide a default behavior of passing requests that have no validation rules defined. Adding code like this is also a great example of the open-closed principle: It was possible to extend the behavior of the validator without changing the code of the validator. So, there ya go. Default, passing validation using the null object pattern.

As always, the code is on GitHub.

Toggling With Windsor

Preface

There are a number of times when we've all had to implement new features or modify the implementation of an existing code base. An intern recently asked me how to feature toggle something using Castle.Windsor. This post will show how to use some Castle.Windsor features to toggle implementations. The example code can be found on GitHub.

Primitive Dependencies

The toggle will be an app setting in the application's config file. It will be loaded by Castle.Windsor by using a custom dependency resolver. This resolver is taken from Mark Seeman's post on AppSettings.

The Service

We'll be using a simple interface as our service definition. There will be two implementations. One represents an old implementation, the other a new.


It's useful to note that this is a common way to achieve some branching by abstraction. This is done by replacing calls to a service with an interface. This interface is the abstraction. Once the calls to the old service are replaced, you are free to implement a new service. When ready, the new service can be substituted for the old without the consumers being aware since they depend on the interface not the concrete.

Typed Factory Selector

Castle.Windsor comes with a handy little bit: the typed factory facility. The typed factory facility lets Castle.Windsor create a factory implementation from an interface defined by you. This relieves you of the task of implementing the factory on your own. It is especially useful if you want to defer the creation of the object.


Our class will use this factory to get an instance of our service, and call the .Print() method. The default for this object will be the first one registered in the container. This behavior can be overridden by implementing a custom selector.


The typed factory and selector must both be registered with the container. The selector must also be specified in the configuration of the typed factory. This is done on lines 21 and 22 of the ContainerFactory class.


Using IHandlerSelector

Mike Hadlow provides a very good example of using a custom IHandlerSelector.

We can use a similar technique to pull in a config value and supply the appropriate implementation at run time. The custom IHandlerSelector uses the config value to select the appropriate handler. If no handlers are found it throws an exception. This handler is then returned.


This service handler must be registered and added to the container's kernel. Line 19 of the ContainerFactory class show the registration. Line 26 shows the selector being added to the kernel. While there is only one selector in this example, the snippet shows how to add more than one.

Running the Console

Changing the config value and running the console app shows that the selectors are functioning correctly.

Wrapping It Up

Feature toggles and branching by abstraction are powerful ways to control whether new code is being used in production. They provide a way to replace old behavior with new, while maintaining the integrity of the product's build. Hopefully these two examples will help you integrate feature toggling into your builds.

Basics: Removing the 'if' (Using Polymorphism)

Preface

Performing transformations of one object type to another type is a very common task in programming. It might be publishing an event based on a command, or an externally known DTO from an internal DTO. It's pretty common to see some use of the if or switch keywords to determine the code flow. I thought I'd take a minute to show how we can go from a typical implementation using if statements to one which uses Linq and AutoMapper to reduce the coupling in the implementation.

The example code uses the following tools:

• NUnit
• NUnit Test Adapter for VS2012 and VS2013
• Fluent Assertions
• FakeItEasy
• AutoMapper

The Interface

For this example, we'll have three different publishers. Each will implement a common interface: IPublisher. The implementations will be responsible for accepting a Command object and publishing the associated Event object. We'll be using two commands and events: Start -> Started, Stop -> Stopped.

The Publish method on the IPublisher interface is intentionally not using a generic declaration.

Overloading

The first Publisher accepts a command. It checks the type of the command received, and calls the appropriate overload. Each overloaded method creates the appropriate event, and publishes it.


This works, but it has a few problems. It both uses an if to determine which type to publish, and manually maps the inbound command to the outbound event. That means this class is responsible for both determining what kind of event to publish and creating that event.

Adding AutoMapper

AutoMapper removes the responsibility of creating the event from the publisher class. AutoMapper Profiles could be used to map more complex associations, but the DynamicMap method works just fine here. Our publisher class is relieved of this responsibility, limiting it to just sorting out the type of event to be published.

It still has the problem of using the if statement to determine the type of command received (and event to be be published).

Removing the 'if'

Introducing a map from the commands and a Linq query allows us to remove the if statements. The class is still responsible for selecting the appropriate action. The concept of associating commands to events is distilled into the dictionary. This leaves the class' methods to simply select the appropriate action and execute it.

The Dictionary was left inside the publisher class to keep everything in once class. It wouldn't take much to move the mappings out of the class. This would further reduce the coupling on the Publisher. More complex mappings could be introduced by changing the Dictionary out for a custom type.

Wrapping It Up

This was a quick demonstration of removing two concerns from a class. The manual mapping of one class to another by introducing AutoMapper. The if statement was removed by introducing a map between the two types. I hope this helped describe a different was of building classes with reduced responsibilities.