can

you

find

the

ghosts?

click me and be prepared

Send a screen shot if you find them to : innovate@domaindotnet.com and the best image will recieve some swag! Non-trivial swag as well! 

Related Posts

• Business Data Catalog
• SharePoint RSS Alpha
• Templates for SharePoint
• Creating a Web Part with Custom Properties
• SharePoint Creating XSLT DataView
• SmartPart for SharePoint – Home
• Windows SharePoint Services 3.0

from tag C#

• Video Favorites
• S#arp Architecture: ASP.NET MVC with NHibernate – Home
• Skills Matter OPEN SOURCE .NET
• Home :: Castle Project
• NHibernate 2.0 gold Released – Must Wait for ‘Linq to NHibernate’ « ubiquitous fluency

(more..)

What’s your opinion on the genesis for a new software process/method entering the mainstream?

They don’t emerge as collective practice from air.

What made waterfall become a shared mental belief system in millions of minds yet fail so consistently, and indeed provide the only level of ‘predictability’ in software at that time (an expectation of predictable disaster).

I could continue on the facts above for a few pages but it merely is an example of our need to carefully choose a belief system that actually has the added value of being correct.

in terms likely unseen (and unknown by most)    What do you believe the memes for successful adoption are? Why did waterfall dominate for so long and at such pain?

For this discussion, your opinion of why Agile succeeded yet while it also brings impassioned opposition from others is a focus of thought.

Yet indeed while software is filled with so many amazing minds why do we fail on the most basic and fundamental topics more as a rule then exception (code reuse for example is still an utter failure for no reason other than human constraints. The technology has existed for well over a decade if not more).

Mental models/belief systems around software development drive ‘methods/methodologies’ for implementation. Unfortunately, many fallacies define the standard for how others perceive ‘how the work should be performed’.

This has been the shame of software for decades, as we seem to have a terrible time growing into a mature field of science. There is far too much failure, lack of predictability, and utter frustration at the highest levels of corporations globally. What happened and why is Agile / Wicked Problem thinking so helpful?

What are some of the reasons this disconnect exists?

Enter Agile

Agile was and mostly is the best approach to resolve the most painful aspects of fitting an artisan’s model into an assembly line mentality. Agile reconciles organizational needs in software around previously unrealistic demands around predictability, verifiable estimations, and planning.

Buzzwords and industry hyperbole aside, Agile has moved our industry forward in fundamental ways. We believe this is mostly around its focus on team-centric behaviors and simply not watering down the statement of facts around software most in the ‘methods’ world could never do as it would sever their ability to be embraced! Agile simply told the truth and said ‘enough is enough’ let us do this using some common sense.

‘Wicked Problems’

Now consider a model already used in many domains that goes back to 1973. This model has already defined the very drivers of ‘what is software development’ we assert. Consider the following:

NOTE: For a more complete listing and discussion please see [3]

If you cannot admit 90% of these statements below about software, why would you believe Agile would work for you?

Every solution to a wicked problem is a “one-shot operation,” every attempt has consequences.

This is not around all code being written from scratch in the least. This means that the delivered software for a wicked problem is unique but almost always composed of other ‘non-wicked’ solutions..

Who would even consider delivering a strategic application where nothing is leveraged from others?

Wicked problems have no given alternative solutions.

There may be no solutions, or there may be a host of potential solutions that are devised, and another host that are never even considered. This is the nature of solving complex and ‘moving target’ type concerns

Mock Frameworks are a Mandate

Although a side discussion, we believe it merits your attention as we simply have no idea how anyone can succeed at Agile without a Mock-Centric world.

Consider an ‘API’ you require but is now ill defined and not even built. Do succeed this API must ‘appear to exist and indeed appear to be working’ to our system as we evolve items around it. For example, a complex API for financial calculations is assumed, but at iteration three we only required a few aspects of that API.

For more discussion on this critical yet later stage topic go here for our deeper discussion

Facts will not Save Us

1) It is irrelevant how correct the above is until external, internal or a combination of forces move the collective thought.

Typically, the competitive market and legislation protecting shareholders should resolve the glaring nature of this issue. However, it is ignorant for we technologists to believe the ‘objective truth’ of our knowledge will directly drive cultural shifts. It does not happen that way in most cases. If it did we would all have OS/2 running on our machines and beta-max machines.

2) Most cannot actually act on these facts or even acknowledge their reality.

This is of course due to their having to ‘change everything’ in how their fiefdoms execute, not to mention trying to explain this to the other stakeholders of IT. I see disastrous practices perpetuated rather than trying to actually be viewed strategically in the value they could offer their organization.

Conclusion for Now

Ask yourself these questions at the most fundamental:

• Does your culture understand and nurture ‘software as iterative learning and Wicked Problem’.
• If so can you only acknowledge this fact with your team (that’s fine, as good luck explaining this to most C-Level execs. In fact we recommend not mentioning Agile unless they mention it to you first.
• How do you treat short-term failures from iterative development (assuming a strong set of feedback loops allowing constant course correction and validation)?

Anyway here’s an idea of how the framework API is shaping up. Expect this to move dramatically toward’s Fowler’s fantastic coverage of ‘Expression Builder’.

We hope that this will evolve into a full Linq provider, and due to the XML foundation this is not all that unrealistic a goal.
 

namespace CCStatisticsTDD
{
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using CCStatistics.Domain.Api;
    using CCStatistics.Domain.Api.Interfaces;
    using NUnit.Framework;
    using Rhino.Mocks;
    using TDDViewOutputStatus;
    using ThoughtWorks.CruiseControl.Core;

    /// <summary>
    /// This is a monolitic 'Smoke Test'. There are many fine grained tests to be writtem!
    /// </summary>
    [TestFixture]
    public class MainTestFixtureBaselineSmokes : TestBase
    {
        /// <summary>
        /// Gets the mock list.
        /// </summary>
        /// <param name="mockBuildName">Name of the mock build.</param>
        /// <param name="repositoryBuildCount">The repository build count.</param>
        /// <returns></returns>
        /// Documentation Created 6/23/2008
        private static List<string> GetMockList(string mockBuildName, int repositoryBuildCount)
        {
            // Add the build name to the array tht is expected to be available for ALL builds
            // from the repositoy. Here we use 10

            return
                Enumerable.Repeat(mockBuildName + " {0}", repositoryBuildCount).Select(
                    s => String.Format(s, DateTime.Now.TimeOfDay.TotalMilliseconds/1000)).ToList();
        }

        protected override ICCStatsProject Prepare()
        {
            // We need a name for this project
            const String mockBuildName = "CCStatisticsBuildMock";

            // Simulate 100 legacy build results
            const int repositoryBuildCount = 100;

            var repository = mocks.Stub<IIntegrationRepository>();
            var buildNames = GetMockList(mockBuildName, repositoryBuildCount);

            // OK set this up to return our builds
            repository.Stub(x => x.GetBuildNames()).Return(buildNames.ToArray());

            var currentProject = mocks.Stub<ICCStatsProject>();

            currentProject.Stub(x => x.IntegrationRepository).Return(repository);
            currentProject.Stub(x => x.StatisticsForPublisher).Return(MockStats);

            currentProject.StubEx(x => x.Name, mockBuildName).
                StubEx(x => x.BuildLogDirectory, DefaultDirectory).
                StubEx(x => x.ArtifactDirectory, DefaultDirectory).
                StubEx(x => x.WorkingDirectory, DefaultDirectory).
                Stub(x => x.StatsPostPublisher).Return(InitializePublisher(currentProject));

            return currentProject;
        }

        private IMetricPublisher InitializePublisher(ICCStatsProject project)
        {
            // OK now it's interesting.. We have our own IMetricPublisher which is a stand-in for the
            // StattisticsPublisher concrete class in CruiseControl. We need to add a new overload
            // and obviously make this behave in ways it was not intended to (as we are doing all the legacy builds)
            var _statisticsPublisher = mocks.Stub<IMetricPublisher>();

            // Here we use the mock array of stats created above. This is a duplicate of StatisticsForPublisher
            // Need to look into consolodating that.
            _statisticsPublisher.Stub(x => x.ConfiguredStatistics).Return(MockStats);

            // Core Validation is Below
            _statisticsPublisher.Expect(x => x.ProcesLogFile(project.Name)).
                IgnoreArguments().Repeat.Times(project.BuildCount);

            return _statisticsPublisher;
        }

        /// <summary>
        /// NOTE: This is a FULL CYCLE Mock Smoke test. We need many more finely grained
        /// tests. This is just to ensure all is well in the application. Each subcomponent
        /// will need to have tests added as this evolves. In the mean time this works
        /// well to ensure we fundamentally work and this is called from OUR BuildServer
        /// (how META is that?) to ensure this is working.
        /// This should be your first stop to undersand this code
        /// </summary>
        [Test]
        public void ShouldSmokeTestProjectAPI()
        {
            using (mocks.Record())
                Assert.IsTrue(ForStatistics.RefreshValues(MockTDDView.New, Prepare()));
        }
    }

    public static class MockExtensions
    {
        /// <summary>
        /// Stubs the type defined
        /// </summary>
        /// <typeparam name="TType">The type of the type.</typeparam>
        /// <param name="currentProject">The current project.</param>
        /// <param name="funcMock">The func mock.</param>
        /// <param name="returnString">The return string.</param>
        /// <returns></returns>
        /// Documentation Created 6/20/2008
        public static TType StubEx<TType>(this TType currentProject, Func<TType, String> funcMock,
                                          String returnString)
            where TType : class, ICCStatsProject
        {
            currentProject.Expect(funcMock).Return(returnString).Repeat.Any();
            return currentProject;
        }
    }
}

Keep in mind, this post is not focused on DSL topics (one any developer cannot really afford to miss now), so one could debate the merits of the link I make. My argument is this:

My definition of a DSL is very simply :

A Domain Specific Language is a way, typically using a combination of visual and textual methods, or just textual, to solve a limited but typically non-trivial set of problems. It is typically a smaller stand-alone ‘language’ or language-extension to a larger more horizontal language such as C# (the point I make here is C# has evolved to allow you to expand C# with DSLs you write in C#. This is very different the the DSL Toolkit and I have never had this opinion until I finally got to be immersed deeply in functional style programming enhancements to C# and the ease of extending just about any API with extensions..

I continue the discussion of what a DSL is here. I will be adding to this section of the site over time but if you just want the listed info, no need to go on a tangent now.

Now that new .NET 3.5 C# 3.0 features are allowing us to make the Framework API of .NET work in ways not present ‘out of the box’ the scope of what is possible can be deep and broad.

Part 1 : The DSL Perspective – A Real Example

I’ll use a project we are working on now, which is quite specifically aimed to

• Minimize the configuration overhead of using an Inversion of Control container
• Provide the benefit of ‘provider/container’ independence
• Allow the consumer to change ‘container provider’ vendors with a simple change in ‘meta configuration’

In other words, we are trying to create an abstraction on top of an abstraction, as we believe this particular area is now mature enough (and indeed this is how software engineering historically more then not progresses over time) to do so.

This project has no intention of BEING an inversion of control container nor a dependency injector, rather its aim is to allow people to make use of these critical concepts in much easier ways while eliminating hesitation related to provider lock-in.

It’s no secret we are massive fans of the work of the Castle Project and they are indeed the first ‘plug-in’ adapter we are on schedule to deliver. However the architecture (something this post is NOT trying to explain) of our solution models the DOMAIN of this area, and is not specific to any particular offering (although to be blunt I would say if any bias existed it would be from the conceptual framework that exists now in the Windsor Container/MicroKernal trunk).

A domain specific example

In the area of ASP.NET one technique for loading our solution into memory is via the IHttpModule implementation we provide. Here is what the custom global.asax.cs code looks like:

public abstract class IntegrationPoint : HttpApplication, IDomainApplication {

private static IZeroConfigDI _container;

/// <summary>
/// Executes custom initialization code after
/// all event handler
/// modules have been added.
/// Also acquire a reference to the HttpModule
/// for IoC / DI if present as a module
/// </summary>
 public override void Init() {        

The interesting areas which are not provided by Microsoft include:

1. IDomainApplication interface
2. IZeriConfigDI Interface static reference
3. Modules.GetIoCModule()

The first two have more to do with our solution then the topic of this post, so I’ll focus on #3.

As this code is inside the HttpApplication, we have access to the Modules property which is read-only. Here is the documentation:

Your first reaction if your familiar with this collection is that it does not have anything even close to the GetIoCModule() method we are calling above.

Indeed this is our ‘domain specific’ extension to the .NET Framework API (one of many for this solution).

Here is the implementation of this ‘DSL’ extension:

/// <summary>
/// Provides ‘DSL specific’ extensions to the ASP.NET environment for
/// the IoC Zero Config environment which is not new (after all this becomes a static)
/// </summary>
/// The entire point for extension methods is ‘Fluency in API’ as it allows ‘shorthand’ code

public static class ZeroConfigInstanceResolver {

private const string ZeroConfigTypeString = "IoCConfigModule.IHttpModel";

 /// <summary>
/// Gets the IZeroConfigDI reference
/// (using domain specific logic and the 
/// generic logic extending the container)
/// </summary>
/// <param name="container">The container.</param>
/// <returns></returns>

public static IZeroConfigDI GetIoCModule(this HttpModuleCollection container) {

                        return
                              container.FindInstance(
                                () =>
                                        container.Get(ZeroConfigTypeString)
                                                    as IZeroConfigDI);

}

As you can see above we are very specific in our intent to add this to the HttpModuleCollection. There is nothing that interesting above except for the method call FindInstance(Func<TResult>) off the target ‘container’. This is a peek into the non-DSL related extension that represents. You might be aware that there is no such FindInstance available off the HttpModuleCollection.

One of our many goals in creating a fluent, easily used API is reducing complexity and providing very maintainable code. Here you can see the Lambada expression required to locate our service is hidden from the developer completely, yet this code is not a ‘one-off’ as we have provided both a DSL level extension in concert with a far more generic extension.

Part 2: Extending the Horizontal to Empower the Domain Specific

In looking at the HttpModuleCollection, and in our desire to facilitate other sources of resolving our module, it inherits from the abstract parent NameObjectCollectionBase.

Here is the definition of what inherits from this abstract parent:

The issue arises as this abstract base leaves it up to its concrete implementer to provide the indexer (or say Get() method) to access the object desired. Therefore a nice way to enhance this abstract type to work across all of the children is to literally pass in the method to resolve the object desired.

Part 3 : The Generic Extension

Here is the trivial extension method used to ‘open up’ the NameObjectCollectionBase and which is used above:

/// <summary>
/// Finds the instance in the NameObjectCollectionBase
/// using the Functor
/// </summary>
/// <typeparam name="TContainer">The type of the container.</typeparam>
/// <typeparam name="TResult">The type of the result.</typeparam>
/// <param name="container">The container.</param>
/// <param name="finder">The finder.</param>
/// <returns></returns>
public static TResult FindInstance<TContainer, TResult>
       (this TContainer container, Func<TResult> finder)
              where TContainer : NameObjectCollectionBase where TResult : class {

                    return finder();

}

Since we cannot abstract up the get method, we take a page from Functional Programming and literally pass in the code to execute (which we do above).

A common pattern is to expose methods like the above, but add additional versions that allow more parameters into the Func<TResult>. This is well covered well by many of the Linq references and books available.

Part 4: How this allows us to embrace deep/wide change

A simplistic example is supporting the acquisitions of our target reference (IZeroConfigDI) if it is set as ApplicationState or as an IHttpModule. We could make the design decision to allow our find method to reside literally as an extension method of our HttpApplication (and indeed this turns out to be a reasonable choice as the valid inheritors from NameObjectCollectionBase are sealed and therefore invalid as generic constraints). Here is one approach (not ideal and not our final code but this illustrates the point.) Why not offer your optimized solution?

CODE:

public abstract class IntegrationPoint : HttpApplication, IDomainApplication {

private static IZeroConfigDI _container;

/// <summary>
/// Executes custom initialization code after
/// all event handler
/// modules have been added.
/// Also acquire a reference to the HttpModule
/// or Application State
/// for IoC / DI if present
/// </summary>
public override void Init() {

                  _container = this.GetIoCModule(); base.Init();

}

And the extension on HttpApplication:

public static class ZeroConfigInstanceResolver {

private const string ZeroConfigTypeString = "IoCConfigModule.IHttpModel";

/// <summary>
/// Gets the IZeroConfigDI reference if possible (null if not found)
/// using domain specific logic and the
/// generic logic extending the container.
/// </summary>
/// <param name="container">The container.</param>
/// <returns></returns>
public static IZeroConfigDI GetIoCModule(this HttpApplication container) {

                       var modules = container.Modules;
                       var applicationState = container.Application; 
                     

                        var resultVar = modules.ToInstance(
                                   () => modules.Get(ZeroConfigTypeString)
                         )
                        ?? 
                        applicationState.ToInstance(
                                   () => applicationState.Get(ZeroConfigTypeString)
                         );

                       return resultVar as IZeroConfigDI;

}

No change is required on the extension to the FindInstance extension.

Can you see how the above can be improved?

We expand this to make the ‘rule for finding the IZeroConfigDI external. To be specific, above we have a business rule implemented that we typically would not want ‘hard coded’. In fact more then not all things considered we want this. So we do the4 following:

• Determine the Delegate signature for the business rule.
• For the above, we need as input (at a minimum) an HttpApplication instance as that is the container for the models and state, and those two items lack a shared interface or base (or we could use that).
• The signature returns (in all cases) an instance cast where it implement IZeroConfigDI to resolve this business rule.
• Therefore the business rule signature is Func<HttpApplication ,IZeroConfigDI>

The world’s most common extension method and an attempt at a definitive version.

In this post I’ll be digging deeper into doing things I’ve always wanted to have in .NET yet didn’t. For example, I know I am not alone in wanting an IList (be it an IList<T> or a simple non-generic IList) to provide me a .foreach method as you get in a List<T>). This is just one of many things we address here (and more).

From the concrete to the abstract

I was having problems with entries in my path environmental variable, and I wanted an easy way to flag bad entries and also allow me to move the entries in the list up or down easily. This was made apparent by my use of a tool called ‘Dependency Walker’. It’s hard to live without but that’s another post. See the screen shot below:

Click here to go to ‘Dependency Walker’

Test to Code

    /// <summary>
    /// Unit Test to Validate that the logic related to     /// 'show combined extensions using paths' is functioning
    /// </summary>
    /// Documentation Created 5/22/2008
    [Test]
    public void shouldShowCombinedExtensionsUsingPaths()
    {
        var _directoriesInPath = EnvironmentVariableTarget.             Process.GetParsedValues("PATH");

        Assert.That(_directoriesInPath, Is.                TypeOf(typeof(String[])));
        Assert.IsTrue(!_directoriesInPath.IsNullOrEmpty());

        _directoriesInPath.ForEach(
            x =>
            {
                if (!Directory.Exists(x))
                    Console.WriteLine("MIssing Directory! " + x);
            });
    }

Above is the test (that also works). Before I launch into why this is interesting, what looks different?

1) EnvironmentVariableTarget : This is Microsoft’s type not mine. It’s the enum used to specify the target search on the environmentals.

2) _directoriesInPath : This is a string[] as you can see from the Assertion. But how does it support ForEach?

3) GetParsedValues() : What is that exactly?

*** As this is so common I decided to try to craft a definitive version of ForEach that would work across non-generic collections, generic collection, and specific type specification on the targeted items in the collection for generics .  ***

In order to enable ForEach behavior I needed to this code which is by far the most common extension method in all the C# 3.0 code alive today.

    /// <summary>
    /// Requires generic IEnumerable and is most common now
    /// </summary>
    /// <typeparam name="TContainedType">The type of the contained type.</typeparam>
    /// <param name="baseEnum">The base enum.</param>
    /// <param name="actionPerform">The action perform.</param>
    /// <returns></returns>
    /// Documentation Created 7/20/2008
    public static IEnumerable<TContainedType> ForEach<TContainedType>(
           this IEnumerable<TContainedType> baseEnum,
               Action<TContainedType> actionPerform) {

        return ForEachInner(baseEnum, actionPerform);
    }

    /// <summary>
    /// The most explicit case
    /// </summary>
    /// <typeparam name="TBase">The type of the base.</typeparam>
    /// <typeparam name="TContainedType">The type of the contained type.</typeparam>
    /// <param name="baselineEnumerable">The baseline enumerable.</param>
    /// <param name="actionPerform">The action perform.</param>
    /// <returns></returns>
    /// Documentation Created 7/20/2008
    public static IEnumerable<TContainedType> ForEach<TBase, TContainedType>(
              this TBase baselineEnumerable, Action<TContainedType> actionPerform)
                   where TBase : IEnumerable {

        return ForEachInner(baselineEnumerable.OfType<TContainedType>(), actionPerform);
    }

    /// <summary>
    /// The least restrictive case
    /// </summary>
    /// <param name="baseEnum">The base enum.</param>
    /// <param name="actionPerform">The action perform.</param>
    /// <returns></returns>
    /// Documentation Created 7/20/2008
    public static IEnumerable<Object> ForEach(this IEnumerable baseEnum,                Action<Object>  actionPerform) {

        return ForEachInner(baseEnum.OfType<Object>(), actionPerform);
    }

    /// <summary>
    /// Do the actual work
    /// </summary>
    /// <typeparam name="TContainedType">The type of the contained type.</typeparam>
    /// <param name="baseEnum">The base enum.</param>
    /// <param name="actionPerform">The action perform.</param>
    /// <returns></returns>
    /// Documentation Created 7/20/2008
    private static IEnumerable<TContainedType> ForEachInner<TContainedType>(
        IEnumerable<TContainedType> baseEnum,
                       Action<TContainedType> actionPerform) {

        var getEnumerate = baseEnum.ToList();
        getEnumerate.ForEach(actionPerform);

        return getEnumerate;
    }

Notice that anything which implements IEnumerable is caught above, regardless of what the collection.

This content assumes you have an introductory knowledge of C# 3.0 language features in .NET 3.5 and mastery of legacy C# 2.0 Generics, Generic Type Constraints, Anonymous Delegates and related material.

I use a ‘pair programming’ approach with continued refactoring as this is how I would discuss it if you were coding with me, with an unfortunate one-way delivery.

The Brief Strategic View

Microsoft has slowly been moving C# in a very productive direction (this is not new, as these features existed in 2.0 although not nearly as well integrated) to provide ‘Functional Language’ features. If you don’t know or care about language semantics, just know that Linq and especially Lambada Expressions are about empowering you to use executable code like a variable, aka to leverage the power of functional programming. For more on this, read this MSDN article by Joel Pobar (former CLR Team) or read the next set of posts (part 2 onward) as I will go into the depths of this.

Painless Intro

I’ll start with a fairly trivial, yet important example (I use it every day). Many times when comparing Strings I want to ignore case and culture (the InvariantCulture). This is provided by an overload as such as you likely know:

[Test,Category("BaselineCore")]
public void shouldAsserStringCloneInvokeEqual() {

var baselineString = “This is a TEST CASE to IgnOrE Casing”;

var stringUpper = baselineString.ToUpperInvariant();

Assert.IsTrue(baselineString.Equals(stringUpper,
StringComparison.InvariantCultureIgnoreCase));

The test results are shown above. Pass. Ok so I really like short, concise code that is understandable at a glance. Also it’s a pain to always (even with ReSharper) use this (and I have seen people use RegEx for this! RegEx is awesome but overkill for this issue).

Refactoring

• Create an extension method on String
• Decide on a good name for the method (this is SO important and for most an afterthought!)

I’ve settled on calling this new method on String ‘EqualsCore’ as that is what we are doing, making the conditions for a match ’simpler’ and seeing ‘just the core values’ are the same (anyway it makes sense to me)., I suppose this could be ‘EqualsRelaxed’ or whatever..

Here is the test case (no code yet):

[Test, Category("BaselineCore")]
public void shouldAsserStringsEqualUsingExtension() {
var baselineString = “This is a TEST CASE to IgnOrE Casing”;
Assert.IsTrue(baselineString. EqualsSimple(baselineString.ToUpperInvariant())); }

Now we write the code. Here is the container for the extension method:

public static class StringExtensions {

public static bool EqualsSimple(this string sTarget, string compare) {

return sTarget.Equals(compare, StringComparison.InvariantCultureIgnoreCase); }

}

Indeed they both pass:

Since every type inherits from Object, and Equals is defined on Object, all instances should support this approach, and I could be early bound by using Generics…. Hmm…

I tried this (note: I gave it a new new ‘EqualsThis’ to separate them.

public static bool EqualsThis<TTarget>(this TTarget sTarget, TTarget compare){

return sTarget.Equals(compare);

}

Functionally not that interesting at all, but a test. So I typed in the following and wow… It works from Intellisense’s view… Ok it compiled! Wait…..FAIL! But why:?

OK here is the new test:

[Test, Category("BaselineCore")]

public void shouldAsserANYTHINGEqualUsingExtension() {

const String baselineString =”This is a TEST CASE to IgnOrE Casing”;

var sb = new StringBuilder(baselineString);

Assert.IsTrue(sb.EqualsThis(new StringBuilder(sb.ToString())));

}

Interesting…

Here is the documentation for what Equals means by default from Microsoft:

So our code fails using the extension yet this returns true:

[Test] public void shouldAssertStringBuilderExplicit() {

const String baselineString = “This is a TEST CASE to IgnOrE Casing”;

var sb = new StringBuilder(baselineString);

var sb2 = new StringBuilder(baselineString); Assert.IsTrue(sb.Equals(sb2));

}

So Reflector to the rescue once again. I could see in Reflector what I believed the issue was. Indeed the StringBuilder class has an overloaded Equals, and even making the extension method cast to the generic type directly was a no go.

This exposes one of the dangers that we must be incredibly careful with. It has always been poor design in my opinion to encourage developers to override common methods such as ToString() and Equals(object X) with their own behaviors as you force consumers of the API to understand IMPLEMENTATION. You cannot ensure your OK simply from a contract. This is known to be evil….

Of course this is a legacy style and will be slowly phased out.

Spin up reflector and look at the code for the OVERLOAD that StringBuilder has:

 How do we get people to commit to something when everyone has a different perspective based on role or even political leanings for larger companies” “Our stakeholders are petrified to commit to a final stake in the ground for fear they will not be able to modify it based on I.T.’s inflexibility and inability to change”

I though perhaps these were problems we had overcome in our industry, as the solutions are fairly well understood. I was wrong. What is the bottom line break-down between I.T. and the Business? That is a long and colorful debate. However I offer this:

• 
  
  Most organizations punish those who are bad at PREDICTING THE FUTURE in software engineering. Of course these are smart people but they continue to demand clairvoyance with a lack of statistics.


• 
  
  What happens? Padding of the schedule and bitter fights for every last month, when nobody has any idea what the effort is, not can they quantify the quality, and more importantly maintainability requirements around a software initiative.


• 
  
  The fact is you cannot know at a micro level about a project until you are well into building it. This is something I have never met a receptive audience for, however it proves itself time and time again.


• 
  
  “How do we give people the freedom to change their minds, yet deliver concrete estimates on time and cost to our managers?”


• 
  
  “How much will this cost us and how can we estimate it knowing we will likely change our minds during the project based on previous projects, and how can we get early and consistent involvement in the development process?”


• 
  
  “How can we ensure that necessary quality is met when we always enter risk mode at the end and quality goes out the window? Can’t we manage with quality at every single stage, even every day?!”

Whatever you do (in my opinion anyway) Don’t say Agile…Nobody on the business side cares. In fact for whatever reason, software causes perfectly rational people to loose their mind (I cannot really blame them).

What I have to say on the matter is accompanied by deep battle-scars. What I have to say is definitely not what most want to hear, and even the technologists want to argue that they CAN come close to prediction. However what I say has the ‘added moral quality of being true’. This is hard to argue in light of the facts. That still often changes nothing, as companies firmly plant their head in the sand again and hope for the best with no real way of knowing what will happen and no practical way of really tracking in real time how their exposure to risk is changing.

I know… I know… Agile is 100% about that but not in the way (typically) this group wants to hear, no that is why a knockout one-two punch is as follows (to be insanely overly simplistic):

1) Heavy Statistical Metrics for the ‘Planning’ View Inclined

2) Of course, Agile Practices to Provide the Inputs to (1) and to Optimize your Chances on the Execution Side

This article is about some detail around (1) that far too many miss in my opinion and why it doesn’t serve you to mix the terminology of these two groups (one of the only occurrences where I recommend that as I am typically the one pushing everyone into ‘one common Domain Driven language’)

The Four Critical Dimensions

When addressing the concerns of a new custom software project these are the four key elements that must always be addressed (there are others such as security, scalability, etc. but those are included in ‘Scope’ as functional and non-functional requirements):

• 
  
  Time (cannot be made)


• 
  
  Scope/Features (you cannot know it until your done, so now what?


• 
  
  Quality (not typically discussed, as who wants ‘bad or sort of low’ quality)


• 
  
  Cost (Since you have almost nothing to go on here, what to do? Your agile consultant says the system is ‘done when it is done, not when some arbitrary date is set’ and he is 100% correct. However the business gets to decide when it is ‘done enough’ not I.T. (with exceptions of course)

As Quality is not usually debated (I have never had a client say that the software could be poor quality), that leaves Time, Scope and Cost as the main moving parts to work with.

The diagram above is where many stop. That is missing the entire point, as the DRIVERS that facilitate your ability to do well are missing. For example:

1) A lack of a continuous integration environment with regression tests will severely hurt your ability to achieve quality (all things being equal)

2) A bad process / culture kills just about all of it

3) Bad team morale is the same coin, different side.

4) A process that cannot manage the inevitable change in scope as we ‘discover’ what the project is all about, will kill just about all of it.

5) A team lacking in solid design skills (design for change) such as a mastery of the important design patterns (Strategy Pattern Anyone?) as well as a lack of a Dependency Injection Container (there is just no good reason not to use one)

6) Lack of an Object to Relational Mapping Abstraction Layer. Deal Breaker as of today (perhaps not a year ago).

There is no real doubt by anyone (except for a few people who also are not so sure on the whole evolution thing) that Agile practices provide the common sense way to address the same concerns most of us have around software.

You know it, I know it, we all know it. However they don’t care (and really have likely heard some bad things about it).

My assumption is you are a person on the software side in a managerial capacity of some sort. I made this mistake for you many times. Resist the urge to ‘set them straight’ or to explain the wonders of Agile when asked these questions (or even when not) and instead… I’ll be getting to that….

Anyway, they would probably just think ‘how 2004’ and ask about SOA or something….

There are some real deal-breaker assumptions around project planning with or without Agile execution (and without agile is very well might be an execution.. OK enough biased injection for one article).

There is often an unspoken (and unrecognized) characteristic that you should be able to ‘tell the future’. And many will think that the worse you are at it, the more likely it is you just might be lazy and/or stupid. I’ve seen it over and over and over. Those who spin, those who pad, those who create fluff to buy time win. Those who really try and don’t play games get nailed to the wall.

Count me in the ‘dumb and lazy’ group because I am terrible at predicting the future.

In high school I was certain this really cute cheerleader would accept my invitation to the prom. I was also certain a few years ago that a system could be built for W dollars, in X months with Y people and Z quality. After all, that is all I could get from the customer as they were taking no input from vendors. Sure it was aggressive but we were so smart! We could do anything!

Both had similar endings.

“In order to succeed in complex software planning, just like in Agile, you have to give up. You have to completely accept the fact that you cannot possible know what you are really doing until you are already well into a solution, at least not at any detailed level”

Sound familiar? But you know you need these Agile ideas on the ‘other side’ and you also know it would be self-defeating to wave that flag..

Nobody wants to admit they lack power, especially in areas they have devoted their life to mastering. But why does this not only empower you, but create the bedrock of extreme competence and success? Is this like the alcoholic finally admitting he is powerless over the bottle? Only then can things get better? I doubt it. This is only software.

·

The Monetary Stakeholders

And for they people either funding and/or profiting from this exercise the concerns can be quite different.

“How can we ensure profitability on our work when we know change will occur? (For internal IT they may be managed and accountable for profitability if they are run like a profit center). If not profitability then “How can we possibly commit to a date and cost when we know this will be a moving target as we evolve the solution?”

“We have an abysmal record of missed deadlines, cost over-runs, quality problems and our users not being happy with our delivered systems. What can we do to improve this?”

“How do we manage the scope changes and ‘feature creep’ placed on us by the business people/stakeholders? This is what typically destroys our profits and/or time commitments and cost commitments, often resulting is a severe quality issues as well. How can we give people what they want but manage this change?”

This article is about bring these two groups together and keeping them just apart enough. This article is targeted to the people who must deliver software to their clients (be it internal IT or an outside Agile consulting firm doing the work). I assume these concepts would work in a non-Agile Process environment but I have cannot remember much luck (or for anyone I knew) in those days without resorting to what I call ‘white tricks’ so I will not comment.

A Framework for Project Statistical Success Optimization

We know that these two very distinct groups will need to work closely together in an Agile environment if we want to optimize our chances. The days of separating the business experts from IT is long gone. In fact, for an Agile project to succeed it is typical to actually have a business domain expert on the development team as a fully fledged member.

‘High Ceremony’ software processes may work (it is really dependent on who is doing the work as an amazing team will almost always succeed – see Peopleware by DeMarco and Lister) but at a large opportunity cost in almost all cases.

Any assigned domain expert should have the reach to get answers to questions quickly outside their domain knowledge when required. For a non-trivial project it will likely cover many domains and it may not be possible for that individual to have a mastery of all of them, however this person should be able to expeditiously get answers from others when required, This is a critical success factor to any Agile project.

I have evolved over many years this approach to software profitability and risk management in a very well tested and disciplined way. It has evolved since 1998 when I was CTO of a niche financial services software firm, which was sold in 2004. I then started the firm agilefactor and significantly evolved these ideas further as I still do literally almost every day! It’s 2007 and there are many war wounds.

If any of this content seems like ‘academia’ or ‘theory’ I can assure you these are all proven real-world strategies, having been applied by myself and many others for years and years.

At the most basic, one could say everything eventually comes down these four critical dimensions…

It is CRITICAL you never let a client box you in on all of the three of these items if it can be avoided or you may have to ‘walk away’ from a project (many in internal IT do not have that luxury but at least this article will show you how to push back in a way that is hard to be disputed by the forces trying to box you in to possible assured failure).

Your client gets to drive only two of the three items at most (such as Time and Cost), while you driven the third (Scope in this example) or the remaining items you control (you must always have at least one that the client explicitly gives you control over).

This is critical to profitable software sales both for commercial and custom one-off development but many fail in this area in their eagerness to close deals (this is especially a problem with salespeople who are not put in check on what they can promise the client, and will promise you right into a massive loss – a lesson I learned firsthand early in my career).

For example, if a client says:

· We are allowing vendors to work until October 1st to achieve these 120 individual scope items in the attached 600 page scope document we have assembled (the first major sign of trouble).

· Vendors have $1,900,000 to build this system. This is non-negotiable and the software must be extreme production quality, with severe contractual penalties for down-time, with scaling up to 10000 concurrent sessions per machine with an average response time of one second or less “

Do we walk away? My initial reaction is absolutely..

This violates the principle described above as you are not in control of any of the four critical elements. The answer is sometime you must walk away!

I will describe here a way to help determine if you should and how you can effectively push back on this, as we know there will be significant deltas on that scope document (both academically from studies and practically from our experience).

We really have no idea if we should accept these terms. There are ways however where we can at least assess the risk and possible accept the terms when we learn these tools.

We all know it is an agile imperative deliver a working build to your stakeholders for each iteration that is ‘real production’ although a subset most of the time

The client drives at most two dimensions, and you must drive the third (again ignoring the Quality variable) or you risk a disaster for everyone.

And after all, THEY are deviating from ‘the plan’ they committed to with confident assurances that there would be little to no need for ‘change orders’ (this is a common occurrence in my experience with a correlation to the larger the company the most convinced they are the scope will not change.

Establish Risk Tolerance Before Defining the Time, Scope and Cost Constraints

Based on the extensive research and project database built by the industry luminaries and incredibly esteemed experts Tom DeMarco and Tim Lister, we now know what the probability distribution curve is for new custom software projects. It is lognormal with a high peak and a long curve to the right. This is absolutely critical for everyone to understand and I highly recommend you read the book “Waltzing with Bears” for a detailed discussion (see the end of this chapter for more information).

This is the probability distribution curve we must embrace:

You establish the client’s willingness to accept the unavoidable possibility of failure across any of the four dimensions (we cannot predict the future) and then find that point on the curve moving up from the X axis point you have for the client (the X axis is probability of success) and take the area to the left of that point.

The total area under the curve is 100% so the farthest right point would indicate 100% confidence (again basically impossible to achieve where most organizations assume 100% by default, something that must change if our industry will ever ‘grow up’. As we cannot control the future, we can only manage it and the risks, we must know how ‘risky’ the client wants to be (again by asking them or inferring it ourselves – often manifested in a ‘public’ date and an ‘internal’ date which is past the public date if we know the client will not cancel the project or penalize us tangibly for missing the public date).

It is a typical ‘risk/reward’ situation. If the customer takes on more risk we might save them money, or deliver early. If they want less risk, we must significantly increase our cumulative resources attached to a project with the knowledge that ‘adding people to a late project makes it later’. This we know as fact so adding people is really only an option in the early phases. This has a large impact on early project planning.

· We need to understand if a customer is OK assuming a 15% risk of failure for example.

· After all, don’t all future events have unknown outcomes?

· Is the client willing to as much as double their budget for a say 10% increase to 95% probability of future success across all dimensions?

A 95% success promise is not only near reckless to offer, it is an incredible shifting of risk anyone should demand a rich reward for taking on.

Studies are behind me on this as well with the well established fact that the best developers are as much as 28 x more productive then the worst). It quite literally is often a 100% increase in total resources allocated for a project to move from 85% to 95% even though it is a 10% gain in probability, hence the very long tail. The highest I have ever had a client go is 90% and we actually had two teams in two locations for redundancy (they didn’t know about each other). One team almost failed but delivered a solution, however the other team’s solution was superior so that is what we went with. This gave us our high probability due to the redundancy and the client paid dearly for this safety.

For example, the development team may say:

“We had a terrible two weeks. Our lead Cryptography expert was sick so we moved those features into the next iteration and we severely underestimated the complexity of the mainframe integration and User Profile Customization.

Therefore we could only achieve 50% of what we had planned for this iteration. I would say from our agreed upon 85% confidence we are now closer to 75%. We can see if we make it up in the next iteration or we can take measures A, B and C to get back to 85%. What do you want to do?”

A better question I have for you is, would you rather not know this? Almost all organizations are nowhere near this level of integrated risk assessment, quantitative measurement and adjusting action execution.

This is typically a decision for a high level individual (typically the project manager with the optional input of the customer) and hopefully with input from as many people as possible covering all dimensions.

Adding people will typically hurt you, not help you. You should already have done the obvious things like have moved the team into a ‘war room’ setting (almost always a sure way to improve productivity) or many other solutions, so if you missed any do it now as these techniques are proven and it is much more painful to fail.

As for the client, they can also be blissfully unaware of how we manage the process of our internal development, but this well established process makes it incredibly hard to fail.

Here is an example with risk tolerance. Based on negotiations with the client they have agreed to a risk tolerance of 85% (a 15% risk of failure). Based on this, you can now estimate knowing your margin for error. The customer has scope requirements, and cost requirements but is fairly open to time.

You come back to the client and say:

“Based on an 85% confidence we feel we can deliver your scope at the cost you describe at some point between April 1 2006 and July 1 2006’”. The client may protest, saying (for the first time) they were hoping for a March 1 delivery. You confer with your colleagues and respond that this would be possible at a 70% Confidence (30% chance of failure). And so the negotiations follow…But at least now you have very concrete ‘dials to turn’ and it is hard for the customer to argue with the fact base supporting you.

I assert that custom software is a ‘wicked problem’ (to see a great definition of ‘wicked problem’ see http://www.cognexus.org/id42.htm or the text at the end of this article.

Conclusion

To help ensure a positive outcome for your client and for your development effort it is critical to consider first the Risk Tolerance of your client, and then negotiate the one dimension you typically have control over after the client informs you of the two (two from Time, Scope and Cost). In some cases you must walk away and this article should help you determine when that is the case.

For a much more detailed discussion of these topics and what this author considers critical:

“Waltzing with Bears”, Dorset House Publishing Company, Incorporated (March, 2003), ISBN: 0932633609

Supplemental Material: Wicked Problems

A wicked problem is one for which each attempt to create a solution changes the understanding of the problem. Wicked problems cannot be solved in a traditional linear fashion, because the problem definition evolves as new possible solutions are considered and/or implemented. The term was originally coined by Horst Rittel.

Wicked problems always occur in a social context — the wickedness of the problem reflects the diversity among the stakeholders in the problem.

Most projects in organizations — and virtually all technology-related projects these days — are about wicked problems. Indeed, it is the social complexity of these problems, not their technical complexity, that overwhelms most current problem solving and project management approaches. (See graphic of wicked problem solving process below.)

Some specific aspects of problem wickedness include:

1. You don’t understand the problem until you have developed a solution. Indeed, there is no definitive statement of “The Problem.” The problem is ill-structured, an evolving set of interlocking issues and constraints.

2. Wicked problems have no stopping rule. Since there is no definitive “The Problem”, there is also no definitive “The Solution.” The problem solving process ends when you run out of resources.

3. Solutions to wicked problems are not right or wrong, simply “better,” “worse,” “good enough,” or “not good enough.”

4. Every wicked problem is essentially unique and novel. There are so many factors and conditions, all embedded in a dynamic social context, that no two wicked problems are alike, and the solutions to them will always be custom designed and fitted.

5. Every solution to a wicked problem is a “one-shot operation,” every attempt has consequences. As Rittel says, “One cannot build a freeway to see how it works.” This is the “Catch 22″ about wicked problems: you can’t learn about the problem without trying solutions, but every solution you try is expensive and has lasting unintended consequences which are likely to spawn new wicked problems.

6. Wicked problems have no given alternative solutions. There may be no solutions, or there may be a host of potential solutions that are devised, and another host that are never even thought of.

(from Rittel and Webber, 1973)

Not to be a terrible bore, but brush up on your Wittgenstein lately?! Ok don’t go that far just yet…(grin).

 First, a Pattern Language (P.L.) is not some academic concept, although many would like you to believe that.

	‘Software as a Service – The Pattern Language Approach’
	‘Software as a Service – The Pattern Language Approach’
	‘Software as a Service – The Pattern Language Approach’

NOTE: I take issue with the word design below… It should be far more broad in most cases, but it often gets boxed into the design aspect of a problem. I would say ‘best practices’ which brings up its own semantic mess…

A pattern language is a structured method of describing [best] good design practices within a field of expertise. It is characterized by

1. Noticing and naming the common problems [and positively executed solutions proven over time] in a field of interest [domain in my words]
2. Describing the key characteristics of effective solutions for meeting [one or more] some stated goal[s], [Often patterns are combined and are multi-faceted]
3. Helping the designer [domain stakeholder] move from problem to problem [and challenge to challenge] in a logical way, [in an attempt to reduce and even eliminate ambiguity] and allowing for many different paths through the design process.

- Wikipedia 

1. This ties in perfectly with my claim that complex software engineering is a ‘wicked problem’. If you disagree then you can stop reading, as none of my assumptions will work. However, I would love to hear your arguments since only around 20-30% of all software projects ’succeed’…. There is no doubt there is dreadful crisis which I know you know about so enough said.

If the fundamental issue here is NOT that people cannot/will not see software engineering for what it is, then what?!

Wicked Problem

FYI: This is also why your pretty much nuts to not be Agile at this point, with some critical exceptions.

OK just to get you started chew on these 10 basic characteristics and tell me how any of them and the failure of senior stakeholders to understand that indeed software fits these characteristics is likely a key systemic disaster which we see and live every day in this industry. Click here for more.

 Before I was able to ‘take this deck on the road’ which I have done now with success (and why I am revisiting this post to offer insights), I had to get the audience (I hate to call them that… The collaborators is better) knowing (for example) that SOA was not a damn bunch of web services! I had to get people understanding exactly how distributed objects and components ARE NOT THE SAME but are similar. etc. etc.

That ASMX pages in .NET are not even close to what you want to be doing, even with WSE 3.0 and ‘Contract First’ techniques (most notably the WSCF add-in from the always amazing thinktecture).

There is nothing more necessary than truth, and in comparison with it everything else has only secondary value.
This absolute will to truth: what is it? Is it the will to not allow ourselves to be deceived? Is it the will not to deceive?
One does not want to be deceived, under the supposition that it is injurious, dangerous, or fatal to be deceived.

Friedrich Nietzsche, 1890

Thanks Herr Nietzsche… None of us want that. So “Listen for the speakers intended meaning, not your perceived definition”.

In this rather philosophical vein, I offer you here an attempt at the beginnings of a unified P.L. covering many best practices we have evolved in the SOA/SaaS space.

As stated, this presentation is born from necessity as it has become almost impossible to have a meaningful conversation on many topics in this domain due to ‘definition overloading’.

This presentation not only attacks that issue, it also brings together into a cohesive pattern language Object Orientation, Distributed Objects, Components and Services while being careful to not make overly broad generalizations that would harm any SaaS initiative.

For example, it is a best practice to evolve a Domain Driven API which is (at a minimum) the core APU which your services will use to fulfil their contracts.

The presentation is here and feedback is encouraged. This also assumes you have embraced either the SCA environment (for Java development) or WCF for .NET. These are (in our guidance to clients) the only relevant implementation frameworks for strategic large scale organizational transformations to SaaS.

Otherwise in real-world terms there are not resources (developers, or the dramatically increased effort with very little benefit) available to support other approaches. Many ‘purists to the extreme’ might attempt a hand-crafted ‘build it all over again’ approach with hand-crafted contracts (typically XSDs).

As these environments support full extensibility, indeed it is incredibly rare that one of these environments will not completely address your needs. Of course there are always exceptions, but this is addresses to 95% of the market.

Due to the immaturity of this space, it is alarming how many companies believe they are not part of that 95%!! Typically they are being driven by parties (internal or external) that have motivations not in line with the business drivers.

Here is a great launch point to get your hands dirty from the master

Web Services Enhancements (WSE) 2.0 SP1 for Microsoft .NETWeb Services Enhancements 2.0 SP1 for Microsoft® .NET (WSE) is a supported add-on to Microsoft Visual Studio® .NET and the Microsoft .NET Framework that enables developers to build secure Web services based on the latest Web services protocol specifications.