Generating Sitecore code without TDS and with Rainbow

About a year ago, I posted about a way to generate code from your Sitecore data without the need to use TDS. Since then, a new version of Unicorn was released, that supports a new serialization format called Rainbow.

The description on Rainbow’s GitHub page: An advanced serialization and comparison system for Sitecore. Designed for easy merging with YAML-based serialization, and a generic merge/deserialization framework. Works well with Unicorn, or by itself.

Unfortunately, the code generator didn’t support this format. But with a little tweaking, I’ve now added support for it!

You can follow the same instructions as described before. And after that, apply some specific changes to enable Rainbow support.

Happy code generating!

Post content audit messages to Slack from Sitecore

The Sitecore community has certainly embraced Slack. If you want to join your fellow Sitecore developers, be sure to follow the link and get it up and running. It’s great to see so many people from the community sharing knowledge about all things Sitecore in this awesome chat application.

We’ve been using Slack with our team internally at our company as well. And we love having great integrations with our build server, JIRA, Pingdom, Sentry and others (some of which are custom built).

It was about time that we helped our content editors out by providing an integration with Sitecore itself. That’s why I built the Sitecore.Slack.IncomingWebhookForEvents module. Whenever someone edits a content item, publishes or installs a package, others will know about it through Slack!

Follow the instructions on the GitHub page to install this module (using NuGet) for your project.

Monitor your Sitecore server’s health

Having a website up and running reliably requires that you monitor its health. You should always be using a monitoring tool like Pingdom (there are many others). That way, you can get notified when the website is down.

Now, there are several ways of making sure that things are ok at runtime. For example:

  • Sitecore’s HeartBeat.aspx for Load Balancer Health Checks – Standard page in Sitecore that does some limited checks. It’s a good idea to monitor this page.
  • Sitecore Diagnostics Toolset – A desktop application that does various checks to see if your Sitecore implementation is correctly working, with regard to things like security, indexing, performance and scalability. It’s a smart idea to use this tool periodically, but it won’t be helpful for monitoring purposes.

Personally, I like to monitor more than just if a website is up. I also want to know if key parts of the system are functioning properly, like connections to other systems, if certain content is available and if my indexes are ok. These checks may be very specific to my implementation.

That’s why I created a simple drop-in .aspx script that you can easily configure to check these things. You can find it in this Gist on Github. Example output:

It’s quite easy to configure and extend. Just read through the implementation of the Page_Load(…) method. Some more details here:

Security

You’ll want to limit who gets to use the monitoring page, to prevent abuse and prevent hackers from getting details about your implementation. This is especially important if the checks you are doing may impact system performance significantly. Make sure your monitoring tool’s requests to the page match your security settings here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
        // Optional: uncomment whichever ways you feel are appropriate for security
        // Ensure that your monitoring tool matches these
        if (!(true
            //&& Request.IsSecureConnection // Checks if using HTTPS
            //&& Request.QueryString["authkey"] == @"secret"
            && Request.UserHostAddress == "127.0.0.1"
            //&& Sitecore.Context.IsLoggedIn
            //&& Sitecore.Context.IsAdministrator
            //&& Sitecore.Context.User.IsInRole(@"sitecore\Developer")
            ))
        {
            Response.StatusCode = (int)System.Net.HttpStatusCode.Forbidden;
            Response.Write("You are not authorized to view this page");
            Response.TrySkipIisCustomErrors = true;
            Response.End();
            return;
        }

 

Adding checks

You can add checks to do when the page is requested by adding them to the ChecksRun collection. Some examples are in the code below. They include checking specific items’ presence in databases and having versions and layout/presentation. They also include checking if search indexes contain enough items, if certain url’s are reachable from the server and if configuration is specified correctly.

It’s also easy to implement your own specific types of checks. Add them to the region “actual checks” as classes that implement the ISitecoreCheck interface (or use the SitecoreCheckBase class).

1
2
3
4
5
6
        // Add your checks here
        ChecksRun.Checks.Add(new ItemExistsCheck("Folder template", "{A87A00B1-E6DB-45AB-8B54-636FEC3B5523}", new[] { "master", "web" }, null, false));
        ChecksRun.Checks.Add(new ItemExistsCheck("Launchpad has layout", "{6B846FBD-8549-4C91-AE6B-18286EFE82D2}", new[] { "core" }, new string[] { "en" }, true));
        ChecksRun.Checks.Add(new SearchIndexSizeCheck("Master index", "sitecore_master_index", 1000, null));
        ChecksRun.Checks.Add(new UrlCheck("Google is reachable", "http://www.google.com/"));
        ChecksRun.Checks.Add(new ConfigCheck("Check if master database is available", "/sitecore/databases/database[@id='master']", 1));

 

Tags

You may not always want to run the same checks. Perhaps some checks should only be performed at night and some checks may be specific to your DTAP or CM/CD servers specifically. You can use tags to limit which tests are run, as demonstrated in the example below. And then you can specify which tags are applicable by specifiying them |-separated in the url parameter “tags”. The url //SitecoreChecks.aspx?tags=fail would result in the following:

1
2
3
4
        if (Tags.Contains("fail", StringComparer.InvariantCultureIgnoreCase))
        {
            ChecksRun.Checks.Add(new AlwaysFailCheck());
        }

 

Let me know if you have any questions/suggestions.

Generating Sitecore code without TDS

In the past, I’ve used my own Compiled Domain Model module on projects. It generates wrapper classes based on your Sitecore templates (among other things). But recently, I’ve been working with Glass Mapper.

Glass Mapper doesn’t wrap Sitecore items, but rather maps Sitecore items to objects. The result is somewhat similar; you can work with strong-typed classes/interfaces and extend them if you want.

There are several ways you can configure Glass Mapper. One of the ways is by using attributes. The attributes map the classes/interfaces and their properties to Sitecore templates and fields. This works very well and is very flexible, but it can be a bit tedious to do manually.

That’s why generating the code is a good option. If you’re using TDS, then you can easily generate code.

But if you can’t use TDS (e.g. because of the costs), then here’s a good alternative. Kern Herskind Nightingale did something similar before. But my implementation supports:

  1. Generate individual files for each template
  2. Configure which paths to generate classes/interfaces for
  3. Generate interfaces for Glass Mapper (but it is easily adaptable to other frameworks)

For more information, look at the documentation here, and you can clone/fork the project on GitHub here.

IDog interface example generated using Sitecore.CodeGenerator

IDog interface example generated using Sitecore.CodeGenerator

Sitecore Code Challenge 1

Someone once told me about Pex for fun. And ever since, I occasionally visit the website to solve some C# coding puzzles. Essentially, it requires you to implement some code and have unit tests for it pass. But you don’t know the implementation of the unit tests.

I thought about how we could do something similar with Sitecore, and came up with a way to do it. If you follow the steps below, you’ll have a .sln with a project that compiles and has unit tests. But the tests all fail.

It’s your job to make sure the code passes the unit tests. But the relevant code in the unit tests is hidden inside an obfuscated DLL (can’t be easily decompiled).

The failing assertions  in the unit tests, will give you detailed hints though. And if you succeed in getting the unit tests to pass, your project can be deployed to Sitecore and will actually be usable!

Since this is quite a new way of getting to know Sitecore in a fun way, I’m bringing this to the community as an experiment. I’m very interested to learn your experiences:

  • Are you able to finish the puzzle?
  • Is it fun trying to solve the puzzle?
  • Any feedback for creating a second challenge (if this one turns out to be a success)?

Follow these steps to get started:

  1. Download this ZIP file and extract it.
  2. Copy “Sitecore.Kernel.dll” to the folder “Binaries” (I’ve tested with version 7.2 rev 140526, but it will probably work with other versions as well).
  3. Open the project in Visual Studio and build the project.
  4. Run the unit tests using a runner that supports xUnit (I’m using xUnit.net VS runner, which is free)
  5. The tests are ordered into steps. Go through the steps in  order and look at the reported errors for hints on how to solve the puzzle.

Let me know if you are able to finish this first code challenge! Have fun!

Validating your code references to serialized data

Sitecore developers often need to reference content items from their code. E.g.: a template or branch ID for an import function or reading a global setting that’s stored in the content tree. Using IDs is usually preferred over using paths, because paths may change. It’s considered good practice to keep those ID references in a class with constants, like this:

1
2
3
4
5
public static class Constants
{
    public static readonly ID SampleTemplateId
        = new ID("{76036F5E-CBCE-46D1-AF0A-4143F9B557AA}");
}

After recently contributing some serialization logic to Sitecore.FakeDb, I got the idea to write unit tests that validate if these constants have values that are actually in Sitecore. We can use TDS or Unicorn serialized data to validate this.

The following unit test does just that (uses NUnit, FluentAssertions, Sitecore.FakeDb and Sitecore.FakeDb.Serialization):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[Test(Description = "Checks if Sitecore IDs in a class with constants can be found in the serialized data")]
public void ShouldIdsBeFoundInSerializedData(
    [Values(typeof(Constants))]
    Type typeWithIdConstants)
{
    foreach (FieldInfo field in typeWithIdConstants
        .GetFields(BindingFlags.Public | BindingFlags.Static)
        .Where(f => f.FieldType == typeof (ID)))
    {
        ID constantValue = field.GetValue(null) as ID;
        if (((object) constantValue) != null)
        {
            constantValue
                .FindFilePath("master")
                .Should()
                .NotBeNullOrWhiteSpace(
                    "Item with ID {0} (as defined by {1}.{2}) should be in the serialized data",
                    constantValue,
                    field.DeclaringType.Name,
                    field.Name);
        }
    }
}

Just pass in the types (that have Sitecore IDs as static properties) using the “Values” attribute.

But I didn’t stop there. At my company, we’re currently working on a project that uses Glass Mapper. Glass Mapper can use attributes to link mapped classes to the correct Sitecore templates. And in a similar way, properties can be linked to field IDs.

1
2
3
4
5
6
[SitecoreType(TemplateId = "{76036F5E-CBCE-46D1-AF0A-4143F9B557AA}")]
public interface ISampleItem
{
    [SitecoreField(FieldId = "{75577384-3C97-45DA-A847-81B00500E250}")]
    string Title { get; set; }
}

I wanted to validate all those IDs as well, and also check if the field IDs are on the right templates.

That’s why I wrote the following unit test. It finds all the types that have attributes like this and deserializes the corresponding templates. It also checks if the fields are there.

To use it, you need to pass in one type from the assembly that you want to test (or multiple) using the “Values” attribute.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
[Test(Description = "Checks if all Glass mapper template and field IDs can be found in serialized data")]
public void ShouldAllTemplatesAndFieldsBeFoundInSerializedData(
    [Values(
        typeof(SomeType), // From assembly MyProject
        )]
    Type typeFromAssembyUnderTest)
{
    Assembly assemblyUnderTest = Assembly.GetAssembly(typeFromAssembyUnderTest);

    // Find all templates and fields that need to be checked in the serialized data
    List<ID> sitecoreTypes = GetAllTemplateIds(assemblyUnderTest.GetTypes());
    List<KeyValuePair<ID,ID[]>> sitecoreFields = GetAllSitecoreFieldsFromAssembly(assemblyUnderTest);

    Console.WriteLine(
        "Now checking {0} template ids and {1} fields (from assembly: {2}) in serialized data",
        sitecoreTypes.Count,
        sitecoreFields.Count,
        assemblyUnderTest.GetName().Name);

    using (var db = new Db())
    {
        // Find all the templates and deserialize them into the FakeDb
        List<DsDbTemplate> templates = sitecoreTypes.Select(t => new DsDbTemplate(t)).ToList();
        foreach (DsDbTemplate template in templates)
        {
            db.Add(template);
        }

        // Iterate all the templates and check if they are correctly deserialized and are actual templates
        foreach (ID sitecoreType in sitecoreTypes)
        {
            Item item = db.GetItem(sitecoreType);

            // check if the template was actually deserialized
            item.Should().NotBeNull();
            item.TemplateID.ShouldBeEquivalentTo(TemplateIDs.Template);
            item.Paths.FullPath.Should().StartWith("/sitecore/templates/");
        }

        // Iterate all the fields and check if they are correctly deserialized and valid for the deserialized templates
        foreach (var sitecoreField in sitecoreFields)
        {
            // Check if the field is valid on the template(s) for the class that references it
            templates
                .Where(t => sitecoreField.Value.Contains(t.ID))
                .Any(t => t.Fields.Any(f => f.ID == sitecoreField.Key))
                .Should()
                .BeTrue(
                    "Field with ID {0} should be available on any of the templates with ID's {1}",
                    sitecoreField.Key,
                    string.Join(", ", sitecoreField.Value.Select(v => v.ToString())));
        }
    }
}

/// <summary>
/// Find all template IDs that are defined on all of the types that are passed.
/// </summary>
/// <param name="types"></param>
/// <returns></returns>
private static List<ID> GetAllTemplateIds(IEnumerable<Type> types)
{
    return (from t in types
            let attributes = t.GetCustomAttributes<SitecoreTypeAttribute>(true)
            where attributes != null && attributes.Any()
            select attributes)
        .SelectMany(a => a)
        .Where(a => a.TemplateId != null)
        .Select(a => ID.Parse(a.TemplateId))
        .ToList();
}

/// <summary>
/// Finds all fields marked with the SitecoreFieldAttribute in the whole assembly.
/// Returns a list of pairs where the key is the field ID and the value is an array of template IDs that the field might belong to.
/// </summary>
/// <param name="assembly"></param>
/// <returns></returns>
private static List<KeyValuePair<ID, ID[]>> GetAllSitecoreFieldsFromAssembly(Assembly assembly)
{
    return assembly.GetTypes().SelectMany(t => t.GetProperties())
            .Where(p => p.IsDefined(typeof (SitecoreFieldAttribute), true))
            .Select(p => new
                {
                    Type = p.ReflectedType,
                    Attr = p.GetCustomAttribute<SitecoreFieldAttribute>(true)
                })
            .Where(a => a.Attr.FieldId != null)
            .Select(a => new KeyValuePair<ID, ID[]>(
                ID.Parse(a.Attr.FieldId),
                GetAllTemplateIdsIncludingFromBaseTypes(a.Type).ToArray()))
            .ToList();
}

/// <summary>
/// Finds all of the template IDs that are declared on the type or any base class or interface that it implements.
/// </summary>
/// <param name="type"></param>
/// <returns></returns>
private static List<ID> GetAllTemplateIdsIncludingFromBaseTypes(Type type)
{
    return GetAllTemplateIds(GetBaseTypes(type).Distinct().Concat(new[] {type}));
}

/// <summary>
/// Finds all of the base types and interfaces of a type.
/// </summary>
/// <param name="type"></param>
/// <returns></returns>
private static IEnumerable<Type> GetBaseTypes(Type type)
{
    if (type.BaseType == null) return type.GetInterfaces();

    return Enumerable.Repeat(type.BaseType, 1)
                        .Concat(type.GetInterfaces())
                        .Concat(type.GetInterfaces().SelectMany<Type, Type>(GetBaseTypes))
                        .Concat(GetBaseTypes(type.BaseType));
}

Since I’m quite new to using Glass Mapper, I’d be interested to get some feedback on this. I’m pretty sure there’s more that can be tested.

Bye!

Unit testing with Sitecore.FakeDb and deserialized data

Recently, I watched the excellent SVUG presentation by Pavel Veller on YouTube. I was impressed by the way Sitecore.FakeDb works. It’s much easier to configure than my FixtureDataProvider, has tighter isolation and is (because of that) much faster.

Even though the syntax for setting up your test data is very easy and short with Sitecore.FakeDb, I think that it can be a lot of work to setup things for a unit test. Besides, those of us using TDS or Unicorn are already putting a lot of information about the structure of items under source control. It would be a shame if we could not use that information.

That’s why I propose a hybrid approach: use DbItem and DbTemplate to setup your tests and where needed, deserialize individual items/templates. This can have the benefit of not having to initialize as much, as well as bringing your serialized data into the test scope.

On that last note: if a template field is renamed and then synchronized with TDS, then if the code you are unit testing relies on that field name… it will fail. Which can be very helpful during development.

I’ve added a project called Sitecore.FakeDb.Serialization to the FakeDb project. It’s usage is entirely optional and adds 2 important classes: DsDbItem and DsDbTemplate (Ds = Deserialized). They inherit from their regular Sitecore.FakeDb counterparts. But instead of having to define the field values, they load them from the filesystem.

You can easily mix these approaches. Here’s an example of a unit test using both:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[TestCase]
public void TestCombined()
{
    using (Db db = new Db()
        {
            new DsDbTemplate(
                "/sitecore/templates/Sample/Sample Item"),
            new DsDbItem("/sitecore/content/home", true)
                {
                    new DbItem("Child")
                        {
                            {"Some field", "Some field value"}
                        }
                }
        })
    {
        // Assert data that was deserialized from the filesystem
        Item home = db.Database.GetItem("/sitecore/content/home");
        Assert.IsNotNull(home);
        Assert.AreEqual("Hello!", home["Title"]);

        // Assert data that was added using the regular approach
        Item child = home.Children["Child"];
        Assert.IsNotNull(child);
        Assert.AreEqual("Some field value", child["Some field"]);
    }
}

 

You can find these changes in my fork on GitHub. To use them, you need to do the following afer you’ve setup Sitecore.FakeDb:
- Add a reference from your unit test project to Sitecore.FakeDb.Serialization.
- Edit the App.config file and add the following section (change the folders to point to your serialized data, or add/remove folders as needed):

1
2
3
4
5
    <szfolders>
      <folder name="core" value="c:\somefolder\data\serialization\core\" />
      <folder name="master" value="c:\somefolder\data\serialization\master\" />
      <folder name="web" value="c:\somefolder\data\serialization\web\" />
    </szfolders>

 

Have fun coding!

Index page editor proof pages with Sitecore 7

After competing in the Sitecore hackathon last weekend, I got excited. So when Kyle Heon posted a particular Tweet today, I immediately knew what I would be doing this evening; create a new (small) project on GitHub.

And here you can find the result. Basically, I created a computed field that actually does a http request and downloads the page that you are indexing.

The reason you would want to do this, is because of how content composition in the page editor works. You can add components to a page that reference separate Sitecore items as associated content (a.k.a. datasources). But the content of those referenced items is indexed separately and therefore difficult to search for using Sitecore.ContentSearch (Sitecore 7 search) – if you want ‘pages’ to be the result, like regular site searches usually need.

You can find more information, including installation instructions, on the GitHub page for Sitecore Html Crawler (just scroll down a little).

There are some limitations to this approach that you should know about.

  1. This doesn’t really work well with wildcard items, because they can only be indexed once; not for every item that you want to display on the wildcard item’s page. You would need to change the actual indexer to support that, or handle the difficulty of it in your search query.
  2. I’ve only tested this with Lucene. I think changing the configuration a little could make it work for other providers such as Solr.

Turn any Sitecore package into a NuGet package

I recently posted this idea on Twitter:

And it didn’t take me that long to actually implement it. Follow these steps to turn any Sitecore package into a NuGet package and share it with the world.

  1. Download the CreateSitecoreNuGetPackage.exe file here or build it yourself with the code from GitHub.
  2. Put the files “Sitecore.Kernel.dll” and “Sitecore.Zip.dll” in the same folder.
  3. Create your package in the Package Manager and export the ZIP file.
  4. Run the following command from the command line:
    CreateSitecoreNuGetPackage [PATH_TO_SITECORE_PACKAGE]
  5. Use the NuGet Package Explorer to open the generated  .nuspec file and improve the metadata if needed.
  6. Publish the package to a repository, like nuget.org.

Now, everyone will be able to install/uninstall your package through NuGet. Remember that you need to install Sitecore Rocks and attach your project before installing the package.

Some links to pages that helped me creating this:

I will be very interested to hear your feedback, as I haven’t been able to test this very well. If this utility catches on, I might make a version that you can use from the package manager directly.

Happy NuGetting :-P

CompiledDomainModel 1.0 released!

This was a little overdue; my Sitecore code generation module, CompiledDomainModel, has been used in many projects and the code has matured enough to justify a 1.0.0.0 version number. A very important addition was made in this release.

Release 1.0.0.0:

  • Added “Platform Mode”, that can be used if generated code needs to be in separate files (e.g. when there are multiple DLL’s where generated code is needed)
  • NuGet support added

Platform Mode

This is a feature that has been requested by several people. If you have multiple projects or logic that is used in several DLL’s, then you need a way to divide the generated classes up into multiple files.

To make this possible, you should first check the “Platform Mode” checkbox on the settings item (/sitecore/system/Modules/CompiledDomainModel/Settings):

Note that the wrapper classes need to be resolved globally when using the platform mode. To make that possible, there is a dependency on the module itself. So you can’t use the “Remove dependencies” function when you want to use the platform mode.

If you generate the code now, you’ll see that a comment is generated:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
// PLATFORM MODE IS ON - please check the CompiledDomainModel settings if this was unintentional
//
// Please use the following url and change it to your needs:
// http://sitecoredemo/sitecore/shell/Applications/CompiledDomainModel/CodeGenerator.aspx?platformsets=<PIPE_SEPARATED_SET_NAMES>
//
// Use one or more of the following names to generate for the part of the platform you need (exclude the brackets):
// [Core] (should be used in a project that all other projects have a dependency with)
// [PlatformTemplatesSet] (domain objects)
// [SomeProjectTemplatesSet] (domain objects)
// [PlatformFixedPaths] (fixed paths)
// [SomeProjectFixedPathsSet] (fixed paths)
//
// As a starting point, this is what the URL would look like if you would want to generate everything
// http://sitecoredemo/sitecore/shell/Applications/CompiledDomainModel/CodeGenerator.aspx?platformsets=Core|PlatformTemplatesSet|SomeProjectTemplatesSet|PlatformFixedPaths|SomeProjectFixedPathsSet

As is explained, you must now make url’s to generate code that is specific to your project. In this case, you could generate code for the platform project and the specific “SomeProject” project. Using the following url’s, you can do that:

  1. http://sitecoredemo/sitecore/shell/Applications/CompiledDomainModel/CodeGenerator.aspx?platformsets=Core|PlatformTemplatesSet|PlatformFixedPaths
  2. http://sitecoredemo/sitecore/shell/Applications/CompiledDomainModel/CodeGenerator.aspx?platformsets=SomeProjectTemplatesSet|SomeProjectFixedPathsSet

These url’s will only generate the code for the DomainObjectSets and FixedPathSets (check the documentation for an explanation about these sets) that are referenced in the url.

NuGet support

You can now install CompiledDomainModel through NuGet. I based the package on this excellent article.

Steps:

  1. You need to have Sitecore Rocks installed in Visual Studio
  2. You need to connect your project to your Sitecore website
  3. Open the Package Manager Console and run Install-Package CompiledDomainModel

Of course you are not required to install CDM this way. You can still use the regular package on the Releases page.

Happy CDM-coding!

UA-4216957-1