During the recent Tridion MVP summit, the subject of Javascript came up. That's not surprising, as we were busy with building some GUI extensions to run on the SDL Tridion 2011 CTP, which is a pretty Javascript intensive activity. Most Tridion people reading this won't be too surprised if I say that Javascript isn't really in the comfort zone for many of us. We've done templating in VBScript, and although Tridion has always supported Javascript (or more strictly, JScript) for templating, we've always avoided it. Tridion has never shipped a JavaScript version of the default templates, and I suspect that's why, very early on, VBScript became the VHS to JavaScript's BetaMax. Most people accept that Betamax lost to VHS despite being technologically better, and for the sake of this discussion I'm going to ignore the dissenting opinions that have surfaced recently. In my view JavaScript is a superior technology to VBScript in many ways, but VBScript achieved critical mass in our world, and that's that. Or is it?

Well to start with, I'm not about to suggest that anyone should start to write Tridion component templates in JavaScript. These days, if you're defecting from VBScript templating, you'll be going to a .NET language; probably C#, and I'm all in favour of that. On the other hand, for many of us, JavaScript clearly has a place in our future. If you're writing web applications these days, a solid grasp of js is essential, and as noted already, if you want to customise the Tridion GUI, you'll be elbow-deep in the stuff before you get anything useful done.

For myself, I started using Javascript for simple update scripts and the like quite some time ago, and this mostly means running the scripts from the command line on the server using the cscript processor built-in to Windows. (These days, I also use PowerShell scripting for some of the more ad hoc work, but that's another story.)

The first obvious point about the language is that it has several styles or idioms in common use. For those of you that are familiar with libraries such as JQuery, you'll know that the style can be very similar to what you might encounter in languages with a functional flavour. To gain expertise in this style, I'd strongly recommend jumping in the deep end with John Resig's "Learning Advanced Javascript". (It's a pretty deep deep end. Note to self: have another go soon, and try to understand it this time!)

The other two idioms I'd mention are the object-oriented style, and what I can only describe as Microsoft style. For the object-oriented style, and how to achieve it, you can do no better than to view Doug Crockford's lecture series hosted at Yahoo. I suppose the best place to learn the Microsoft style is MSDN. In the days of the Atlas project, Microsoft tried to use JavaScript as just another layer in the ASP.NET stack, and with some success. For Tridion guys, this style is most notable because some of this flavour is to be seen in the Tridion GUI - or at least that's how it appears to me based on a fairly peremptory poke around. (I'd love someone to tell me if if I'm wrong. Really! Although comments won't show up, as I have to moderate to prevent spam. That's my only moderation criterion, though. If you take the trouble to write a real comment, I'll publish it.)

I hope to return to to the subject of Tridion and Javascript in future posts, but for now, I'd just like to start with a simple example of why it works for me. But please be kind; I'm not a Resig or a Crockford.

It's fairly often useful to be able to recurse through a folder structure and process each item within a given hierarchy. Usually, the processing involves two parts. Firstly, filtering: am I interested in this item or not? Secondly, for the items I'm interested in, I want to run some code, either to report on the item or to alter it in some way. This pattern comes up a lot, and for a lot of the small- to medium-scale jobs, the recurse-filter-process logic is a significant part of the work. Maybe the actual payload only amounts to a couple of lines of code. If you had a simple way to re-use the recurse-filter-process part, you could do a lot of quick-and-dirty jobs, erm... pretty quickly.

In Javascript, I can paste the following function in to a template or a script file, and the recurse-filter-process part is done:

function recurseItems( orgItem, filter, process){
  var items = orgItem.GetItems();
  var eItems = new Enumerator(items);
  for (;!eItems.atEnd();eItems.moveNext()){
    var itemType = eItems.item().ItemType;
    if (itemType === TDSDefines.ItemType.Folder || itemType === TDSDefines.ItemType.StructureGroup){
        recurseItems(eItems.item(), filter, process)
    } else {
      if (filter(eItems.item())) {
          process(eItems.item());
      }
    }
  }
}

I'm quite sure you could tidy this up a bit, but whatever... What makes this so straightforward to re-use is the fact that in JavaScript, functions are first-class objects, and it's very easy to pass functions as arguments to another function, and invoke them from within that function. The recurseItems() function expects to be passed an organizational Item as its first argument. (OK - as written, this won't work for categories, or for the sake of argument, publications, but bear with me...)

The "filter", and "process" arguments are functions.

Let's say I want to list all the components in a given hierarchy. I could write something like this (by which I mean this code isn't tested, it's for illustration purposes, right?):

function isComponent(item){return item.ItemType === TDSDefines.ItemType.Component;}
function outputTitle(item){WScript.Echo(item.Title + "\n";))
var topFolder = tdse.GetObject("tcm:1-1234-2", TDSDefines.OpenMode.View);
recurseItems(topFolder, isComponent, outputTitle);

So with 4 lines of code, I've listed the items I'm interested in. Not bad, eh? OK - I cheated. But how? Take a look at the attached file TDSDefines.js. It's a port of the standard TDSDefines constants to JavaScript. The cheating part is that I instantiate the "tdse" object in there, which somewhat breaks the purity of having a TDSDefines file, but you're always going to want tdse, so wtf not? Anyway - this file is what allows me to type things like "TDSDefines.ItemType.Component", or TDSDefines.OpenMode.View, instead of 16, or 1. JavaScript lends itself very well to this kind of nested data structure, in ways that VBScript would struggle with.

Assuming you are using the cscript host on your Tridion server, and that TDSDefines.js is in the same directory as your script, you'll need to use a couple of lines of code to import your "defines".

    var fso = new ActiveXObject("Scripting.FileSystemObject");
    eval(fso.OpenTextFile(fso.BuildPath(fso.GetParentFolderName(WScript.ScriptFullName), "TDSDefines.js"), 1).ReadAll());

Inside Tridion, of course, TDSDefines.js just becomes a template building block, and gets included in the normal way.

Funnily enough, I never got round to porting the default template code to JavaScript, and now, presumably, I never will. All the same, using JavaScript for this kind of work has allowed me to practice and get more familiar with the language, and to have a toolkit that allows for very, very fast creation of quick-and-dirty recursion scripts among others. It doesn't end there. JavaScript, or perhaps JSON, allows you to take a script-as-data approach which will get you to your desired result much quicker than, for example, having to write scripts that crunch through XML data files. Perhaps that would make a good subject for a future post.

In the meantime, I hope this gives you yet another excuse to hone your JavaScript skills. Those skills will definitely come in useful.

I don't know exactly when it was that Tridion started shipping the XML Schema files for the content delivery configuration files. For what it's worth, I only really became aware of it within the last few months. In that short time, schema validation has saved my ass at least twice when configuring a Tridion Content Delivery system. What's not to like? Never mind "What's not to like?" - I'll go further. Now that the guys over at Tridion have gone to the trouble of including these files as release assets - it is positively rude of you not to validate your config files.

Being a well-mannered kind of guy, I figured that I'd like to validate my configuration files not just once, but repeatedly. All the time, in fact. Whenever I make a change. The trouble is that the typical server where you find these things isn't loaded down with tools like XML Spy. The last time I validated a config file, it involved copying the offending article over to a file share, and then emailing it to myself on another machine. Not good. Not easy. Not very repeatable.

But enter our new hero, Windows 2008 Server - these days the deployment platform of choice if you want to run Tridion Content Delivery on a Windows box. And fully loaded for bear. At least the kind of bears you can hunt using powershell. Now that I can just reach out with powershell and grab useful bits of the .NET framework, I don't have any excuse any more, or anywhere to hide, so this afternoon, I set to work hacking up something to validate my configuration files. Well - of course, it could be any XML file. Maybe other people will find it useful too.

So to start with - I thought - just do the simplest thing. I needed to associate the xml files with their relevant schemas, and of course, I could have simply done that in the script, but then what if people move things around etc., so I decided that I would put the schemas in a directory on the server, and use XMLSchema-instance attributes to identify which schema belongs with each file.

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="schema.xsd"

OK - so I'd have to edit each of the half-dozen or so configuration files, but that's a one-off job, so not much trouble. The .NET framework's XmlReader can detect this, and use it to locate the correct schema. (although if it isn't correctly specified, you won't see any validation errors even if the file is incorrect. I'll hope to fix that in a later version of my script.)

I created a function in powershell, like this:

# So far this silently fails to catch any problems if the schema locations aren't set up properly
# needs more work I suppose. Until then it can still deliver value if set up correctly
function ValidateXmlFile {
    param ([string]$xmlFile       = $(read-host "Please specify the path to the Xml file"))
    "==============================================================="
    "Validating $xmlFile using the schemas locations specified in it"
    "==============================================================="
    $settings = new-object System.Xml.XmlReaderSettings
    $settings.ValidationType = [System.Xml.ValidationType]::Schema
    $settings.ValidationFlags = $settings.ValidationFlags `
            -bor [System.Xml.Schema.XmlSchemaValidationFlags]::ProcessSchemaLocation
    $handler = [System.Xml.Schema.ValidationEventHandler] {
        $args = $_ # entering new block so copy $_
        switch ($args.Severity) {
            Error {
                # Exception is an XmlSchemaException
                Write-Host "ERROR: line $($args.Exception.LineNumber)" -nonewline
                Write-Host " position $($args.Exception.LinePosition)"
                Write-Host $args.Message
                break
            }
            Warning {
                # So far, everything that has caused the handler to fire, has caused an Error...
                Write-Host "Warning:: Check that the schema location references are joined up properly."
                break
            }
        }
    }
    $settings.add_ValidationEventHandler($handler)
    $reader = [System.Xml.XmlReader]::Create($xmlfile, $settings)
    while($reader.Read()){}
    $reader.Close()
}

With this function in place, all I have to do is have a list of lines like the following:

ValidateXmlFile "C:\Program Files\Tridion\config\cd_instances_conf.xml"
ValidateXmlFile "C:\Program Files\Tridion\config\live\cd_broker_conf.xml"

If I've made typos or whatever, I'll pretty soon find them, and this can easily save hours. My favourite mistake is typing the attributes in lower case. Typically in these config files, attributes begin with a capital letter. Once you've made a mistake like that, trust me, no amount of staring at the code will make it obvious. You can stare straight at it and not see it.

So there you have it - as always - comments or improvements are always welcome, particularly if anyone knows how to get the warnings to show up!

In recent times, SDL Tridion have been putting out marketing videos on their very own YouTube channel. Each one takes a fictitious (or perhaps anonymised) use-case for their products and presents a fairly non-technical view of it by means of animated graphics, with voice-overs in disconcertingly generic varieties of British English. Never mind British, it's English English, of a sort of carefully not-too-posh-but-not-too-common-either, home-counties kind.

It's a bit strange for me. Obviously, I'm not the target audience. The videos are aimed at non-technical people who are not familiar with Tridion. Over the years I've got to know Tridion pretty well, both as a company and as a product suite. From so close up, the videos have an almost surreal aura. The multitudinous mismatches between the marketeers' in-world reality, and my own, leap out at me.

Why British English, when it's a Dutch company, whose biggest growth market is the US? Why this particularly strange variant of British English? (OK - full disclosure here; I'm a Geordie!) Is this the SDL influence, or does this come from US marketeers?

And are the examples real? I'm curious.

And so on....

As for the content, at least the most recent one (SDL Tridion WCM Platform for Syncronizing Online Content) had me jumping up and down in a couple of places. Like this: "... I can always revert back to previous versions of any content. It is important for our compliance department that we have proof of how a web site looked at any date in the past". Talk about a confusing message. Firstly, normal versioning in Tridion is not intended for that, and won't meet that need. They have a separate product called Archive Manager, which could be used to implement a solution for this problem.

Of course, if their marketing department were to put out a message aimed at someone like me, they would almost certainly deserve to be fired on the spot. It just hits my weirdness button. Know what I mean?

I have to admit, I nearly choked when I saw this. Bear with me a moment, and I'll tell you all about it.

Not so very long back, I was trying to set up the SDL Tridion Content Porter to work as part of an automated "build" system. One of the requirements was that I'd be able to save content and then re-import it into various different publications. After all, you need a go-to-production option as well as being able to support development and test work. After beating my brains out trying to figure out the publication mapping features in Content Porter, I asked around a bit, and found that lots of people have trouble with this, and one well-favoured option is just to go round the problem and run regexes against the entire intermediate file set to swap publication names. Oh-kaay - so a couple of hours later I'd hacked out some javascript that would do this, and solving the mapping problem the official way promptly went on the back burner, perhaps for ever (or at least until the pan boils dry). Moral of the story: doing it the official way is just too stupidly hard.

So on with the tale: This morning I was installing SDL Tridion Webforms on my research image. This was the first time I've installed WebForms, so I was stepping gently through the documentation when I realised that the WebForms installation relies on Content Porter. OK then - a quick excursion to install Content Porter, configure the Business Connector etc., and back to the main plot. That's when I nearly choked. The documentation for the WebForms installation contains this little gem:

When you perform this import, Content Porter creates a Publication called WebForms. Once created, this Publication contains the items that you need in order to use WebForms Designer and WebForms Field Type Editor. Alternatively, you can also import WebForms items into an existing Publication. Note To import WebForms into an existing Publication, rename the Publication to which you want to import the items to WebForms before you run Content Porter. Then, after you have used Content Porter, rename the Publication back to its original name.

So there you have it: according to Tridion, the correct way to solve this problem is to rename the publication, and then rename it back again afterwards. Let's hope that's an OK thing to do in your environment.

To tell the truth, I quite like this as a "hack". It's robust and solid, and very definitely gets the job done. In fact, it's about as nice a job of working around Content Porter's limitations as you'll find. I wish I'd thought of it myself. In fact, part of the reason for this post is as a public service announcement for anyone who doesn't happen to spend their Sunday mornings reading the WebForms installation manual.

But please, Tridion. Isn't this a wake-up call? When your own product installation guides have to give out workarounds like this. I know there's a new version of Content Porter on the roadmap, and I very sincerely hope that it's going to come with batteries included. While I'm on the subject - this is what the WebForms installation guide says a couple of lines further down:

Important:
Due to dependencies between items that you are importing, you will have to run the Content Porter twice in order to import all items used by SDL Tridion WebForms Designer. The first time that you run the Content Porter, you will receive error messages during the import process. These messages are not critical. You can click "Skip All" to continue.

The Content Porter should manage this. If the import of an item fails because its dependencies aren't there yet, and the dependencies are there in the package, then just wait until the end and redo it. Automatically! Rinse and repeat. Why inflict this misery on end users? There's a very real use case for Content Porter where you want to produce a package to give to someone else to import, and you don't want them worried by this kind of nonsense.  If there's any reason for the existence of Content Porter, it's the managing of depencies between the items being imported.

End-user misery aside - I should be able to use the Content Porter as part of an automated solution, and that just won't fly while I have to know in advance that a particular package requires two or more attempts to succeed.

/rant

Those of you who read my previous post will remember that I accessed Tridion's configuration by instantiating a TDSXGit.Configuration object. People who've worked with Tridion for a while may remember that it used to be quite common to edit the configuration in a file called cm_cnfg_git.xml. This file is still there, but without the xml extension, and these days it's encrypted so it doesn't make much sense to try to edit it directly.

To an Englishman like myself, this name TDSXGit is vaguely funny, because "git" in British English is a mild term of abuse. It's not uncommon for me to come out with phrases like: "Which stupid git broke the build"? It's definitely abuse, but fairly mild; you can say it to someone that you like. 

But to the point: Back in the R4 days, Tridion's configuration data was kept in the registry, which was all well and good, but had it's own problems. When R5 was designed, there was so much XML around the place that it seemed much more sensible to keep the configuration in an XML file. The problem with this was that all that disk IO would have been a total performance killer. We needed a memory cache. Good idea, you might think, but in a COM-based web application, how do you do that? The design we ended up with makes use of a couple of fairly obscure features of COM. (By the way - I'm not claiming any credit for this, just describing what was designed by other members of the team.)

The idea is to get an object to remain in memory, and to provide a mechanism whereby any code within the application can grab a reference to the object. In COM, a reference to an object is always a pointer to an interface. Memory access in COM is controlled by "apartments" - objects running in one apartment can't directly access objects running in another apartment.  In particular, if you have an interface pointer for an object in one apartment, you can't just use that pointer from a different apartment. The interface pointer needs to be "marshalled" across the apartment boundary; in other words, if you should be talking to a proxy that's local to your apartment, you'll get a pointer to that instead.  The mechanism for doing this is called the global interface table, hence the acronym GIT.

The GIT is visible from anywhere in the process, and if you register an interface with the GIT, that immediately takes place of the first problem, that of keeping the object in memory. In COM, memory management is done by reference counting. An object keeps track of how many other objects currently have a reference to it, and if that number drops to zero, the object will self-destruct, thereby freeing any memory it was using. As soon as you register an object with the GIT, well the GIT has a reference to it, and therefore it isn't going to self destruct, so you have your memory cache.

When you register an object with the GIT, the API hands you back a "cookie". A cookie in this context is just a number. If you know this number, you can ask the GIT for an interface pointer that references the object. You can keep doing this as many times as you like, unless there's been an explicit call to release the object from the GIT. The interface pointer you get back will work in the apartment you are in.

There's one more thing that you need to make this all work, and that's a way of making sure the cookie is always available when you need to get hold of your memory cache. For this, you can use another obscure COM feature: the shared properties manager (SPM). This just allows you to save a value by name and retrieve it. (The SPM also takes care of a couple of other things, like grouping the properties to prevent name collisions, and locking to control access contentions.)

So when a Tridion process first accesses the configuration, the configuration file will be decrypted and loaded into a DOM object, and the DOM will be registered with the GIT. The cookie is then stored in the SPM. Any subsequent accesses for the life of the process will be simply a matter of grabbing the cookie from the SPM and using it to get the interface pointer from the GIT.

There are other techniques that could be used, but this has the advantage not only of eliminating disk IO, but also repeated parsing of the XML to create the DOM.

This should explain why when you update some configuration value, you have to shut down each of the processes that make use of the configuration. The GIT and SPM are each specific to a process. It is technically possible to get TDSXGit.Configuration to release the DOM from the GIT, but none of Tridion's application code actually does this. That's a reasonable design for a server application that isn't re-configured very often.

In theory (at least according to the theory I just described), it should only be necessary to restart a process if it is affected by the configuration value that just changed, but my own experience flies in the face of this, and I always restart all the processes. It's a bit of a cargo cult thing I suppose, but I'll keep doing it. Actually, I'd love someone to point out where my reasoning is flawed. I hate that cargo cult thing.

"Which stupid git forgot to restart the processes after that configuration change?"

One morning this week I woke up to a very strange combination of life and programming. Kind of a "things that make ya go hmm..." moment. Here's how it went:

Some considerable time ago, I began work on a re-write of a Tridion events system. The existing system was so gunged up with layer upon layer of cruft that we decided it was unmaintainable and that we wouldn't touch it unless absolutely necessary. The fickle winds of project management have now blown us back onto the jagged rocks; we ended up dispatching from the new system back to the old code, which turns out to be fairly broken, and I have to mend it. Ouch!

No big deal, you may think, until you realise that the broken piece is a thirty page long stretch of Visual Basic 6. That's thirty pages all in one function. This code handles a Tridion OnComponentSavePost event, and in the process, saves the component four separate times. Each of these saves causes a recursion into the handler, each time from a different place and with a different data state. Depending on the state, various if-then-else blocks either fire or don't, leading to the code being executed in an order very different from what you see laid out as you hit page-down thirty-odd times. As you can imagine, before making even minor changes to this stuff, you have to marinate in the code for quite some time, tracing the various paths of illogic through the smoke-trails of your predecessor's brain-damaged nightmare. One thing's for sure - you know if you touch anything, there's a huge risk that you'll break something else.

If you've ever done this kind of work, you'll know that once you've lived in the code for a few days, there's every likelihood that you'll spend your nights dreaming of it too.

During this time, our seven-month old son, Finlay has had a virus, resulting in some fairly disrupted sleep for us all. Perhaps once every hour or two through the night, the poor lad would cry out, and you'd find yourself awake in response. After a night of this, it becomes increasingly hard to come up out of deep sleep, and one morning at five or six o'clock, and  I found myself in the middle of a dream of scrolling code, hearing a cry, but hearing it as a bug. Huh? How does that work? You hear of people with psychological disorders hearing colours and so forth, and I suppose in the dream world there are no rules. I heard a cry, and experienced it as a sort of "Oh shit, there's another bug." moment.

Finlay's back to health now, and (mostly) sleeping through the night. Maybe some time soon, I will too.

Pulling website information out of the IIS metabase

using System;
using System.Collections.Generic;
using System.Text;
using System.DirectoryServices;

namespace Hinttech.Dotnet.Samples.WebSiteDumper
{
    class Program
    {
        static void Main( string[] args )
        {
            using (DirectoryEntry webServers = new DirectoryEntry("IIS://localhost/W3SVC"))
                {
                foreach( DirectoryEntry server in webServers.Children)
                {
                    PropertyValueCollection serverComment = server.Properties["ServerComment"];
                    if ( serverComment.Value != null && serverComment[0].ToString() == "Tridion Content Manager")
                    {
                        Console.WriteLine("The Tridion web site is running on: ");
                        Console.WriteLine("===================================");
                        // If you want the https sites too, you need to do the same thing for "SecureBindings"
                        foreach ( string serverBinding in server.Properties["ServerBindings"] )
                        {
                            string[] serverBindingParts = serverBinding.Split(':');
                            string ipAddress = serverBindingParts[0];
                            string port = serverBindingParts[1];
                            string hostHeader = serverBindingParts[2];
                            if (string.IsNullOrEmpty(ipAddress))
                            {
                                ipAddress="Default";
                            }
                            Console.WriteLine(
                                "\tIP Address = {0}\n\tPort= {1}\n\tHostHeader= {2}",
                                ipAddress, port, hostHeader );
                            Console.WriteLine("===================================");
                        }                                            
                        server.Dispose();
                        break;                        
                    }
                server.Dispose();
                }
	    }

#if DEBUG
            Console.ReadKey();
#endif
        }
    }
}

I was recently asked a question about XSLT templating in Tridion. The question was about how you create component linking code from within your XSLT component template. The tricky part is that you need a publication ID and a component template ID. My solution involved getting these in the page template and passing them in to the XSLT as parameters. OK - job done - it's just coding from there eh? Well yes - but it didn't stop being interesting at that point, because my questioner in the meantime had come up with his own solution based on some sample code that he'd found on the Tridion forum. He hadn't needed to add the parameters himself, because Tridion adds some parameters for you as if by magic.

It turns out that if you include any of the following three global parameters at the top of your XSLT, Tridion will add the corresponding DOMs as parameters as if by magic.

<xsl:param name="tcm:Page"/>
<xsl:param name="tcm:Publication"/>
<xsl:param name="tcm:ComponentTemplate"/>

This is very cool, and saves you the effort of coding it, but I don't know whether it's a supported feature. I suppose it should be, because it's really just the standard behaviour for a component template translated to XSLT. I just can't find it anywhere in the documentation.

There is a gotcha, mind you! Although the parameters are declared in the "tcm:" namespace, which resolves to "http://www.tridion.com/ContentManager/5.0", don't be tempted to use a prefix other than "tcm:" for this namespace. If you do, you'll find that there's something broken; the parameters don't seem to be added automatically. (Tested on R51SP4) Still - as it's an undocumented, and perhaps unsupported feature, we can't really complain.

It would make a good enhancement request though. Perhaps Tridion can be persuaded to document this and support it. In the meantime, perhaps it's better to be on the safe side and explicitly add the parameters you need to the Component Presentation from the Page Template.

Tridion behaves badly when publishing ASPX