I've put another new post up on the Indivirtual blog. This time it's about how to create an HTML validation server

http://www.indivirtual.nl/blog/create-run-w3c-validation-server/

I've used Dyndns's free dynamic dns service for years now. Granted, in recent times they have made it progressively more difficult to use the free service. I suppose they want to get paid. I've now been notified that tomorrow they will stop the free service altogether. Oh well - plenty more free dns providers in the sea - at least I hope so. A quick Google found me www.noip.com - so this evening I switched over. All seems well so far.

Anyway - Dyndns were early players in this game, back in a time when the Internet was a different place. Much kudos to them for providing this service over the years. I suppose eventually commercial interest had to come to the fore, but we had a good run. Thanks Dyndns.

I've just published a new post on the Indivirtual blog that explains How to set Rights and Permissions using the SDL Tridion core service API

I've just published another blog post at Indivirtual. This time it's about the ImportExport API. As you might guess - I had to have a go using the powershell. Here's the link:

http://www.indivirtual.nl/blog/sdl-tridions-importexport-api-end-content-porter/

When designing and implementing web-content-managed web sites with Tridion, the usual arrangement is to have at least four distinct environments, designated for specific purposes. Development, Test, Acceptance, Production. Often we refer to this as the DTAP street. Each environment has its own peculiarities. The production environment serves web pages to the visiting public, so will have at least some servers in the "demilitarized zone". There will probably be multiple web servers behind a load balancer, and particular attention will be paid to defences against the ne'er-do-wells of the Internet. The acceptance environment will be used for the final testing of software releases before they are allowed on to the production system. If the production system is load-balanced, so will the acceptance system be, and lots of attention will be paid to ensuring that the A-environment is a truly representative copy of P. The hardware will be close to identical, and all software will be patched to exactly the same levels as on the production system (unless, of course, a patch is being rolled out, in which case this will take place first in A). The other two environments belong to the development team. The Test environment is used for testing during a development project, to ensure that the necessary quality levels are achieved before moving on to acceptance testing. New versions of the software may be frequently deployed - perhaps daily or several times a day. In general, the environment will be maintained to be a good representation of the Production environments, but not to quite the same levels of obsession as for the Acceptance environment. The Development environment will be quite similar to the Test environment, but is likely to have extra software installed for use in development. Programming software, automated build and test software - that kind of thing. Typically the programmers will have more access privileges in the Development environment than in the other environments. Depending on the organisation, they may well be system administrators in D, and have significant privileges in T. This makes sense, because often they will need to try new approaches, and set up new configurations, or perhaps they might need to attach a debugger to the running software to analyse its processing.

All of this adds up to a significant investment. There will be an entire team of people busy for quite a while to get this set up and to maintain it. Hardware (although often virtualised these days - still complex enough), software - operating systems, databases, security, etc, etc, Then you have to add in all the work of simply managing the whole thing. It's not cheap, and then on top of that, the licenses usually aren't free. So there's a temptation to cut corners. This can mean missing out an environment entirely, or even two - although even the most miserly will usually draw the line at doing development work on the same system that serves the public. It can also mean taking shortcuts in configuration expenses. Maybe you can't afford to have your system administrator spend his time making special configurations for the development environment. The thing is, making sure everything is done right can be unpalatably expensive. So, of course, the first thing to do is ensure that such an expensive set-up is a good fit for your needs. For the vast majority of web sites, you definitely don't need a high-end enterprise web content management system like Tridion. If you do, however, then it's probably a pretty good sign that the expense of running a proper DTAP street is also worth it.

But what if you want all that goodness without having to pay for it? Well in that case, the responsible technicians need to make it clear what the trade-offs are. You can save money, but it's a gamble. The problem is that getting these things working doesn't just cost money. In an emergency, you can usually get more of that. The trouble is, that it also costs time, and the definition of an emergency is that you don't have any of that spare. So - imagine a situation where you would like to be able to debug a problematic piece of software, but your security requirements are pretty heavy, and cast in procedural concrete. So you attempt to set up the necessary tools in your development system, but it doesn't work. To get it working, you estimate you'll need to spend a couple of days of research (say - a day each for a developer and a sysadmin). Maybe it's twice that, maybe it's half, and maybe you need to write a report on all the possible approaches, and have it approved by a committee of architects. Whatever - it's more expensive than you'd like... so you choose not to do it. This is the point at which clear communication is essential.

Living, as I do, in Amsterdam, I can't help feeling just a bit smug as I listen to the news on the radio. In England, the Somerset Levels have been flooded for a long time, and it's still raining. The amount of rain landing on the South of England, and on Wales just now is more than normal - that is to say, it only comes down like that a few times in a century. The people of the Somerset Levels are complaining vociferously that maybe they'd have stood for being flooded for a week, but the water won't go away so they've had it for weeks and weeks. Now some people near London are getting their feet wet too, so suddenly it's important. :-) On my way home, I felt some of that weather. The rain was lashing down, with a pretty solid wind behind it. Was I worried? Not in the slightest, even though I live at least as far below sea level as the people of the levels. You see, here, if the weather gets like that, the only real effect you'll see, is perhaps some smoke coming out of the chimney at your local friendly pumping station. The whole landscape is littered with places for water to go. Every little canal or pool has big sloping sides that will accommodate several times the normal amount of water.

So - when it rains in Somerset (nothing personal, folks), they get their feet very wet indeed, and have some very uncomplimentary things to say about the government's Environment Agency, whose job it is to build and maintain the DTAP street. Various government agencies turn up to provide sandbags. When it rains here, the pumps kick in, and we're good. Sometimes I find it hard to articulate to budget holders exactly why I'd like them to spend money on the odd pumping station that's never really going to get used, is it? I mean come on, what are the chances? Did I say pumping station?

Seriously - if you're going to cut corners on your infrastructure, make sure all your stakeholders know the difference between Somerset and Amsterdam.

Anyone who works with Tridion content delivery will be familiar with the fact that Logback is used as the logging framework. Recently I found myself looking into this more than I had previously, so here are a couple of observations that might be interesting. The first is that you can use the groovy scripting language instead of XML to write your configuration files. (I'll get to exactly how useful, or otherwise, this might be in a bit...) Anyway - the following is a machine translation of the logback.xml file that ships with Tridion, Now - proponents of the groovy approach will tell us that groovy can be much terser than the XML equivalent. At first sight it doesn't look much different, but I imagine you could factor out the creation of all those appenders to some sort of factory, and then it would look a lot shorter. Can I leave that as "an exercise for the student"? :-)

import ch.qos.logback.classic.encoder.PatternLayoutEncoder
import ch.qos.logback.core.rolling.RollingFileAppender
import ch.qos.logback.core.rolling.TimeBasedRollingPolicy
import java.nio.charset.Charset

import static ch.qos.logback.classic.Level.${LOG.LEVEL}
import static ch.qos.logback.classic.Level.OFF

scan()
def log.pattern = "%date %-5level %logger{0} - %message%n"
def log.history = "7"
def log.folder = "c:/tridion/log"
def log.level = "ERROR"
def log.encoding = "UTF-8"
appender("rollingTransportLog", RollingFileAppender) {
  rollingPolicy(TimeBasedRollingPolicy) {
    fileNamePattern = "${log.folder}/cd_transport.%d{yyyy-MM-dd}.log"
    maxHistory = "${log.history}"
  }
  encoder(PatternLayoutEncoder) {
    charset = Charset.forName("${log.encoding}")
    pattern = "${log.pattern}"
  }
  prudent = true
}
appender("rollingDeployerLog", RollingFileAppender) {
  rollingPolicy(TimeBasedRollingPolicy) {
    fileNamePattern = "${log.folder}/cd_deployer.%d{yyyy-MM-dd}.log"
    maxHistory = "${log.history}"
  }
  encoder(PatternLayoutEncoder) {
    charset = Charset.forName("${log.encoding}")
    pattern = "${log.pattern}"
  }
  prudent = true
}
appender("rollingMonitorLog", RollingFileAppender) {
  rollingPolicy(TimeBasedRollingPolicy) {
    fileNamePattern = "${log.folder}/cd_monitor.%d{yyyy-MM-dd}.log"
    maxHistory = "${log.history}"
  }
  encoder(PatternLayoutEncoder) {
    charset = Charset.forName("${log.encoding}")
    pattern = "${log.pattern}"
  }
  prudent = true
}
appender("rollingCoreLog", RollingFileAppender) {
  rollingPolicy(TimeBasedRollingPolicy) {
    fileNamePattern = "${log.folder}/cd_core.%d{yyyy-MM-dd}.log"
    maxHistory = "${log.history}"
  }
  encoder(PatternLayoutEncoder) {
    charset = Charset.forName("${log.encoding}")
    pattern = "${log.pattern}"
  }
  prudent = true
}
appender("rollingSessionPreviewLog", RollingFileAppender) {
  rollingPolicy(TimeBasedRollingPolicy) {
    fileNamePattern = "${log.folder}/cd_preview.%d{yyyy-MM-dd}.log"
    maxHistory = "${log.history}"
  }
  encoder(PatternLayoutEncoder) {
    charset = Charset.forName("${log.encoding}")
    pattern = "${log.pattern}"
  }
  prudent = true
}
logger("com.tridion", ${LOG.LEVEL})
logger("com.tridion.transport", ["rollingTransportLog"])
logger("com.tridion.transport.HTTPSReceiverServlet", ["rollingDeployerLog"])
logger("com.tridion.transport.transportpackage", ["rollingDeployerLog"])
logger("com.tridion.transformer", ["rollingDeployerLog"])
logger("com.tridion.deployer", ["rollingDeployerLog"])
logger("com.tridion.tcdl", ["rollingDeployerLog"])
logger("com.tridion.event", ["rollingDeployerLog"])
logger("com.tridion.monitor", ["rollingMonitorLog"])
logger("Tridion.ContentDelivery", ${LOG.LEVEL}, ["rollingCoreLog"])
logger("com.tridion.preview", ["rollingSessionPreviewLog"])
logger("com.tridion.storage.persistence.session", ["rollingSessionPreviewLog"])
root(OFF, ["rollingCoreLog"])

So why might this be interesting to Tridion infrastructure specialists? Well it isn't. Not at all. At least not right now - because doing it this way requires the groovy runtime to be available, and that isn't in a standard Tridion content delivery setup. I attempted a trivial hack by dropping a couple of the groovy jars in place, but no joy. Realistically, this would only be a practical approach if Tridion decided to build it into the product and support it. I imagine the dev team puts quite some effort into keeping their dependency tree as clean as possible, so this might come under the heading of stuff that would only get added if people really, really wanted it!

Anyway - I love the smell of XML in the morning, so it's all the same to me. So on with the useful part of this post. If you check out exactly how logback gets its configuration settings, you'll see that before it picks up logback.xml, it first looks for a file called logback-test.xml. I'm happy to say that this does work out-of-the-box. This means that when you come across a server where you need to debug a problem, and its standard logging settings need to be boosted up to DEBUG, you don't have to edit the existing config file. Just drop your insanely debuggy logback-test.xml file in next to logback.xml (and restart things) and Bob's your uncle. When you're done, just delete (and restart). Even the restarting might be optional - another feature of logback is that you can configure it to scan for configuration changes, although I have no clue whether it would then pick up the existence of logback-test.xml)

Ok - this is such a minor benefit over copying and renaming that it hardly justifies the deaths of all those IP packets that were bravely lost in transmission during the serving of this web page. Whatever.... that's the thing with research, eh? Negative results are also important to report. In short - logback.groovy looked cool, but won't work - and maybe carrying a customised logback-test.xml around in your toolkit might be handy, but then again, maybe not.

I'll sign off with one more public service announcement. I recently saw someone using a logback configuration that specified a logging level of ON. Apparently they had been advised to do so by someone who ought to have checked first. The possible values are OFF, ERROR, WARN, INFO, DEBUG and TRACE. Anything other than that will not be recognised, and you'll get DEBUG logging, which is the default if that happens.

I'd been following the Gentoo Wiki guidance on upgrading Grub, and had been taking it very carefully. I'd worried about getting this right, as getting it wrong would leave me with a brick, so I'd been very pleased to see the notes on using the old bootloader to chain load the new one. That way I could check that my configuration was correct before taking the plunge of installing the new version into the Master Boot Record. I didn't want to automatically generate the new config file, as I didn't trust it. (Rightly so as it turned out, because my initrd files didn't follow the strict naming requirements, so weren't picked up by the config generation script) Anyway - the hand-written config was half a dozen lines long, and the generated one was utterly incomprehensible.

So anyway - I managed to create the config file, and get everything set up for chain loading. I rebooted the server, and bingo - there was the chain loader entry in my "old" boot screen, and when I followed it, I got the new menu and could boot the server. Great stuff! Now it should have been a simple question of running grub2-install, and I'd be finished. So I did this, and then.... the computer wouldn't start. Fortunately I had a grub prompt, so grub was "working" - but it obviously couldn't find its config file. I already knew that with the right incantations it might be possible to get the thing to boot without a config file, and after a bit of googling, I got enough clues to attempt it. (For the record, what I think I'd done wrong was to fail to remount /boot after my chain test and before running grub2-install, with the result that grub then didn't know how to correctly find /boot.)

It took a few attempts, but the command line completion in grub helps a lot. This is what I eventually ended up typing at the grub prompt to get a working boot.

grub > set root=(hd0,1)
grub > linux /kernel-gen-newudev-3.3.8-gentoo root=/dev/sda3
grub > initrd /initramfs-gen-newudev-3.3.8-gentoo
grub > boot

Note that the root for the boot loader is different from the root of the operating system, so you have to specify them separately. Obviously YMMV for the names of the kernel and initrd files, not to mention device identifiers.

But the real advice here is to avoid missing out that crucial mount operation!!

So there I was - just for fun building my new Gentoo system, when all of a sudden, I wasn't. Building, that is. I wasn't building anything. In fact, part of the motivation for a clean build had been that emerging new things was getting tiresomely fragile. Anyway - here's what happened when I tried an emerge. The interesting part is where it says: Fatal Python error: Failed to open /dev/urandom

>>> Emerging (1 of 18) sys-libs/glibc-2.17
 * Fetching files in the background. To view fetch progress, run
 * `tail -f /var/log/emerge-fetch.log` in another terminal.
 * glibc-2.17.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                            [ ok ]
 * glibc-2.17-patches-8.tar.bz2 SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                 [ ok ]
make -j2 -s glibc-test
make -j2 -s glibc-test
>>> Unpacking source...
 * Checking gcc for __thread support ...                                                                                             [ ok ]
 * Checking kernel version (3.3.8 >= 2.6.16) ...                                                                                     [ ok ]
 * Checking linux-headers version (3.9.0 >= 2.6.16) ...                                                                              [ ok ]
>>> Unpacking glibc-2.17.tar.xz to /var/tmp/portage/sys-libs/glibc-2.17/work
>>> Unpacking glibc-2.17-patches-8.tar.bz2 to /var/tmp/portage/sys-libs/glibc-2.17/work
 * Applying Gentoo Glibc Patchset 2.17-8 ...
 *   0035_all_glibc-2.16-i386-math-feraiseexcept-overhead.patch ...                                                                  [ ok ]
 *   0059_all_glibc-2.19-make-4.0.patch ...                                                                                          [ ok ]
 *   0065_all_glibc-2.18-qecvt-guards.patch ...                                                                                      [ ok ]
 *   0070_all_glibc-2.18-localedef-page-align-1.patch ...                                                                            [ ok ]
 *   0071_all_glibc-2.18-localedef-page-align-2.patch ...                                                                            [ ok ]
 *   0072_all_glibc-2.18-localedef-page-align-3.patch ...                                                                            [ ok ]
 *   0085_all_glibc-disable-ldconfig.patch ...                                                                                       [ ok ]
 *   0090_all_glibc-2.17-arm-ldso.cache.patch ...                                                                                    [ ok ]
 *   1005_all_glibc-sigaction.patch ...                                                                                              [ ok ]
 *   1008_all_glibc-2.16-fortify.patch ...                                                                                           [ ok ]
 *   1040_all_2.3.3-localedef-fix-trampoline.patch ...                                                                               [ ok ]
 *   1055_all_glibc-resolv-dynamic.patch ...                                                                                         [ ok ]
 *   1505_all_glibc-nptl-stack-grows-up.patch ...                                                                                    [ ok ]
 *   1506_all_glibc-2.17-hppa-fpu.patch ...                                                                                          [ ok ]
 *   1507_all_glibc-2.17-hppa-ldso-flag.patch ...                                                                                    [ ok ]
 *   1507_all_hppa-ia64-DL_AUTO_FUNCTION_ADDRESS.patch ...                                                                           [ ok ]
 *   1508_all_glibc-2.17-hppa-futex.patch ...                                                                                        [ ok ]
 *   1508_all_hppa-fanotify_mark.patch ...                                                                                           [ ok ]
 *   3020_all_glibc-tests-sandbox-libdl-paths.patch ...                                                                              [ ok ]
 *   5063_all_glibc-dont-build-timezone.patch ...                                                                                    [ ok ]
 *   6024_all_alpha-fix-signal-thunk-unwind-info.patch ...                                                                           [ ok ]
 *   6230_all_arm-glibc-hardened.patch ...                                                                                           [ ok ]
 * Done with patching
 * Using GNU config files from /usr/share/gnuconfig
 *   Updating scripts/config.sub                                                                                                     [ ok ]
 *   Updating scripts/config.guess                                                                                                   [ ok ]
>>> Source unpacked in /var/tmp/portage/sys-libs/glibc-2.17/work
Fatal Python error: Failed to open /dev/urandom
/usr/lib/portage/bin/phase-functions.sh: line 87:  4204 Aborted                 "${PORTAGE_PYTHON:-/usr/bin/python}" "${PORTAGE_BIN_PATH}"/ilter-bash-environment.py "${filtered_vars}"
 * ERROR: sys-libs/glibc-2.17::gentoo failed (unpack phase):
 *   filter-bash-environment.py failed
 *
 * Call stack:
 *            ebuild.sh, line 714:  Called __ebuild_main 'unpack'
 *   phase-functions.sh, line 993:  Called __filter_readonly_variables '--filter-features'
 *   phase-functions.sh, line 137:  Called die
 * The specific snippet of code:
 *      "${PORTAGE_PYTHON:-/usr/bin/python}" "${PORTAGE_BIN_PATH}"/filter-bash-environment.py "${filtered_vars}" || die "filter-bash-enviroment.py failed"

So what was going on here? Well as it turned out, my system has three versions of python loaded, and Gentoo's portage system (of which emerge is part) seems to rely on you not using python 3. After a short bit of fiddling with "eselect python list" and "eselect python set", to get the default python back to 2.7, the build ran like a charm.

So anyway - this has got to count as the most bizarrely mis-reported error in my most recent years. "/dev/urandom" was working fine. I could start it and stop it ("/etc/init.d/urandom stop" and so forth) and I could use it to access randomness. Why then did I get the "failed to open" message with one version of python, and not with another. Answers on a postcard? Whatever - this was a public service announcement.

Most likely - when you hear the word Slavery, you think of the atrocities of the past. The transatlantic slave trade must be the best known, but throughout history people have enslaved each other. Every single one of us has slave ancestors and slaver ancestors. The history of slavery is not just the hugely visible mass-operations that we learned about in school. It's also gone on at a much smaller scale throughout the history of the human race.

So why am I writing this? Because that history continues. Because slavery is still among us in the world of today, and on a shocking scale. That is to say - there's no huge headline slave trade, but over the world, there's enough of it going on to be truly shocking. There are significant numbers of people in the world today who live as slaves. Frankly - no civilised person can tolerate this without doing something to help. The trouble is that so much rhetoric has been wasted on the wrongs of the past that the very same thing going on right now in this world of ours today is almost invisible to us.

But it's not all bad news. Somebody just sent me a link to this story about major corporations using big data to help identify and help with the problem.

When I became aware of the whole issue of modern slavery, I wondered what I could do to help. Nothing, right? Invisible problem - definitely invisible solution. So I looked into it a bit, and discovered that Anti-slavery.org is working hard to find solutions. Not only that, but they've been doing it for 170 years. In all that time, they have never reached a moment when anyone could say "OK guys - job done, we can disband and go and do something more useful instead".

So I've decided to dedicate my run in this year's Egmond half marathon to raising money for this cause. Please join me by visiting my sponsorship page and making a donation.

Thank you.

Most of my blog output is to be found right here, but this week I published an article on the Indivirtual web site.

http://inside.indivirtual.nl/2013/10/the-razor-mediator-for-tridion-in-practice/

The article goes into the background of the Razor Mediator for Tridion, and our thinking behind using it on a customer project. I hope you find it interesting.