The title of this post is a tribute to the fantastic Conor vs. SQL blog.

I was working on a database with several triggers on each table. The triggers audited INSERTs, UPDATEs and DELETEs by inserting records to shadow tables in a separate logging database. The database was backed up from one environment and restored to another in which the name of separate logging database was different. I wrote the following script to automatically update all the triggers, replacing any occurrence of the old logging database name with the correct new name.

Heads up: you could really do some damage with this script. Backup and test a restore from that backup before using it!

DECLARE @replaceThis VARCHAR(8000)
DECLARE @withThis VARCHAR(8000)
SET @replaceThis = 'OldLogDatabaseName'
SET @withThis = 'NewLogDatabaseName'
DECLARE c Cursor
FOR
    SELECT
        object_id,
        SCHEMA_NAME(schema_id) AS schema_name,
        name AS object_name
    FROM sys.objects
    where type_desc = 'SQL_TRIGGER'
    ORDER BY modify_date;
OPEN c
DECLARE @triggerObjectId AS INT
DECLARE @triggerSchema AS NVARCHAR(4000)
DECLARE @triggerName AS NVARCHAR(4000)
DECLARE @triggerDefinition AS NVARCHAR(MAX)
DECLARE @sql AS NVARCHAR(MAX)
FETCH NEXT FROM c INTO @triggerObjectId, @triggerSchema, @triggerName
WHILE (@@FETCH_STATUS <> -1)
BEGIN
    SELECT    @triggerDefinition = m.definition
    FROM    sys.sql_modules m, sys.triggers t
    WHERE    m.object_id = t.object_id
        AND t.object_id = @triggerObjectId
    PRINT 'Dropping ' + @triggerName + '...'
    SET @sql = 'DROP TRIGGER ' + @triggerSchema + '.' + @triggerName
    EXEC sp_ExecuteSql @sql;
    PRINT 'Recreating ' + @triggerName + '...'
    SET @sql = REPLACE(@triggerDefinition, @replaceThis, @withThis)   
    EXEC sp_ExecuteSql @sql;
    FETCH NEXT FROM c INTO @triggerObjectId, @triggerSchema, @triggerName
END
CLOSE c
DEALLOCATE c

Cool code snippet of the week: We flew to the moon. Computers did a lot of the heavy lifting. Source code for the Command and Lunar modules, as part of an effort to emulate the machines, is up on Google Code. Here's the project home page.

This post’s about a simple extension to the Dictionary<TKey, TValue> type in .NET. It won’t change the world, but it’s a nice tool to have in the shed. I’ve quite possibly reinvented the wheel, here, too – if there’s something already floating around in the BCL that I’ve missed, please let me know :-).

What does it do?

The extension simplifies working with Dictionaries that contain zero, one or many Values corresponding to each potential TKey.

How does it do it?

The naive (and simplest!) approach is to declare a Dictionary<TKey, List<TValue>>, and do the plumbing manually when adding a new value for a key. That is, when you’re adding a new value V for a key K, you need to detect the case where there is no List<TValue> for that K, and construct a List<TValue> and add it to the Dictionary so you can then add the V to the end of your List<TValue>.

Well, I’ve written that code three times over the last six months, so I’ve made a generic extension to Dictionary<TKey, TValue> which does the above.

 

Why bother?

For a recent project, I had a whole buncha legacy code parsed into a Abstract Syntax Trees. I wanted to traverse the trees just once each (to improve locality of reference), applying transformations from a large-ish library based on the type of the current node. So a Dictionary<NodeType, ITransformer> gives me some of what I need, but falls down when I want to have multiple ITransformers for a single NodeType.

So, instead of using a Dictionary<NodeType, ITransformer>, I use a MultValueDictionary<NodeType, ITransformer>, which provides all the functionality of Dictionary<NodeType, List<ITransformer>> as well as a convenient Add() overload which takes care of constructing a new List in the (most common) case where there’s no existing List for that value of NodeType.

 

The addition of colour support to the .NET console in 2.0 made me happy. A lot of the coding I do these days is console based: I've written a bunch of utilities that trawl through the big (~1.3 million LOC) code base our vendor's delivering, generating reports that highlight possible issues. Tasteful, understated use of colour in the output reports really improves their usability. Sure, I want a console version of the <blink/>and <marquee/> tags, but colour will keep me quiet for a while.

The problem comes when you want to archive coloured text from a console window. If you launch a console app from a command prompt and redirect its output to a text file (e.g. "myapp.exe > myappoutput.txt"), all you get is a plain text file; the colour information is lost. The same happens when you mark a region in the console window and copy to the clipboard; your text makes it to the clipboard, but the colouring is left behind.

The solution's pretty simple. I wrote a thin wrapper around System.Console which adds a new feature: the ability to echo console output to an HTML file, with formatting and colouring preserved.

So now when my console apps use colour (a little something like this):

I get HTML export with colouring for free:

Simple, but effective. Sure, I'm a little ashamed I'm not dynamically injecting instructions into an IL stream or anything, but on the other hand, this works. And it's multi-coloured. Top that.

Sample code

HtmlExportConsole

If you race on iRacing.com, then you can use iStatistician to chart your performance over time.

Until the release of v2.7, iStatistician was horribly broken for a couple of weeks. Sorry if you were one of the affectees. Briefly, iStatistician reads your driving record straight from the iRacing.com website, and since iRacing.com was re-organised fairly extensively a couple of weeks ago in the inter-season break, my app was looking for all sorts of information in all the wrong places.

Unfortunately, there’s not a lot I can do to protect against this kind of occasional breakage, without talking to the iRacing.com team to get access to some kind of test environment so that I can upgrade iStatistician ahead of any major breaking releases. Obviously, there’s no guarantee they’d be up for that, and also, I’m not sure having two weeks warning (say) would actually mean I’d find the time to make the necessary modifications in time for a website upgrade :-).

While iStatistician has its fair share of horrible, crufty, brittle web scraping code, there are also a few nice elements I should write up at some point. It’s a WPF app running on the user’s machine, using NHibernate over a SQL Server Compact Edition database for persistent storage. There’s some use of LINQ over the domain model, and a nice charting API (with bitmap export, zooming scrolling, etc.) that’s quite LINQ-friendly too, and moderately reusable.

All that’s below the surface though, I guess. The whole point is that you can generate charts like the one below, which tells me that while I have been improving just a little over time, Sunday at 11:15pm is not the best time for me to race. Apparently.

Problem

Say you're comparing two XML files - say web.configs taken from two installations of a website, looking the cause of an environment-specific problem. There's nothing more annoying than trawling through a thousand semantically-neutral white-space differences flagged by your  text-based diff tool, looking for a meaningful difference.

For example, these two chunks of XML:  

 

and

mean pretty much the same thing, but to a text comparison tool, they look very different.

Solution

One solution is to transform the XML to a more "canonical" form before diffing. (I know "more canonical" is a bit like "more pregnant", but you get my meaning).

The following chunk of source compiles into a light tool which takes two parameters:

1. Input XML filename
2. Output XML filename

The tool loads the input file, and then writes a "canonical" version of its XML.

The output XML is:

• Uniformly indented based on the number of ancestor elements
• One attribute per-line (helps out text differs that are very line-oriented)
• More normalised as far as whitespace like tabs, spaces and newlines goes.
  

Basically, it's just a sneaky way of eliminating some (but not all) the irrelevant (i.e. non-meaningful, i.e. semantically-neutral) variation between two textual representations of what should be similar underlying XML.

Just in case: the canonicalization I'm talking about here has different goals and applies a different set of transformations than the canonicalization described here.

Source

 

Update 2009-10-29:

iRacing have implemented a check to restrict use of this technique... more here.

 

This post’s about a nice trick I came across to inject your own code into a third party app that’s not designed to be extensible.

The app in question is iRacing, a racing simulator. The extension is Stephane’s Telemetry HUD Plugin, which adds extra timing and telemetry info as a HUD on top of the standard 3d view that iRacing renders.

 

The Telemetry HUD Plugin ships as a single DLL, d3d9.dll, which you copy into your main iRacing program folder (the folder which contains iRacing.exe).

When I first saw this arrangement, I assumed the trick worked like this:

1. iRacing asks to load the “real” Direct3D d3d9.dll.
2. Windows applies its defined search orderfor DLLs, and decides to load Stephane’s d3d9.dll (since it’s in the local app folder), instead of the “real” dll that’s off in %windir%\system32.
3. Stephane’s d3d9.dll somehow (maybe by calling LoadLibrary? Hey, I’m not Stephane, alright?) causes the real d3d9.dll to be loaded as well so that iRacing can still get at the Direct3D functions it needs. (But what about naming collisions? Is it even possible to have two DLLs with the same name loaded? Answers below…)

I wanted to find out whether my hunch was on the right track, and how the details really worked. So first I fired up Dependency Walker and loaded iRacingSim.exe (the main iRacing executable) while the plugin was installed:

The top-left panel lists the DLLs that iRacingSim.exe depends on. I’ve highlighted D3D9.DLL – notice that it’s loaded from the iRacing folder, since I’ve got the Telemetry HUD Plugin installed, and Dependency Walker is smart enough to apply the same DLL resolution rules as Windows.

Because I’ve highlighted D3D9.DLL, the second-from-the-top-on-the-right panel is listing all the functions it exports, and the top-right panel is listing just those exported functions which are actually called by iRacingSim.exe. We can see that D3D9.DLL only exports Direct3DCreate9(), and that’s the only function which iRacingSim.exe actually calls. Contrast that with a view of the “real” D3D9.dll:

Here you can see that the real D3D9.DLL exports a whole bunch of functions (in the panel second from the top on the right), but Dependency Walker still says that only one of them is actually called by iRacingSim.exe.

So at this point, it looks to me as though Stephane either:

1. chose to fake a DLL that iRacingSim.exe called very few exports from (saving time and effort); or
2. chose D3D9.DLL because a call to Direct3DCreate9() was a convenient time to hook in and do his own initialisation (since by that point, he knows that the graphic subsystem is being initialised).

One question remains: When iRacing calls Direct3DCreate9() in what it thinks is Direct3D but is actually Stephane’s fake DLL, how does Direct3D actually get initialised? Does Stephane just pass the call through to the real D3D9.DLL? Fire up Dependency Walker and load the Telemetry HUD Plugin d3d9.dll:

Well, we can tell from the tree at the top left that Dependency Walker can’t see a dependency from the fake D3D9.DLL to the real one. Maybe Dependency Walker can only see static, declared (declarative) inter-DLL dependencies, though – perhaps if Stephane’s code imperatively sets up the dependency by calling LoadLibrary, passing a path to the real D3D9.dll, he can then call to the real thing? Maybe we could check that by checking the string table in his DLL?

Fire up devenv.exe and open the fake D3D9.dll using the binary editor. Ctrl-F to look for occurrences of “d3d9” – maybe we can find some clue as to how the real D3D9.dll gets loaded?

That’s a pretty interesting string – “\d3d9.dll”. And immediately before and after it, there’s talk of something called PROXYDLL, which looks a bit library-ish. Google is our friend, and finds us “Intercept Calls to DirectX with a Proxy DLL”. That’s an article on CodeGuru, including sample code, which uses a pretty familiar-sounding technique to inject code into a Direct3D 8 or 9 app and render extra stuff on the screen.

After downloading the sample source, I run into the following chunks of code…

Well, given that “PROXYDLL: InitInstance called.” shows up in the binary screenshot above character for character, I’m pretty sure that Stephane’s work’s based off this sample, so we can be confident the D3D9.dll technique used by the sample’s also used by Stephane’s fake D3D9.dll.

Here we go, how does LoadOriginalDll() work?

And there’s the resolution (see what I did there?): a quick call to GetSystemDirectory tells us where system32 is, and we ask LoadLibrary to load d3d9.dll from that folder.

None of this was insanely complicated, but it’s something I wouldn’t have ever tried, I think.

For starters, I would have been sceptical that it’d be so easy to subvert the DLL resolution rules for what is effectively a system library. Having said that, I see how there’s no sensible measure to prevent a user running whatever code they want on their machine, of course.

Second, I’m really surprised that iRacing, an online multiplayer game (where you’d expect some cheat/hack countermeasures on the part of the developers), doesn’t have parasite detection code that picks up either unrecognised or suspicious-looking DLLs, or at least funky stuff like “hey, wait a sec, I have TWO d3d9.dlls loaded in my process space! uncool!”.

It’ll be interesting to see how iRacing respond to this plug-in. Traditionally, iRacing have patched their software only in the one week break between quarterly seasons; only the most egregious bug exploits (for example, cuttable corners on the track) have been patched “out-of-band”. Given that the inter-season break was last week, and that Stephane released his tool a few months back, it looks like they’ve chosen to tolerate his hack-let for now.

This tool is grey-hat. It surfaces information to the driver which otherwise isn’t available: extremely detailed and precise, real-time feedback on whether you’ve fallen behind or pulled ahead of your best lap ever up to this point on the current lap – incredibly useful, with overtones of mild cheatiness.

My concern is that it’s not that far from here to a genuine black-hat tool which observes your acceleration and accelerator levels, reasons that you must be spinning the rear wheels, and eases off the accelerator by filtering the values fed from your wheel and pedals to iRacing (using a similar technique to that found above).

That class of hacks would basically ruin the key characteristic of iRacing that makes it the greatest ever online racing league: a competitive and even playing field populated by (almost universally) non-cheating (mostly) non-idiots.

I finally cracked it with the hassle of maintaining my own blog engine. The ten minutes a year I spent thinking about the five minutes a year I spent actually fixing critical bugs were really starting to bring me down. So, today, I ported some of the less embarrassing posts across to the shiny new installation of BlogEngine.NET you see before you. (That's right, the posts you see below are the ones I'm not embarrased by).

This post isn't about BlogEngine.NET, however; it's a little eulogy/rhapsody/elegy to the tiny blog engine that sputtered along, half-built, for well over five years like a Pinocchio wot ain't got no arms or sumfink because its Gepetto has way too many other commitments on the go at the moment, sorry.

I started hacking my blog engine together in early 2003, on version 1.1 of the .NET Framework. I wrote it as a way of learning ASP.NET. And .NET. And C# and ADO.NET. The codebase started out horrible, and got worse over time.

On day one, my pages hit the data layer by preparing literal SQL strings to be executed against a Jet back-end database. When I was a boy, etc. etc. After six years of experimentation, quick ten-liner experiments to prove a concept for a real day-job project, and random walk technique drift, I ended up with a selection of six different data access strategies, ranging all the way up to use of NHibernate on some pages. NHibernate 1.2. They're up to 2.0.1GA now! Fancy that, when I was a boy, etc. etc.

The security model was built around ASP.NET Membership, which lets you lock down folders or files by applying group membership requirements in your web.config files. My web.config files still bear the scars (in the form of oddly and very-precisely ordered elements) of an old ASP.NET security vulnerability - certain valid sets of permission demands would be misinterpreted by ASP.NET, meaning that unauthorised users would be granted access to content when they shouldn't. The workaround was to re-arrange your web.config so the dodgy ASP.NET code path would not be exercised.

It's nice how much of an app's history you can reconstruct just looking at the final shape of its source code. 

The UI design was originally retro (by which I mean ugly). The underlying HTML and CSS was originally very pragmatic and just a little verbose (by which I mean apallingly ugly). Things did not improve in either respect. I'm perversely proud that after six years and three substantial UI redesigns which all stuck to the black, white and gray theme, I've now managed to move to a new blog engine which itself is almost completely monochromatic. 

When I think back on this little chunk of source code, I think my dominant emotion will contine to be shame. Normally, I find something to be proud of in even the most pedestrian application I'm involved in building. That's how I choose to get through dark, pedestrian days - by finding a place to hide a nicely-fashioned little something for the next guy or girl to enjoy.

And yet there's nothing remarkable about this software except the way I managed not to do anything, anywhere in it that I am actually genuinely proud of. There's nothing to build an excited "new technique" blog post around. I didn't even open source it as a cautionary tale so people could learn by seeing the errors of my ways. I could at least have sent send in any of the several Daily WTF-worthy code snippets that resulted from the left and right halves of a single line of code being written in snatched moments on either side of a batch of F1 GP pitstops.

You may have noticed that I've been writing this entry in the tone of a column in the Daily Mail by a Top Gear compere wherein he craps on in a negative way about some car for six paragraphs and then summarises with the line "but actually, i quite like it". Well, sure, I have been writing in that style, but it was all masterful misdirection because I'm feeling dark today.

Goodbye, un-named blog engine #9492341. You didn't deserve to exist for as long as you did, and it's all my fault.

I'm implementing a 6502 emulator in C# as part of a project to emulate the Nintendo Entertainment System.

I've implemented all the 6502 opcodes, and now I'm going back through and implementing cycle counting, so that each instruction consumes the correct number of clock cycles.

On the 6502, some instructions take one extra cycle if a "page boundary is crossed". Different references use slightly different wording, but none which I've found so far really clearly specify exactly what constitutes a page boundary crossing. Take Post-indexed indirect addressing:

The CPU takes the byte following the opcode. The CPU goes to the specified zeropage address and reads a word. It adds the value of the Y register to that. It reads a byte from the location specified by the resulting 16 bit value.

1. There could be a page boundary between the opcode and the operand.
2. I could cross a page boundary going to the zero page from the operand (or the opcode, or the start of the next instruction).
3. The word I read there could span a page boundary by starting at address 0x**FF.
4. Adding the Y register to that word could cause a page boundary crossing.

Of those, only 3 and 4 seem likely candidates for the "page boundary crossing" we're interested in.

Reading 64doc,  the section "Absolute indexed addressing" describes what the CPU's up to at each cycle during one of the absolute indexed addressing read instructions. It implies that the "type 4" page boundary crossing in the list above is the type that causes the CPU to take an extra cycle for the instruction: if adding an offset to an address causes a page boundary crossing, then the instruction takes 1 extra cycle.

This model has good explanatory power too. The LDA (load accumulator) instruction has 6 opcodes, each for a different addressing mode. Their cycle timings are 3, 4, 6, 5+,4, 4+ (where '+' means an optional extra cycle if a page boundary is crossed). The linked document above says that the 6502 optimistically fetches from the calculated address assuming there was no page boundary crossing, and then spends the extra cycle, if necessary, re-fetching from the correct page if it guessed wrong. Look now at the STA (store accumulator) instruction. Its cycle timings for the same addressing modes are 3,4,6,6,4,5. Notice that the timings are identical, except that we always get charged the extra cycle. Because STA writes to memory, the CPU can't optimistically write to its best guess of the desired address, and then write again to the correct address if it turns out it guessed wrong; very sensibly, the store instructions always wait for the address to be calculated with certainty before going ahead and writing to the bus.

It's easier to optimise a correct program than it is to correct an optimised program.