Tag Archives: Debug Engine

VSTorqueScript #6

I've spent the last two days working on the Debug Engine and I've made some progress:

  • Tracepoint Support
  • Breakpoint Support
  • Support for the Watch window (I might have enlisted this already once but now support is more complete)

I'm not going to post any screenshots this time, instead I'll try to create an alpha package for you to test. This will probably take some time, too, because I want to make sure that you can install and uninstall it cleanly. I've been using the experimental hive so far, so I have to make sure it doesn't corrupt anyone's installation accidentially... :-)

A few implementation notes for interested people as always:

  • If you ever have to marshal an object to an IntPtr, be very careful, because doing the wrong thing will certainly crash your application - I had that happen to me and it took me quite some time to figure out why Visual Studio was suddenly  'experiencing internal problems' and had to be closed' all the time:
    When you have to convert an IntPtr to a wrapped object, use

    MyObject = (MyType) GetObjectFromIUnknown( intPtr ); // or As MyType;
    

    As you see, the intPtr is a COM object and every COM object implements IUnknown, so the call is pretty safe. The conversion afterwards is already 'managed', so safe, too.
    If you want to convert an IntPtr from an object, you have to be more careful though. The Marshal namespace offers at least two methods for it:

    • public static IntPtr GetComInterfaceForObject(
      	Object o,
      	Type T
      )
      
    • public static IntPtr GetIUnknownForObject(
      	Object o
      )
      

    Don't use the latter for anything except if you really need an IUnknown interface IntPtr pointer!
    The method really only returns a pointer to the IUnknown interface of the object, which usually isn't what the unmanaged code expects - which probably will result in misinterpreting some uninitialized memory as a vtable that is not there.
    You have to use GetComInterfaceForObject and don't forget this. Because you're only working with IntPtrs in the end, there is nothing to warn you about your misstep.

  • If you want to use an object with System.Collection.Generic.Dictionary<K,V>, it's not sufficient to implement IEquatable<K> as stated in the documentation - you also have to override GetHashCode (and override the normal Equals, too, then while you're at it), because Dictionary uses GetHashCode for the bucket assignments and you can implement IEquatable as hard as you want, GetHashCode might simply assign two 'equal' objects to different buffers and you have a problem..
  • One of the benefits of COM is that no one cares about your implementation.. take advantage of it!
    I had to remember this the hard way, when I implemented my breakpoint code. At first I wrote a PendingBreakpoint class, which implemented IDebugPendingBreakpoint2, and then a BoundBreakpoint one, which implemented IDebugBoundBreakpoint2. It was unnecessary messy and today I sighed and rewrote the breakpoint code to use just one Breakpoint class that implements both interfaces and it's a lot more elegant now.
  • Funny fact: An enum that is even mentioned in MSDN and the SDK docs is missing from the Interops library:
    public enum enum_BP_UNBOUND_REASON
    {
        BPUR_UNKNOWN = 0x0000,
        BPUR_CODE_UNLOADED = 0x0002,
        BPUR_BREAKPOINT_REBIND = 0x0003,
        BPUR_BREAKPOINT_ERROR = 0x0004
    };
    
  • The initialization process for the DE is as follows:
    1. DebugEngine and Program Creation Events
    2. Load Complete Event
    3. Call CreatePendingBreakpoint for all known breakpoints
    4. First Continue Call
    5. Entry Point Event
    6. Either the second Continue call or the SDM enters break mode

I had to write a few wrapper classes, too, which I might post about next time around. It's enough for today though.

Cheers,
Black

VSTorqueScript #5

It took me quite long to make progess in the last days. Partially this is due me being in Munich again and working in my uni job part-time again - and I'm stunned each time how tired I am when I come home.

It also is due to me experimenting with my TelnetDebuggerClient implementation that connects my debug engine to Torque's Telnet Debugger. I've tried quite a few different approaches for synchronous and asynchronous communication before I could settle for the simplest and the more advanced ones reside in the repository's history. I'll probably have to switch to them sooner or later because I only implement asynchronous communication at the moment which is fine for most things except when you need more advanced debugging info like for displaying objects properly in the debug window for example.

I've also had to read the "Definitive ANTLR Reference" in one night because I had to return it to uni's library the next day, so I was busy for 7 hours reading 350 pages of a parser reference. It was interesting though and tonight I'll try to write a QuakeC grammar - that's the plan at least.

However yesterday, I had time to work on the debug engine some more and here's a picture that tells it all:

VSTorqueScript now supports displaying the callstack, too

VSTorqueScript now supports displaying the callstack, too

As you see you can now break a program and it will retrieve the callstack from Torque and update the callstack window automatically and open the right file and jump to the correct file position, too - or rather as I've just noticed one-line off because VS line indicies have base 0 and TorqueScript uses base 1 >_>...

A few implementation notes as always:

  • I t seems, if your Debug Engine doesn't implement IDebugEngineLaunch2, your Program Provider has to implement IDebugProgramProvider2.WatchForProviderEvents
  • Moreover, if you don't implement IDebugEngineLaunch2, which in turn forces you to implement IDebugProcess2 (and maybe IDebugProcess3, too), suddenly some "deprecated" methods in your Program class (your IDebugProgram2 implementation) will be called, which, if you follow MSDN and the debug engine sampe, will result in failed asserts and the nice fail message "This function is not called by the debugger." :-)
    It's kind of logical that IDebugProgram2.Execute() is going to be called because the debugger has no IDebugProcess2 interface to call Execute on..
  • What is enum_FRAMEINFO_FLAGS_VALUES? It's undocumented and I suspect it's used for the undocumented m_dwFlags field in FRAMEINFO...
  • I'm totally stunned by how stable Visual Studio is. Most the interfaces I implement are only half implemented and return E_NOTIMPL on most calls and Visual Studio runs and runs well and only displays the information it can in a clean way.
    I guess this is one of the advantages of the COM programming/design paradigma and I think it's great. Being able to implement interfaces bit by bit and add functionality slowly makes debugging easier and also helps you understand things better - which is good because even though a lot is documented, you still need some practice and trial & error to get things right eventually.

Enough is enough though for now, and I'll head to uni in a bit.

Only one last remark: I've started reading the book "The Pragmatic Programmer" and so far it's really entertaining and, I guess, good.
I'll write more on it, when I've finished it and can actually assess its qualities..

Cheers,
Andreas

VSTorqueScript #4

My little pet project is actually called VSTorqueScript, so I've decided to use the correct title for my posts, too.

Today has been a somewhat productive day, although not as productive as I've hoped, because I was stuck with a very stupid issue for 3 or 4 hours. I'm not proud of it, but it made me go to the VSX forum and become active there.

I've worked more on the debug engine afterwards (well, actually I went jogging and swimming - in that order.. think about it ) and made some progress. Quite a bit of the stub stuff is implemented now and I think I'll start implementing the real thing tomorrow.

No screenshots this time, I'm sorry, but for the SDK users, I have a little helper file. If you look at the managed debug example, you'll find an AD7Events file that wraps the debug event interfaces.
I wasn't easily able to port it over, so I decided to write my own version of it (while keeping the comments).

I've appended it to the post, if you want to take a look.

On other news I'll probably switch this blog over to my own webspace soon to get some advantages (like expandable sourcecode amongst things), but I think I'll stick to wordpress.com for a few more weeks.
It's really a pity that you can't redirect from wordpress to your domain, but only vice-versa :(

Anyway, off to work some more on my code.

Cheers,
Andreas

Read more »

Visual TorqueScript #3.5

Small update: After hours and hours of error and trial (and implementing a few interfaces by using Copy&Paste wisely), I've finally managed to get Visual Studio to load my debug engine.

This looks gorgeous, doesn't it?

A few notes for those interested in writing Debug Engines:

  • See my previous post for the ProvideDebugEngine helper attribute
  • There exists a really nice managed debug engine example, that isn't part of the SDK.
    You can find it here.
  • You are required to set a PortSupplier, even if the documentation says otherwise (this was a major source of frustration in the last hour).
  • Since ProvideDebugEngine requires a Type for PortSupplier, I've been using this stub class:
        /// <summary>
        /// The class is just a stub for use with ProvideDebugEngine.
        /// It seems it's mandatory to provide a PortSupplier GUID with a debug engine.
        /// </summary>
        [Guid("708C1ECA-FF48-11D2-904F-00C04FA302A1")]
        internal sealed class DefaultPortSupplier
        {
        }
    
  • I've also added another attribute (ProvideClass) for convenience to register a class directly (it's the same as using ProvideObject on the current class type).
    Since the code is quite short, I'm going to supply it directly:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    
    namespace Microsoft.VisualStudio.Shell
    {
        /// <summary>
        /// This is just a wrapper for ProvideObject to provide the class the attribute is used on
        /// </summary>
        [AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
        class ProvideClass : RegistrationAttribute
        {
            public override void Register(RegistrationAttribute.RegistrationContext context)
            {
                ProvideObjectAttribute objectProvider = new ProvideObjectAttribute(context.ComponentType);
                objectProvider.Register(context);
            }
    
            public override void Unregister(RegistrationAttribute.RegistrationContext context)
            {
                ProvideObjectAttribute objectProvider = new ProvideObjectAttribute(context.ComponentType);
                objectProvider.Unregister(context);
            }
        }
    }
    

And yes, I love the tag :-)
Cheers,
Andreas

Visual TorqueScript #3

I've spent a part of the last two days cursing the managed project framework (it's a bliss, too, and I'm thankful that it's available at the same time), because it is too different from what I need, and I'll probably end up rewriting most of it to use different approaches and not use MSBuild anymore.

I've spent more time, however, on reading up on Debug Engines in Visual Studio 2008 and I've started work on my own yesterday. To be honest, I haven't really started to work on it, I've rather spent the evening and most of today figuring out how to register a debug engine at all and write attributes for that, so that it becomes an automated process, just like it's done for Packages and Language Services by using attributes like ProvidePackage and ProvideLanguageService.

The code has turned out pretty nicely and now you can use the attributes like this, for example, to register a debg engine:

    [ProvideDebugEngine("{5887F58F-3B7F-401d-95CE-A7BADA1E4D7D}", "TorqueScript Debug Engine",
        Attach = true, ProgramProvider = typeof(VSTorqueScript_DebugEngine))]
    [ProvideIncompatibleEngineInfo( "{5887F58F-3B7F-401d-95CE-A7BADA1E4D7A}" )]
    [ProvideIncompatibleEngineInfo( "{5887F58F-3B7F-401d-95CE-A7BADA1E4D7F}" )]
    [ProvideAutoSelectIncompatibleEngineInfo( "{5887F58F-3B7F-401d-95CE-A7BADA1E4D7F}" )]
    [ProvideObject(VSTorqueScript_DebugEngine)
    [Guid(GuidList.guidVSTorqueScript_DebugEngineString)]
    public sealed class VSTorqueScript_DebugEngine
...

In my opinion this looks pretty neat - and it works.. \o/
I really love the whole concept of attributes and the way they make coding a lot easier (even though it's hard to find information about useful attributes, as you only find out they exist when you stumble over them or write them on your own).

You can find the code here: http://www.icculus.org/~black/weblog/VS_DebugEngine/

A few notes:

  • The implementation was done making heavy use of copy&paste and quick replace (I copied the table containing the help description of the metrics into the source file and reworked it using RE magic to look the way it does now)
  • At first I used Type.GetProperties to enumerate all properties and write them out into the registry but that bloated the resulting registry entry, of course. Because of this I later switched to the HashTable approach that I first found in SDK's implementation of ProvideLanguageServiceAttribute.
  • The restriction that all attribute parameters must be constant expressions forced me to make a few changes - at first I had Guid properties and for the incompatible engines list I expected a Guid[], which didn't work. I changed the Guids to Strings and added the Provide*EngineInfo attributes instead of the incompatible engines list (same for the autoselect one, of course).
  • The Provide*EngineInfo attributes contained their registry code at first. This was ugly because they kind of required a ProvideDebugEngine attribute to exist, too, which I couldn't enforce. Later I changed them to be pure value holders and moved all registry code into ProvideDebugEngine.
    Now nothing happens if you don't supply a ProvideDebugEngine attribute, too.

Now I'll try to start to work on the actual Debug Engine for TorqueScript.

Cheers,
Andreas