Improvements in the CLR Core in .NET Framework 4.5

Alongside all the exciting advents in Windows 8 and Metro apps, the .NET CLR is marching on. The next version of the CLR will feature several “internals” improvements, mostly in the performance area. Read on to learn about changes to the garbage collector, the JIT, and the native image generator engine in the next CLR.

Background mode for Server GC
Background GC is a neat feature introduced in CLR 4.0 to the Workstation GC flavor. It’s a little hard to explain without any background (pun intended), but the general gist is the following: while performing a full gen2 collection, the GC checkpoints at well-defined locations to see if a gen0/gen1 GC has been requested*. If a lower generation GC has been requested, the gen2 collection is paused, and the low-generation collection runs to completion before the gen2 collection resumes. This allows the code that caused the low-generation collection to resume its execution.

The new feature in the next CLR is that background GC is now supported on Server GC. (Recall that under Server GC, there is a GC heap for each processor and a separate GC thread for each processor.) There are additional changes in the GC, including a work-stealing mode in Server GC that allows GC threads to steal parts of the graph for marking if the workload is not distributed evenly.

Managed Profile Guided Optimization
If you have done any C++ work since Visual Studio 2003, you probably know about Profile Guided Optimization (PGO). In the C++ compiler, PGO is a special multi-step optimization mode, which relies on exercising the application through a set of scenarios with data collection enabled, and then recompiling the application with this data to optimize based on runtime information.

The same idea is now applied to managed code that has been NGEN-ed. A special command-line tool called mpgo.exe (which I can see integrated into Visual Studio like the C++ counterpart) instruments your .exe and embeds into it some runtime data that is subsequently used by ngen.exe to generate an optimized image.

You might wonder what kind of optimizations are enabled by MPGO. The only two that were disclosed at the time have to do with reducing the number of pages loaded from disk by organizing hot code together on the minimal number of pages, and with reducing the number of copy-on-write pages by organizing together on the minimal number of pages any data that will likely be overwritten at runtime.

Automatic NGEN
Integrating NGEN in your application’s installer might be a challenge; auto-NGEN addresses this challenge by performing NGEN automatically if runtime-generated profiling information deems fit. When your .NET 4.5 application runs, the runtime generates assembly usage logs which are examined by an automatic maintenance task that runs in the background when the system is idle, and generates native images for frequently used assemblies. (This maintenance task might also choose to evict native images for assemblies that have not been used for a long time.)

Multi-core background JIT
Developers can now opt-in to an optimized JIT that uses otherwise unused processing time to perform multi-threaded JIT compilation of frequently used methods speculatively. The first time the application is launched, data is collected on frequently used methods, and on subsequent launches this data is used to guide the background JIT engine.

This feature requires adding a few lines of code to your application – calls to the ProfileOptimization.SetProfileRoot() and ProfileOptimization.StartProfile(filename) methods. However, this feature is enabled by default in Silverlight 5 and ASP.NET applications.

Re-JIT
This last feature allows recompiling a method (running the JIT again) at runtime. This is a feature most likely suitable for profilers that replace the IL code with an instrumented version at runtime to collect information without restarting the process. It is exposed through the ICorProfileInfo4::RequestReJIT and ICorProfilerCallback4::GetReJITParameters methods.

* How come a low-generation GC was requested while a full collection was running? Recall that Workstation GC can be non-blocking (concurrent), enabling application threads to run during significant parts of the collection itself, including most of the GC mark phase.

Metro .NET Framework Profile (“Windows Tailored”)

The amount of confusion generated by the first two keynotes at BUILD was immense. The blogosphere and Twitter were brimming with bold rumors of the “.NET is dead” kin. I even heard someone discuss seriously the possibility that C# Metro apps will be compiled directly to native WinRT bindings, bypassing IL and JIT altogether, so that clr.dll won’t even be loaded.

Even though I disproved these rumors two days ago, it was still great to hear Krzysztof Cwalina explain in detail how .NET remains a fully-featured framework for developing Metro applications in C# and VB.NET. As you may have heard or noticed, not every .NET API is available to Metro applications – and the subset of APIs available to Metro applications is called the “Windows Tailored” or Metro profile.

Even before Metro, .NET had numerous target profiles – the Silverlight profile, the Windows Phone 7 profile, the .NET Client profile, and of course the full profile. The way profiles are implemented is a compiler trick. Visual Studio ships with a bunch of reference assemblies that you can find (on a default install) under C:\Program Files\Reference Assemblies\Microsoft\Framework. These assemblies contain only metadata and no executable code – indeed, they are not necessary at runtime because when your application runs, .NET assemblies are loaded from the GAC. To project only a part of the available APIs from the implementation assembly, the reference assembly simply omits whatever namespace, types, or methods that are not included in the profile.

Clearly, there are ways to circumvent this mechanism, such as binding directly to the GAC assemblies instead of using the Visual Studio-provided references. Another option is to use reflection at runtime. These are valid possibilities – with the minor setback that your application will not pass Windows Store certification if you use unsupported APIs. (This idea, of course, is not new, and has been exercised by Apple in iOS apps for quite some time.)

The same idea exactly is used for the Metro profile. It executes at runtime using the same assemblies as the full desktop CLR, but only a portion of it is available to Metro applications. What kinds of APIs were removed?

1. Irrelevant APIs were removed – think ASP.NET, WPF, Console access.
2. Obsolete and dangerous APIs were removed.
3. Duplicate APIs were removed.
4. Windows API wrappers were removed, in favor of the WinRT types.
5. Badly designed APIs were removed.

Specifically, the following APIs that are not part of the Metro profile you are most likely to miss:

1. System.Data – removed entirely. There is an OData component, but no LINQ to SQL or anything of that sort.
2. Remoting – removed entirely.
3. AppDomains – removed entirely. There is only one AppDomain in each Metro application process.
4. Reflection.Emit – removed entirely.
5. Private reflection – you cannot invoke private methods, internal methods on another assembly, access private members and so on. (Frankly, I am not so upset about this change because I’ve grown sick of “clever” solutions using some undocumented private member which breaks in a year or two.)
6. System.Thread – must use Task.Run instead for all threading purposes. (Creating a new thread is roughly equivalent to creating a task with  TaskCreationOptions.LongRunning.)

What kinds of APIs are included in the profile, or will require porting to use the profile?

1. HTTP client stack is included.
2. XML (through XML LINQ mostly) is included.
3. Serialization is included (DCS).
4. MEF is included.
5. WCF client stack is included.

The end result is a very slim subset of the .NET framework, with only 15 implementation assemblies and ~1000 types spanning 60 namespaces. (This does not include the WinRT APIs and the Windows.UI.Xaml stack.)

Obviously, one of the reasons the resulting framework is so slim is that so much is provided by WinRT. Life has not just become easier – quite the opposite, because you have to go and learn WinRT. However, the interop story with WinRT is incredible – or rather, completely transparent – due to the brilliant type mappings from the WinRT type system to managed languages. (Quite possibly the most incredible automatic language projection is the way WinRT operations are projected to .NET Tasks and JavaScript promises.)

Under the Covers of WinRT Using C++

The WinRT type system relies strongly on WinRT components, which are COM objects implementing a specific set of interfaces and adhering to a certain ABI (Application Binary Interface). We will examine here this ABI and how C++ compiler extensions help reference that ABI without exposing the nitty-gritty details of dealing with COM interfaces and COM activation.

A WinRT component implements the IInspectable interface, which derives from IUnknown (however, WinRT components do not have dual interfaces, i.e. they do not implement IDispatch). The IInspectable interface contains three methods: GetIids, which returns the interfaces implemented by the component; GetRuntimeClassName, which returns the fully-qualified name of the component; and GetTrustLevel, which returns the component’s trust level that can be used to control component activation.

IInspectable is used by the dynamic language bindings from JavaScript. Given a reference to a WinRT component, the JavaScript interpreter can call the IInspectable::GetRuntimeClassName method and obtain the fully-qualified class name of the WinRT component. Using the namespace name the interpreter can ask Windows to load the metadata file (.winmd) describing the component, and from the metadata determine how to use the interfaces implemented by the component.

When WinRT components are accessed from C++ or from .NET languages, there is no need for the IInspectable interface – IUnknown::QueryInterface suffices for all intents and purposes. In fact, .NET “interop” with WinRT relies on simple COM interop – an RCW created by the CLR manages the lifetime of the underlying WinRT component. For C++ language bindings, the story is somewhat more complicated, and the language extensions come into play.

The C++ language extensions (enabled by the /ZW switch) map WinRT to standard C++ patterns like constructors, destructors, class methods, and exceptions. For example, the language extensions hide the COM-style activation (RoActivateInstance) behind a constructor call (albeit with the ref new keyword), and hide reference counting (IUnknown::AddRef and Release) by automatically managing references. The syntax of these extensions is very similar to C++/CLI – if you know the latter, you will certainly feel at home with the former. Some highlights:

public ref class SomeClass sealed – defines a COM object implementing IInspectable (and IUnknown). You can implement interfaces by deriving from an “interface class”.

property SomeType Prop { SomeType get(); void set(SomeType); } – defines a property, very similar to a property in C++/CLI.

delegate SomeType Foo(SomeOtherType …) – defines a delegate, which is a glorified function pointer, that can subsequently be used to define events.

…and the list goes on and on – on one hand, the syntax is very similar to C++/CLI, and is indeed inspired by .NET in every detail, and on the other hand, the result is a native C++ class that implements pure COM interfaces through a lot of language magic.

The C++ compiler generates inline functions on the consumer side and the component side, as necessary: on the consumer side, COM HRESULTs are converted to exceptions and COM “retval” arguments are converted to return values; on the component side, WinRT and C++ exceptions are converted to HRESULTs and return values are converted to “retval” arguments.

//Consumer-side stub pseudo-code
inline int Computer::Compute(int first, int second) {
    int result;
    HRESULT hr = ___impl_Compute(this, first, second, &result);
    if (hr != 0) ___impl_throw_for_hr(hr);
    return result;
}
//Component-side stub pseudo-code
HRESULT __stdcall ___impl_Compute(Computer* cmp,
  int first, int second, int* result) {
    try { *result = cmp->Compute(first, second); }
    catch(Platform::Exception^ e) { return e->HResult; }
    return S_OK;
}

Note that the only HRESULT, which does not represent an exceptional condition, is S_OK (numeric value 0). No more positive HRESULTs (such as S_FALSE) that need to be checked explicitly by the caller. By the way, the inline wrappers can be called directly, bypassing the C++ language extensions.

An even more hardcore alternative is to reach directly into the vtable behind the WinRT object reference. Indeed, the ABI dictates that WinRT object references are simply pointers to a bunch of vtables, so all that remains is choosing the right vtable and indexing within it the desired method:

//Assuming 32-bit pointers
Computer^ computer = ref new Computer;
int* vtable_array = (int*)computer;
int* icomputer_vtable = (int*)vtable_array[0];
int* compute_will_be_fptr = (int*)icomputer_vtable[6];
typedef HRESULT (__stdcall *compute_fptr_t)(Computer*,
  int, int, int*);
compute_fptr_t compute_fptr = (compute_fptr_t)compute_will_be_fptr;
//…use compute_fptr freely :-)

Another way of putting it is: “a hat is a pointer to a pointer to an array of function pointers”. You can inspect the precise layout of the WinRT component’s C++ class in memory using the /d1reportSingleClassLayout<CLASSNAME> undocumented compiler switch. For example, for the default component created by the Visual Studio C++ WinRT Component project template, this switch reports the following layout and vtable structure:

//The WinRT class:
public ref class WinRTComponent sealed {
    int _propertyAValue;
public:
    WinRTComponent();
    ~WinRTComponent();
    property int PropertyA {
        int get() { return _propertyValue; }
        void set(int propertyAValue) { _propertyAValue = propertyAValue; }
    }
    int Method(int i);
    event SomeEventHandler^ someEvent;
};

//The output of the /d1reportSingleClassLayoutWinRTComponent switch:
class WinRTComponent    size(40):
    +---
    | +--- (base class __IWinRTComponentPublicNonVirtuals)
    | | +--- (base class Object)
0   | | | {vfptr}
    | | +---
    | +---
    | +--- (base class __IWinRTComponentProtectedNonVirtuals)
    | | +--- (base class Object)
4   | | | {vfptr}
    | | +---
    | +---
    | +--- (base class IDisposable)
    | | +--- (base class Object)
8   | | | {vfptr}
    | | +---
    | +---
    | +--- (base class Object)
12  | | {vfptr}
    | +---
16  | _propertyAValue
20  | EventSource <backing_store>someEvent
32  | __cli_MultiThreadedRefCount __cli_refcount
36  | __cli_disposed
    | <alignment member> (size=3)
    +---

WinRTComponent::$vftable@__IWinRTComponentProtectedNonVirtuals@:
     | -4
0    | &thunk: this-=4; goto WinRTComponent::__cli_QueryInterface
1    | &thunk: this-=4; goto WinRTComponent::__cli_AddRef
2    | &thunk: this-=4; goto WinRTComponent::__cli_Release
3    | &thunk: this-=4; goto WinRTComponent::__cli_GetIids
4    | &thunk: this-=4; goto WinRTComponent::__cli_GetRuntimeClassName
5    | &thunk: this-=4; goto WinRTComponent::__cli_GetTrustLevel

WinRTComponent::$vftable@__IWinRTComponentPublicNonVirtuals@:
     | &WinRTComponent_meta
     |  0
0    | &WinRTComponent::__cli_QueryInterface
1    | &WinRTComponent::__cli_AddRef
2    | &WinRTComponent::__cli_Release
3    | &WinRTComponent::__cli_GetIids
4    | &WinRTComponent::__cli_GetRuntimeClassName
5    | &WinRTComponent::__cli_GetTrustLevel
6    | &WinRTComponent::__cli_WinRTComponentDll1___IWinRTComponentPublicNonVirtuals____cli_get_PropertyA
7    | &WinRTComponent::__cli_WinRTComponentDll1___IWinRTComponentPublicNonVirtuals____cli_set_PropertyA
8    | &WinRTComponent::__cli_WinRTComponentDll1___IWinRTComponentPublicNonVirtuals____cli_Method
9    | &WinRTComponent::__cli_WinRTComponentDll1___IWinRTComponentPublicNonVirtuals____cli_add_someEvent
10    | &WinRTComponent::__cli_WinRTComponentDll1___IWinRTComponentPublicNonVirtuals____cli_remove_someEvent
11    | &WinRTComponent::Method

WinRTComponent::$vftable@IDisposable@:
     | -8
0    | &thunk: this-=8; goto WinRTComponent::__cli_QueryInterface
1    | &thunk: this-=8; goto WinRTComponent::__cli_AddRef
2    | &thunk: this-=8; goto WinRTComponent::__cli_Release
3    | &thunk: this-=8; goto WinRTComponent::__cli_GetIids
4    | &thunk: this-=8; goto WinRTComponent::__cli_GetRuntimeClassName
5    | &thunk: this-=8; goto WinRTComponent::__cli_GetTrustLevel
6    | &thunk: this-=8; goto WinRTComponent::__cli_Platform_IDisposable____cli_<Dispose>
7    | &thunk: this-=8; goto WinRTComponent::<Dispose>

WinRTComponent::$vftable@Object@:
     | -12
0    | &thunk: this-=12; goto WinRTComponent::__cli_QueryInterface
1    | &thunk: this-=12; goto WinRTComponent::__cli_AddRef
2    | &thunk: this-=12; goto WinRTComponent::__cli_Release
3    | &thunk: this-=12; goto WinRTComponent::__cli_GetIids
4    | &thunk: this-=12; goto WinRTComponent::__cli_GetRuntimeClassName
5    | &thunk: this-=12; goto WinRTComponent::__cli_GetTrustLevel

Finally, another option is to use RoActivateInstance and work directly with COM interfaces. This is a rather masochistic approach, but it seems like the only way to target WinRT components from a language that “only speaks COM”. The following C++ code is a simple example of that:

//Error-checking elided for clarity
LPCWSTR str = L”ComputingStuff.Computer”;
HSTRING hstr;
HRESULT hr = WindowsCreateString(str, wcslen(str), &hstr);
IInspectable* ptr;
hr = RoActivateInstance(hstr, &ptr);
WindowsDeleteString(hstr);
IComputer* comp;
hr = ptr->QueryInterface(IID_IComputer, (void**)&comp);
int result;
hr = comp->Compute(42, 42, &result);
comp->Release();
ptr->Release();

The C++ bindings are very high-performance when the source code of both sides is available. For example, a cross-interface cast (of a component that implements multiple interfaces) will be identical to a C++ dynamic_cast, requiring only a shift through adjustor thunks. However, if the component is behind the full ABI wall, a cross-interface cast will be compiled to a QueryInterface call, as is always the case with COM.

There are also quite a few interesting things to be said about the WinRT type system. Although not covered in the sessions I attended so far, the docs are pretty clear on this subject, and expose a large variety of types that map partially to existing C++ types and are somewhat similar to the .NET type system. For example, all WinRT components are derived from Platform::Object, and all WinRT exceptions are derived from Platform::Exception. Primitive types, however, can be boxed by casting them to Platform::Object^. Even primitive types, however, feature the ToString() method, which converts them to their string representation. I am not ready at this time to write more about these topics, but will be exploring them further in the future.

Windows To Go

After experiencing it first hand, I can only say that Windows To Go feels like magic. The potential uses are truly overwhelming. Read on to learn why.

Windows To Go allows you to capture a fully configured Windows environment on a USB stick and use it to boot on any host. This is not just booting Windows from USB – which was theoretically possible before – this is about specializing the image for each host you boot from it. It’s about taking your operating system on the go.

Some of the enterprise scenarios this enables are downright incredible. Think about a corporate employee being able to take home his work system and pick up exactly where he started. Imagine a police officer booting the police car PC off a personalized Windows USB stick. Consider a software consultant who doesn’t have to bring in his laptop to the client or to a classroom or to a shared machine.

Consider also the consumer scenarios. You could come into an Internet café with “your own Windows” and leave no trace of your activity on the shared host. You could come over to your parents over the weekend carrying only a USB stick.

Windows To Go works off standard USB sticks, and can support USB 2.0 and USB 3.0 drives. During first boot on any system, the image from the stick is specialized for the specific hardware – and then remembered for future use. (This specialization process took just under 2 minutes on my MacBook Air.)

Another awesome feature is what happens when the boot drive is surprise-removed, say yanked out when you accidentally misplace your laptop. When this happens, the kernel freezes the display and you have 60 seconds to plug the drive back in. After you do that, you can continue working normally. (I wish the screen could read something like “Boot drive disconnected. You should reconnect it immediately.” – but it is apparently very difficult to implement.)

As you may have guessed, everyone who attended Steve Silberberg’s session on Windows To Go got a Kingston USB 3.0 32GB drive with Windows 8 To Go:

Ten minutes later it was plugged into my MacBook Air and “specializing” for its hardware. Alex was so kind to video this for me:

Fifteen minutes later I was logged onto Windows 8 with my Live ID and enjoying all the settings (including my login tile picture!) I configured previously on my Samsung tablet.

This is one of these unique experiences with a computing technology that leaves you with your mouth gaped. This is magic.

SELA Blogs and Tweets @ BUILD

The group of 19 experts we have here at BUILD have had some things to say about the conference. You can follow us on the Twitter list or read our blogs. Here are some highlights from the last two days:

Yuval Mazor: It is time for BUILD!

Elad Shaham: My Impressions from BUILD Keynote

Sasha Goldshtein: Windows 8 Announcement Executive Summary

Yuval Mazor: Impressions of the First Day of BUILD 2011

Sasha Goldshtein: Some Thoughts on Windows 8

Ran Wahle: Windows 8 For Developers - What's New?

Shai Raiten: Windows 8 Developer Preview Is Now Available For Download

Shai Raiten: Visual Studio 11 and TFS 11 Available For MSDN Subscribers

Shai Raiten: Team Foundation Server on Windows Azure: A Preview Is Available!

Sasha Goldshtein: Sessions at BUILD Day 2 and Windows 8

Sasha Goldshtein: WinRT and .NET in Windows 8

WinRT and .NET in Windows 8

The Web today is full of rumors about the demise of Silverlight, .NET, Win32, and nearly anything else that doesn’t align immediately with Metro-style apps. Indeed, it seems sometimes that we are so eager to focus on “what’s dead” that we forget to look at the new announcements and try to figure out “what’s alive”.

From a brief analysis of the Windows 8 Developer Preview, Visual Studio 11 Developer Preview, and whatever bits of information delivered at the conference sessions, I think I have a pretty decent mental picture of what’s going on.

First of all, a managed Metro application (e.g. written in C#) will still load the CLR. I confirmed this by launching a Metro application and inspecting it with WinDbg – I was even able to load SOS and run basic commands. The CLR version bundled with the preview is 4.0.30319.17020, and that’s the version that gets loaded inside a Metro app. (It’s curious to note that even though the Visual Studio setting dictates “Any CPU” and the Windows version is 64-bit, the actual process is 32-bit.)

Symmetrically, a “fully” .NET application (e.g. a WinForms app) can reference the WinRT metadata assemblies and use WinRT APIs. This will be a necessity in some cases, for example to tap into the WinRT sensor APIs.

Next, a C++ Metro application will still load Win32 DLLs such as kernel32 and ntdll. Moreover, the WinRT APIs call into the Win32 DLLs – so they are not a replacement but rather a wrapper, an API flavor, on top of Win32. (Historical note: Windows used to have a feature called “environment subsystems”, which can be roughly described as API flavors. WinRT is not an environment subsystem – it is a library on top of the Win32 environment subsystem.)

What about the limitations, then, that a Metro application has in terms of API surface? Indeed, a Metro application written in C++ will be able to access only a part of the Win32 API surface, and this is accomplished through a bunch of preprocessor definitions. For example, the MessageBox function is not available for Metro-style apps, and its declaration is protected in the header file as follows:

#pragma region Desktop Family
#if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)
WINUSERAPI int WINAPI MessageBoxA(…);
…

For managed applications, the Metro projects in Visual Studio are targeting a special subset of the .NET framework, and will not have access to all the classes – not even all the classes of the .NET Client Profile. For example, the System.IO.FileStream class is not available.

Although these limitations may seem artificial, annoying, and counterproductive, there is probably a good reason for having them. The Metro UI and Metro apps are a fresh start for Windows applications and their developers. There is no trivial way to port an existing UI app to Metro without a considerable UI redesign. It’s also a great opportunity to enforce additional limitations that make sure Metro apps respect the user’s privacy, conserve battery power, integrate well with the UI guidelines, and live in harmony within the application sandbox.

To summarize, WinRT does not replace .NET, nor does it compete with .NET – and the same applies to Win32. You can write server and client applications using .NET, including WPF and Silverlight. You can write server and client applications using C++, including Win32 and MFC. However, if you choose to believe in the Metro way, you can write Metro applications for Windows 8 using .NET languages or using C++ or using JavaScript, with a consistent set of APIs exposed through WinRT.

Sessions at BUILD Day 2 and Windows 8

The second day of the BUILD conference was dedicated mostly to breakout-style sessions on the various topics covered only in brief during the keynotes. Some of the new stuff I’ve seen today in pseudo-random order:

Visual Studio 11

• Productivity Power Tools (with over a million downloads)! are going to be built-in in Visual Studio 11.
• You can right-click a piece of code in Visual Studio and select “Find Matching Clones” for a similarity analyzer that detects copy-paste across the board.

• Visual Studio will feature a revamped image editor and a viewer for 3D models.
• TFS will be available through Windows Azure as a SaaS offering. This includes a nice web front-end with a Scrum board (including post-in notes :-)).

Metro Apps

Metro applications adhere to an application lifecycle model that is very reminiscent of Windows Phone (and other mobile operating systems). Namely, when a Metro app is launched, the previous application goes off the screen and becomes suspended. Prior to suspension, the app gets 5 seconds to save transient state. Suspended apps are not removed from memory, but all their threads are suspended.

After an application has been suspended, the user can return to it, which will resume it from the suspended state. However, it’s also possible that the system will terminate the app because of a low memory condition. In this case, when the application is relaunched, it has to restore its state from persistent storage and appear to the user as if it has never been gone.

In addition to the simple activation model whereas the user launches the app from a start screen tile, applications can be launched through contracts – an app-to-app communication mechanism. For example, if an app can send information to your Facebook friends, and you use the “Share” charm within any other app, Windows will present to the user your application and it might be invoked with the user-provided data. Your app will have to check its activation context and retrieve from it the custom data to share.

Even though most applications will be suspended when the user is not interacting with them, it is still possible to engage in certain background activities without forgoing the Metro-style UI. For example, your app can play background audio while in the background. Another option is to be a “lock-screen app” which can periodically run code in the background and maintain persistent TCP connections (this would obviously be required of an email or instant messaging app).

WinRT

The goal of WinRT is to provide a native foundation for Metro-style applications. It is not a replacement for all Windows programming frameworks, and it is not a replacement for .NET – it is only intended as a framework for Metro-style applications to interact with.

WinRT is a native framework implemented as a bunch of C++ classes. To bind these classes to other languages (C++, JavaScript, C# and VB.NET) it takes metadata – .winmd files – which describe the contents of WinRT modules. These .winmd files have the same format as standard CLR assemblies. From .winmd files, language projections are automatically generated for the supported languages, and these projections are customized for each language. (For example, asynchronous operations will be exposed as promises in JavaScript and as Tasks in .NET languages.)

Possibly the biggest change is in the way WinRT components are consumed and implemented in C++. Considering that WinRT components are glorified COM objects, one might expect the horrors of native C++ consuming COM, what with smart interface pointers, managing lifetime through scopes, and so on. Fortunately, the C++ compiler supports a new switch, /ZW, which compiles new syntactic extensions to code that manages WinRT objects. For example, a ^ is a reference to a reference-counted WinRT object, so whenever you manipulate a Platform::String^, for example, the compiler inserts the necessary IUnknown::AddRef and Release calls. (This is akin to the way Objective C on Mac OS X 10.5 implements automatic garbage collection.)

Some Thoughts on Windows 8

Following a day full of announcements, I’ve had some thoughts brewing inside for a while about Windows 8 and its application model. While the things we’ve seen today are immersive, cool, and even downright amazing considering the timeframe, I’d keep in the excitement until I’ve had a chance to play with the platform a little bit.

The Windows Store model announced today pretty much seals the fate of cowboy free-style application development we were used to on Windows. No longer a bunch of APIs and some docs and go ahead to build your app with no restrictions: all Metro-style applications (read: all Windows 8 apps with a serious future) will be restricted in what they can access, will be sandboxed and isolated from one another, will use only documented and isolated APIs dictated by the application model, will be subject to stringent UI guidelines, will go through a rigorous submission and validation process, and will be signed to verify that they are coming from a developer you trust.

This, all of this, is generally a good thing. Especially for the users, but also for developers who don’t have to struggle for control over the system or bother to learn APIs they shouldn’t care about. Still, it’s somewhat sad, too – this day marks the end of an era. In 10-15 years, developers will look up to us and admire the software giants for whom the term “operating system” consisted of not much more than an abstraction of hardware.

Packaged applications with dependencies, abandoning installers and custom registry manipulations, application manifests, capabilities, pre-declared features, user approval, respecting the user’s privacy – such obvious choices that could have been made decades ago and the boldness of making such a move on the most popular consumer operating system is incredible.

The other thing I kept thinking about during the day is whether the Metro UI is a UI paradigm that I like, and if it fits the desktop the same way it fits mobile phones. I don’t have an answer yet – and 30 minutes of experience dragging tiles around on the developer device doesn’t change that – but it does seem well-thought-of, integrated, suitable for fat fingers, vivid, fluid, active, and most of all – modern. I’m going to attend a couple of sessions on the Metro design guidelines, touch conventions, and common gestures; from what I saw on the keynote stage, it’s going to be a great ride.

Lastly, the controversial topic of application “backgrounding” (suspension). When a Metro app leaves the screen, the operating system gives it a chance to save state and then suspends it. This is similar to many mobile operating systems, including Windows Phone and iOS (and to some extent, Android as well).

There is an obvious reason why this approach is now extended to the desktop as well – with ultra-portable notebooks and tablets less than 1cm thick and lighter than 1kg it’s probably next to impossible to get more than a couple of hours of battery life out of the device without some serious compromises. Suspended applications is one such compromise.

All in all, Metro-style apps seem like a great opportunity to start from scratch. To design your applications afresh with new, modern user interface guidelines. To use clean new APIs that span all major programming languages. To believe in the Microsoft dream of reimagining Windows.

Is Windows 8 better than Mac OSX Lion, iPad 2, Android Honeycomb, or anything else? It’s really too early to tell. What I can tell is that the atmosphere, the motivation, the buzz around the new platform are nothing like what I’ve seen around a Microsoft technology during the last decade. And that means something – something about Microsoft’s commitment and something about the excitement of us developers.

Does Windows 8 change everything? I don’t know yet. But I am certainly going to find out.

Windows 8 Announcement Executive Summary

Well, the first day of the BUILD conference certainly started with a blast. I just came back to the hotel after sitting through a full day of keynote presentations introducing Windows 8 – the operating system, the development platform, and the user experience.

The purpose of this post is to summarize briefly the Windows 8 announcements and the way they reflect on existing and future development. This is just an executive summary – if you are looking for the nitty-gritty details, follow this space (and other spaces :-)) for future posts.

Windows 8 and Metro Apps

Windows 8 builds upon the success of Windows 7, and has complete application compatibility with Windows 7. Absolutely all applications and hardware designed for Windows 7 should work with Windows 8.

WinRT – the Windows 8 development framework – is an unmanaged library that wraps some (not all) Windows native APIs. WinRT has natural bindings to the major programming languages, including C++, C#/VB.NET, and JavaScript.

Windows 8 features two types of apps – Metro-style Windows 8 apps, and “standard” apps. (To appreciate the Metro-style UI, watch the keynote or the following 4-minute video: http://www.youtube.com/watch?v=p92QfWOw88I) Metro-style apps as well as legacy apps can be delivered through the Windows Store. There is a paid, trial, and free model.

Unfortunately, there is no automatic wizard for porting existing applications to Metro-style apps. In fact, most apps will need a UI redesign (they call it “reimagining”). Another thing is that Metro-style apps are more sandboxed and controlled than their “legacy” counterparts. This is a part of the obvious move every OS vendor is making towards a controlled application store.

HTML and JavaScript Hype

The whole HTML5 + JavaScript hype means only one thing: you can (if you want) write Metro-style apps using HTML and JavaScript. However, you can write them in C++ or C# or VB.NET as well. There is no inherent advantage in using JavaScript, and the result is NOT cross-platform.

The Development Platform

Metro apps are built using XAML and a programming language of your choice (C++/C#/VB.NET/JS). Any WPF/SL developer will feel immediately at home. There are two development stacks – using standard .NET and native technologies like WPF, SL, MFC; or using WinRT bindings from C++, C#/VB.NET, and JavaScript.

The development concepts, user interface guidelines, styles, controls, and many other things are borrowed from or at least strongly influenced by Windows Phone 7. (This does not mean that Windows 8 apps will run on Windows Phone 7 without modification.) The development platform is Visual Studio 11 and Expression Blend.

For testing the new bits and developer tools, all BUILD attendees are getting a developer machine – 11.6” Samsung tablet with a bunch of sensors, preloaded with Windows 8 Developer Preview and the development tools.

What’s Next?

There is no release timeline yet: the timeline will be quality-based.

All in all, a very exciting day with lots of announcements – a new OS, new application model, new development platform, and new hardware on which to test it all. Stay tuned!

9/11

A couple of days ago I got a phone call from my dad at 7:30AM. He asked “Are you guys OK?” and immediately scenarios of terrorist acts or missile launches started running through my head. It scared me – this unnatural but immediate reaction to an innocent question.

The reality of living in Israel during the last 20 years has conditioned all of us to think in terms of surviving yesterday’s bombing or tomorrow’s war, and the periods of relative peace sometimes spanning months in a row aren’t enough to wipe away the instincts and fears.

Whenever I use public transportation, I “evaluate” all the passengers for terrorist threats and examine every new face boarding the car. Whenever I hear the siren of an ambulance I wonder if it’s someone having a heart attack or a bombing in the middle of a shopping center. I know from conversations with my friends and family that they are wired that way, too.

Ten years ago it was 3:45PM in Israel and I was watching a show on TV with a friend. A few minutes later, every news channel was broadcasting the unbelievable images of the burning twin towers, and my friend had difficulty believing that it wasn’t footage from a poorly-executed action flick.

We will never forget 9/11. But ten years later, today more than ever, with the very real threat of terror engrained deeply into our instincts, we should refuse to be terrorized.

Don’t Use Workflow Bookmarks in Transaction Scopes

It’s been a while since I posted about a workflow-related bug, and it’s not because of my mad WF skillz – it had more to do with Manu’s expertise in the big WF project I’m involved in. The following bug slipped through the cracks and is worth sharing.

One of our custom activities sends a query to the user and waits for a response. This can take days – a human operator is not always available to answer the question, and some questions may require escalation. In one of our recent workflows, two queries had to be sent simultaneously, and the first query answered should cancel the other. This is the Pick activity’s natural habitat. What we also needed is to issue one of the queries within a transaction to record its results transactionally. This leads to the following workflow, with the transaction scope marked in a red box:

Unfortunately, running this workflow as part of our integration tests, we discovered that the first branch is never executed – in fact, it never returns from the Delay activity. However, the tracking information seemed to suggest that it is not cancelled, either – so it couldn’t be the case that the second branch has already completed.

It turns out that our activity’s implementation relies on a bookmark – it creates a bookmark and waits for an external mechanism (a WCF service host) to resume it from the bookmark.

public class QueryActivity : NativeActivity
{
    protected override bool CanInduceIdle
    {
        get { return true; }
    } 
    protected override void Execute(
        NativeActivityContext context)
    {
        context.CreateBookmark(
            "MyBookmark", BookmarkSet);
    }
 
    private void BookmarkSet(
        NativeActivityContext context,
        Bookmark bookmark, object value)
    {
        Console.WriteLine("Done");
    }
}

Creating a bookmark and waiting for resumption in a transaction scope leads exactly to the situation described above, and it is a bad idea anyway – a transaction isn’t supposed to last several days.

To solve the problem, we will need to use a custom mechanism to bring the workflow transactionally to a point where it has recorded the query answer (by resuming the bookmark, persisting the workflow within the external transaction, and then letting it proceed). This is how a seemingly simple task turned out to be quite complex.

I have been recently posting short updates and links on Twitter as well as on this blog. You can follow me: