chamildezoysa

Good Bye MFC!! (Throwing MFC Out of Windows)

2007-04-24T23:08:00.000-07:00

For years, Qt has advertised itself as a better MFC than MFC. Thanks to the Qt 4 Visual Studio Integration, it is now easier than ever for a Windows developer to develop cross-platform, high-performance C++ graphical user interface applications from his favorite IDE.

Qt is a C++ application development framework developed by Trolltech, a Norwegian software company. While Qt mainly focuses on enabing the development of graphical user interfaces, it also provides excellent support for networking, multithreading, SQL operations (including ODBC), XML, and Unicode.

For Windows developers, Qt offers the following advantages:

*Enhanced C++performance. Qt is written in C/C++ (with parts in assembly language) and is highly-optimized to minimize memory usage and maximize speed. Applications based on Qt are native, compiled C++ applications that outperform those applications developed with Java and C#.

*Stability. Since its introduction in 1995, Qt has been actively developed and maintained by Trolltech. The current version, Qt 4.1, was released in December 2005 and supports advanced features, such as semi-transparent controls, antialiased drawing, and bidirectional text rendering.

*Ease of use. Qt has a clean, object-oriented, C++ design, and provides a powerful signal and slot mechanism similar to C#’ s events and delegates. Signals and slots are flexible, fully object-oriented and implemented in C++ through a separate tool called moc (the meta-object compiler).

*Portability. Qt provides a uniform application programming interface for Microsoft Windows, from Windows 98 to Windows XP, Mac OS X, Linux, Solaris, HP-UX, IRIX, AIX, and many other Unix variants. Qt’s" write once, compile everywhere" philosophy lets you develop applications on Windows and deploy them later on other platforms.

Internally, Qt emulates the various platforms’ controls using platform-specific APIs such as GDI+ and the Windows XP theming engine. This enables Qt to accurately reproduce the look-and-feel of each supported platform, and allows programmers to extend or customize Qt’s built-in controls by re-implementing virtual functions. It’s even possible to implement a custom style to give all controls a custom look-and-feel, and to distribute it as a plug-in.

Although Qt is cross-platform, it includes a few Windows-specific modules, notably an MFC-to-Qt migration framework and a COM/ActiveX integration (to create and embed ActiveX controls). Today, Qt is used by companies as diverse as AT& T, IBM, NASA and Xerox, and mass-market applications such as Adobe Photoshop Album and Google Earth are developed using it. In addition, the open source edition of Qt is the foundation of KDE, one of the two major Linux/Unix desktop environments.

Overview of the Qt 4 Visual Studio Integration

The Visual Studio integration, introduced with Qt 4.0, takes advantage of Visual Studio’s highly extensible architecture to make C++ application development on Windows faster, easier, and more intuitive than ever before. The integration includes the following features:

*Fully integrated form editor with layout support. Since version 2.2, Qt includes a powerful visual user interface design tool called Qt Designer, with support for automatic layouts and dynamic, XML-based user interfaces. The integration encapsulates Qt Designer in Visual Studio, with Qt properties displayed in the standard Property Browser (see Figure One).

FIGURE ONE: The integrated form editor

*Wizards for creating new Qt projects. The wizards let you specify the Qt modules you want to use and generate skeleton classes to get you started (see Figure 2).

FIGURE TWO: The Qt Application wizard

*Automated build setup for Qt-specific build steps. Qt includes three tools that generate C++ code behind the scenes: moc (the meta-object compiler), uic (the user interface compiler), and rcc (the resource compiler). These are invoked automatically by the integration when necessary.

*An integrated resource management system. Adding new resources to a Qt project is comparable to adding resources to a standard C++ project. The main difference is that cross-platform .qrc files (Qt resource files) are used rather than Windows .rc files.

*Integrated Qt documentation. Qt’s comprehensive API documentation is integrated with the Visual Studio online help (see Figure Three).

FIGURE THREE: The Qt reference documentation

A Regular Expression Tester

To illustrate how the Visual Studio integration works, let’s create a very small application to let the user try a regular expression against a string. The application, depicted in Figure Four, is a dialog box with three QLabels, three QLineEdits, and two QPushButtons. The user can enter a regular expression pattern and some text on which the regular expression is run; the part of the text that matches is shown in the bottom-most QLineEdit.

FIGURE FOUR: RegExpTester application

The first step is to create a skeleton project. Invoke the New Project dialog in Visual Studio and click the Qt Projects folder (see Figure Five). Next, select the Qt Application item and type" RegExpTexter" as the project name. This pops up the wizard shown in Figure Two.

FIGURE FIVE: Creating a new Qt project

The wizard includes a page to specify which Qt modules you want to link against, followed by a page that enables you to specify the name of the skeleton class to be generated by the wizard. For this example, change the base class of the generated class so that it’s QDialog, not QMainWindow, because the application is a simple dialog-style application with no menus or toolbars.

We’ll now design the dialog using the integrated Qt Designer form editor. To invoke the form editor, we click on the regexptester.ui file in the Visual Studio Solution Explorer. Then we drag the controls we need from the Qt Toolbox onto the form and position them approximately as shown in Figure 6. (Qt’s layout system will lay them out precisely later on.)

FIGURE SIX: Placing controls on a form

The next step is to edit the controls’ properties using the Property Browser. Table One lists the properties that need to be set.

TABLE ONE: Setting the controls’ properties

Control	Property	Value
First label	ObjectName	“patternLabel”
	Text	“Pattern:”
Second label	ObjectName	“textLabel”
	Text	“Text:”
Third label	ObjectName	“resultLabel”
	Text	“Result:”
First line edit	ObjectName	“patternLineEdit”
Second line edit	ObjectName	“textLineEdit”
Third line edit	ObjectName	“resultLineEdit”
	Editable	false
First button	ObjectName	“resetButton”
	Text	“Reset”
Second button	ObjectName	“closeButton”
	Text	“Close”

You need to add layouts to the form to make it look better and to make it resizable. First, add a horizontal layout to position the Reset and Close buttons side by side. You also need a spacer to push the buttons to the right of the layout. To add a spacer, drag the Vertical Spacer item from the Qt Toolbox onto the form, next to the push buttons. Then select the spacer and the buttons and click Form Editor|Layout Horizontally from the Qt entry in the menu bar.

The entire window also needs a layout that will take care of positioning the other controls as well as the button sub-layout. To add this layout, select the labels, the line edits, and the button layout, then click Form Editor& gt; Layout in a Grid. This gives the form shown in Figure Seven.

FIGURE SEVEN: The form with layouts

Qt’s layouts automatically assign reasonable positions and sizes to the controls they are responsible for, based on their needs. This is especially useful in internationalized applications: With fixed sizes and positions, the translation text is often truncated; with layouts, the child widgets are automatically resized. Layouts can also run right-to-left, to accommodate languages such as Arabic and Hebrew.

You can click Form Editor& gt; Preview Form at any time to preview the form without compiling it. If you build the application now, the result is the dialog depicted in Figure Four.

However, the dialog is inoperative. Let’s start by making the Close button work. You can connect the button’s clicked() signal to the form’s accept() slot. This can be done through the form editor.

1.Click the Edit Connections toolbar button to enter the connection mode.

2.Click the form’s Close button and hold the left mouse button pressed, then move the cursor to an empty area of the form and release the mouse button (see Figure Eight). This invokes the Configure Connection dialog, which lists the available signals and slots.

3.Choose the clicked() signal and the accept() slot and click OK.

FIGURE EIGHT: Connecting a signal to a slot visually

Qt controls emit" signals" when state changes take place, for example when the text changes in a line edit, or when a new item is selected in a list box. These signals can be" connected" to slots (member functions), so that when a signal is emitted, any slots it is connected to are called. Controls that are connected to one another are independent components since they don’t need to know anything about each other.

If you build and run the application now, the Close button closes the window and causes the application to terminate.

More Fun With Dialogs

Let’s continue and implement the rest of the dialog functionality. The contents of the Result line need to be updated whenever the user edits the text in one of the two other line edits. This is achieved by adding a slot called updateResult() to the form and by connecting the two line edits’ textChanged() signals to that slot. The signals are emitted whenever the text changes; the connection ensures that the slot is invoked to recompute the Result line edit’s contents based on the new text in the Pattern and Text fields.

For that, you must edit the regexptester.h and regexptester.cpp files that were generated by the Qt Application wizard. Listing One shows the header file after adding the slot declaration. Notice the particular syntax for declaring a slot; this is Qt-specific and is converted into standard C++ by the C++ preprocessor.

LISTING ONE: The form’s header file

#ifndef REGEXPTESTER_H
#define REGEXPTESTER_H

#include
#include "ui_regexptester.h"

class RegExpTester : public QDialog
{
Q_OBJECT

public:
RegExpTester(QWidget *parent = 0, Qt::WFlags flags = 0);
~RegExpTester();

private slots:
void updateResult();

private:
Ui::RegExpTesterClass ui;
};

#endif // REGEXPTESTER_H

The RegExpTester class inherits from QDialog (the class you specified in the project wizard) and represents the entire dialog.

The first unusual thing to notice about the class definition is the Q_OBJECT macro. This signifies that the class has some Qt extensions to C++ that will be discussed shortly. The private slots are private member functions that can also be called by Qt’s signals and slots mechanism.

The header file includes a file called ui_regexptester.h. That file is generated by the uic program based on the regexptester.ui, an XML file that stores the form we designed using the form editor. The uic- generated ui_regexptester.h file declares a class called Ui::RegExpTester. In the private section of the RegExpTester class, there is a member variable of type Ui::RegExpTester called ui.

The updateResult() slot needs to be implemented and connected to the Pattern and Text line edit’s textChanged() signals. This is done in the regexptester.cpp file, listed in Listing Two. Again, the code that needs to be added to the wizard-generated skeleton is shown in bold.

LISTING TWO: The form’s source file

#include "regexptester.h"

RegExpTester::RegExpTester(QWidget *parent, Qt::WFlags flags)
: QDialog(parent, flags)
{
ui.setupUi(this);
connect(ui.patternLineEdit, SIGNAL(textChanged(QString)),
this, SLOT(updateResult()));
connect(ui.textLineEdit, SIGNAL(textChanged(QString)),
this, SLOT(updateResult()));
}

RegExpTester::~RegExpTester()
{
}

void RegExpTester::updateResult()
{
QRegExp regExp(ui.patternLineEdit->text());
if (regExp.indexIn(ui.textLineEdit->text()) != -1)
ui.resultLineEdit->setText(regExp.cap(0));
else
ui.resultLineEdit->setText("");
}

The constructor calls setupUi() on the Ui::RegExpTester object to create the form’s controls and layouts. Then, the Pattern line edit’s textChanged() signal is connected to the updateResult() slot. The same is done with the Text line edit.

In the updateResult() slot, the regular expression pattern is applied to the specified text. If the regular expression matches, the matched text is shown in the Result line edit; otherwise, it shows"& lt;no match& gt;".

The dialog is almost fully functional. The only remaining issue is that the Reset button still does nothing. The solution is to add a reset() slot and connect it to the Reset button’s clicked() signal in the constructor. The slot is implemented as follows:

void RegExpTester::reset()

{

   ui.patternLineEdit->clear();

   ui.textLineEdit->clear();

   updateResult();

}

This completes the small sample application. But what about Q_OBJECT, private slots, and other Qt keywords? The C preprocessor turns them all into pure C++ so the compiler never sees them. The machinery to handle Qt’s extensions is generated behind the scenes by moc, which reads your source files and generates some additional source files to implement what we’ve used.

Goodbye, MFC

Microsoft is giving up MFC and is preparing to replace its successor, Windows Forms, with the Avalon framework. The uncertainty surrounding these three APIs has alienated many Windows developers, who are now looking at Qt as a more stable alternative.

With the Qt 4 Visual Studio Integration, a Windows developer can easily leverage his or her existing C++ skills to develop cross-platform, high-performance object-oriented, GUI applications.

Moving to Windows Vista x64 - .NET 3.0

2007-02-02T02:24:00.000-08:00

Introduction
x64 Section
Vista Section
.NET Framework 3.0
Conclusions

Introduction

This is an introduction to Windows Vista and the x64 architecture. Writing an article like this is always uneasy, because there's plenty to talk about, but on the other hand it's an article, not a book. I tried to focus on some important aspects, but it goes without saying it that I had to cut out a lot (e.g. the User-Mode Driver Framework, and I'm very sorry for that). This is just a general overview on certain topics, if you want to learn more, then you should really consider turning to specific guides. Also, I won't talk about some obvious matters of the x64 architecture, like the fact that applications can now access a larger memory range etc. This article should be considered a quick upgrade for x86/XP developers.

At the time I write this article I've been using Windows Vista for a month and its official release is scheduled for January 30th (so, in another month). I moved to x64 with XP some months ago and at the time I did I was surprised that I found all the drivers for my devices. But, as we know, Windows Vista requires drivers to be certified, and in order to get the certification companies have to supply a x64 version of the driver. No certification will be released for x86-only drivers. However, at the moment I write, a lot of applications like virtual drive encrypters don't provide drivers for Vista (since x64 versions haven't got a certificate). If you didn't know about the certification, don't worry, I'll talk about it later and you'll see that it's still possible to run drivers without it. I just wanted to say that hardware compatibility is no longer an issue like it was one year ago, and by switching to Windows Vista x64 you're not taking too much chances.

I tried to organize this article in two sections, one about the changes brought us by x64 and then by Vista. I tried as hard as possible to separate these two things, because the x64 technology already existed under Windows XP, so it was important to me that the reader was given a clear distinction between those things that affect only Vista and those ones which affect both topics.

x64 Section

x64 Assembly

In this paragraph I'll try to explain the basics of x64 assembly. I assume the reader is already familiar with x86 assembly, otherwise he won't be able to make heads or tails of this paragraph. Moreover, since this is just a very (but very) brief guide, you'll have to look into the AMD64 documentation for more advanced stuff. Some stuff I won't even mention, you'll see by yourself that some instructions are no longer in use: for instance, that the lea instruction has completely taken place of the mov offset.

What you're going to notice at once is that there are some more registers in the x64 syntax:

8 new general-purpose registers (GPRs).
8 new 128-bit XMM registers.

Of course, all general-purpose registers are 64 bits wide. The old ones we already knew are easy to recognize in their 64-bit form: rax, rbx, rcx, rdx, rsi, rdi, rbp, rsp (and rip if we want to count the instruction pointer). These old registers can still be accessed in their smaller bit ranges, for instance: rax, eax, ax, ah, al. The new registers go from r8 to r15, and can be accessed in their various bit ranges like this: r8 (qword), r8d (dword), r8w (word), r8b (low byte).

Here's a figure taken from the AMD docs:

Applications can still use segments registers as base for addressing, but the 64-bit mode only recognizes three of the old ones (and only two can be used for base address calculations). Here's another figure:

And now, the most important things. Calling convention and stack. x64 assembly uses FASTCALLs as calling convention, meaning it uses registers to pass the first 4 parameters (and then the stack). Thus, the stack frame is made of: the stack parameters, the registers parameters, the return address (which I remind you is a qword) and the local variables. The first parameter is the rcx register, the second one rdx, the third r8 and the fourth r9. Saying that the parameters registers are part of the stack frame, makes it also clear that any function that calls another child function has to initialize the stack providing space for these four registers, even if the parameters passed to the child function are less than four. The initialization of the stack pointer is done only in the prologue of a function, it has to be large enough to hold all the arguments passed to child functions and it's always a duty of the caller to clean the stack. Now, the most important thing to understand how the space is provided in the stack frame is that the stack has to be 16-byte aligned. In fact, the return address has to be aligned to 16 bytes. So, the stack space will always be something like 16n + 8, where n depends on the number of parameters. Here's a small figure of a stack frame:

Don't worry if you haven't completely figured out how it works: now we will see a few code samples, which, in my opinion, always make the theory a lot easier to understand. Let us take for instance a hello-world application like:

int WINAPI _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                   LPSTR szCmdLine, int iCmdShow)
{
  MessageBox(NULL, _T("Hello World!"), _T("My First x64 Application"), 0);
  return 0;
}

This code disassembled would look like:

.text:0000000000401220 sub_401220 proc near       ; CODE XREF: start+10E p
.text:0000000000401220
.text:0000000000401220 arg_0= qword ptr 8
.text:0000000000401220 arg_8= qword ptr 10h
.text:0000000000401220 arg_10= qword ptr 18h
.text:0000000000401220 arg_18= dword ptr 20h
.text:0000000000401220
.text:0000000000401220    mov [rsp+arg_18], r9d
.text:0000000000401225    mov [rsp+arg_10], r8
.text:000000000040122A    mov [rsp+arg_8], rdx
.text:000000000040122F    mov [rsp+arg_0], rcx
.text:0000000000401234    sub rsp, 28h
.text:0000000000401238    xor r9d, r9d            ; uType
.text:000000000040123B    lea r8, Caption         ; "My First x64 Application"
.text:0000000000401242    lea rdx, Text           ; "Hello World!"
.text:0000000000401249    xor ecx, ecx            ; hWnd
.text:000000000040124B    call cs:MessageBoxA
.text:0000000000401251    xor eax, eax
.text:0000000000401253    add rsp, 28h
.text:0000000000401257    retn
.text:0000000000401257 sub_401220 endp

The stack pointer initialization is all about the things I said earlier. Since we are calling a child-function with parameters we need the space for all four parameter registers (0x20, this value is already aligned to 16 byte) and the return address (0x08). Thus, we'll have 0x28. Remember that if the stack-value is too small or is not aligned, your code will crash at once. Also, don't wonder why there's no ExitProcess in this function: compiling the code above with Visual C++ adds always a stub (WinMainCRTStartup) which then calls our WinMain. So, the ExitProcess is in the stub code. But what happens when the code before the MessageBox calls a function which take seven parameters instead of four?

.text:0000000000401180 sub_401180 proc near        ; CODE XREF: sub_4011F0+4 p
.text:0000000000401180                             ; sub_4011F0+11 p
.text:0000000000401180
.text:0000000000401180 var_28= qword ptr -28h
.text:0000000000401180 var_20= qword ptr -20h
.text:0000000000401180 var_18= qword ptr -18h
.text:0000000000401180
.text:0000000000401180    sub rsp, 48h
.text:0000000000401184    lea rax, unk_402040
.text:000000000040118B    mov [rsp+48h+var_18], rax
.text:0000000000401190    lea rax, unk_402044
.text:0000000000401197    mov [rsp+48h+var_20], rax
.text:000000000040119C    lea rax, unk_402048
.text:00000000004011A3    mov [rsp+48h+var_28], rax
.text:00000000004011A8    lea r9, qword_40204C   ; __int64
.text:00000000004011AF    lea r8, qword_40204C+4 ; __int64
.text:00000000004011B6    lea rdx, unk_402054    ; __int64
.text:00000000004011BD    lea rcx, aAa           ; "ptr"
.text:00000000004011C4    call TakeSevenParameters
.text:00000000004011C9    xor r9d, r9d               ; uType
.text:00000000004011CC    lea r8, Caption        ; "My First x64 Application"
.text:00000000004011D3    lea rdx, Text          ; "Hello World!"
.text:00000000004011DA    xor ecx, ecx           ; hWnd
.text:00000000004011DC    call cs:MessageBoxA
.text:00000000004011E2    add rsp, 48h
.text:00000000004011E6    retn
.text:00000000004011E6 sub_401180 endp

As said, the child function takes 7 parameters, making it necessary to provide space for 3 extra parameters on the stack. So, 7 * 8 = 0x38, which aligned to 16byte is 0x40. Providing, then, space for the return address makes it 0x48, our value indeed. I think you have understood the stack-frames logic by now, it's actually quite easy to understand it, but it needs a second to revert from the old x86/stdcall logic to this one. But now enough of this, now that we've seen how the x64 code works, we'll try compiling an assembly source by ourselves.

Before we start, I have to make something clear. There are some assemblers over the internet which make the job easier, mainly because the initialize the stack by themselves or they create code that is easy to converto from/to x86. But I think that is not the point here in this article. In fact, I'm going to use the microsoft assembler (ml64.exe), which requires you to write everything down, just like in the disassembly. Another option could be compiling the with another assembler and then link it with ml64. I think the reader should really make these decisions on his own. As far as I am concerned, I don't believe that much code should be written in assembly and avoided whenever it could be done. This new x64 technology is a good opportunity to re-think about these matters. In the last years I always wrote 64-bit compatible code in C/C++ (I mean unmanaged, of course) and when I had to recompile a project of 70,000 lines of code for x64, I didn't had to change one single line of code (I'll talk about the C/C++ programming later). Despite of all the macros an assembler offers, I seriously doubt that people who wrote their whole code in assembly will be able to switch so easily to x64 (remember one day even the IA64 syntax could be adopted). I think in most cases the obvious choice will be not converting to the new technology and stick to x86, but this isn't always possible, it depends on the software category.

The Microsoft assembler is contained in the SDK and in the DDK (WDK for Vista). Right now, I'm using Vista's WDK, which I freely downloaded from the msdn. The first sample of code I'm going to show you is a simple Hello-World messagebox application.

extrn MessageBoxA : proc
extrn ExitProcess : proc

.data
body db 'Hello World!', 0
capt db 'My First x64 Application', 0

.code
Main proc
sub rsp, 28h
xor r9d, r9d        ; uType = 0
lea r8, capt        ; lpCaption
lea rdx, body       ; lpText
xor rcx, rcx        ; hWnd = NULL
call MessageBoxA
xorecx, ecx        ; exit code = 0
call ExitProcess
Main endp

end

As you can see, I didn't bother unwinding the stack, since I call ExitProcess. The syntax is very similar to the old MASM one, although there are a few dissimilarities. The ml64 console output should be something like this:

The command line to compile is:

ml64 C:\...\test.asm /link /subsystem:windows
  /defaultlib:C:\WinDDK\6000\lib\wnet\amd64\kernel32.lib
  /defaultlib:C:\WinDDK\6000\lib\wnet\amd64\user32.lib /entry:Main

If the libs are not in the same directory as ml64.exe, you'll have to provide the path like I did. The entry has to be provided, otherwise you would have to use WinMainCRTStartup as main entry.

The next sample of code I'm going to show you displays a window calling CreateWindowEx. What you're going to learn through this code is structure alignment and how integrating resources in your projects. Like I said earlier, I don't want to encourage you to write your windows in assembly, but I believe that this sort of code is good for learning. Now the code, afterwards the explanation.

Collapse

extrn GetModuleHandleA : proc
extrn MessageBoxA : proc
extrn RegisterClassExA : proc
extrn CreateWindowExA : proc
extrn DefWindowProcA : proc
extrn ShowWindow : proc
extrn GetMessageA : proc
extrn TranslateMessage : proc
extrn DispatchMessageA : proc
extrn PostQuitMessage : proc
extrn DestroyWindow : proc
extrn ExitProcess : proc

WNDCLASSEX struct
cbSize            dd      ?
style             dd      ?
lpfnWndProc       dq      ?
cbClsExtra        dd      ?
cbWndExtra        dd      ?
hInstance         dq      ?
hIcon             dq      ?
hCursor           dq      ?
hbrBackground     dq      ?
lpszMenuName      dq      ?
lpszClassName     dq      ?
hIconSm           dq      ?
WNDCLASSEX ends

POINT struct
x                 dd      ?
y                 dd      ?
POINT ends

MSG struct    
hwnd              dq      ?
message           dd      ?
padding1          dd      ?      ; padding
wParam            dq      ?
lParam            dq      ?
time              dd      ?
pt                POINT   <;>
padding2          dd      ?      ; padding
MSG ends

.const
NULL equ 0
CS_VREDRAW equ 1
CS_HREDRAW equ 2
COLOR_WINDOW equ 5
; WS_OVERLAPPEDWINDOW = (WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | 
; WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX) 
WS_OVERLAPPEDWINDOW equ 0CF0000h
CW_USEDEFAULT equ 80000000h
SW_SHOW equ 5
WM_DESTROY equ 2
WM_COMMAND equ 111h
IDC_MENU equ 109
IDM_ABOUT equ 104
IDM_EXIT equ 105

.data
szWindowClass db 'FirstApp', 0
szTitle db 'My First x64 Windows', 0
szHelpTitle db 'Help', 0
szHelpText db 'This will be a big help...', 0

.data?
hInstance qword ?
hWnd qword ?
wndclass WNDCLASSEX <;>
wmsg MSG <;>

.code

WndProc: ; proc hWnd : qword, uMsg : dword, wParam : qword, lParam : qword
mov [rsp+8], rcx        ; hWnd (save parameters as locals)
mov [rsp+10h], edx      ; Msg
mov [rsp+18h], r8       ; wParam
mov [rsp+20h], r9       ; lParam
sub rsp, 38h
cmp edx, WM_DESTROY
jnz @next1

xor ecx, ecx          ; exit code
call PostQuitMessage
xor rax, rax
ret

@next1:
cmp edx, WM_COMMAND
jnz @default

mov rbx, rsp
add rbx, 38h
mov r10, [rbx+18h]     ; wParam
cmp r10w, IDM_ABOUT
jz @about
cmp r10w, IDM_EXIT
jz @exit
jmp @default

@about:
xor r9d, r9d
lea r8, szHelpTitle
lea rdx, szHelpText
xor ecx, ecx
call MessageBoxA
jmp @default

@exit:
mov rbx, rsp
add rbx, 38h
mov rcx, [rbx+8h]       ; hWnd
call DestroyWindow

@default:
mov rbx, rsp
add rbx, 38h
mov r9, [rbx+20h]       ; lParam
mov r8, [rbx+18h]       ; wParam
mov edx, [rbx+10h]      ; Msg
mov rcx, [rbx+8]        ; hWnd
call DefWindowProcA
add rsp, 38h
ret

MyRegisterClass:  ; proc hInst : qword
sub rsp, 28h
mov wndclass.cbSize, sizeof WNDCLASSEX
mov eax, CS_VREDRAW
or eax, CS_HREDRAW
mov wndclass.style, eax
lea rax, WndProc
mov wndclass.lpfnWndProc, rax
mov wndclass.cbClsExtra, 0
mov wndclass.cbWndExtra, 0
mov wndclass.hInstance, rcx
mov wndclass.hIcon, NULL
mov wndclass.hCursor, NULL
mov wndclass.hbrBackground, COLOR_WINDOW
mov wndclass.lpszMenuName, IDC_MENU
lea rax, szWindowClass
mov wndclass.lpszClassName, rax
mov wndclass.hIconSm, NULL
lea rcx, wndclass
call RegisterClassExA
add rsp, 28h
ret


InitInstance: ; proc hInst : qword
sub rsp, 78h       
mov rax, CW_USEDEFAULT
xor rbx, rbx
mov [rsp+58h], rbx            ; lpParam
mov [rsp+50h], rcx            ; hInstance
mov [rsp+48h], rbx            ; hMenu = NULL
mov [rsp+40h], rbx            ; hWndParent = NULL
mov [rsp+38h], rbx            ; Height
mov [rsp+30h], rax            ; Width
mov [rsp+28h], rbx            ; Y
mov [rsp+20h], rax            ; X
mov r9d, WS_OVERLAPPEDWINDOW  ; dwStyle
lea r8, szTitle               ; lpWindowName
lea rdx, szWindowClass        ; lpClassName
xor ecx, ecx                  ; dwExStyle
call CreateWindowExA
mov hWnd, rax
mov edx, SW_SHOW
mov rcx, hWnd
call ShowWindow
mov rax, hWnd                 ; set return value
add rsp,78h
ret


Main proc
sub rsp, 28h
xor rcx, rcx   
call GetModuleHandleA
mov hInstance, rax
mov rcx, rax
call MyRegisterClass
test rax, rax
jz @close              ; if the RegisterClassEx fails, exit 

mov rcx, hInstance
call InitInstance
test rax, rax
jz @close              ; if the InitInstance fails, exit 

@handlemsgs:             ; message processing routine
xor r9d, r9d         
xor r8d, r8d
xor edx, edx
lea rcx, wmsg
call GetMessageA
test eax, eax
jz @close
lea rcx, wmsg
call TranslateMessage
lea rcx, wmsg
call DispatchMessageA
jmp @handlemsgs

@close:
xor ecx, ecx  
call ExitProcess
Main endp

end

As you can see, I tried to stay as low level as I could. The reason why I avoided for other functions other than the main the proc macro is that the ml64 puts a prologue end an epilogue, which I didn't want, by itself. Avoiding the macro made it possible to define my own stack frame without any intermission by the compiler. The first thing to notice scrolling this code is the structure:

MSG struct    
hwnd              dq      ?
message           dd      ?
padding1          dd      ?      ; padding
wParam            dq      ?
lParam            dq      ?
time              dd      ?
pt                POINT   <;>
padding2          dd      ?      ; padding
MSG ends

It requires two paddings which the x86 declaration of the same structure didn't. The reason, in a few words, is that qword members should be aligned to qword boundaries (this for the first padding). The additional padding at the end of the structure follows the rule that: every structure should be aligned to its largest member. So, being its largest member a qword, the structure should be aligned to an 8-byte boundary.

To compile this sample, the command line is:

ml64 c:\myapp\test.asm /link /subsystem:windows
  /defaultlib:C:\WinDDK\6000\lib\wnet\amd64\kernel32.lib
  /defaultlib:C:\WinDDK\6000\lib\wnet\amd64\user32.lib
  /entry:Main c:\myapp\test.res

test.res is a file I took from a VC++ wizard project, I was too lazy to make on by myself. Anyway, making a resource file is very easy with the VC++, but no one forbids you to use the notepad, it just takes more time. To compile the resource file all you need to do is to use the command line: "rc test.rc".

I think the rest of the code is pretty easy to understand. I didn't cover everything with this paragraph, but now you should have quite a good insight into x64 assembly. Let's move on.

C/C++ Programming

Writing x64 compatible code in C/C++ is very easy. All what it takes is to follow some basic rules. The most common mistake that make that makes 99% of the old 32bit sources uncompatible is wrong casting. For Instance:

ptr1 = (DWORD) (sizoef (x) + ptr2);  // <-- WRONG!

This line of code assumes that pointers are 32bit long, but on x64 pointers are 64bit long and the line of code above basically truncates the pointer making it invalid. So, always cast like this:

ptr1 = (ULONG_PTR) (sizoef (x) + ptr2);  <-- RIGHT!

It doesn't matter if you use ULONG_PTR, LONG_PTR, DWORD_PTR or whatever. The important thing is that you use one of these defines (or directly by pointer type: (void *)).

Keep in mind that all handles and handle derivates are qwords. HANDLE, HKEY, HICON, HBITMAP, HINSTANCE, HMODULE, HWND etc. etc. These are all 64bit long, even though they're not all the same handle (HINSTANCE, for example, is just a pointer, not a real handle). Even WPARAM and LPARAM are now 64bit long. There's no rule to follow, just don't assume these types are 32 or 64bit long: write code that is compatible with both conditions:

HWND *hWndArray = (HWND *) malloc(sizeof (DWORD) * n);  <-- WRONG!

Instead write:

HWND *hWndArray = (HWND *) malloc(sizeof (HWND) * n);  <-- RIGHT!

As you can see this isn't a rule, just good sense.

The defines to use for writing architecture-dependent code are:

`_M_IX86`	x86 code only.
`_M_AMD64`	x64 code only.
`_M_IA64`	Itanium code only.
`_WIN32`	32bit code (x86, maybe ARM for WINCE).
`_WIN64`	64bit code (x64, Itanium).

if you want to write, for example, a piece of code for x86 only, you could write:

#ifdef _M_IX86
  // x86 only code
#endif

Now that you know all the rules, you just have to compile your project for x64. Keep in mind that every project in VC++ (nowadays) starts with a x86 configuration: it's your job to add a project configuration to the project, but don't worry it's very easy. All you have to do is open the configuration manager (Build -> Configuration Manager) and then under "Active solution platform" click New, just like this:

A dialog box will pop up where you can choose the new platform which for to create a new project configuration. There's nothing more to do, except to build.

Inline Assembly

Bad news! Microsoft completely removed the support for inline assembly in C/C++, both for user and kernel mode. If you try to compile a code sample like this on x64/Itanium:

#include "stdafx.h"
#include 

int _tmain(int argc, _TCHAR* argv[])
{
  __asm int 3;
  return 0;
}

It will give you more than just one error. Being the __asm keyword no longer supported, the __naked declspec was removed as well (since it doesn't make sense without inline assembly).

Now, prepare for the good news. Before you start thinking about using external asm files or stuff like that, you should know that the VC++ offers some very powerful assembly intrinsics. The header to include to use these intrinsics is "intrin.h". Let's take for a code sample the intrinsics _ReturnAddress() and _AddressOfReturnAddress(). The first one gives us the return address of the current function and the second one the address of the return address itself. Let's analyze this little code sample that I took from the MSDN:

int _tmain(int argc, _TCHAR* argv[])
{
  void* pvAddressOfReturnAddress = _AddressOfReturnAddress();
  printf_s("%p\n", pvAddressOfReturnAddress);
  printf_s("%p\n", *((void**) pvAddressOfReturnAddress));
  printf_s("%p\n", _ReturnAddress());

  return 0;
}

The second and the third printf_s will show the same output, since both display the return address of the current function. These intrinsics are very powerful, and nothing can stop us from doing some of the old tricks we did with inline assembly. For instance, having the address of the return address could give me the possibility of changing it and making the function return somewhere else. Let's try that:

ULONG_PTR OldAddress = 0;

void f1()
{
  printf_s("Hello there!\n");

  ULONG_PTR *pAddressOfReturnAddress = (ULONG_PTR *)
      _AddressOfReturnAddress();

  if (OldAddress == 0)
  {
      OldAddress = *pAddressOfReturnAddress;
      *pAddressOfReturnAddress =  (ULONG_PTR) &f1;
  }
  else
  {
      *pAddressOfReturnAddress = OldAddress;
  }

}

The output of this function is:

Hello there!
Hello there!

That's because, as you can see from the code, I changed the return address of the current function making it execute again. I put a condition to make it execute again just once, otherwise it would have brought to an endless loop. An important thing to know is that this sample works in Release mode only if you disable code optimization, otherwise the VC++ will remove the line of code which sets the new return address. I'm sure there are ways to trick the VC++ not to do this, but the problem is that if the function is called just by one caller like this one, the VC++ will put the code of the function directly in the caller one, so setting a new return address under these conditions is a bit risky. Disabling optimization is, I believe, the safest way to act.

Enough of this trivia. Here's a list of the intrinsics for x64 taken from the MSDN (many of them are supported on x86 as well):

`_AddressOfReturnAddress`	Provides the address of the memory location that holds the return address of the current function. This address may not be used to access other memory locations (for example, the function's arguments).
`__addgsbyte, __addgsword, __addgsdword, __addgsqword`	Add a value to a memory location specified by an offset relative to the beginning of the GS segment.
`__assume`	Passes a hint to the optimizer.
`_BitScanForward, _BitScanForward64`	Search the mask data from least significant bit (LSB) to the most significant bit (MSB) for a set bit (1).
`_BitScanReverse, _BitScanReverse64`	Search the mask data from most significant bit (MSB) to least significant bit (LSB) for a set bit (1).
`_bittest, _bittest64`	Generates the bt instruction, which examines the bit in position b of address a, and returns the value of that bit.
`_bittestandcomplement, _bittestandcomplement64`	Generate the btc instruction, which examines bit b of the address a, returns its current value, and sets the bit to its complement.
`_bittestandreset, _bittestandreset64`	Generate the btr instruction, which examines bit b of the address a, returns its current value, and resets the bit to 0.
`_bittestandset, _bittestandset64`	Generate the bts instruction, which examines bit b of the address a, returns its current value, and sets the bit to 1.
`__debugbreak`	Causes a breakpoint in your code, where the user will be prompted to run the debugger.
`_disable`	Disables interrupts.
`__emul, __emulu`	Performs multiplications that overflow what a 32-bit integer can hold.
`_enable`	Enables interrupts.
`__faststorefence`	Guarantees that every preceding store is globally visible before any subsequent store.
`__getcallerseflags`	Returns the EFLAGS value from the caller's context.
`__inbyte`	Generates the in instruction, returning one byte read from the port specified by Port.
`__inbytestring`	Reads data from the specified port using the rep insb instruction.
`__incgsbyte, __incgsword, __incgsdword, __incgsqword`	Add one to the value at a memory location specified by an offset relative to the beginning of the GS segment.
`__indword`	Reads one double word of data from the specified port using the in instruction.
`__indwordstring`	Reads data from the specified port using the rep insd instruction.
`__int2c`	Generates the int 2c instruction, which triggers the 2c interrupt.
`_InterlockedAnd, _InterlockedAnd64`	Used to perform an atomic AND operation on a variable shared by multiple threads.
`_interlockedbittestandreset, _interlockedbittestandreset64`	Generate the lock_btr instruction, which examines bit b of the address a and returns its current value.
`_interlockedbittestandset, _interlockedbittestandset64`	Generate the lock_bts instruction, which examines bit b of the address a and returns its current value.
`_InterlockedCompareExchange, _InterlockedCompareExchange64, _InterlockedCompare64Exchange128, _InterlockedCompare64Exchange128_acq, _InterlockedCompare64Exchange128_rel`	Provides compiler intrinsic support for the Win32 Platform SDK InterlockedCompareExchange function.
`_InterlockedCompareExchangePointer`	Perform an atomic exchange operation, which copies the address passed in as the second argument to the first and returns the original address of the first.
`_InterlockedDecrement, _InterlockedDecrement64`	Provides compiler intrinsic support for the Win32 Platform SDK InterlockedDecrement function.
`_InterlockedExchange, _InterlockedExchange64`	Provide compiler intrinsic support for the Win32 Platform SDK InterlockedExchange function.
`_InterlockedExchangeAdd, _InterlockedExchangeAdd64`	Provide compiler intrinsic support for the Win32 Platform SDK _InterlockedExchangeAdd Intrinsic Functions function.
`_InterlockedExchangePointer`	Perform an atomic exchange operation, which copies the address passed in as the second argument to the first and returns the original address of the first.
`_InterlockedIncrement, _InterlockedIncrement64`	Provide compiler intrinsic support for the Win32 Platform SDK InterlockedIncrement function.
`_InterlockedOr, _InterlockedOr64`	Perform an atomic operation (in this case, the OR operation) on a variable shared by multiple threads.
`_InterlockedXor, _InterlockedXor64`	Used to perform an atomic operation (in this case, the exclusive or XOR operation) on a variable shared by multiple threads.
`__invlpg`	Generates the x86 invlpg instruction, which invalidates the translation lookaside buffer (TLB) for the page associated with memory pointed to by Address.
`__inword`	Reads data from the specified port using the in instruction.
`__inwordstring`	Reads data from the specified port using the rep insw instruction.
`__ll_lshift`	Shifts a 64-bit value specified by the first parameter to the left by a number of bits specified by the second parameter.
`__ll_rshift`	Shifts a 64-bit value specified by the first parameter to the right by a number of bits specified by the second parameter.
`__load128, __load128_acq`	Loads a 128-bit value atomically.
`_mm_cvtsd_si64x`	Generates the x64 extended form of the Convert Scalar Double-Precision Floating-Point Value to 64-Bit Integer (cvtsd2si) instruction, which takes the double in the first element of value and converts it to a 64-bit integer.
`_mm_cvtsi128_si64x`	Generates the x64 extended form of the movd instruction, which extracts the low 64-bit integer from an __m128i structure.
`_mm_cvtsi64x_sd`	Generates the Convert Double Word Integer to Scalar Double-Precision Floating-Point Value (cvtsi2sd) instruction.
`_mm_cvtsi64x_si128`	Generates the x64 extended form of the movd instruction, which copies a 64-bit value to a __m128i structure, which represents an XMM register.
`_mm_cvtsi64x_ss`	Generates the x64 extended version of the Convert 64-Bit Integer to Scalar Single-Precision Floating-Point Value (cvtsi2ss) instruction.
`_mm_cvtss_si64x`	Generates the x64 extended version of the Convert Scalar Single Precision Floating Point Number to 64-bit Integer (cvtss2si) instruction.
`_mm_cvttsd_si64x`	Generates the x64 extended version of the Convert with Truncation Scalar Double-Precision Floating-Point Value to 64-Bit Integer (cvttsd2si) instruction, which takes the first double in the input structure of packed doubles, converts it to a 64-bit integer, and returns the result.
`_mm_cvttss_si64x`	Emits the x64 extended version of the Convert with Truncation Single-Precision Floating-Point Number to 64-Bit Integer (cvttss2si) instruction.
`_mm_set_epi64x`	Returns the __m128i structure with its two 64-bit integer values initialized to the values of the two 64-bit integers passed in.
`_mm_set1_epi64x`	Provides a way to initialize the two 64-bit elements of the __m128i structure with two identical integers.
`_mm_setl_epi64`	Returns the lower 64 bits of source argument in the lower 64 bits of the result.
`_mm_stream_si64x`	Writes the data in Source to a memory location specified by Dest, without polluting the caches.
`__movsb`	Generates a Move String (rep movsb) instruction.
`__movsd`	Generates a Move String (rep movsd) instruction.
`__movsq`	Generates a repeated Move String (rep movsq) instruction.
`__movsw`	Generates a Move String (rep movsw) instruction.
`__mul128`	Multiplies two 64-bit integers passed in as the first two arguments and puts the high 64 bits of the product in the 64-bit integer pointed to by HighProduct and returns the low 64 bits of the product.
`__mulh`	Returns the high 64 bits of the product of two 64-bit signed integers.
`__outbyte`	Generates the out instruction, which sends 1 byte specified by Data out the I/O port specified by Port.
`__outbytestring`	Generates the rep outsb instruction,which sends the first Count bytes of data pointed to by Buffer to the port specified by Port.
`__outdword`	Generates the out instruction to send a doubleword Data out the port Port.
`__outdwordstring`	Generates the rep outsd instruction, which sends Count doublewords starting at Buffer out the I/O port specified by Port.
`__rdtsc`	Generates the rdtsc instruction, which returns the processor time stamp. The processor time stamp records the number of clock cycles since the last reset.
`_ReadBarrier`	Forces memory reads to complete.
`__readcr0, __readcr2, __readcr3, __readcr4, __readcr8`	Read the control registers. These intrinsics are only available in kernel mode.
`__readfsbyte, __readfsdword, __readfsqword, __readfsword`	Read memory from a location specified by an offset relative to the beginning of the FS segment. These intrinsics are only available in kernel mode.
`__readgsbyte, __readgsdword, __readgsqword, __readgsword`	Read memory from a location specified by an offset relative to the beginning of the GS segment. These intrinsics are only available in kernel mode.
`__readmsr`	Generates the rdmsr instruction, which reads the model-specific register specified by register and returns its value. This function may only be used in kernel mode.
`__readpmc`	Generates the rdpmc instruction, which reads the performance monitoring counter specified by counter.
`_ReadWriteBarrier`	Effectively blocks an optimization of reads and writes to global memory.
`_ReturnAddress`	The _ReturnAddress intrinsic provides the address of the instruction in the calling function that will be executed after control returns to the caller.
`__shiftleft128`	Shifts a 128-bit quantity, represented as two 64-bit quantities LowPart and HighPart, to the left by a number of bits specified by Shift and returns the high 64 bits of the result.
`__shiftright128`	Shifts a 128-bit quantity, represented as two 64-bit quantities LowPart and HighPart, to the right by a number of bits specified by Shift and returns the low 64 bits of the result.
`__store128, __store128_rel`	Stores a 128-bit value atomically.
`__stosb`	Generates a store string instruction (rep stosb).
`__stosd`	Generates a store string instruction (rep stosd).
`__stosq`	Generates a store string instruction (rep stosq).
`__stosw`	Generates a store string instruction (rep stosw).
`__ull_rshift`	on x64, shifts a 64-bit value specified by the first parameter to the right by a number of bits specified by the second parameter.
`_umul128`	Multiplies two 64-bit unsigned integers passed in as the first two arguments and puts the high 64 bits of the product in the 64-bit unsigned integer pointed to by HighProduct and returns the low 64 bits of the product.
`__umulh`	Return the high 64 bits of the product of two 64-bit unsigned integers.
`__wbinvd`	Generates the Write Back and Invalidate Cache (wbinvd) instruction.
`_WriteBarrier`	Forces memory writes to complete and be correct according to program logic at the point of the call.
`__writecr0, __writecr3, __writecr4, __writecr8`	Write the control registers. These intrinsics are only available in kernel mode.
`__writefsbyte, __writefsdword, __writefsqword, __writefsword`	Write memory to a location specified by an offset relative to the beginning of the FS segment. These intrinsics are only available in kernel mode.
`__writegsbyte, __writegsdword, __writegsqword, __writegsword`	Write memory to a location specified by an offset relative to the beginning of the GS segment. These intrinsics are only available in kernel mode.
`__writemsr`	Generates the Write to Model Specific Register (wrmsr) instruction. This function may only be used in kernel mode.

There are also some 3D intrinsics (called 3DNow) which will be useful for game/3D coders. I left those intrinsics out of the list since they were too many and you'd need to include another header file to use them: "mm3dnow.h".

If these intrinsics are not enough, you might need to use an external asm file. On the other hand, if you're really lazy and you just need something on the fly, there's a quick way to embed assembly code in your C/C++ files.

#include "stdafx.h"
#include 

unsigned char BitSwapAsm[7] =
{
  0x48, 0x8B, 0xC1,        // mov rax, rcx
  0x48, 0x0F, 0xC8,        // bswap rax
  0xC3            // retn
};
__int64 (*BitSwap)(__int64 Value) =  (__int64 (*)(__int64))
                                                   (ULONG_PTR) BitSwapAsm;

int _tmain(int argc, _TCHAR* argv[])
{
  //
  // I have to change the page protection, otherwise the code would crash
  //
  DWORD dwOldProtect;
  VirtualProtect(BitSwap, sizeof (BitSwapAsm), PAGE_EXECUTE_READWRITE,
                 &dwOldProtect);

  printf_s("%p\n", BitSwap(0xDDCCBBAA));
  getchar();
}

This code relies on function pointers and I had to change the page protection flags in order to make it execute. It's really a dumb method, but in some case it could be time saving.

Windows On Windows

Of course, compatibility for 32bit applications has to be provided on x64 (and Itanium as well) and this is what WOW64 (Windows on Windows 64) is all about. When we look at the modules loaded by a 32bit application with a 32bit version of the Task Explorer we see this:

Seems pretty regular, except, of course, for the system files path, which in our case is syswow64 instead of the old common System32. It's easy to understand why it is this way: the System32 folder is now reserved for the 64bit environment and the 32bit files had to be placed somewhere else. But look what happens when I open the same process with an x64 version of the Task Explorer:

Suddenly, all the 32bit modules are gone and what remains are the WOW64 emulation modules. Here's the description the MSDN gives us of these modules:

The WOW64 emulator runs in user mode, provides an interface between the 32-bit version of Ntdll.dll and the kernel of the processor, and it intercepts kernel calls. The emulator consists of the following DLLs:

Wow64.dll provides the core emulation infrastructure and the thunks for the Ntoskrnl.exe entry-point functions.
Wow64Win.dll provides thunks for the Win32k.sys entry-point functions.
Wow64Cpu.dll provides x86 instruction emulation on Itanium processors. It executes mode-switch instructions on the processor. This DLL is not necessary for x64 processors because they execute x86-32 instructions at full clock speed.

Along with the 64-bit version of Ntdll.dll, these are the only 64-bit binaries that can be loaded into a 32-bit process.At startup, Wow64.dll loads the x86 version of Ntdll.dll and runs its initialization code, which loads all necessary 32-bit DLLs. Almost all 32-bit DLLs are unmodified copies of 32-bit Windows binaries. However, some of these DLLs are written to behave differently on WOW64 than they do on 32-bit Windows [...].

Instead of using the x86 system-service call sequence, 32-bit binaries that make system calls are rebuilt to use a custom calling sequence. This new sequence is inexpensive for WOW64 to intercept because it remains entirely in user mode. When the new calling sequence is detected, the WOW64 CPU transitions back to native 64-bit mode and calls into Wow64.dll. Thunking is done in user mode to reduce the impact on the 64-bit kernel, and to reduce the risk of a bug in the thunk that causes a kernel-mode crash, data corruption, or a security hole. The thunks extract arguments from the 32-bit stack, extend them to 64 bits, then make the native system call.

32bit applications have a maximal 2GB space (4GB if explicitly required) and the rest of the space is handled by the system. This doesn't change much of course, since on x86 user mode applications had 2GB of virtual memory space out of 4GB (the other 2GB were reserved for kernel mode). On x64 these two other GB can now be accessed by 32bit applications. In order to achieve this, the IMAGE_FILE_LARGE_ADDRESS_AWARE flag has to be set in the File Header's Characteristics field. You can do this programmatically or manually with a normal PE editor like the CFF Explorer, just like this:

I've seen this done by 3D-games players in order to increase performances. Of course, it's only useful for very heavy memory consuming applications.

A very useful function to determine whether a process is running under WOW64 or not is:

BOOL IsWow64Process(
HANDLE hProcess,   // [in] Handle to a process. 
PBOOL Wow64Process // [out] Pointer to a value that is set to TRUE if the 
                   // process is running under WOW64. Otherwise, the value 
                   // is set to FALSE. 
);

The work done by Wow64Cpu.dll on x64 is zero, because x64 supports x86 natively. I was first tempted to look how the calling sequence works in order to make one myself and provide a way to use x86 components from x64 in the same address space, but, on second thought, even if it could be implemented, it wouldn't work on Itanium. And this brings us to one of the next paragraphs, because under normal conditions a 32bit application cannot load a 64bit dll and a 64bit application cannot load a 32bit dll. So, interprocess communication becomes an important aspect on 64bit systems. Anyway, before that, I have to talk about file system and registry redirection, since they are strictly related to WOW64, but deserve an extra paragraph for their importance.

File System And Registry Redirection

Since the System32 path is reserved to 64bit files, any time a 32bit application tries to access this directory it is redirected to SysWow64 one. However, there are some subdirectories of System32 that are shared between 32bit and 64bit applications and so no redirection is needed. These subdirectories are:

%windir%\system32\catroot
%windir%\system32\catroot2
%windir%\system32\drivers\etc
%windir%\system32\logfiles
%windir%\system32\spool

Also, there are some functions related to the WOW64 file system redirection:

`GetSystemWow64Directory`	Retrieves the path of the system directory used by WOW64. This directory is not present on 32-bit Windows.
`Wow64DisableWow64FsRedirection`	Disables file system redirection for the calling thread. File system redirection is enabled by default.
`Wow64EnableWow64FsRedirection`	Enables or disables file system redirection for the calling thread. This function may not work reliably when there are nested calls. Therefore, this function has been replaced by the Wow64DisableWow64FsRedirection and Wow64RevertWow64FsRedirection functions.
`Wow64RevertWow64FsRedirection`	Restores file system redirection for the calling thread.

I think it's easy to understand how to use these functions. However, I add a little code sample (you can find almost the same one on the MSDN):

Collapse

int _tmain(int argc, _TCHAR* argv[])
{
  BOOL bIsWOW64Enabled;

  if (IsWow64Process(GetCurrentProcess(), &bIsWOW64Enabled))
  {
      if (bIsWOW64Enabled == TRUE) // we run under WOW64
      {
          PVOID pOldValue;
          DWORD FileSize;

          HANDLE hFile = CreateFile(_T("c:\\windows\\system32\\notepad.exe"),
              GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);

          FileSize = GetFileSize(hFile, NULL);

          CloseHandle(hFile);

          _tprintf(_T("File Size: %d Bytes\n"), FileSize);

          Wow64DisableWow64FsRedirection(&pOldValue); // disable redirection

          hFile = CreateFile(_T("c:\\windows\\system32\\notepad.exe"),
                             GENERIC_READ, FILE_SHARE_READ, NULL,
                             OPEN_EXISTING, 0, NULL);

          FileSize = GetFileSize(hFile, NULL);

          CloseHandle(hFile);

          _tprintf(_T("File Size: %d Bytes\n"), FileSize);

          Wow64RevertWow64FsRedirection(pOldValue);  // restore redirection

          getchar();
      }
  }

  return 0;
}

The output of this program is:

File Size: 151040 Bytes
File Size: 169472 Bytes

The file size changes because one time the program opens the 32bit notepad and one time the 64bit one. Of course, remember when you're using these functions, always use them along with GetProcAddress, otherwise your code won't work on older systems which don't provide them.

Let's move on to the registry. As for the file system the registry is being redirected as well, or better some keys of it. These keys are:

HKEY_LOCAL_MACHINE\Software
HKEY_USERS\*\Software\Classes
HKEY_USERS\*_Classes

You can find every one of these keys duplicated for 32bit applications in their WOW node: any of these keys has a subkey called Wow6432Node, which contains a duplicate of the parent key. For instance:

Some of these WOW64 redirected keys have subkeys which are reflected. Reflection in this case means that when I change a reflected key in the 32bit node the change is being reflected on the 64bit key as well and viceversa. This is necessary, because some keys need to remain in synch. This is quite different from just sharing the keys between 64bit and 32bit mode, because the reflection can be filtered and also disabled. These are the reflected keys:

HKEY_LOCAL_MACHINE\Software\Classes
HKEY_LOCAL_MACHINE\Software\Microsoft\COM3
HKEY_LOCAL_MACHINE\Software\Microsoft\EventSystem
HKEY_LOCAL_MACHINE\Software\Microsoft\Ole
HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc
HKEY_USERS\*\Software\Classes
HKEY_USERS\*_Classes

The functions to handle reflection are:

`RegQueryReflectionKey`	Determines whether reflection has been disabled or enabled for the specified key.
`RegDisableReflectionKey`	Disables registry reflection for the specified key. Disabling reflection for a key does not affect reflection of any subkeys.
`RegEnableReflectionKey`	Restores registry reflection for the specified disabled key. Restoring reflection for a key does not affect reflection of any subkeys.

They work just like the WOW64 file system functions, so I don't think a code sample is necessary. There are also some shared keys between 64bit and 32bit applications:

HKEY_LOCAL_MACHINE\SOFTWARE\Classes\HCP
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\Calais\Current
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\Calais\Readers
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\Services
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CTF\SystemShared
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CTF\TIP
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DFS
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Driver Signing
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\EnterpriseCertificates
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSMQ
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Non-Driver Signing
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RAS
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Software\Microsoft\Shared Tools\MSInfo
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\TermServLicensing
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Transaction Server
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontDpi
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontMapper
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontSubstitutes
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\NetworkCards
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Ports
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Control Panel\Cursors\Schemes
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Group Policy
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OC Manager
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Telephony\Locations
HKEY_LOCAL_MACHINE\SOFTWARE\Policies

As said, these keys are shared, so any change made to them will affect both 32bit and 64bit applications, and there's no way to avoid this like for reflected keys.

But what if a 32bit applications wants to access the 64bit registry or viceversa? Don't worry! As I discovered when I was dealing with the same problem, Microsoft provides a very simple way to do the job. The flags KEY_WOW64_64KEY and KEY_WOW64_32KEY can be used with these functions: RegCreateKeyEx, RegDeleteKeyEx and RegOpenKeyEx.

`KEY_WOW64_64KEY`	Access a 64-bit key from either a 32-bit or 64-bit application.
`KEY_WOW64_32KEY`	Access a 32-bit key from either a 32-bit or 64-bit application.

What I needed to do was to access the subkeys of a 64bit key from a 32bit application, which translated in code is just:

RegOpenKeyEx(HKEY_LOCAL_MACHINE, MyKey, 0, KEY_READ | KEY_WOW64_64KEY, &hKey);

Easy, isn't it?

All in all, the documentation provided by Microsoft on file system and registry redirection is very good and I just reported what I first found on the MSDN. I don't think these redirections are going to be much of a problem for programmers.

Interprocess Communication

As mentioned in the Windows On Windows paragraph, interprocess communication becomes an important aspect on x64, since a 64bit application might need to use a 32bit component and viceversa. The MSDN suggests these ways for process to communicate between each other:

Handles to named objects such as mutexes, semaphores, and file handles can all be shared.
Handles to windows (HWND) can be shared.
RPC.
COM LocalServers.
Shared memory can be used if the contents of the shared memory are not pointer-dependent.
The CreateProcess and ShellExecute functions can launch 32-bit and 64-bit processes from either 32-bit or 64-bit processes.
The CreateRemoteThread function is special-cased for specific functions, allowing 64-bit debuggers to break into 32-bit processes.

Using CreateProcess or ShellExecute means that you could communicate through arguments and output reading. If you need something more sofisticated (and professional), you have no choice but to use RPCs (Remote Procedure Calls) or COM objects. For RPCs you need to learn a bit about the MIDL (Microsoft Interface Definition Language), but eventually every code sample I tried wasn't working on Vista x64, so I gave up on RPCs. I would suggest you to use a COM, writing them in MFC is very easy (comparing to writing them without MFC, I mean). There's a very good series of articles on CodeProject about writing ActiveXs. Actually, the guide is about how writing ActiveXs in plain C (I had to reduce the size of my ActiveX, so I couldn't use MFC), but the theory is the same and these articles are well written and could save you from the effort of reading a book. If you have never written COM objects before, you will eventually discover that it can be annoying.

Shared memory is not really an option. If you are looking for a solution between CreateProcess and COM objects, you may use pipes or things like that. Actually, you could implement your own pipes through shared memory and mutexes. This is what I have done in some projects:

The " *32" next to the process name is the way of the Task Manager to tell us which are 32bit processes. As you can see the Server is a 64bit process and the Client a 32bit one. The two processes communicate with each other without problems. However, don't get too excited, there are some problems and I'll explain later what they are about. For now, let's see a code sample (communication.zip download available at the top of the article)

Here's the Client code:

Collapse

#include 
#include 

#define BUF_SIZE  256 * sizeof (TCHAR)

TCHAR MyEvent[] = _T("Global\\SharedMemoryEvent");

TCHAR szName[]= _T("Global\\MyFileMappingObject");

int WINAPI _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                  LPSTR szCmdLine, int iCmdShow)
{
  //
  // Create the event to communicate between server and client
  //

  HANDLE hEvent = CreateEvent(NULL, FALSE, FALSE, MyEvent);

  //
  // Start server process
  //

  PROCESS_INFORMATION pi = { 0 };
  STARTUPINFO si = { 0 };

  if (!CreateProcess(_T("Server.exe"), NULL, NULL, NULL, FALSE, 0, NULL,
                     NULL, &si, &pi))
      return 1;

  //
  // Wait for the server to complete the job
  //

  WaitForSingleObject(hEvent, INFINITE);

  //
  // Access shared memory object
  //

  HANDLE hMapFile = OpenFileMapping(
      FILE_MAP_ALL_ACCESS,   // read/write access
      FALSE,                 // do not inherit the name
      szName);               // name of mapping object 

  if (hMapFile == NULL) return 1;

  LPCTSTR pBuf = (LPTSTR) MapViewOfFile(
      hMapFile,              // handle to map object
      FILE_MAP_ALL_ACCESS,   // read/write permission
      0,                  
      0,                  
      BUF_SIZE);                 

  if (pBuf == NULL) return 1;

  //
  // Shows Server Output
  //

  MessageBox(NULL, pBuf, _T("Server Output"), MB_OK);

  UnmapViewOfFile(pBuf);

  CloseHandle(hMapFile);

  //
  // Tell the server that the object isn't used any longer
  //

  SetEvent(hEvent);

  return 0;
}

And here's the Server code:

Collapse

#include 
#include 

#define BUF_SIZE  256 * sizeof (TCHAR)

TCHAR MyEvent[] = _T("Global\\SharedMemoryEvent");

TCHAR szName[] = _T("Global\\MyFileMappingObject");
TCHAR szMsg[] = _T("Message from server process");

int WINAPI _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                   LPSTR szCmdLine, int iCmdShow)
{
  //
  // Create the memory shared object
  //

  HANDLE hMapFile;
  LPCTSTR pBuf;

  hMapFile = CreateFileMapping(
      INVALID_HANDLE_VALUE,    // use paging file
      NULL,                    // default security 
      PAGE_READWRITE,          // read/write access
      0,                       // max. object size 
      BUF_SIZE,                // buffer size  
      szName);                 // name of mapping object

  if (hMapFile == NULL) return 1;

  pBuf = (LPTSTR) MapViewOfFile(hMapFile,   // handle to map object
      FILE_MAP_ALL_ACCESS,             // read/write permission
      0,                 
      0,                 
      BUF_SIZE);         

  if (pBuf == NULL) return 1;

  CopyMemory((PVOID) pBuf, szMsg, (_tcslen(szMsg) + 1) * sizeof (TCHAR));

  //
  // Wait for event before closing file object
  //

  HANDLE hEvent = OpenEvent(EVENT_ALL_ACCESS, FALSE, MyEvent);

  SetEvent(hEvent);

  WaitForSingleObject(hEvent, INFINITE);

  UnmapViewOfFile(pBuf);
  CloseHandle(hMapFile);

  return 0;
}

What these applications do is:

The client creates a communication event.
The client starts the server and waits for the communication event to be set.
The server creates a shared memory object and fills it with an output.
The server sets the communication event in order to tell the client to process the output.
The server waits for the client to clear the shared memory.
The client processes the server's output.
The client tells the server that it can now clear the shared memory.

I believe it's easier to understand the code itself than this list. The problem I mentioned earlier is that in order to share a memory object (or an event) between processes, I have to create it in the "Global\\*" section. What happens with Vista is that only applications with admin privileges can access this section with CreateFileMapping (no problems with mutexes or events, though), and since usually applications run in Vista with user privileges, you have to explicitly tell Vista to run the Client application with admin privileges, which is not very professional. The solution to this problem could be to share the memory through a temporary file or even the registry (for small data).

Portable Executable

If your software has anything to do with Portable Executables it won't be too hard to move to x64 (if you haven't done it already). Basically, what in PE64 changes is the size of virtual addresses (VAs), which are now 64bit wide. Keep in mind that not all the fields described as virtual addresses really are such, most of the time they're just relative virtual addresses (RVAs), which are, like in the PE32, 32bit wide. What changes, in short, is the Optional Header (which has some 64bit wide fields like the ImageBase), Import Directory thunks (the two thunk arrays. OFTs and FTs, are now 64bit wide, since thunks were built to contain virtual addresses among the other things), the Load Config Directory and the TLS Directory.

Let's take, for instance, the old PE32 Optional Header:

Collapse

typedef struct _IMAGE_OPTIONAL_HEADER {
  //
  // Standard fields.
  //

  WORD    Magic;
  BYTE    MajorLinkerVersion;
  BYTE    MinorLinkerVersion;
  DWORD   SizeOfCode;
  DWORD   SizeOfInitializedData;
  DWORD   SizeOfUninitializedData;
  DWORD   AddressOfEntryPoint;
  DWORD   BaseOfCode;
  DWORD   BaseOfData;

  //
  // NT additional fields.
  //

  DWORD   ImageBase;
  DWORD   SectionAlignment;
  DWORD   FileAlignment;
  WORD    MajorOperatingSystemVersion;
  WORD    MinorOperatingSystemVersion;
  WORD    MajorImageVersion;
  WORD    MinorImageVersion;
  WORD    MajorSubsystemVersion;
  WORD    MinorSubsystemVersion;
  DWORD   Win32VersionValue;
  DWORD   SizeOfImage;
  DWORD   SizeOfHeaders;
  DWORD   CheckSum;
  WORD    Subsystem;
  WORD    DllCharacteristics;
  DWORD   SizeOfStackReserve;
  DWORD   SizeOfStackCommit;
  DWORD   SizeOfHeapReserve;
  DWORD   SizeOfHeapCommit;
  DWORD   LoaderFlags;
  DWORD   NumberOfRvaAndSizes;
  IMAGE_DATA_DIRECTORY DataDirectory[IMAGE_NUMBEROF_DIRECTORY_ENTRIES];
} IMAGE_OPTIONAL_HEADER32, *PIMAGE_OPTIONAL_HEADER32;

And the PE64 one:

Collapse

typedef struct _IMAGE_OPTIONAL_HEADER64 {
  WORD        Magic;
  BYTE        MajorLinkerVersion;
  BYTE        MinorLinkerVersion;
  DWORD       SizeOfCode;
  DWORD       SizeOfInitializedData;
  DWORD       SizeOfUninitializedData;
  DWORD       AddressOfEntryPoint;
  DWORD       BaseOfCode;
  ULONGLONG   ImageBase;
  DWORD       SectionAlignment;
  DWORD       FileAlignment;
  WORD        MajorOperatingSystemVersion;
  WORD        MinorOperatingSystemVersion;
  WORD        MajorImageVersion;
  WORD        MinorImageVersion;
  WORD        MajorSubsystemVersion;
  WORD        MinorSubsystemVersion;
  DWORD       Win32VersionValue;
  DWORD       SizeOfImage;
  DWORD       SizeOfHeaders;
  DWORD       CheckSum;
  WORD        Subsystem;
  WORD        DllCharacteristics;
  ULONGLONG   SizeOfStackReserve;
  ULONGLONG   SizeOfStackCommit;
  ULONGLONG   SizeOfHeapReserve;
  ULONGLONG   SizeOfHeapCommit;
  DWORD       LoaderFlags;
  DWORD       NumberOfRvaAndSizes;
  IMAGE_DATA_DIRECTORY DataDirectory[IMAGE_NUMBEROF_DIRECTORY_ENTRIES];
} IMAGE_OPTIONAL_HEADER64, *PIMAGE_OPTIONAL_HEADER64;

Of course, ULONGLONG are 64bit wide fields. As you can see, the AddressOfEntryPoint remains, as every RVA, a dword. Conversely, ImageBase, being a Virtual Address, becomes a qword.

Distinguishing between PE32 and PE64 should be done by checking the Magic field in the Optional Header. This field can be one of these values:

#define IMAGE_NT_OPTIONAL_HDR32_MAGIC      0x10b
#define IMAGE_NT_OPTIONAL_HDR64_MAGIC      0x20b
#define IMAGE_ROM_OPTIONAL_HDR_MAGIC       0x107

It is your choice to either double write every time the code to handle both PE32/64 or write a class to handle them automatically.

Exception Handling

Remember the old days when you set the SEH in your code? Well, with x64/Itanium they're gone. Exception Handlers are now stored as structured in the PE64 Exception Directory. The basic structure is this:

typedef struct _RUNTIME_FUNCTION {
  DWORD BeginAddress;
  DWORD EndAddress;
  DWORD UnwindData;
} RUNTIME_FUNCTION, *PRUNTIME_FUNCTION;

All three fields are RVAs (otherwise there wouldn't be dwords).

`BeginAddress`	Points to the start address of the involved part of code.
`EndAddress`	Points to the end address of the same part of code.
`UnwindData`	Points to an UNWIND_INFO structure.

The UNWIND_INFO structure tells how the portion of code should be handled. Here's the declaration I found on MSDN:

typedef union _UNWIND_CODE {
  struct {
      UBYTE CodeOffset;
      UBYTE UnwindOp : 4;
      UBYTE OpInfo   : 4;
  };
  USHORT FrameOffset;
} UNWIND_CODE, *PUNWIND_CODE;

typedef struct _UNWIND_INFO {
  UBYTE Version       : 3;
  UBYTE Flags         : 5;
  UBYTE SizeOfProlog;
  UBYTE CountOfCodes;
  UBYTE FrameRegister : 4;
  UBYTE FrameOffset   : 4;
  UNWIND_CODE UnwindCode[1];
/*  UNWIND_CODE MoreUnwindCode[((CountOfCodes + 1) & ~1) - 1];
*   union {
*       OPTIONAL ULONG ExceptionHandler;
*       OPTIONAL ULONG FunctionEntry;
*   };
*   OPTIONAL ULONG ExceptionData[]; */
} UNWIND_INFO, *PUNWIND_INFO;

Here's the description of the UNWIND_INFO structure members taken directly from the MSDN:

`Version`	Version number of the unwind data, currently 1.
`Flags`	Three flags are currently defined: `UNW_FLAG_EHANDLER` The function has an exception handler that should be called when looking for functions that need to examine exceptions. `UNW_FLAG_UHANDLER` The function has a termination handler that should be called when unwinding an exception. `UNW_FLAG_CHAININFO` This unwind info structure is not the primary one for the procedure. Instead, the chained unwind info entry is the contents of a previous RUNTIME_FUNCTION entry. See the following text for an explanation of chained unwind info structures. If this flag is set, then the UNW_FLAG_EHANDLER and `UNW_FLAG_UHANDLER` flags must be cleared. Also, the frame register and fixed-stack allocation fields must have the same values as in the primary unwind info.
`SizeOfProlog`	Length of the function prolog in bytes.
`CountOfCodes`	This is the number of slots in the unwind codes array. Note that some unwind codes (for example, `UWOP_SAVE_NONVOL`) require more than one slot in the array.
`FrameRegister`	If nonzero, then the function uses a frame pointer, and this field is the number of the nonvolatile register used as the frame pointer, using the same encoding for the operation info field of `UNWIND_CODE` nodes.
`FrameOffset`	If the frame register field is nonzero, then this is the scaled offset from RSP that is applied to the FP reg when it is established. The actual FP reg is set to RSP + 16 * this number, allowing offsets from 0 to 240. This permits pointing the FP reg into the middle of the local stack allocation for dynamic stack frames, allowing better code density through shorter instructions (more instructions can use the 8-bit signed offset form).
`UnwindCode`	This is an array of items that explains the effect of the prolog on the nonvolatile registers and RSP. See the section on `UNWIND_CODE` for the meanings of individual items. For alignment purposes, this array will always have an even number of entries, with the final entry potentially unused (in which case the array will be one longer than indicated by the count of unwind codes field).
`ExceptionHandler`	This is an image-relative pointer to either the function's language-specific exception/termination handler (if flag `UNW_FLAG_CHAININFO` is clear and one of the flags `UNW_FLAG_EHANDLER` or `UNW_FLAG_UHANDLER` is set).
`Language-specific handler data (ExceptionData)`	This is the function's language-specific exception handler data. The format of this data is unspecified and completely determined by the specific exception handler in use.
`Chained Unwind Info (ExceptionData)`	If flag `UNW_FLAG_CHAININFO` is set then the `UNWIND_INFO` structure ends with three `UWORDs`. These `UWORDs` represent the `RUNTIME_FUNCTION` information for the function of the chained unwind.

The possible values of the Flags field are:

#define UNW_FLAG_EHANDLER  0x01
#define UNW_FLAG_UHANDLER  0x02
#define UNW_FLAG_CHAININFO 0x04

Let's take for instance this code:

#include 
#include 
#include 

int APIENTRY _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                LPTSTR lpCmdLine, int nCmdShow)
{
 __try
 {
    __debugbreak();
 }
 __except (EXCEPTION_EXECUTE_HANDLER)
 {
    MessageBox(0, _T("Hello!"), _T("SEH"), 0);
 }

 return 0;
}

The dissassembly would be:

.text:0000000000401000 wWinMain proc near  ; CODE XREF: __tmainCRTStartup+18C p
.text:0000000000401000    sub rsp, 28h     ; BeginAddress 
.text:0000000000401004    int 3            ; Trap to Debugger
.text:0000000000401005    jmp short loc_401021
.text:0000000000401007 ; ---------------------------------------------------
.text:0000000000401007    xor r9d, r9d     ; ExceptionHandler
.text:000000000040100A    lea r8, Caption         
.text:0000000000401011    lea rdx, Text           
.text:0000000000401018    xor ecx, ecx            
.text:000000000040101A    call cs:__imp_MessageBoxW
.text:0000000000401020    nop
.text:0000000000401021
.text:0000000000401021 loc_401021:         ; CODE XREF: wWinMain+5 j
.text:0000000000401021    xor eax, eax
.text:0000000000401023    add rsp, 28h
.text:0000000000401027    retn
.text:0000000000401027 wWinMain endp       ; EndAddress (+ alignment)

If you need to generate code dinamically and set for it an exception handler, you can use the function RtlAddFunctionTable, which takes as paramater an array of RUNTIME_FUNCTION structures. This means, of course, that you'll have to fill one or more UNWIND_INFO structure/s by yourself. It's certainly a bit more complicated than on x86, but my guess is that a lot of software protections are going to use this method.

.NET Framework

Both the x86 and x64 .NET frameworks can coexist peacefully on a x64 Windows. In case you install both, there will be two things of everything: two directories in the .NET directory, two directories in the global cache and two main registry keys. Since .NET assemblies don't contain native code, why could it be useful to have both frameworks on the same computer?

.NET assemblies can call native code through the System.Runtime.InteropServices namespace. Of course, a .NET assembly which runs on the x64 .NET framework is a 64bit process, therefore it can't call x86 components functions. Viceversa, assemblies executed on x86 can't use x64 components. You can explicitly tell the Visual Studio to compile your assembly for a specific platform (x86, x64, Itanium) by just going on Project -> Properties -> Build.

64bit PEs are always built for a specific platform, only PE32 assemblies are allowed to run on every framework (x86 systems wouldn't be able to execute a PE64). Anyway, it's possible to make PE32 assemblies run just on the x86 framework by just setting one flag in the .NET Directory (IMAGE_COR20_DIRECTORY). The flag is COMIMAGE_FLAGS_32BITREQUIRED. By setting this flag you'll force to execute the given assembly as a 32bit process even on 64bit platforms.

There are also some differences between the 32bit and 64bit .NET framework. I noticed that the 64bit one is very serious about alignments and integrity cheks in assemblies, and the new 3.0 framework has even more checks.

Vista Section

Editions

Windows Vista is shipped in various editions, before you install the new system, you should be aware of the features missing in some editions. Here's the official features table given by Microsoft:

Features	Home Basic	Home Premium	Business	Ultimate
Most secure Windows ever with Windows Defender and Windows Firewall
Quickly find what you need with Instant Search and Windows Internet Explorer 7
Elegant Windows Aero desktop experience with Windows Flip 3D navigation
Best choice for laptops with enhanced Windows Mobility Center and Tablet PC support
Collaborate and share documents with Windows Meeting Space
Experience photos and entertainment in your living room with Windows Media Center
Enjoy Windows Media Center on TVs throughout your home with Xbox 360™ and other devices
Help protect against hardware failure with advanced business backup features
Business networking and Remote Desktop for easier connectivity
Better protect your data against loss or theft with Windows BitLocker™ Drive Encryption

But there are other things that the common user might not notice. For instance, the Business and Ultimate edition allow virtualization (using the system as a virtual machine). The End-User License Agreement for the Vista Home Basic and Premium reads:

4. USE WITH VIRTUALIZATION TECHNOLOGIES. You may not use the software installed on the licensed device within a virtual (or otherwise emulated) hardware system.

Apparentely, Microsoft put special checks in the home editions, in order to prevent them from working in emulation. I read that VMWare was quite disappointed by this policy adopted by Microsoft. This controversy might change things, but I think many programmers and companies should know this before buying one or another Vista edition.

Microsoft Visual Studio

I chose this as the second paragraph of the Windows Vista section, because what every programmer first does when he has a brand new system is to install and set up his compilers. The problem is, since many things change with Vista, the only compatible Visual Studio platform is the 2005 one. Compatibility for Visual Studio 6 and Visual Studio .NET 2003 is no longer provided (although Visual Basic 6 seems to be supported). Not only that, in order to make the VS.NET 2005 work, you'll need to download and install the Service Pack 1 for it.

For a lot of programmers like myself it's not a big deal, since maintaining the code up to date is very important, but who has ever worked with little companies knows that a lot of them have no interest in doing the same, with the result that, for instance, many solutions are still developed for the .NET framework 1.0. Those solutions will, of course, still run on Vista, but there won't be any tool to compile them. I don't think Microsoft is going to solve this issue. Thus, for many companies it will take some time to switch to Vista.

P.S. The setup of Microsoft Visual Studio's Service Pack 1 will take much time. Don't worry, it's normal.

User Account Control

What stands in the way of working properly for most applications on Windows Vista is the User Account Control (UAC), also known as Limited User Access (LUA). As we saw in the Interprocess Communication paragraph, admin rights to create shared memory objects are necessary, but Windows Vista runs every process (except system processes) with user rights. Incompatibilities are, most of the times, generated by programmers false assumption that their code will run on admin level. Programs which worked without problems with user rights on Windows NT 4-5.1 won't have any problems running properly on Vista. However, common mistakes (or bad habits) like:

Modifying files in their own Program Files directory.
Writing in the HKEY_LOCAL_MACHINE to store settings.

Are no longer problems, since on Vista there's a thing called virtualization. Basically, every file modified in a system directory, like Program Files, is actually stored in a directory called Virtual Store. My path for this directory is C:\Users\Daniel\AppData\Local\VirtualStore. The Windows Explorer will show you those files in the system directory, but actually they are all in the VirtualStore, which, and it goes without saying, is unique for every user. The files are just stored like this in the Virtual Store:

The directory hierarchy in the Virtual Store is exactly the same it would be without virtualization. I guess you have already figured out how it works.

Just like for the File System, there's also a registry virtualization. Everything written in the key HKEY_LOCAL_MACHINE\Software will actually be stored under HKEY_CLASSES_ROOT\VirtualStore\Software.

What you should keep in mind is that eveyrthing that might affect other users is no longer possible in a standard execution of your software. This means, apart from the things said above, you won't be able to load drivers, modify certain files (maybe not even read them), modify or read specific registry keys/values, access certain global objects, enumerate or modify the memory of processes which run with higher privileges than you do, etc. And you won't even be able to enumerate or send messages to windows created by those processes. This prevents exploits we have already seen in the past. There's a very good MSDN paper (Developer Best Practices and Guidelines for Applications in a Least Privileged Environment) about the UAC and its conseguences for developers, it's a detailed explanation of everything I tried to say here in short.

Compatibility Verification

There's a tool to verify if your application is compatible with the UAC. This tool is called Microsoft Application Verifier and it can also detect other issues, but right now we're not interested in those ones. Using it is very easy, all you need is to add an application to the list and select what kind of checks the verifier should do. In our case, we'll select the LUA (aka UAC) compatibility check.

The next step is running the application you added to the list. When you're done, you save the log and then, as you can read from the screenshot above, you open it with the Microsoft Standard User Analyzer. Don't even think about opening it with another application, depending on the size of your software the verifier will generate a gigantic xml log, which would make every other program consume a big part of your ram. For this test I used the Task Explorer which, of course, tries to adjust its token to SeDebugPrivilege in order to list even system processes. In fact:

Since this application is not running with admin rights, it won't be able to acquire debug privileges. I wouldn't recommend you to use always this tool, since following good design rules should be enough. However, it might be useful.

Obtaining Admin Rights

Of course, it's still possible to run an application with admin rights. You can either do that manually or through code. To do it manually, you can just right-click on the application an then click "Run as administrator" or check the "Run this program as administrator" box under Properties -> Compatibility. However, programmers cannot expect users to do these operations by themselves, so they'll need another way.

A very easy way is to tell the system the requested execution level through a manifest file. I suppose the reader knows what a manifest file is and how to integrate it in an application. There are plenty of guides about this subject and I don't think it should be re-discussed here.

The schema of such a manifest file should be like this one (this is actually for x86, as you see under processorArchitecture):





Description of your application

The available values for level are:

`asInvoker`	The application runs with the same token as the parent process.
`highestAvailable`	The application runs with the highest privileges the current user can obtain.
`requireAdministrator`	The application runs only for administrators and requires that the application be launched with the full token of an administrator.

And for uiAccess:

`false`	The application does not need to drive input to the UI of another window on the desktop. Applications that are not providing accessibility should set this flag to false. Applications that are required to drive input to other windows on the desktop (on-screen keyboard, for example) should set this value to true.
`true`	The application is allowed to bypass UI protection levels to drive input to higher privilege windows on the desktop. This setting should only be used for UI Accessibility applications.

I wrote a very small application (which just waits for an input key) to demonstrate the use of a manifest file. The download can be found at the top of this article.

There's also a function, CredUIPromptForCredentials, to request specific credentials from a user and can be used to obtain admin rights, but it's not very comfortable for the user himself.

Disable It

If you are annoyed by all these dialogs asking you for permissions, you can disable the UAC. There are two simple ways I'm aware of. The first one described in detail on the UAC Team Blog is about going to Control Panel -> Administrative Tools -> Local Security Policy. In the tree on the left click Local Policies and then Security Options. Scroll until you find: "User Account Control: behavior of the elevation prompt for administrators". Change the current value to: "Elevate without prompting". Just like this:

The other method is more drastic. It's about removing completely the UAC and requires a machine's reboot. In Administrative Tools, instead of clicking on Local Security Policy, click on System Configuration. Choose the tab "Tools" in the dialog and scroll until you find "Disable UAC" under "Tool Name". To execute the command, click on Launch.

As you can see from the command line, it simply executes reg.exe to add a key/value. The full command line is:

C:\Windows\System32\cmd.exe /k %windir%\System32\reg.exe
ADD HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
/v EnableLUA /t REG_DWORD /d 0 /f

To re-enable the UAC, it sets the EnableLUA dword to 1.

I don't know whether it's a good idea or not to disable the UAC. Sure is that a lot of programmers will be too annoyed to keep it enabled. There's been a lot of criticism about this security system. However, I personally believe it's the only one to guarantee a minimum of security on an account where one has still admin rights. If one needs more security, one shall switch to a normal user account.

Address Space Layout Randomization

In short ASLR. There's an interesting weblog about this new feature on Vista. Randomization, as you know, is useful to prevent buffer overflow exploits and has already been implemented on other operating systems. On Vista randomization is done on various levels:

Image Base
Stack
Heap

With the new Service Pack 1 for Visual Studio 2005 the option /dynamicbase has been introduced. This options makes it possible to relocate executables just like dynamic libraries, just by adding a relocation section to the PE header. All executables on Windows Vista have been compiled with this option, making it impossible for a programmer to assume where specific data is placed. Randomization for images has 256 variations. The reason for this are explained by Michael Howard:

On a final note, it is true that we don't have as much randomization as PaX and other more aggressive ASLR implementations. For instance, image randomization is only 8 bits (1 of 256 variations). Images have to be 64K aligned, and so on a 32-bit system we could have theoretically randomized images by up to 15 bits (1 of 32, 768 variations), but the incremental security gain is small - if you navigate to a Website and your browser crashes, will you go back to that site another 255 times - and would have come at the expense of fragmenting the entire address space, thereby reducing the contiguous memory available to applications and degrading system performance? We think we hit a nice balance.

I did some tests and it seems that on x86, unlike on x64, the image base changes only when rebooting. On x64 the imagebase seems changing with every execution. By the way, x64 shouldn't have problems with contiguous memory availability, since the address space is enormously larger than on x86. Unfortunately, I can't do more tests, because, at the moment, I'm running on Vista x86.

On the other hand, the stack should have 16,384 possible variations (14 bits). I wrote, for testing purposes, a little application to calculate stack variations on N executions. I wrote it very quick, so the implementation is also very rudimental. Basically, one executable calls another executable, compiled with the /dynamicbase option, N times and gets from it the stack address of a local variable. It then evaluates if the stack address has already been used or not; if not, it increments the number of variations. To communicate the stack address to the parent, the dynamic-base executable uses a SendMessage (which is enough, since wparam and lparam are the same size of a pointer). This application was compiled for both x86 and x64.

Here's the dynamic-base executable source:

#include 
#include 

#define WM_STACKADDRESS        (WM_USER + 100)

int APIENTRY _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
           LPTSTR lpCmdLine, int nCmdShow)
{
  int x;

  return (int) SendMessage(FindWindow(NULL, _T("RandTest")),
      WM_STACKADDRESS, 0, (LPARAM) &x);
}

And this is the main executable:

Collapse

#include 
#include 
#include "resource.h"

#define WM_STACKADDRESS        (WM_USER + 100)

ULONG_PTR *pAddresses = NULL;
UINT nAddresses = 0;

//
// Test Loop thread: not blocking for GUI
//

void TestLoop(ULONG_PTR p)
{
  UINT nExec = (UINT) p;

  STARTUPINFO si = { 0 };
  PROCESS_INFORMATION pi = { 0 };

  for (UINT x = 0; x < class="cpp-string">"DynBaseApp.exe"), NULL, NULL, NULL, FALSE, 0,
          NULL, NULL, &si, &amp;pi);

      WaitForSingleObject(pi.hProcess, 3000);
  }

  delete pAddresses;

  //
  // We're through with testing, show number of variations
  //

  TCHAR szMsg[100];

  wsprintf(szMsg, _T("The number of variations on %d executions is: %d."),
      nExec, nAddresses);

  MessageBox(NULL, szMsg, _T("Test Result"), MB_ICONINFORMATION);

  ExitThread(0);
}

LRESULT CALLBACK DlgProc(HWND hDlg, UINT uMsg, WPARAM wParam, LPARAM lParam)
{


  switch (uMsg)
  {

  case WM_CLOSE:
      {
          EndDialog(hDlg, FALSE);
          break;
      }

  case WM_COMMAND:
      {
          switch ((WORD) wParam)
          {

          case IDC_TEST:
              {
                  //
                  // If executions != 0 start the loop
                  //

                  TCHAR nStrExec[20];

                  GetDlgItemText(hDlg, ED_EXECUTIONS, nStrExec, 20);

                  UINT nExec = _tcstoul(nStrExec, NULL, 10);

                  if (nExec == 0) break;

                  pAddresses = new ULONG_PTR [nExec];
                  nAddresses = 0;

                  CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE) &TestLoop,
                      (LPVOID) (ULONG_PTR) nExec, 0, NULL);

                  break;
              }
          }

          break;
      }

  case WM_STACKADDRESS:
      {
          //
          // It's a stack address being sended by DynBaseApp.exe
          // add it to the list if not already there
          //

          ULONG_PTR Addr = (ULONG_PTR) lParam;

          for (UINT x = 0; x < class="cpp-keyword">if (Addr == pAddresses[x]) return FALSE;
          }

          pAddresses[nAddresses] = Addr;
          nAddresses++;

          break;
      }
  }

  return FALSE;
}

int APIENTRY _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
           LPTSTR lpCmdLine, int nCmdShow)
{
  return (int) DialogBox(hInstance, (LPCTSTR) IDD_RANDTEST, NULL,
                        (DLGPROC) DlgProc);
}

The results were pretty good. On 16,385 execution the minimum I got was 4026 (on x64). With less executions the results were, of course, even better (100/100, 199/200, 296/300).

To compile executables with the /dynamicbase option, just add "/dynamicbase" (without brackets) in the linker command line (Project -> Properties -> Linker -> Command Line -> Additional options).

Driver Signing

The big change with Vista is that drivers require now to be certificated to run (at least on 64bit), and in order to obtain the certification you have to ship an x64 version of your driver. x86-only submission are no longer accepted by WHQL (Windows Hardware Quality Labs). If you have no clue how to sign a driver there's a good doc on MSDN and, of course, the official page. Nevertheless, it's still possible to disable driver certification for testing and debugging reasons. Reboot your system and press F8 to get the Advanded Boot Options. Select "Disable Driver Signature Enforcement".

However, there's a lot more to be discussed if you're a device driver programmer or want to be one. In that case, I advise you to read the NT Insider and for this particular issue this article. It describes all the steps that have to be done to set up a Vista machine for driver debugging and testing.

Patch Guard

This paragraph (and child) shouldn't be for Vista only. However, there's been a lot of talking about Vista's Patch Guard. Patch Guard isn't news, it was first introduced for x64 (it's not available for x86) with Windows XP and 2003. I chose to put this paragraph in the Vista section, because the messing around with this technology brought some changes in Vista. What Patch Guard means is that it's no longer possible to patch kernel data, not even by trusted components. In fact, some companies, like Symantec, protested against this technology and said that Microsoft was using it in order to prevent third-part developed security solutions from working. This is non-sense, of course. Microsoft security products don't patch Vista's kernel either and use instead documented interfaces as everyone else. Basically, Patch Guard checks the integrity of system data, and if it's corrupted it calls KeBugCheckEx causing the system to shut down. Things that trigger this behavior are:

Modifying system service tables (SDT).
Modifying the interrupt descriptor table (IDT).
Modifying the global descriptor table (GDT).
Using kernel stacks that are not allocated by the kernel.
Patching any part of the kernel (detected only on AMD64-based systems).

Patch Guard is disabled only when debugging the system. If your product relies on patching the kernel, you might be able to use an alternative method like suggested in this weblog.

Clearly, customers demand effective security solutions, and they can be developed without relying on kernel patching techniques. Some of the alternatives to kernel patching are:

Windows Vista includes the "Windows Filtering Platform", which enables software to perform network oriented activities such as packet inspection and other activities necessary to support firewall products.
The file system mini filter model allows software to participate in file system activities, which can be used by Anti-Virus software.
Registry notification hooks, introduced in Windows XP, and recently enhanced in Windows Vista, allow software to participate in registry related activities in the system.

I'll discuss the third alternative in the Registry Filtering paragraph.

Attacks

Some efforts have been made in this direction, since Patch Guard is a software implementation. Uninformed bypassed the Patch Guard protection on Windows XP x64. Joanna Rutkowska bypassed Vista RC1's Patch Guard by patching (on disk) the pagefile of a driver. This attack was done in user mode and therefore requires the ability to open a disk through CreateFile with write permissions and then modify data through WriteFile. It seemed that Vista's RC2 solved the issue by preventing the disk to be modified from user mode (even with high privileges). However, I've done some tests and CreateFile still returns a valid handle. The modification of some disk parts, like the boot sector, is still allowed as I could see. Nevertheless, if your utility relies on raw write access to the disk, you might encounter some problems. Fact is, this limitation doesn't even solve the issue, because theoretically the same trick could still be used in kernel mode.

Registry Filtering

Since it's no longer possible patching the Service Descriptor Table (SDT) on x64, one might be wondering how Mark Russinovich's Regmon (alias the new Process Monitor) works on x64. The answer is simple, starting with Windows XP there's no need to hook the SDT anymore. In fact, with Windows XP a new official and documented way has been introduced to filter the registry. This method relies on three functions: CmRegisterCallback (supported on XP and Vista), CmRegisterCallbackEx (supported on Vista only) and CmUnRegisterCallback. CmRegisterCallback/Ex registers a callback function for every registry operation. The callback function looks like this:

NTSTATUS RegistryCallback(
  IN PVOID  CallbackContext,
  IN PVOID  Argument1,
  IN PVOID  Argument2
  );

These are the parameters:

CallbackContext	The value that the driver passed as the Context parameter to CmRegisterCallback or CmRegisterCallbackEx when it registered this RegistryCallback routine.
Argument1	A REG_NOTIFY_CLASS-typed value that identifies the type of registry operation that is being performed and whether the RegistryCallback routine is being called before or after the register operation is performed.
Argument2	A pointer to a structure that contains information that is specific to the type of registry operation. The structure type depends on the REG_NOTIFY_CLASS-typed value for Argument1, as shown in the following table. For information about which REG_NOTIFY_CLASS-typed values are available for which operating system versions, see REG_NOTIFY_CLASS.

Argument1 carries the registry operation and Argument2 is a pointer to a structure. Here's the list of operations and their structures:

Operation	Structure
`RegNtDeleteKey`	`REG_DELETE_KEY_INFORMATION`
`RegNtPreDeleteKey`	`REG_DELETE_KEY_INFORMATION`
`RegNtPostDeleteKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtSetValueKey`	`REG_SET_VALUE_KEY_INFORMATION`
`RegNtPreSetValueKey`	`REG_SET_VALUE_KEY_INFORMATION`
`RegNtPostSetValueKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtDeleteValueKey`	`REG_DELETE_VALUE_KEY_INFORMATION`
`RegNtPreDeleteValueKey`	`REG_DELETE_VALUE_KEY_INFORMATION`
`RegNtPostDeleteValueKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtPostDeleteValueKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtSetInformationKey`	`REG_SET_INFORMATION_KEY_INFORMATION`
`RegNtPreSetInformationKey`	`REG_SET_INFORMATION_KEY_INFORMATION`
`RegNtPostSetInformationKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtRenameKey`	`REG_RENAME_KEY_INFORMATION`
`RegNtPreRenameKey`	`REG_RENAME_KEY_INFORMATION`
`RegNtPostRenameKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtEnumerateKey`	`REG_ENUMERATE_KEY_INFORMATION`
`RegNtPreEnumerateKey`	`REG_ENUMERATE_KEY_INFORMATION`
`RegNtPostEnumerateKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtEnumerateValueKey`	`REG_ENUMERATE_VALUE_KEY_INFORMATION`
`RegNtPreEnumerateValueKey`	`REG_ENUMERATE_VALUE_KEY_INFORMATION`
`RegNtPostEnumerateValueKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtQueryKey`	`REG_QUERY_KEY_INFORMATION`
`RegNtPreQueryKey`	`REG_QUERY_KEY_INFORMATION`
`RegNtPostQueryKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtQueryValueKey`	`REG_QUERY_VALUE_KEY_INFORMATION`
`RegNtPreQueryValueKey`	`REG_QUERY_VALUE_KEY_INFORMATION`
`RegNtPostQueryValueKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtQueryMultipleValueKey`	`REG_QUERY_MULTIPLE_VALUE_KEY_INFORMATION`
`RegNtPreQueryMultipleValueKey`	`REG_QUERY_MULTIPLE_VALUE_KEY_INFORMATION`
`RegNtPostQueryMultipleValueKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreCreateKey`	`REG_PRE_CREATE_KEY_INFORMATION`
`RegNtPreCreateKeyEx`	`REG_CREATE_KEY_INFORMATION`
`RegNtPostCreateKey`	`REG_POST_CREATE_KEY_INFORMATION`
`RegNtPostCreateKeyEx`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreOpenKey`	`REG_PRE_OPEN_KEY_INFORMATION`
`RegNtPreOpenKeyEx`	`REG_OPEN_KEY_INFORMATION`
`RegNtPostOpenKey`	`REG_POST_OPEN_KEY_INFORMATION`
`RegNtPostOpenKeyEx`	`REG_POST_OPERATION_INFORMATION`
`RegNtKeyHandleClose`	`REG_KEY_HANDLE_CLOSE_INFORMATION`
`RegNtPreKeyHandleClose`	`REG_KEY_HANDLE_CLOSE_INFORMATION`
`RegNtPostKeyHandleClose`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreFlushKey`	`REG_FLUSH_KEY_INFORMATION`
`RegNtPostFlushKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreLoadKey`	`REG_LOAD_KEY_INFORMATION`
`RegNtPostLoadKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreUnLoadKey`	`REG_UNLOAD_KEY_INFORMATION`
`RegNtPostUnLoadKey`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreQueryKeySecurity`	`REG_QUERY_KEY_SECURITY_INFORMATION`
`RegNtPostQueryKeySecurity`	`REG_POST_OPERATION_INFORMATION`
`RegNtPreSetKeySecurity`	`REG_SET_KEY_SECURITY_INFORMATION`
`RegNtPostSetKeySecurity`	`REG_POST_OPERATION_INFORMATION`
`RegNtCallbackContextCleanup`	`REG_CALLBACK_CONTEXT_CLEANUP_INFORMATION`

According to the MSDN, pointers in these structures should be accessed in try/except blocks. The callback can prevent operations from being performed as well (it's a real filter). To do that on XP, it just has to return a value different from STATUS_SUCCESS. Unfortunately, by doing this, the thread which originally called the registry function will get the same error as well. That's why on Vista a new value is supported: STATUS_CALLBACK_BYPASS. By returning this value, the registry operation isn't actually performed, but the thread won't get an error value. This is very useful for security solutions.

I wrote a small (very small) registry filter (see download link for MyRegFilter at the top of the article) to show how this new method works. Don't get too excited about it, I wrote it in 20 minutes and it's not that good, but maybe it's helpful for someone.

Collapse

#include 

WCHAR DeviceName[] = L"\\Device\\MyRegFilter";
WCHAR SymLinkName[] = L"\\DosDevices\\MyRegFilter";

UNICODE_STRING usDeviceName;
UNICODE_STRING usSymbolicLinkName;

typedef struct _DEVICE_CONTEXT
{
  PDRIVER_OBJECT  pDriverObject;     
  PDEVICE_OBJECT  pDeviceObject;

  LARGE_INTEGER RegCookie;
}
DEVICE_CONTEXT, *PDEVICE_CONTEXT, **PPDEVICE_CONTEXT;

PDEVICE_OBJECT  g_pDeviceObject  = NULL;
PDEVICE_CONTEXT g_pDeviceContext = NULL;

#define FILE_DEVICE_MYREGFILTER 0x8000


NTSTATUS DriverInitialize(PDRIVER_OBJECT  pDriverObject,
                        PUNICODE_STRING pusRegistryPath);
NTSTATUS DriverEntry(PDRIVER_OBJECT  pDriverObject,
                   PUNICODE_STRING pusRegistryPath);

NTSTATUS RegistryCallback(PVOID CallbackContext, PVOID Argument1,
                        PVOID Argument2);

#ifdef ALLOC_PRAGMA

#pragma alloc_text (INIT, DriverInitialize)
#pragma alloc_text (INIT, DriverEntry)

#endif


NTSTATUS DeviceDispatcher(PDEVICE_CONTEXT pDeviceContext, PIRP pIrp)
{
  PIO_STACK_LOCATION pisl;
  NTSTATUS ns = STATUS_NOT_IMPLEMENTED;

  pisl = IoGetCurrentIrpStackLocation(pIrp);

  switch (pisl->MajorFunction)
  {

  case IRP_MJ_CREATE:
  case IRP_MJ_CLEANUP:
  case IRP_MJ_CLOSE:
       case IRP_MJ_DEVICE_CONTROL:
      {
          ns = STATUS_SUCCESS;
          break;
      }
  }

  pIrp->IoStatus.Status = ns;
  pIrp->IoStatus.Information = 0;

  IoCompleteRequest(pIrp, IO_NO_INCREMENT);

  return ns;
}

NTSTATUS DriverDispatcher(PDEVICE_OBJECT pDeviceObject, PIRP pIrp)
{
  return (pDeviceObject == g_pDeviceObject ?
      DeviceDispatcher(g_pDeviceContext, pIrp)
      : STATUS_INVALID_PARAMETER_1);
}


VOID DriverUnload(PDRIVER_OBJECT pDriverObject)
{
  //
  // Stop filtering the registry
  // Shouldn't be placed in the unload
  //

  CmUnRegisterCallback(g_pDeviceContext->RegCookie);

  IoDeleteSymbolicLink(&usSymbolicLinkName);
  IoDeleteDevice(pDriverObject->DeviceObject);
}

NTSTATUS DriverInitialize(PDRIVER_OBJECT pDriverObject,
                        PUNICODE_STRING pusRegistryPath)
{
  PDEVICE_OBJECT pDeviceObject = NULL;
  NTSTATUS ns = STATUS_DEVICE_CONFIGURATION_ERROR;

  RtlInitUnicodeString(&usDeviceName, DeviceName);
  RtlInitUnicodeString(&usSymbolicLinkName, SymLinkName);

  if ((ns = IoCreateDevice(pDriverObject, sizeof (DEVICE_CONTEXT),
      &usDeviceName, FILE_DEVICE_MYREGFILTER, 0, FALSE,
      &pDeviceObject)) == STATUS_SUCCESS)
  {
      if ((ns = IoCreateSymbolicLink(&usSymbolicLinkName,
          &usDeviceName)) == STATUS_SUCCESS)
      {
          g_pDeviceObject  = pDeviceObject;
          g_pDeviceContext = pDeviceObject->DeviceExtension;

          g_pDeviceContext->pDriverObject = pDriverObject;
          g_pDeviceContext->pDeviceObject = pDeviceObject;

      }
      else
      {
          IoDeleteDevice(pDeviceObject);
      }
  }

  return ns;
}

NTSTATUS DriverEntry(PDRIVER_OBJECT pDriverObject,
                   PUNICODE_STRING pusRegistryPath)
{
  PDRIVER_DISPATCH *ppdd;
  NTSTATUS ns = STATUS_DEVICE_CONFIGURATION_ERROR;

  if ((ns = DriverInitialize(pDriverObject, pusRegistryPath))
                                                          == STATUS_SUCCESS)
  {
      ppdd = pDriverObject->MajorFunction;

      ppdd[IRP_MJ_CREATE                  ] =
      ppdd[IRP_MJ_CREATE_NAMED_PIPE       ] =
      ppdd[IRP_MJ_CLOSE                   ] =
      ppdd[IRP_MJ_READ                    ] =
      ppdd[IRP_MJ_WRITE                   ] =
      ppdd[IRP_MJ_QUERY_INFORMATION       ] =
      ppdd[IRP_MJ_SET_INFORMATION         ] =
      ppdd[IRP_MJ_QUERY_EA                ] =
      ppdd[IRP_MJ_SET_EA                  ] =
      ppdd[IRP_MJ_FLUSH_BUFFERS           ] =
      ppdd[IRP_MJ_QUERY_VOLUME_INFORMATION] =
      ppdd[IRP_MJ_SET_VOLUME_INFORMATION  ] =
      ppdd[IRP_MJ_DIRECTORY_CONTROL       ] =
      ppdd[IRP_MJ_FILE_SYSTEM_CONTROL     ] =
      ppdd[IRP_MJ_DEVICE_CONTROL          ] =
      ppdd[IRP_MJ_INTERNAL_DEVICE_CONTROL ] =
      ppdd[IRP_MJ_SHUTDOWN                ] =
      ppdd[IRP_MJ_LOCK_CONTROL            ] =
      ppdd[IRP_MJ_CLEANUP                 ] =
      ppdd[IRP_MJ_CREATE_MAILSLOT         ] =
      ppdd[IRP_MJ_QUERY_SECURITY          ] =
      ppdd[IRP_MJ_SET_SECURITY            ] =
      ppdd[IRP_MJ_POWER                   ] =
      ppdd[IRP_MJ_SYSTEM_CONTROL          ] =
      ppdd[IRP_MJ_DEVICE_CHANGE           ] =
      ppdd[IRP_MJ_QUERY_QUOTA             ] =
      ppdd[IRP_MJ_SET_QUOTA               ] =
      ppdd[IRP_MJ_PNP                     ] = DriverDispatcher;
      pDriverObject->DriverUnload           = DriverUnload;

      //
      // Filter the registry
      //

      ns = CmRegisterCallback(RegistryCallback, g_pDeviceContext,
                              &g_pDeviceContext->RegCookie);

      if (!NT_SUCCESS(ns)) IoDeleteDevice(g_pDeviceObject);
  }

  return ns;
}

//
// Registry Filter Callback
//

NTSTATUS RegistryCallback(PVOID CallbackContext, PVOID Argument1,
                        PVOID Argument2)
{
  PDEVICE_CONTEXT pContext = (PDEVICE_CONTEXT) CallbackContext;
  REG_NOTIFY_CLASS Action  = (REG_NOTIFY_CLASS) Argument1;


  switch (Action)
  {

  case RegNtPreDeleteKey:
      {
          //
          // Pre DeleteKey
          //

          PREG_DELETE_KEY_INFORMATION pInfo
                              = (PREG_DELETE_KEY_INFORMATION) Argument2;

          DbgPrint("Delete Key\n");

          //
          // You can prevent this operation from happening
          // Without having the thread noticing it
          // Only on Windows Vista
          //
          //
          // return STATUS_CALLBACK_BYPASS;
          //

          break;
      }

  case RegNtPreCreateKeyEx:
      {
          //
          // Pre CreateKey
          //

          PREG_CREATE_KEY_INFORMATION pInfo
                              = (PREG_CREATE_KEY_INFORMATION) Argument2;

          DbgPrint("Create Key\n");

          break;
      }


  default:
      {
          //
          // Return STATUS_SUCCESS
          //

          break;
      }
    
  }

  return STATUS_SUCCESS;
}

In this code sample I use DbgPrint to be notified of registry operations. On Vista the output of DbgPrint is disabled by default. If you want to enable it, follow these instructions.

Power Management

Power Management was improved in Vista, not only because new things have been introduced, but also because the Power Management for device driver programmers has been made easier. These WinHec docs are a very good source to start from. One of the big news is the introduction of the Hybrid Sleep (the default off-mode). In this sleep mode the system image is written on a hybernation file on disk from where the system can be resumed. Drivers are notified of entering the Hybrid Sleep (S4 state) by IRP_MN_SET_POWER (Parameters.Power.State == PowerSystemHybernate). Use the SYSTEM_POWER_STATE_CONTEXT structure (Parameters.Power.SystemPowerStateContext) to determine the state transition process.

Also, it might not be very important, but I happened to read what follows. Silent shutdown cancellations (in user mode) are no longer allowed on Vista. This means that if your application gets a WM_QUERYENDSESSION and doesn't return TRUE (in order to let the system shut down), Vista will show a dialog box informing the user of this behavior.

.NET Framework 3.0

The .NET Framework 3.0 is shipped along with Vista. However, it can be installed on XP SP2 as well. To use the new technologies introduced by this new framework with your Visual Studio 2005, you'll need two extensions. One for the Windows Presentation Foundation (WPF) and Windows Communication Foundation (WCF). And one for the Windows Workflow Foundation (WWF). I cannot discuss these technologies extensively, of course, but I can try to give an insight to programmers who have never worked with them.

Windows Presentation Foundation

I'm very enthusiastic about this technology, but it takes a moment or two for old fashioned C/C++ programmers (like myself) to understand how it works. Basically, it's a new way of creating GUIs for desktop applications and web pages. The main difference from the old way, is that these GUIs are created through XAML (eXtensible Application Markup Language), a language based on XML. The advantages of using the WPF are many. You can use 2D/3D, audio, video, animations etc. in seconds. There are no more HWNDs, and all the work is delegated to the GPU. On MSDN TV there are a few demonstrations of how through the WPF you can design beautiful and advanced GUIs. The WPF offers a very good separation between GUI development and the internal code implementation. Also, a lot of things can be achieved through XAML without having to use C#/VB code (smallWPF.zip download available at the top of the article)

In this little code sample I bind sliders to values of a listbox in order to change its appearance (position and shadow). What's so stunning is that the listbox can still be used, you can scroll it, select items, etc. I don't want to say that rotating a listbox is useful, but this is just a sample of what can be done. As I said, the slider are bound to values, this means that I didn't use code. Everything this application does is written in XAML. Here's all the code:

Collapse



  
    
      Hello,
      how are you?
      This
      is
      a
      3D
      ListBox!

I used this code to bind a value to a slider:

AngleX="{Binding ElementName=MySlider1, Path=Value}"

ElementName is the name of the control to bind and Path is the property of the bound control which should be used to fill the value. In this case, the position of MySlider1 fills the AngleX field. I could also bind a control's behavior to C# code. But, of course, this is not the place to discuss every property of this technology. I just hope that this paragraph got you intrigued enough to make you want to read more about it.

Windows Communication Foundation

The WCF is an interface to design services. The background idea is unified programming model for already existing technologies: COM+ / .NET Enterprise services, MSMQ, .NET Remoting, ASP.NET Web Services, Web Services Enhancements (WSE). Moreover, the ability of intercommunication between these technologies handled by the WCF, without having the programmer to worry about it, through reliable and secure ways. From what I could read, it seems a very good final solution to all the problems we know from the past, since from now on the programmer doesn't have to think about the communication process itself, which is handled by the WCF, meaning he doesn't have to worry about which technology he is communicating with and where from. For more information, this is the official Windows Communication Foundation hompage. However, there are even more practical samples on codeproject.

Windows Workflow Foundation

The WWF is a good way of formalizing workflow-based activities through visual items which are bound to code. Ok, this sounds strange, I'll try again. Basically, if you are a company and have to formalize a process of activities and want the comfort of having a visual model, then the WWF is what you are looking for. I can't talk about this subject extensively because I haven't used this technology extensively myself. However, since many programmers might wonder what this technology is all about, I'll try a simple understanding approach. Here's a workflow graph I made:

As you can see, the graph is divided in single activities (very few, because I have no imagination). The activities can be bound through declarative rules. This means that if a have a condition that needs to be satisfied, I can declare the condition as a property. There are a many workflow components, each of them has its own properties. For instance, the code-workflow component can define a code function in a C# file, which is executed when it's the component turn of activity. In this little sample, the first activity is to wait for the user's input. After that, a declarative rule is set, which sub-divides the workflow in two separate activity flows.

I hope that, in spite of my terrible workflow model, you got the general purpose of the Windows Workflow Foundation. If you're interested in learning more, check out the official homepage where you can also find a lot of code samples. As usual, there's plenty of guides about this subject.

A Vector Type for C#

2007-02-02T02:19:00.000-08:00

Download source - 6.9 Kb

Introduction

For years I have seen people struggle with vector mathematics. This guide should walk you through the creation of a reusable Vector type and the mathematics behind it all. The resulting code is not designed to be fast or efficient but is to be as simple and understandable as possible.

I havel used the Cartesian coordinate system in three-dimensions (i.e. three perpendicular axis of x, y and z) and Euclidian geometry. Don't worry about these terms, they are just the formal names for some of the maths covered at senior school. The vector space is volumetric (cube); note that you can use other vector spaces, such as a cylindrical space where one axis (usually z) relates to the radius of the cylinder.

You may have guessed that computers are quite slow with this type of math. Matrix mathematics is more efficient but much harder to understand. You will need a basic grasp of trigonometry and algebra to understand this guide.

Unless stated otherwise I assume that the vector is positional, originating at point (0,0,0). Alternatives to positional vectors are: unit vectors, which can be interpreted as either having no magnitude or an infinite magnitude; and vector pairs where the origin of the vector is another vector, magnitude being a distance from the origin vector.

Please note that this guide is extremely verbose and goes into far too much detail for a normal C# programmer. Please do not be offended if any part of this seems patronising. I have written the guide for a wide audience.

A quick glossary:

Operator, this is the symbol used to define an operation such as plus (+) in (a+b)
Operand, these are the variables used in an operation such as (a) and (b) in (a+b). The left-hand-side (LHS) operand is (a) where as the right-hand-side (RHS) operand is (b).

All of the equations in this guide assume A (or v1) and B (or v2) can be broken down into:

Using the code

To begin with let us define how the vector information will be stored. I don't often create structs when coding but for our Vector this is perfect. If you are reading this article you probably already know that a vector represents values along a number of axes. For this tutorial we will be developing a three-dimensional type so... thee variables and three axis.

public struct Vector
{
  private double x, y, z;
}

What orientation are the axes in? Well, for the type we are building it doesn't matter but I always assume:

You may have noticed that Z is negative as you look down the axis. This is a common convention in graphics libraries such as OpenGL. Again, these axis have no affect on our code.

A quick diversion: Why a struct instead of a class?

The differences between struct and class:

A struct is a value type created on the stack instead of the heap, thus reducing garbage collection overheads.
They are passed by value not by reference.
They are created and disposed of quickly and efficiently.
You cannot derive other types from them (i.e. non-inheritable).
They are only appropriate for types with a small number of members (variables). Microsoft recommends a struct should be less than 16 bytes.
You do not need the new keyword to instantiate a struct.

Basically, it looks like, acts like, and is a primitive type. Although, there is no reason why the vector type could not be created as a class.
A more in-depth article on structs has been written by S. Senthil Kumar at http://www.codeproject.com/csharp/structs_in_csharp.asp

Accessing the variables

Did you notice that the variables were private?

While I have chosen to build a struct, I habitually hide my variables and create public accessor and mutator properties. This is not strictly good practice for structs, but I have created them in case I feel the need to convert to a class at a later date (this is good practice for classes).

In addition to the properties, an array style interface has also been provided. This lets the user call the Vector with myVector[x], myVector[y], myVector[z]. Additionally, the user can get or set all of the components as an array using the Array property, i.e. myVector.Array = {x,y,z}.

Collapse

public double X
{
  get{return x;}
  set{x = value;}
}

public double Y
{
  get{return y;}
  set{y = value;}
}

public double Z
{
  get{return z;}
  set{z = value;}
}

public double[] Array
{
  get{return new double[] {x,y,z};}
  set
  {
     if(value.Length == 3)
     {
        x = value[0];
        y = value[1];
        z = value[2];
     }
     else
     {
        throw new ArgumentException(THREE_COMPONENTS);
     }
  }
}

public double this[ int index ]
{
  get
  {
     switch (index)
     {
        case 0: {return X; }
        case 1: {return Y; }
        case 2: {return Z; }
        default: throw new ArgumentException(THREE_COMPONENTS, "index");
    }
  }
  set
  {
      switch (index)
      {
         case 0: {X = value; break;}
         case 1: {Y = value; break;}
         case 2: {Z = value; break;}
         default: throw new ArgumentException(THREE_COMPONENTS, "index");
     }
  }
}
private const string THREE_COMPONENTS = "Array must contain exactly three components, (x,y,z)";

To construct the type using typical class syntax the following constructir methods have been provided:

Collapse

public Vector(double x, double y, double z)
{
  this.x = 0;
  this.y = 0;
  this.z = 0;

  X = x;
  Y = y;
  Z = z;
}

public Vector (double[] xyz)
{
  this.x = 0;
  this.y = 0;
  this.z = 0;

  Array = xyz;
}

public Vector(Vector v1)
{
  this.x = 0;
  this.y = 0;
  this.z = 0;

  X = v1.X;
  Y = v1.Y;
  Z = v1.Z;
}

We now have a framework for storing, accessing and mutating the Vector and its components (x,y,z). We can now consider mathematical operations applicable to a vector. Let's begin by overloading the basic mathematical operators.

Operator overloading

Overloading operators allows the programmer to define how a type is used in the code. Take, for example, the plus operator (+). For numeric types this would suggest addition of two numbers. For strings it represents the concatenation of two strings. Operator overloading is of huge benefit to programmers when describing how a type should interact with the system. In C# the following operators can be overloaded:

Addition, concatenation, and reinforcement (+)
Subtraction and negation (-)
Logical negation (!)
Bitwise complement (~)
Increment (++)
Decrement (--)
Boolean Truth (true)
Boolean false (false)
Multiplication (*)
Division (/)
Division remainder (%)
Logical AND (&)
Logical OR (|)
Logical Exclusive-OR (^)
Binary shift left (<<)
Binary shift right (>>)
Equality operators, equal and not-equal (== and !=)
Difference\comparison operators, less-than and greater-than(< and >)
Difference\comparison operators, less-than or equal-to and greater-than or equal-to(<= and >=)

Operator overloads for the Vector type

Addition (v3 = v1 + v2)

The addition of two vectors is achieved by simply adding the x, y, and z components of one vector to the other (i.e. x+x, y+y, z+z).

public static Vector operator+(Vector v1, Vector v2)
{
  return
  (
     new Vector
     (
        v1.X + v2.X,
        v1.Y + v2.Y,
        v1.Z + v2.Z
     )
  );
}

Subtraction (v3 = v1 - v2)

Subtraction of two vectors is simply the subtraction of the x, y, and z components of one vector from the other (i.e. x-x, y-y, z-z).

public static Vector operator-(Vector v1, Vector v2 )
{
  return
  (
     new Vector
     (
         v1.X - v2.X,
         v1.Y - v2.Y,
         v1.Z - v2.Z
     )
  );
}

Negation (v2 = -v1)

Negation of a vector inverts its direction. This is achieved by simply negating each of the component parts of the vector.

public static Vector operator-(Vector v1)
{
  return
  (
     new Vector
     (
        - v1.X,
        - v1.Y,
        - v1.Z
     )
  );
}

Reinforcement (v2 = +v1)

Reinforcement of a vector actually does nothing but return the original vector given the rules of addition, (i.e. +-x = -x and ++x = +x).

public static Vector operator+(Vector v1)
{
  return
  (
     new Vector
     (
        + v1.X,
        + v1.Y,
        + v1.Z
     )
  );
}

Comparison (<, >, <=, and >=)

When comparing two vectors we use magnitude. The magnitude of a vector is its length, irrespective of direction.

Magnitude
The length (or magnitude) of a vector can be determined using the formula .

public static double Magnitude(Vector v1)
{
  return
  (
     Math.Sqrt
     (
        v1.X * v1.X +
        v1.Y * v1.Y +
        v1.Z * v1.Z
     )
   );
}

Where possible I have created static methods to extend the programmer's options when making use of the type. Methods which directly affect or are effected by the instance simply call the static methods.

public double Magnitude()
{
  return Magnitude(this);
}

Returning to the comparison operators ...

Less-than (result = v1 <>

Less-than compares two vectors, returning true only if the magnitude of the left-hand-side vector (v1) is less than the magnitude of the other (v2).

public static bool operator<(Vector v1, Vector v2)
{
  return v1.Magnitude() < v2.Magnitude();
}

Less-than or Equal-to (result = v1 <= v2)

Less-than or equal-to compares two vectors returning true only if the magnitude of the left-hand-side vector (v1) is less than the magnitude of the other (v2) or the two magnitudes are equal.

public static bool operator<=(Vector v1, Vector v2)
{
  return v1.Magnitude() <= v2.Magnitude();
}

Greater-than (result = v1 > v2)

Greater-than compares two vectors returning true only if the magnitude of the left-hand-side vector (v1) is greater than the magnitude of the other (v2).

public static bool operator>(Vector v1, Vector v2)
{
  return v1.Magnitude() > v2.Magnitude();
}

Greater-than or Equal-to (result = v1 >= v2)

Greater-than or equal-to compares two vectors returning true only if the magnitude of the left-hand-side vector (v1) is grearter than the magnitude of the other (v2) or the two magnitudes are equal.

public static bool operator>(Vector v1, Vector v2)
{
  return v1.Magnitude() >= v2.Magnitude();
}

Equality (result = v1 == v2)

To check if two vectors are equal we simply check the component pairs. We AND the results so that any pair which is not equal will result in false.

public static bool operator==(Vector v1, Vector v2)
{
  return
  (
     (v1.X == v2.X)&&
     (v1.Y == v2.Y)&&
     (v1.Z == v2.Z)
  );
}

Inequality (result = v1 != v2)

If the operator == (equal) is overridden, C# forces us to override != (not-equal). This is simply the inverse of equality.

public static bool operator!=(Vector v1, Vector v2)
{
  return !(v1==v2);
}

Division (v3 = v1 / s2)

Division of a vector by a scalar number (e.g. 2) is achieved by dividing each of the component parts by the divisor (s2).

public static Vector operator/(Vector v1, double s2)
{
  return
  (
     new Vector
     (
        v1.X / s2,
        v1.Y / s2,
        v1.Z / s2
     )
  );
}

Multiplication (dot, cross, and by scalar)

Multiplication of vectors is tricky. There are three distinct types of vector multiplication:

Multiplication by a scalar (v3 = v1 * s2)
Dot product (s3 = v1 . v2)
Cross product (v3 = v1 * v2)

Only multiplication by scalar and division by scalar have been implemented as operator overloads. I have seen operators such as ~ overloaded for the dot product to distinguish it from cross product; I believe that this can lead to confusion and have choosen not to provide operators for dot and cross products leaving the user to call the appropriate method instead.

Multiplication by scalar is achieved by multiplying each of the component parts by the scalar value.

public static Vector operator*(Vector v1, double s2)
{
  return
  (
     new Vector
     (
        v1.X * s2,
        v1.Y * s2,
        v1.Z * s2
     )
  );
}

The order of operands in multiplication can, of course, be reversed.

public static Vector operator*(double s1, Vector v2)
{
  return v2 * s1;
}

The cross product of two vectors produces a normal to the plane created by the two vectors given.

The formulae for this (where v1 = A and v2 = B) is

The equation should always produce a vector as the result.

The sine of theta is used to account for the direction of the vector. Theta always takes the smallest angle between A and B (i.e. ).

The right hand side of the formula is arrived at by expanding and simplifying the left hand side using the rules:

Sin 0° = 0

Sin 90° = 1

In a matrix style notation this looks like:

You should be aware that this equation is non-commutable. This means that v1 cross-product v2 is NOT the same as v2 cross-product v1.

The C# code for all of this is:

public static Vector CrossProduct(Vector v1, Vector v2)
{
  return
  (
     new Vector
     (
        v1.Y * v2.Z - v1.Z * v2.Y,
        v1.Z * v2.X - v1.X * v2.Z,
        v1.X * v2.Y - v1.Y * v2.X
     )
  );
}

And the instance counterpart of the static method is:

public Vector CrossProduct(Vector other)
{
  return CrossProduct(this, other);
}

Note that this instance method does not affect the instance from which it is called but returns a new Vector object. I have chosen to implement cross product in this fashion for two reasons; one, to make it conistant with dot product which cannot produce a vector, and two, because cross product is usualy used to generate a normal used somewhere else, the origional vecotor needing to be left intact.

[Side note] A quick template for manualy calculating the cross product of two vectors is:

The dot product of two vectors is a scalar value defined by the formulae;

The equation should always produce a scalar as the result.

Cosine theta is used to account for the direction of the vector. Theta always takes the smallest angle between A and B (i.e. ).

The right hand side of the formula is arrived at by expanding and simplifying the left hand side using the rules:

Cos 0° =1

Cos 90° = 0

The C# code for this is:

public static double DotProduct(Vector v1, Vector v2)
{
  return
  (
     v1.X * v2.X +
     v1.Y * v2.Y +
     v1.Z * v2.Z
  );
}

And its counterpart:

public double DotProduct(Vector other)
{
  return DotProduct(this, other);
}

Extended functionality

We now have all the basic functionality required of a Vector type. To make this type really useful I have provided some additional functionality.

Magnitude revisited

We have already seen the method to get the magnitude or length of a vector. Here is a method to alter a vector's magnitude.

public static Vector Magnitude(Vector v1, double newMagnitude)
{
  if (newMagnitude < 0)
  {throw new ArgumentOutOfRangeException("newMagnitude", newMagnitude, NEGATIVE_MAGNITUDE);}

  if (v1 == new Vector(0, 0, 0))
  {throw new ArgumentException(ORAGIN_VECTOR_MAGNITUDE, "v1");}

  return
  (
     new Vector
     (
        v1 * (newMagnitude / v1.Magnitude())
     )
  );
}

public void Magnitude(double newMagnitude)
{
  this = Magnitude(this, newMagnitude);
}

private const string NEGATIVE_MAGNITUDE = "The magnitude of a Vector must be a positive value, (i.e. greater than 0)";
private const string ORAGIN_VECTOR_MAGNITUDE = "Cannot change the magnitude of Vector(0,0,0)";

Normalisation and Unit Vector

A unit vector is one which has a magnitude of 1. To test if a vector is a unit vector we simply check for 1 against the magnitude method already defined.

public static bool IsUnitVector(Vector v1)
{
  return v1.Magnitude() == 1;
}

public bool IsUnitVector()
{
  return IsUnitVector(this);
}

Normalization is the process of converting some vector to a unit vector. The formula for this is:

Collapse

public static Vector Normalize(Vector v1)
{
  // Check for divide by zero errors
  if ( v1.Magnitude() == 0 )
  {
     throw new DivideByZeroException( NORMALIZE_0 );
  }
  else
  {
     // find the inverse of the vectors magnitude
     double inverse = 1 / v1.Magnitude();
     return
     (
        new Vector
        (
           // multiply each component by the inverse of the magnitude
           v1.X * inverse,
           v1.Y * inverse,
           v1.Z * inverse
        )
     );
  }
}

public void Normalize()
{
  this = Normalize(this);
}

private const string NORMALIZE_0 = "Can not normalize a vector when it's magnitude is zero";

The normalization instance method directly affects the instance.

Interpolation

This method takes an interpolated value from between two vectors. This method takes three arguments, a starting point (vector v1), and end point (Vector v2), and a control which is a fraction between 1 and 0. The control determines which point between v1 and v2 is taken. A control of 0 will return v1 and a control of 1 will return v2.

or:

where:

= Current value

= Initial value (v1)

= Final value (v2)

= Control parameter, where , and where, ,

Collapse

public static Vector Interpolate(Vector v1, Vector v2, double control)
{
  if (control >1 || control <0)
  {
     // Error message includes information about the actual value of the argument
     throw new ArgumentOutOfRangeException
     (
         "control",
         control,
         INTERPOLATION_RANGE + "\n" + ARGUMENT_VALUE + control
     );
  }
  else
  {
     return
     (
        new Vector
        (
            v1.X * (1-control) + v2.X * control,
            v1.Y * (1-control) + v2.Y * control,
            v1.Z * (1-control) + v2.Z * control
         )
     );
  }
}

public Vector Interpolate(Vector other, double control)
{
  return Interpolate(this, other, control);
}

private const string INTERPOLATION_RANGE = "Control parameter must be a value between 0 & 1";

Distance

This method finds the distance between two positional vectors using Pythagoras theorem.

public static double Distance(Vector v1, Vector v2)
{
  return
  (
     Math.Sqrt
     (
         (v1.X - v2.X) * (v1.X - v2.X) +
         (v1.Y - v2.Y) * (v1.Y - v2.Y) +
         (v1.Z - v2.Z) * (v1.Z - v2.Z)
     )
  );
}

public double Distance(Vector other)
{
  return Distance(this, other);
}

Absolute

This method finds the absolute value of a vector by applying the Abs method to each of the component parts. This is not the same as magnitude.

public static Vector Abs(Vector v1)
{
  return
  (
     new Vector
     (
        Math.Abs(v1.X),
        Math.Abs(v1.Y),
        Math.Abs(v1.Z)
     )
  );
}

public Vector Abs()
{
  return Abs(this);
}

Angle

This method finds the angle between two vectors using normalization and dot product.

^ refers to a normalized (unit) vector.

|| refers to the magnitude of a Vector.

public static double Angle(Vector v1, Vector v2)
{
  return
  (
     Math.Acos
     (
        Normalize(v1).DotProduct(Normalize(v2))
     )
  );
}

public double Angle(Vector other)
{
  return Angle(this, other);
}

Max and Min

These methods compare the magnitude of two vectors and return the vector with the largest or smallest magnitude respectively.

public static Vector Max(Vector v1, Vector v2)
{
  if (v1 >= v2){return v1;}
  return v2;
}

public Vector Max(Vector other)
{
  return Max(this, other);
}

public static Vector Min(Vector v1, Vector v2)
{
  if (v1 <= v2){return v1;}
  return v2;
}

public Vector Min(Vector other)
{
  return Min(this, other);
}

Yaw

This method rotates a vector around the Y axis by a given number of degrees (Euler rotation around Y).

The hypotenuse (R) cancels out in the equation.

public static Vector Yaw(Vector v1, double degree)
{
  double x = ( v1.Z * Math.Sin(degree) ) + ( v1.X * Math.Cos(degree) );
  double y = v1.Y;
  double z = ( v1.Z * Math.Cos(degree) ) - ( v1.X * Math.Sin(degree) );
  return new Vector(x, y, z);
}

public void Yaw(double degree)
{
  this = Yaw(this, degree);
}

This method directly affects the instance from which the method was called.

Pitch

This method rotates a vector around the X axis by a given number of degrees (Euler rotation around X).

The hypotenuse (R) cancels out in the equation.

public static Vector Pitch(Vector v1, double degree)
{
  double x = v1.X;
  double y = ( v1.Y * Math.Cos(degree) ) - ( v1.Z * Math.Sin(degree) );
  double z = ( v1.Y * Math.Sin(degree) ) + ( v1.Z * Math.Cos(degree) );
  return new Vector(x, y, z);
}

public void Pitch(double degree)
{
  this = Pitch(this, degree);
}

This method directly affects the instance from which the method was called.

Roll

This method rotates a vector around the Z axis by a given number of degrees (Euler rotation around Z).

The hypotenuse (R) cancels out in the equation.

public static Vector Roll(Vector v1, double degree)
{
  double x = ( v1.X * Math.Cos(degree) ) - ( v1.Y * Math.Sin(degree) );
  double y = ( v1.X * Math.Sin(degree) ) + ( v1.Y * Math.Cos(degree) );
  double z = v1.Z;
  return new Vector(x, y, z);
}

public void Roll(double degree)
{
  this = Roll(this, degree);
}

This method directly affects the instance from which the method was called.

Back-face

This method interprets a vector as a face normal and determines whether the normal represents a back facing plane given a line-of-sight vector. A back facing plane will be invisible in a rendered scene and as such can be except from many scene calculations.

If then if

public static bool IsBackFace(Vector normal, Vector lineOfSight)
{
  return normal.DotProduct(lineOfSight) < 0;
}

public bool IsBackFace(Vector lineOfSight)
{
  return IsBackFace(this, lineOfSight);
}

Perpendicular

This method checks if two vectors are perpendicular (i.e. if one vector is the normal of the other).

public static bool IsPerpendicular(Vector v1, Vector v2)
{
 return v1.DotProduct(v2) == 0;
}

public bool IsPerpendicular(Vector other)
{
  return IsPerpendicular(this, other);
}

Sum components

This method simply adds together the vector components (x, y, z).

public static double SumComponents(Vector v1)
{
  return (v1.X + v1.Y + v1.Z);
}

public double SumComponents()
{
  return SumComponents(this);
}

To Power

This method multiplies the vectors components to a given power.

public static Vector Pow(Vector v1, double power)
{
  return
  (
     new Vector
     (
        Math.Pow(v1.X, power),
        Math.Pow(v1.Y, power),
        Math.Pow(v1.Z, power)
     )
  );
}

public void Pow(double power)
{
  this = Pow(this, power);
}

Square root

This method applies the square root function to each of the vectors components.

public static Vector Sqrt(Vector v1)
{
  return
  (
     new Vector
     (
        Math.Sqrt(v1.X),
        Math.Sqrt(v1.Y),
        Math.Sqrt(v1.Z)
     )
  );
}

public void Sqrt()
{
this = Sqrt(this);
}

Usability functions

For completeness a number of standardised methods have been added complete the type.

To get a textual description of the type:

public override string ToString()
{
  string output = null;

  if (IsUnitVector()){output += UNIT_VECTOR;}
  else {output += POSITIONAL_VECTOR;}

  output += string.Format( "( x={0}, y={1}, z={2} )", X, Y, Z );
  output += MAGNITUDE + Magnitude();

  return output;
}

private const string UNIT_VECTOR = "Unit vector composing of ";
private const string POSITIONAL_VECTOR = "Positional vector composing of ";
private const string MAGNITUDE = " of magnitude ";

To produce a hashcode for system use (required in order to implement comparator operations (i.e. ==, !=)):

public override int GetHashCode()
{
  return
  (
     (int)((X + Y + Z) % Int32.MaxValue)
  );
}

Check for equality (standardised version of == operatator):

public override bool Equals(object other)
{
  // Check object other is a Vector object
  if(other is Vector)
  {
     // Convert object to Vector
     Vector otherVector = (Vector)other;
     // Check for equality
     return otherVector == this;
  }
  else
  {
     return false;
  }
}

Comparison method for two vectors which returns:

-1 if the magnitude is less than the others magnitude
0 if the magnitude equals the magnitude of the other
1 if the magnitude is greater than the magnitude of the other

This allows the Vector type to implement the IComparable interface.

Public int CompareTo(object other)
{
  if(other is Vector)
  {
     Vector otherVector = (Vector)other;

     if( this < class="cs-keyword">return -1; }
     else if( this > otherVector ) { return 1; }

     return 0;
  }
  else
  {
     // Error condition: other is not a Vector object
     throw new ArgumentException
     (
        // Error message includes information about the actual type of the argument
        NON_VECTOR_COMPARISON + "\n" + ARGUMENT_TYPE + other.GetType().ToString(),
        "other"
     );
  }
}

private const string NON_VECTOR_COMPARISON = "Cannot compare a Vector to a non-Vector";
private const string ARGUMENT_TYPE = "The argument provided is a type of ";

Standard Cartesian vectors

Finaly four standard vector constants are defined:

public static readonly Vector origin = new Vector(0,0,0);
public static readonly Vector xAxis = new Vector(1,0,0);
public static readonly Vector yAxis = new Vector(0,1,0);
public static readonly Vector zAxis = new Vector(0,0,1);

Summary

We now have a Vector type with the following functionality:

Vector(double x, double y, double z)
Vector(double[] xyz)
Vector(Vector v1)

Indexer
+
-
==
!=
*
/
<
>
<=
>=

CrossProduct
DotProduct
Magnitude
Normalize
IsUnitVector
Interpolate
Distance
Abs
Angle
Max
Min
Yaw
Pitch
Roll
IsBackFace
IsPerpendicular
SumComponents
Sqrt
Pow

Magnitude
Normalize
Yaw
Pitch
Roll
Sqrt
Pow

CrossProduct
DotProduct
IsUnitVector
Interpolate
Distance
Abs
Angle
Max
Min
IsBackFace
IsPerpendicular
SumComponents
CompareTo
Equals
ToString
GetHashCode

Artificial intelligence network load balancing using Ant Colony Optimisation - C# Algorithms

2007-02-02T01:52:00.000-08:00

Download source code - 798 Kb

Introduction

Ants first evolved around 120 million years ago, take form in over 11,400 different species and are considered one of the most successful insects due to their highly organised colonies, sometimes consisting of millions of ants.

One particular notability of ants is their ability to create "ant streets". Long, bi-directional lanes of single file pathways in which they navigate landscapes in order to reach a destination in optimal time. These ever-changing networks are made possible by the use of pheromones which guide them using a shortest path mechanism. This technique allows an adaptive routing system which is updated should a more optimal path be found or an obstruction placed across an existing pathway.

Computer scientists began researching the behaviour of ants in the early 1990's to discover new routing algorithms. The result of these studies is Ant Colony Optimisation (ACO) and in the case of well implemented ACO techniques, optimal performance is comparative to existing top-performing routing algorithms.

This article details how ACO can be used to dynamically route traffic efficiently. An efficient routing algorithm will minimise the number of nodes that a call will need to connect to in order to be completed thus; minimising network load and increasing reliability. An implementation of ANTNet based on Marco Dorigo & Thomas stützle has been designed and through this a number of visually aided test were produced to compare the genetic algorithm to a non-generic algorithm. The report will final conclude with a summary of how the algorithm perform and how it could be further optimised.

Background

Electronic communication networks can be categorised as either circuit-switched or packet-switched. Circuit-switched networks rely on a dedicated connection from source to destination which is made once at start-up and remains constant until the tear-down of the connection. An example of a circuit switched network would be British Telecoms telephone network. Packet-switched networks work quite differently however and all data to be transmitted is divided into segments and sent as data-packet. Data-packets can arrive out of order in a packets-switched network with a variety of paths taken through different nodes in order to get to their destination. The internet and office local area networks are both good examples of packet-switched networks.

A number of techniques can be employed to optimise the flow of traffic around a network. Such techniques include flow and congestion control where nodes action packet acknowledgements from destination nodes to either ramp-up or decrease packet transmission speed. The area of interest in this report concentrates on the idea of network routing and routing tables. These tables hold information used by a routing algorithm to make a local forwarding decision for the packet on the next node it will visit in order to reach its final destination.

One of the issues with network routing (especially in very large networks such as the internet) is adaptability. Not only can traffic be unpredictably high but the structure of a network can change as old nodes are removed and new nodes added. This perhaps makes it almost impossible to find a combination of constant parameters to route a network optimally.

Routing algorithms

Packet-switched networks dynamically guide packets to their destination via routing tables stored in a link state and are selected via a link state algorithm.

The Link state algorithm works by giving every node in the network a connectivity graph of the network. This graph depicts which nodes are directly connected. Values are stored for connected nodes in map which represent the shortest path to other nodes. One such link state algorithm used in network routing is Dijkstras algorithm. When a path between two nodes is found, its weight is updated in the table. Should a shorter path be found the new optimal weight will be updated to the table replacing the old value.

The algorithm allows traffic to be routed around the network whilst connecting to the least number nodes as possible. The system works but doesn't take into account influx of traffic and load balancing.

Introducing ANTNet

By replacing Dijkstras algorithm with a generic algorithm, paths taken by calls could be scored by how short of a path they took, that way if they were queued on a busy network they would perform badly. Consequently other paths would score relative and be chosen. This would work in real time and allow the routing system to adapt as packets are transmitted ANTNet uses virtual pheromone tables much like when an ant follows a path dropping pheromones to re-enforce it. The quicker the ants move down a path the more throughput of ants thus; a greater concentration of pheromones. In the same way pheromone tables in ANTNet allow fast routes to score a higher chance of being selected whilst the less optimal route scores a low chance of being selected.

The idea behind ANTNet is when a call is placed an Ant will traverse across the network using a link-state deterministic algorithm. Every node holds a pheromone table for all other nodes of the network. Each pheromone table holds a list of table entries containing all the connected nodes of the current node.

The algorithm

To begin with, each possible path has an even likelihood of being chosen. An ant is placed on a network of 4 nodes with the source node of 1 and destination node 2. A chance mechanism is invoked and a path is chosen.

Figure 3.1 - The network graph

Next Node	% chance
2	33.33333%
3	33.33333%
4	33.33333%

Table 3.1 - Pheromone table for node 1

In this case node 2 has been selected [figure 3.2] and the ant arrives at its source destination.

The ant then moves and updates the pheromone tables for the visited nodes with higher (and more mathematically biased) value. This would be calculated for figure 3.2 and table 3.2 in the following way:

Node 2 was the final destination
It took 1 hop to get to its destination
Divide 1 (hop) by 100 : 100%
Add 100 to the probability value of node 2 (currently 33.3333) : 133.3333
Add the values of the other nodes to 133.3333 (133.3333+ 33.3333 + 33.3333) : 200 (approximately)
Calculate the ratio : ratio = 100/200 0.5
Set the probability of the node to its current value multiplied by the ratio

Node 2: 133.3333 * ratio (0.5) = 66.6666%
Node 3: 33.3333 * ratio (0.5) = 16.6666%
Node 4: 33.3333 * ratio (0.5) = 16.6666%

Node 2 (66.6666%) + Node 3 (16.6666%) + Node 4 (16.6666%) = 99.9999%

The system isn't 100% accurate as the total will never add up to exactly 100% but it will be close enough to allow accuracy within the level required.

The following diagram depicts how the path and pheromone table after the update has taken place.

Figure 3.2

Next Node	% chance
2	66.6666%
3	16.6666%
4	16.6666%

Table 3.2

The program

For the purpose of this program a bi-directional, un-weighted topological network consisting of 30 nodes has been created and closely resembles the British Synchronous Digital Hierarchy (SDH) network. After a basic number of parameters have been set the simulation is run. Firstly all pheromone tables are defaulted to equal weights and then calls are generated and placed on the network. Initially the routes chosen are random. If a call cannot connect to a node it is forced to wait and the wait counter is enumerated to reflect how long (in timer ticks) this was. Once a node has reached its destination node it will work its way backwards altering the local nodes pheromone table as it traverses. The shorter the route taken the greater increase in probability given to its table entry in the pheromone table. This happens repeatedly until the weight of the fastest node is shifted such that slower routes have a very low chance of being chosen.

NOTE: in order to compile and run the program you will need to download dotnetcharting from dotnetcharting.com

Program features

ANTNet On/Off – Switches the algorithm on and off
Simulation Speed – 1 tick p/s – 1,000 tick p/s (or as near speed as the system can run)
Total calls to make – Number of completed calls before simulation termination
Maximum concurrent calls – number of calls allowed at the same time
Node capacity – The number of calls a node can route at once
Call duration – The length (in ticks) of a call
Reduce I/O – bypasses the network visualisation to increase simulation speed
Return on connection – returns the node immediately to source after connection
Real-time network load visualisation – view Node ID, capacity and routing state (blue = OFF)
Graphing facility with labelling and simulation overlay
Render Pheromone Tables to HTML

Classes

MainForm – The GUI for the application
DrawPanel – The real-time network visualisation custom control
Global – Contains static variables that are accesses by the application
Node – Represents a Node and holds an array of PheromoneTable objects for routing
Call – Represents a call on the network and hold a source and destination node
Simulation – Represents a completed simulation and is used for creating graphs
PheromoneTable - A routing table for a Node

The network contains 30 Nodes and each Node contains an array of PheromoneTable objects, one for every other Node in the network (29). Every PheromoneTable contains an array of TableEntries, one for each Node connected to the current Node.

The following diagram represents the relationships between classes in the program.

Update the pheromone table

        // returns the next Node of the path
       public int ProbablePath(ArrayList VisitedNodes)
       {
           // create a random generator
           Random r = new Random(Global.Seed);

           double val=0;
           double count = 0;
           double Lastcount = -1;
  
           ArrayList tempTEVector=new ArrayList();

           // loop through all the connected nodes
           for(int i=0;i           {
               // has the node been visitied?
               bool v=false;

               //loop through all the visited nodes
               for(int j=0;j               {
                   // if the ID's match then this node has alrady been visited
                   if(tableEntry[i].NodeID==(int)VisitedNodes[j])
                       v=true;
               }
              
               // If v is false then the node hasnt been visited.. so Add
               if(!v)
               {
                   // get the node
                   Node n = Global.Nodes[tableEntry[i].NodeID];

                   // if the node is accepting connections
                   if(!n.FullCapacity)
                   {
                       // add the node as a possible candidate
                       tempTEVector.Add(tableEntry[i]);
                   }
               }
           }
          
           // if all connections have been visited
           if(tempTEVector.Count==0)
           {
               // loop through all the connected nodes
               for(int i=0;i                   tempTEVector.Add(tableEntry[i]);
           }

           // get the ceiling amount for probabilities
           for(int i=0;i               val+= ((TableEntry)tempTEVector[i]).Probablilty;

           //create randon value
           val = r.NextDouble()*val;

           // loop through the temp Table Entryies
           for(int i=0;i           {
               // increment the count on each loop
               count += ((TableEntry)tempTEVector[i]).Probablilty;

               // if the random value falls into delegated range then select that path as the next node
               if(val>Lastcount && val < count)
                   return ((TableEntry)tempTEVector[i]).NodeID;

               // get the value of the last count
               Lastcount=count;
           }

           // method should never return here
           return -1;
       }

Return the next node via the pheromone table

        // updates the probabilities of the pheromone table by multiplying the selected 
       // nodes probability by a radio of  newVal
       public void UpdateProbabilities(double newVal, int EntryTableNodeID)
       {
           TableEntry t;
           double total=0;

           // loop through all the table entries
           // get the total enumeration of probabilities and add the new value.
           // Since this total will be more than 100 a ratio multiplication is
           // applied. Although these values will not equate to exactly 100% floating
           // point calculations will be accurate enough at least 99.99999% which is satisfactory
           for(int j=0;j           {
               t = tableEntry[j];

               // enumerate the total probablility
               total += t.Probablilty;
              
               // if the table entry matches the id of the chosen node path
               if(EntryTableNodeID==t.NodeID)
               {
                   // add the new value to the total
                   total += newVal;
                   t = tableEntry[j];
                   // add the new value the current value of the selected path
                   t.Probablilty += newVal;
               }
           }

           // calculate the ratio for the multiplcation
           double ratio = 100/total;

           // loop through each table entry and multiple the current probability
           // by the new ratio
           for(int j=0;j           {
               tableEntry[j].Probablilty *= ratio;
           }

           // this will enumerate all the values to 99.99999%
       }

       // Constructor takes a node to represent and a list of all connected nodes off the
       // calling node
       public PheromoneTable(Node n, int[] conns)
       {
           this.NodeID = n.ID;
      
           // create a tableEntry array the same length as the number of connections
           this.tableEntry = new TableEntry[conns.Length];

           // create a new tableEntry for each connection
           for(int i=0;i               tableEntry[i] = new TableEntry(conns[i]);

           // set default equal values
           for(int i=0;i               tableEntry[i].Probablilty = (100 / (double)conns.Length);
       }
   }

Simulation results

The following tests will illustrated how the ANTNET algorithm effects the routing of traffic. These tests will show effectiveness of the algorithm against the system running without ANTNET. Since it is possible to switch nodes on and off, a number of test comparisons will be done to show how ANTNET can improve the routing of a network when paths are no longer valid and new routes have to be chosen.

These tests have been run with the following parameters

ANTNet On
Simulation Speed –1,000 tick p/s
Total calls to make – 5000
Maximum concurrent calls – 60
Node capacity – 35
Call duration – 170 The length (in ticks) of a call
Reduce I/O – bypasses the network visualisation to increase simulation speed
Return on connection – returns the node immediately to source after connection

5.1 ANTNet VS non- ANTNet

The first test contains two simulations.

Simulation 1 ( Orange ) – ANTNet algorithm OFF
Simulation 2 ( Blue) – ANTNet ON

From this simulation it is clear that even by the first 500 calls completed, ANTNet has reduced the average number of hops by approximately 1.5 nodes. This is made more apparent by the end of the simulation where the best paths are made more biased as a choice and are re-enforced as the optimal route, resulting in ANTnet improving network performance by almost 3.5 hops

Figure 5.1 – Non-adaptive algorithm ( orange) VS ANTNet algorithm ( blue)

To view the algorithm from a different perspective the following graph depicts the system running with the ANTNet algorithm off and then activated on the 2,000 th call. This can be identified by a label and follows with a decline of average hops by almost 2.

Figure 5.2 – ANTNet activated after the 2000th call

5.2 Loop elimination

Before an Ant returns back to its source node, an optimisation technique of loop elimination can be invoked. The problem with loops is that they can receive several times the amount of pheromone than they should, leading to the problem of self re-enforcing loops.

Figure 5.3 – Loop removal ( Blue) VS non-loop removal ( Orange )

Figure 5.3 shows two simulations:

Simulation 1 ( Orange ) – Loop elimination OFF
Simulation 2 ( Blue) – Loop elimination ON

From this test, loop elimination has reduced the average number of hops by 1 node with a much more stable adaptation. This would mean that when alternative paths must be chosen, the loop elimination algorithm responds much faster that the regular implementation.

Note: Both lines show the actual number of nodes traversed and not the number after loop removal.

5.3 Adaptivity

It is important to simulate how the network adapts when nodes are removed from the network. Static routing tables may hold the shortest path but they don't necessarily take into account network traffic and nodes that are offline. Three simulations have been run on the program to display how the system adapts compared to a non adaptive algorithm.

Figure 5.4 – Adaptive vs non-adaptive algorithm

Simulation 1 ( orange)

This is a normal run of the simulator to create optimised Pheromone tables for the next two simulations.

Simulation 2 ( blue)

Adaptive algorithm switched OFF.

Nodes 14,15 and 17 are switched off as these are the main Northern access hubs into London so traffic needs to be diverted to the west of England . Since the network is non adaptive, the pheromone tables are biased towards nodes that are have been taken offline and subsequently being continuously redirected and taking longer journeys every time. This increase is displayed in figure 6.4 by the blue line.

Simulation 3 ( green)

Adaptive algorithm switched ON.

Nodes 14,15 and 17 are still switched off but since the network is now set to adaptive, the pheromone tables are readjusted and the system learns alternative routes. This can be seen in figure 6.4 by the green line.

Linux on the Nokia 770 Internet Tablet

2007-02-02T01:37:00.000-08:00

The Nokia 770 is an internet tablet designed to connect to the internet with 802.11b/g WLAN or through a phone with Bluetooth. It has a nice 800 x 480 screen (64k colors) and quite a few apps on board. The underlying OS is the Debian GNU/Linux derivative, BusyBox. As it stands, the device is not a phone.

The day after my 770 arrived, I read a review in the Washington Post where the guy really didn't like the unit at all. I like it a lot, so I thought about his reasons. My conclusion is that the reviewer is a Windows guy. As such, he would have been less likely to find the other wonderful stuff that's floating around for the 770. In any case, quite a few people have been working on the 770, which richens the experience considerably.

The Machine

Out of the box, the 770 does quite a few things, including being a music and video player. The official lineup of software includes:

Web Browser, Flash Player version 6, Email Client, Internet Radio, News Reader, Media players, Image viewer, PDF viewer, File Manager, Search, Calculator, World Clock, Notes, Sketch, Games including chess, mahjong, and marbles.

It is pretty nice to sit in your most comfy chair and browse the web with a nice light object that doesn't have to sit in your lap (and fry it as some laptops do). The screen is a good size, although the pleasures might be a bit site-dependant in that if the reading columns are wide you will have to scroll sideways. Actually, that's a gotcha. In the menu under "View," click "Optimized view." This will rearrange the pages to avoid sideways scrolling, even if you increase the font size. The standard font size is small but readable. You can make the fonts larger with a simple button push. The screen is sharp as well, which helps.

It's worth saying a little about the music player. The 770 is a little bigger than an iPod and quite suitable for taking along a limited playlist. For storage, the unit uses Reduced Size MMC and 1GB models available. If you're one of those people with 30,000 MP3s or Ogg Vorbis songs, you can't take the whole song collection. Since you have to select tunes or playlists on the screen, playlists are the best playing mode. Set the screen lock, chuck it your bag, and it will play to the end of the playlist.

Data Entry

You have a choice of a screen keyboard or handwriting recognition. Some people have commented on the slow speed and inaccuracy of the handwriting recognition, but the speed can be set, and when it's on the fastest setting you really have to zip. I found the accuracy to be mostly okay. If you only use this device to enter URLs in the browser or to write short emails, it doesn't present too much of a problem. If you want to do more, there is a lot of help at hand.

That's enough of the technical stuff. Here's the fun part (see Figure 1).

Figure 1. Doom on the Nokia 770.

The Hacks

Because the 770 has been out for a few months there's plenty of porting activity already. Nokia set up maemo.org as the official headquarters. The site has a Nokia 770 Application Catalog that lists what's currently available.

What can you get for your "phone?" Several flavors of XTerm, editors including Vim, Joe, Midnight Commander, some Wi-Fi tools, telnet and FTP clients, and a whole lot more, including ... Doom! This plays pretty well actually and the big wide screen (for a handheld) makes it quite an attractive proposition.

To install new software on the 770, first download the deb package. Then connect the 770 by its USB cable and transfer the files to the RSMMC on the 770. Open the control panel on the 770, choose Install Programs, and away you go.

After you've done that, you may notice that you can't immediately launch apps by typing commands into the console (see Figure 2). This is because the new programs go into the directory /var/lib/install/usr/bin. To fix the problem, make a .profile file and add that path. I use:

export LD_LIBRARY_PATH=/var/lib/install/usr/lib:$LD_LIBRARY_PATH
export PATH=/var/lib/install/usr/bin:$PATH

Figure 2. Browsing the Nokia 770 Filesystem.

At that point you might find that you can't write the .profile file. This is because you're at / by default. Go to /home/user, create the file, and you'll be fine.

Getting root (necessary for some tools) isn't all that straightforward. Fortunately, there are two possibilities: How Do I Become Root on the Nokia 770 and How to obtain root in Nokia 770 - the easy way .

File Manager

The File Manager application only shows files in your /home/user/MyDocs, directory, your RS-MMC card and on Bluetooth-paired devices. However, there are several workarounds.

For simple viewing/browsing all files, just open the web browser and set the URL to /usr/ or similar. Then navigate around as much as you like.

To unlock the full capabilities of the built-in file manager, create a symbolic link. Open X Terminal Emulator and create a link into MyDocs directory:

$ cd ~/MyDocs
ln -s / Root

Root will then appear in the File manager, in Midnight Commander, or whichever shell you prefer.

Change the Logo and Sounds

The big Nokia logo you see on startup comes from the file /usr/share/icons/hicolor/scalable/hildon/qgn_indi_startup_nokia_logo.png. The Nokia hands live at /usr/share/icons/hicolor/scalable/hildon/qgn_indi_nokia_hands.png. The Nokia tunes are in /usr/share/sounds. Replace the files as you like.

(The file manager and logo hacks came from mobile analysis and development's Nokia 770 page.)

Data Entry

The stock data entry methods for the 770 are quite adequate for minimal typing, but the first long email will send you looking for alternatives. The first option is a Bluetooth keyboard. Nokia makes one but apparently, any HID Bluetooth keyboard will do (except HP's Bluetooth keyboard, which doesn't work). A driver is available from Maemo.org and it's pretty much as simple as downloading, installing, and activating it from the menu bar at the top of the screen.

Maybe you'd like something a bit bigger or less expensive since the Bluetooth keyboards generally seem to have a fair price premium. A great hack by Thoughtfix enables the USB port so that you can use an ordinary (or maybe a slightly compact) USB keyboard. The problem here is that the USB port on the 770 doesn't supply power and won't act as a host. You can fix the second part with a simple download and a command ( see Karoliina's Nokia 770 dev howto ) but the other part requires the use of some cables, a battery, a voltage regulator, and a little time putting them together. Thoughtfix has project details in USB power "injector" for USB host mode .

Use Wikipedia

Bleb.org has a hack on using nicer CSS for Wikipedia. The default skin contains a long left-hand column, however by using a Wikipedia account, you can change the skin to one more suited to a device such as the 770.

Hacks of the Near Future

When the news of the 770 first came out, a lot of people said, "Wi-Fi? How about a VoIP client?" As it happens, Nokia might have had these thoughts at the design stage, since there's an inactivated facility for audio input on the current unit. Very recently, they announced a software upgrade that will include a VoIP client.

In the words of Nokia's Ari Virtanen:

The Internet Tablet OS 2006 edition features pre-installed Google Talk and Jabber compatibility for Instant Messaging as well as Google Talk compatibility for Internet calling. This will give Nokia Internet Tablet users the power to instantly communicate with people, both through instant messaging and Voice over IP.

There will also be a built-in Google search facility and support for SIP-based VoIP solutions, which will allow users to make and receive calls from public telephone networks.

Nokia's Nokia 770 Internet Tablet page allows you to sign up for notifications of this upgrade. I also recommend subscribing to Bleb.org's feed of new 770 applications for more information on ports as they come out.

Conclusion

There's a lot more to come for this great little unit. Nokia shows signs of having a clue in shepherding the 770 on its way both in marketing and the way developers have been encouraged, for example they've made an SDK available and have an open source GUI. If you feel like doing some coding or porting, maemo.org is the place to go. Interestingly too, the Windows-only tools that exist for Symbian now have Linux and Mac OS X ports for this Linux-based unit.

There's plenty more to come here!

Using a vista side bar gadget to consume an image feed

2007-01-31T02:43:00.000-08:00

Introduction

When I first installed Windows Vista on my Virtual Server I noticed the sidebar and thought, "Ok, that's pretty cool". But when I found out that all you needed was HTML and Javascript to create one I said, "WOW, that's awesome!". My mind filled with the possibilities. One of the first that came to mind when I saw the Feed Headlines gadget in the Gadget Gallery was an image feed. I figured if the gadget could pull an RSS feed, which is essentially XML, from a remote location and refresh the data on an interval then I figured I could pull custom XML and display images located on a remote server as well.

The goal of this article will take you through the process of creating a Vista Sidebar Gadget for the first time. I have included just about all the steps you need to go through to create any gadget. Plus, I've tried to provide background on each step and point out the Gotchas that I ran into as I was creating this Sidebar Gadget for this article.

As a disclaimer, this article is a bit of a shameless plug for the company I work for because we sell professional sports photos online. But I figured it would be ok, because the point of the article is to show how you can integrate a Sidebar Gadget with your online content. For example, you could take this gadget and adapt it to consume an image feed of the current sale items, or best deals in your online catalog. If the user sees something they like they click the "buy" link and can purchase the item!

What is a Sidebar Gadget?

The Windows Vista Sidebar is an executable that runs on the desktop of the new Windows Vista OS. It hosts "Gadgets" which are small DHTML applications which have almost all the capabilities of a web page running in Internet Explorer. (I say almost all because this is a new technology and it's my first attempt at using it.)

Windows Vista comes with a small sampling of Sidebar Gadgets to give you an idea of what is possible. Some seem pretty useful:

An analog clock
An RSS news reader
A CPU / memory monitor

"What's so useful about an analog clock" you say? Well, it's ok, but what makes it useful is that the Sidebar is capable of running multiple, independent instances of each gadget. And each instance has it's own settings. So if you work in a different time zone than your clients or compatriots you can run a separate instance of the clock for each time zone! Of course your desktop real-estate is limited, but the sidebar is capable of displaying multiple pages of gadgets, so you can navigate through the ones you have open.

A minimal Gadget application consists of the following items:

An XML manifest file named gadget.xml (yes, the name is a requirement)
An HTML file
An "icon" (jpg, gif or png) file

A Gadget application may also include the following items:

script files (.vbs or .js)
stylesheet files (.css)
a settings HTML file
a "flyout" HTML file
globalization files
ActiveX components

Gadgets have two installation directories:

%USER_DATA%\Local\Microsoft\Windows Sidebar\Gadgets - for user gadgets
%SYSTEM_ROOT%\Program Files\Windows Sidebar\Gadgets - for global gadgets

Gadget Development Tools & Resources

What tools do you need to create a Sidebar Gadget? The short answer, "Notepad". Yup, if you are comfortable writing your HTML and Javascript or VBScript in a plain text editor then you can do just that. No compilation is needed. However, if you're planning on something more robust than your first "Hello, World!" gadget I'd recommend an IDE that supports script debugging. Because I do all my HTML in a text editor, I tried to get started with notepad on my first try. But since my window.alert() attempts failed from the get go, I was dead in the water right out of the gate.

So I installed my IDE of choice, Visual Studio 2005, configured IE for script debugging and instantly determined the cause of my first problem and every problem thereafter. If you can do it all in notepad, more power to you. You're more of a man than I. But after my first experience, I don't recommend flying blind.

You don't need to be running Windows Vista either, but I don't recommend that either since you can't really test your gadget without it. I don't know about you, but I'm not quite ready to take that plunge. So I installed Vista as a virtual machine on Virtual Server. For this article, I am working with Windows Vista Ultimate Edition, RC2 Build 5744. The final release is out by now (or at least it's available on MSDN) so you can work with a more current version, but everything seemed to be working fine for me so there was no reason to download and install the entire OS all over again.

Gotcha #1

An important note about updating the installed gadget files. I found that if I'm just updating the .htm file for the Flyout or Settings windows, I can update the file and the next time the window loads it will display the updates. However, if I'm modifying a .js, .css or the main Gadget.htm file, then I have to close all running instances of the Gadget before I can open a new instance and view the changes.

Here are a list of resources I found helpful along the way:

Sidebar Gadget Object Model - for reference
Sidebar Gadget Tutorial - Earth Map - for a more complex example
FileSystemObject online reference - for reading/writing files
W3 DOM Object model reference - for DHTML, XML DOM
List of known bugs - for sanity

Globalization

If you want to make your Sidebar Gadget localized the key is to use declared constants for all your text. If you have images with text on them, you'll need to create additional copies of the image in each language you intend to support. Then create a folder with the language code (en-US for english united states, for example). That folder should contain a duplicate folder structure for the resources (js, css and image files) and the localized resources. In the image of my solution explorer above, you'll see I have a folder named "en-US" with a subfolder named js which contains a javascript file named local.js. Local.js contains all my declared constants for error messages and other text. If I wanted to support other languages I would simply duplicate the contents of en-US with values in the language being supported.

Step 1: The manifest File

The manifest file describes your Sidebar Gadget and contains the settings required by Windows Sidebar to both display it in the Sidebar Gallery and create an instance of it in the Sidebar.



   MaxPreps Gallery Viewer
   Developmentalmadness.Vista.Gadgets
   1.0.0.0
   
                    text="Mark J. Miller's blog"/>
   
   © 2007
   View high school sport's action photos!
   
                    src="images/icons/MaxPreps_Blk_130w.gif" />
   
   
       
           
           Full

There isn't really any documentation on the gadget.xml file schema, all the tutorials simply display an example of the manifest file. It's up to the reader to just copy & paste, then edit the sample. But I'll try and give you a summary here of what I've found.

Except for the hosts element and it's children, most of the schema is used to describe your Sidebar Gadget for the Gadget Gallery. The elements name and icons are used to display an application icon above the name of your Sidebar Gadget in the Gadget Gallery. While version, author, info, copyright and description all are used by the details pane to describe your Sidebar Gadget. The info element has two attributes: url and text. url is exactly what it says it is. The text attribute is optional, but if you want to display something descriptive in place of the url you can use it. The only one I haven't been able to account for is namespace, but by the time I realized it, I figured I had already come up with a namespace and entered it, so I decided to leave it.

Because of the lack of documentation, I'm guessing on some of this, but most of what is below the hosts element stays as it is. However, the src attribute of base indicates the main HTML file for your Sidebar Gadget and defaultImage allows you to specify a drag icon used when you drag your Sidebar Gadget from the Gadget Gallery to the Sidebar. This can be a transparent png and the transparency should be preserved as you drag the icon to the Sidebar. However, I am by no means a graphic artist, so I did not attempt to make my own graphic to test this.

Step 2: The Settings Page

As you create your gadget, it should save you some time if you create your settings page next. The reason I say this is because I did not, and I found that I would have to go back through much of my code that is already in place to accommodate each configuration setting I add. Because I found myself running out of time, I settled for keeping the configuration settings to a single simple option so I could have time to write this article. So unless you are disciplined enough to spec out your entire first project before starting it I recommend building your settings page first. Then as you build the rest of your Sidebar Gadget you can quickly add in settings as you write the application code and you won't have to go back and rewrite sections to accommodate new settings.

Each page (gadget, settings, flyout) is independent and does not share variables. So the Sidebar Gadget object model contains an System.Gadget.Settings object which allows you to persist data while your Sidebar Gadget is running. It is very useful for saving state data and communicating between the different pages of your Sidebar Gadget. Your settings page is a means to allow users to make configuration changes to your Gadget and persist those during the current session.

A settings page is not required, but when you do create one you will see the above icon included below the close button next to your gadget. And when you click on it you'll get something like the image below. As I said before, I am not a graphic designer, so the part I appreciate about the settings page is it's simplicity. Even though you need to include a full HTML page (HTML, HEAD and BODY elements) the only other thing you need is to add a STYLE element to designate the height and width of the body and then drop a couple HTML controls onto the page. Everything around the page, including the "OK" and "Cancel" buttons and their functionality comes pre-built as part of the Sidebar Gadget package. I chose to go one step further and add a DIV tag as a place holder for validation errors on the page.


   
        Settings
       
       
   
   
       List size (#):
                   style="color:Red;font-size:12pt;font-family:Calibri;">

Gotcha #2

A lesson I learned is that certain small details can cause a lot of headaches. When I created my first HTML page for this project I added a SCRIPT tag to the HEAD element to reference an external .js file. But I couldn't figure out why the Sidebar Gadget wouldn't load. (This was when I decided I needed an IDE with script debugging capability if I wanted to continue with this endeavor). It turns out I could not use the following format for my script tag:

In order to use the settings page you'll need the following two lines of code which will be called from your main page:

System.Gadget.settingsUI = "settings.htm";
System.Gadget.onSettingsClosed = settingsUpdated;

The first line should be called from your onload event to tell the Sidebar you are using a settings page. This will add the settings button next to your gadget. When the user clicks this button, it will load your settings page. The second line indicates a method (in this case settingsUpdated) to be called when the settings page successfully closes. It will allow you to read the new settings and update the behavior or layout of your Sidebar Gadget.

function loadSettings()
{
   //set the close event handler
   System.Gadget.onSettingsClosing = onClose;
  
   //read the current setting
   var sMaxCount = System.Gadget.Settings.read(SETTING_MAX_GALLERY_COUNT);
  
   //check to see if it has been set
   if(sMaxCount != "")
       gMaxCount = parseInt(sMaxCount);
      
   //set the value of the HTML control
   maxCount.value = gMaxCount;
}

When your settings page loads, you'll need to do at least two things: set the onSettingsClosing event handler, and read the current settings values to display them in the controls on the settings UI.

The first is straightforward, just create a function with the following signature and pass it to the onSettingsClosing delegate: function (parameter1). The name of the function and the parameter are up to you but here's what my function looks like:

function onClose(event)
{
   if(event.closeAction == event.Action.commit)
   {
       //make sure the value entered was numeric
       if(isNaN(maxCount.value))
       {
           //display error message
           errorMessage.innerHTML = "Please enter an integer value.";
          
           //cancel the 'Ok' action
           event.cancel = true;
          
           //exit function
           return;
       }
      
       //get the integer value
       gMaxCount = parseInt(maxCount.value);
      
       //save settings
       System.Gadget.Settings.write(SETTING_MAX_GALLERY_COUNT, gMaxCount);
      
       //indicate success
       event.cancel = false;
   }
}

The parameter passed to your event handler is the event arguments. This is another case where I did not find documentation other than what was in the code samples I came across. If anyone finds documentation, I will gladly include it in my resource links. The event.Action property has two possible values: commit or cancel. They correspond to the "Ok" and "Cancel" buttons on the settings UI.

In this case, I check to see if the user clicked the "OK" button (Action.commit) and then validate the data to make sure the value is numeric. If it isn't, I display an error message and set event.cancel = true so that the user will be returned to the settings page to either cancel the operation or correct the problem and resave the settings. If everything is valid, I save the settings and set event.cancel = false so the settings page will close and the user will be returned to my Sidebar Gadget.

When you want to persist your settings you have two methods: write(string name, obj value) and writeString(string name, obj value). When you use write the Settings object will try and guess the type of your value. If your value is a string, use writeString to eliminate the guesswork when you can.

This brings us to the second step I mentioned: reading the current settings. The write and writeString methods each have corresponding read methods: read(string name) and readString(string name). When you read from the Settings object, make sure and check the value for an empty string (""). If the setting is empty or has not been set, it will return and empty string instead of a null value. When using boolean values, a tip I learned from a tutorial I read was to cast the value to force it as boolean. You can cast it like this:

var valueFromSettings = System.Gadget.Settings.read("myValue");
var myBool = !!valueFromSettings;

Now lest move on to the actual Gadget functionality

Step 3: The Gadget UI

My goal in creating this gadget was to duplicate the RSS Reader gadget's functionality and pull an XML feed which I would use as a source to build a list of the most recent photo galleries on my site. Something I hadn't mentioned yet, is that because of the architecture of Sidebar Gadgets you can open up any Gadget on your system and view the source code, just like if you were viewing a web page in IE. No, I don't mean you can right-click and select "view source...". But you can navigate to the directory where the gadget is located and view the source files. So when I got started I opened up the RSS Reader and realized they were using the Feed Store built-in to IE. And since I would not be using RSS, Atom or any other standard schema for my XML I had to turn elsewhere.

After some playing around with XSLT, I opted to go with AJAX instead because it would make certain features easier to implement. There are some good AJAX tutorials on the site I used when I used AJAX for the first time, so do a search and read a few of them if you're not familiar with the technology yet.

The next decision I had to make was how to make the XML feed available to the gadget. So in order to simplify things and allow others to use the source code for this gadget on their local machines, I decided to create two static XML files and put them on my local web server. I have included these two xml files and Feed.xml and Feed2.xml in the source files included with the article. You can place them on your local web server or a remote one it doesn't matter. As I was building this gadget, I had the gadget installed on my Vista virtual machine and my feed files stored in my IIS virtual directory on Windows XP. When I wanted to test the auto update ability, I just swapped the files back and forth to imitate a dynamic process.

For brevity sake I haven't included the actual AJAX code here, just the pertinent stuff:

gadget.js

///////////////////////////////////////////////
//  loadMain()
//
//  Summary: called by body onload event from 
//      gadget.htm. Sets Gadget file references
//      and loads XML data
///////////////////////////////////////////////
function loadMain()
{
   // ... other init code here ....
          
   //get settings
   var sMaxCount = System.Gadget.Settings.read(SETTING_MAX_GALLERY_COUNT);
   if( sMaxCount != "" )
       gListMax = parseInt(sMaxCount);
      
 //set flag to resize the height of the
 //gadget after building list
   gResizeGadget = true;
  
   //retrieve data
   makeXmlRequest(DATA_RESOURCE_URI, AJAX_TYPE_GALLERIES);
}

ajax.js

Collapse

/////////////////////////////////////////////////////////////////
//  getXmlContent()
//
//  Summary: called by the onreadystatechanged event, if
//      the request is complete it gets the XML document and 
//      saves it to disk so that it can be accessed by all
//      gadget pages. Then it calls printGalleries() to 
//      update the list of galleries on the gadget.
//////////////////////////////////////////////////////////////////
function getXmlContent()
{
   //check to see if the request is complete
   if(checkReadyState(gRequest))
   {
       //get XML doc
       var ajaxDoc = gRequest.responseXML;
      
   //store XML in global variable
     gXmlDoc = ajaxDoc;
     
       //if this is a galleries xml doc refresh the gadget
       if(gAjaxType == AJAX_TYPE_GALLERIES)
           printGalleries();
          
       //set timer to refresh data again in 5 minutes
       setTimeout("refreshData()", 5 * 60000);
   }
}

/////////////////////////////////////////////////////////
//  refreshData()
//
//  Summary: downloads data resource uri and refreshes 
//      xml data. If xml data is currently being read
//      the process is rescheduled for 1 min later
/////////////////////////////////////////////////////////
function refreshData()
{
   //make sure the data isn't currently being read
   if(gReadingXml)
   {
       //reschedule data refresh
       setTimeout("refreshData()", 1 * 60000);
       return;
   }
  
   //add 'random' parameter to prevent caching
   var dt = new Date();
  
   //resubmit AJAX request
   makeXmlRequest(DATA_RESOURCE_URI + "?t=" + dt.getTime(),
                  AJAX_TYPE_GALLERIES);
}

To tie the missing stuff together here, onLoad calls makeXmlRequest which calls getXmlContent when it completes. Then getXmlContent stores the XML in a global variable and calls printGalleries which uses the global variable to read the XML and build the list on the gadget UI. I haven't included it here because of it's length and there's nothing in printGalleries which is unique to the Sidebar Gadget API. Then a timer is set to requery the remote server for updates to the feed by calling refreshData.

Layout

A design aspect that you will be forced to deal with is the limited space you are given for your gadget. The width of the Sidebar is 130 pixels, which is not much space for anything. But certainly in our case there isn't much room for text. Because I'm pretty new to DHTML, my first attempt to deal with this was to count up the maximum number of characters I could fit using the current font, then use the substring method to chop that string and append an ellipsis(...). But I couldn't get it to look as neat as the RSS Feed reader Gadget. So again, I opened up the code to see what they had done. Now those of you who are more experienced with CSS and DHTML may have seen this one coming a mile away, but bear with me here.

Collapse

//create a row and a cell to display the data
var newRow = table.insertRow(table.rows.length);   
var cell = newRow.insertCell(0);

//build the innerHTML string
var html = "item" + i + "\" title=\"" + node.getAttribute("name")
html += "\" onClick=\"loadFlyout('" + node.getAttribute("id");
html += "');this.blur();\"";
html += " onmouseover=\"this.style.color='Red'\"";
html += " onmouseout=\"this.style.color=''\"";
html += " style=\"font-size:13px;margin-bottom:0px;margin-top:0px;\"> ";
html += node.getAttribute("name") + "
 ";
html += "sport" + i + "\" style=\"color:White;font-size:11px;";
html += " color:#67788a;margin-top:0px;margin-bottom:0px;\"> " ;
html += node.getAttribute("sport") + "
 ";

//set the innerHTML for the table cell
cell.innerHTML = html;

//format the title so that long text gets cut of with an ellipsis
eval("item" + i).style.textOverflow = "ellipsis";
eval("item" + i).style.overflow = "hidden";
eval("item" + i).style.whiteSpace = "nowrap";
eval("item" + i).style.width = 115;

//format the sport so that long text gets cut of with an ellipsis
eval("sport" + i).style.textOverflow = "ellipsis";
eval("sport" + i).style.overflow = "hidden";
eval("sport" + i).style.whiteSpace = "nowrap";
eval("sport" + i).style.width = 115;

//place a border between items to make it easier to read
eval("sport" + i).style.borderBottom = "dotted 1px White";

There are two important steps here to point out. First, you must be sure to set the width of the HTML element which will act as the container for your text. In this case I am using a DIV tag within a TD element. So I have set the width of the DIV tag. Then for each of the text elements, I set textOverflow, overflow and whiteSpace style properties. You don't have to set them in any particular order, as you can see above, but they each need to be set to get the effect of the text running off the side of the page.

So now we have a Gadget which reads from a remote XML feed and then displays a list of the items in the feed. For navigation controls, I borrowed the graphics used by the RSS Reader Gadget. They include previous and next buttons, and a counter to display the start and end index of the items currently in the list. Let's move on now to what we can now do with that data.

Step 4: The Flyout

The flyout is an optional component of the Sidebar Gadget, but it's great when you need more room to display your application. The only limit to the size of your flyout is that of the user's display resolution. With the graphics capabilities required by Windows Vista in the first place it's probably safe to assume that your users are going to have a minimum of 1024x768 resolution.

To open your flyout only takes two lines of code. You need to specify the HTML file used by the flyout and set the show property to true. Like this:

System.Gadget.Flyout.file = "flyout.htm";
System.Gadget.Flyout.show = true;

Where you place these commands isn't so important as the order they are in, before you display the flyout you must indicate the file to be used. You also have the option of registering a method with either or both of the onShow and onHide events. Keep in mind that the Gadget page and the Flyout are independent and cannot communicate directly with one another. This means that the onShow and onHide events are used for updating the Gadget UI or trigger some behavior in the main Gadget page based on those events. We'll come back to these events in a minute.

The best way to communicate with the flyout is to use the System.Gadget.Settings object. In this case we use it to store the id of the gallery that was clicked on the Gadget so the flyout can read from the XML and display the images. Here's where we hit a bit of a snag.

Gotcha #3

The XML feed was too large to pass into the settings object. Whenever I tried to write the XML as a string, it was unsuccessful. There were no errors, the program just kept running as if everything was ok. But when the flyout tried to read the XML from System.Gadget.Settings.readString the result was an empty string. I tried inspecting the settings object from the Gadget as soon as I had written the XML, but still the value wasn't getting stored. I knew I was doing everything correctly because I could read the gallery id value I had passed, just not the XML.

To work around this I decided to write the XML to disk, then I could read it in from the flyout. The next obstacle was that the Sidebar object model has no read and write capability without user interaction. If you want to open or save a file the System.Shell object and it's children provide methods to ask the user where they want to save a file, or which file to open. And it has the ability to inspect the file system objects and to create and delete folders, but not to open files as text or write to them behind the scenes. However, this problem was easily solved by using the Scripting.FileSystemObject.

ajax.js

Collapse

///////////////////////////////////////////////////
//  saveXmlDoc(xmlDoc)
//  Parameters: xmlDoc - XmlDocment object to be saved
//      to local disk
//  Summary: saves the specified XmlDocment object to
//      the local disk to make it available to flyout.htm
///////////////////////////////////////////////////
function saveXmlDoc(xmlDoc)
{
   //set flag so data won't get overwritten while we're using it
   gReadingXml = true;
  
   //get storage path
   var path = getDataPath();
  
   //create/open text file
   var fso = new ActiveXObject("Scripting.FileSystemObject");
   var output = fso.OpenTextFile(path,2,true);
  
   //write XML data to file
   output.WriteLine(xmlDoc.xml);
  
   //close file
   output.Close();
  
   //reset flag
   gReadingXml = false;
}

/////////////////////////////////////////////////////////
//  deleteXmlDoc(path)
//  Parameters: path - path of the XML file to delete
//  Summary: deletes specified file from local disk.
/////////////////////////////////////////////////////////
function deleteXmlDoc(path)
{
   var fso = new ActiveXObject("Scripting.FileSystemObject");
   fso.DeleteFile(path);
}

Now I was able to create save and open functions which used the Scripting.FileSystemObject to allow the XML to be passed back and forth between the Gadget and the Flyout.

Which brings us to permissions. When writing files the Gadget can write files to the file system, but only within the application path and subdirectories. If you try to manipulate files out side that path you'll get permission denied errors. Fortunatelythe Sidebar Gadget object model has a convenient property to give you access to the application path: System.Gadget.path.

But because we're writing files to be passed back and forth we've created two problems for ourselves. First, the System.Gadget.Settings object is instance independent, so if there are multiple instances of your control in the sidebar they don't conflict. But part of that benefit, is that you have no way of knowing how many instances are open or what their settings are. So if I'm writing files back and forth, I need to make sure I'm not conflicting with other instances of my Gadget.

I decided the way around this was to use a file name that would be unique among all instances of my Gadget. And the easiest method available was the javascript Data.getTime method which returns the number of milliseconds since Jan 1, 1970. Since the probability of a user being able to add multiple instances of my Gadget to the Sidebar is nil this works fine for our needs. Then in order to allow the Flyout to retrieve the file all I need to do is pass the path to the file to System.Gadget.Settings.

Event Sequence

I want to backtrack for a moment to the Flyout events onShow and onHide. My first choice was to use these two events to write and clean up the XML file. The onShow seemed a good choice because it was the perfect trigger to tell my Gadget when the file was needed. onHide because I found that the Gadget has no onClose event to allow me to clean up files when I'm done. Because the System.Gadget.Settings object does not persist data after a Gadget is closed, if I don't delete the files then eventually the files will pile up in the users directory over time. Disk space may be cheap, but it's up to the user to determine how to use that space, not me.

As it turns out, the onHide event worked fine for my needs. Unfortunately, I found out that onShow fires after the onLoad event of the Flyout. So now I needed to figure out how to write the file before the Flyout tried to access it in the onLoad event. I finally decided to use the onClick event to first, write the XML file to disk, then open the flyout. This guarantees that the file exists before the Flyout loads.

gadget.js

///////////////////////////////////////////////
//  loadFlyout(galleryId)
//  Parameters: galleryId - (string) the id value
//      of the selected GALLERY element
//  Summary: Called from onclick event of gadget.htm.
//      It stores settings needed by flyout
//      to display selected gallery and opens
//      the flyout
///////////////////////////////////////////////
function loadFlyout(galleryId)
{
   //write the xml to local disk so it can be accessed by flyout.htm
   storeXml();
  
   //display the flyout
       System.Gadget.Flyout.show = true;
}

So, as I mentioned before, when the user clicks on the name of a gallery the onClick event fires and calls loadFlyout and passes the id of the gallery as an argument. The loadFlyout function writes the XML stored in a global variable to disk, then opens the Flyout.

Gotcha #4

The next issue I ran into was a surprise. I had been using an external CSS file for the general formatting of my page, and I wanted to set the background color of the Flyout. But it remained white. The page could view the CSS file, because I could change the height and width of the body, but other settings had no effect. But when I used inline styles, I had no problem. I searched around for a workaround, but only found other complaints of the problem with no solutions. At this point I abandoned the external CSS and used inline styles.

External Links

The last important feature of note was that I wanted a user to be able to go to my website and purchase a print of the image. I thought, there might be a problem with putting a link to an external resource on my flyout page, but there wasn't. In fact it was easier than I imagined. If you put a link to an external resource, a new IE window will open to the resource in the link - exactly the way I wanted it to happen!

Step 5: Packaging Your Gadget

So now, we're finished. Or almost. To deploy our newly completed Sidebar Gadget you have two options. You can simply copy the files to a new folder in the Gadget directory, then rename the folder to include ".gadget" at the end. So if our folder is named "Photogallery", just rename it to "Photogallery.gadget".

The above method may work fine if you are creating your own personal gadget and then just dropping it in the folder locally. But if you have a gadget like ours, and you want to deploy it to other users, this second method is actually easier. Simply create a zip file containing all the files and subfolders (but not the root folder itself), then change the .zip extension to .gadget. And that's it! When a user tries to download or open your compressed .gadget file Windows Vista will automatically install it to their user gadgets folder.

The Finished Product

There you have it, the complete package, from start to finish. How to develop a functional Vista Sidebar Gadget. I struggled a bit towards the end with a list of features I would have liked to implement or some changes I would make now that I better understand the process. But they'll have to wait until version 2.0. I'd be happy to hear your ideas for different features that could make this Gadget better. Here's a few that I'd like to add when I get time to work on this again:

Resize the main Gadget window when the user undocks the Gadget.
Resize the Flyout window when the user clicks a thumbnail in order to make the enlarged view even bigger.
Add more user configurable settings like the location of the feed, the size of the flyout, the number of thumbnails to display at a time, as well as color and font settings.

Exchange Data Securely Without HTTPS/SSL-C# Web Services

2007-01-29T02:31:00.000-08:00

Introduction

In this article, I will show you how to exchange data securely with a Web Service without HTTPS/SSL but without compromising the same protection level of HTTPS/SSL. It will also give you a real life example on the implementation and limitation of the important encryption types as well as Digital Signature that are available in .NET Framework.

Advantages

No Installation and configuration is required in IIS or in the Client.
No Deployment issue (Nothing to embed/ship) in the client.
No Additional Library is required.
Less easier to implement comparing alternates.

Requirements

If you are not familiar with Symmetric, Asymmetric Encryption and Digital Signature. I strongly suggest you read the following articles of cp before you move forward, there are also many reference available in the Web on this topic:

I have created the solution in Visual Studio 2005 and .NET 2.0 but it can be easily ported to Visual Studio 2003 and .NET 1.1.

Background

Symmetric VS Asymmetric

I assume that you already know the difference between the Symmetric and Asymmetric encryption. In short, in Symmetric encryption both the parties (the server and client) use the same key to encrypt and decrypt the data. On the other hand, in Asymmetric encryption a pair of key (known as Public/Private Key) is used to encrypt/decrypt. If the data is encrypted with the Private Key the only way to decrypt it is the Public Key and the vice versa.

Digital Signature

Digital Signature is also a part of the Public/Private Key implementation. The private key is used to sign the data and public key is used to verify that data has not been tampered. However, Digital signature does not retain the secrecy of the data, which means the data always remain plain (Not encrypted).

Alternate Solutions

Share Common Secret Key: The easiest way to implement it to share a common secret key before the Client and Service start communicating.
MS WSE Library: I found it difficult to implement. Maybe it will look easier to you.

The Solution

Initial Proof of Concept

After going thorough few details of the above mentioned articles and of course, MSDN I came up with the following message exchange between the client and server, which looks pretty, rock solid to me.

	The Client	The Server	The Hacker
1.	Client creates random Public/Private Key Pair and sends the Server the Public Part	The Server in turns creates a random Public/Private Key Pair and returns Public Part to Client.	The Hacker can only view both parties Public key.
2.	The Client encrypts the target data with received Server Public Key and sends it back to Server.	The Server receives the encrypted data decrypts it with the previously generated Server Private Key. Server Process the Data. Server encrypts the process result with the previously received Client Public Key and returns back to Client.	The hacker only has the Server Public key. So it is not possible for the hacker to decrypt the data, as it requires the server private key to decrypt. Again, the Hacker only has the Client Public key and thus it is not possible for the Hacker to decrypt the process result.
3.	The Client receives the encrypted process result and decrypts it with the previously generated Client Private Key.
4.	The Client returns to regular processing	The Server Goes to sleep.	The Hacker Cries.

.NET Platform

In .NET platform, there are different algorithms available for both Symmetric and Asymmetric. I am not going to explain the details of these algorithms as it is beyond the scope of this article. The characteristics are:

Symmetric Algorithms

DES: 64 bit encryption - Good.
RC2: 128 bit encryption - Better.
Rijndael: 256 bit also known as AES and the most secure algorithm – The Best.
TripleDES: 192 bit encryption – Much better.

Asymmetric Algorithms

DSA: Does not support Encryption/Decryption. Only allows Digital Signature (So skipping from discussing).
RSA: Supports both Encryption/Decryption and Digital Signature. But it cannot encrypt/decrypt data more than 117 byte. If you want to encrypt more than 117 byte, it will throw you a Cryptographic Exception with a message that Key not valid for use in specified state. There are many references available in web, which explains this limitation.

Implementation

As mentioned above the only asymmetric algorithm that supports Encryption/Decryption is RSA, but it has the limitation of data length. Moreover, in Web Service it is regular case that we are exchanging data more than 117 bytes with those bulky xml tags. We can surly use other mechanism such as compression, use other medium such as JSON instead of XML to reduce the data size, but none of these ensures that the data size will be always less than 117 byte. Here comes the Symmetric Key into the scene. Let me revise the previous proof of concept.

Revised Solution

	The Client	The Server	The Hacker
1.	Client creates random Public/Private Key Pair and sends the Server the Public Part.	The Server in turns creates a random Public/Private Key Pair and returns Public Part to client.	The Hacker can only view both parties Public Key.
2.	Client Creates a random symmetric key, encrypts it with the previously received Server Public Key, and sends it to Server.	The Server receives the client symmetric key decrypts it with the previously generated Server Private Key. Now the Server Creates a Random Symmetric Key encrypts it with the Client Public Key and returns it back to the Client.	Same thing happens for the Hacker as in the initial proof of concept. The Hacker cannot decrypt the data, as it requires the Server Private key. Again, the Hacker only has the Client Public Key and thus it is not possible for the Hacker to decrypt the Server Symmetric Key.
3.	Client Receives the encrypted Symmetric Key of Server. It decrypts it with previously generated Client Private Key. Client Encrypts the target data with the previously generated Client Symmetric key and sends to Server.	Server Receives the Encrypted data, Server decrypts it with the previously received Client Symmetric key. Server Process the Data. Server encrypts the process result with the previously generated Server Symmetric Key and returns back to Client.	The Hacker does not have the Client Symmetric Key and so it is not possible to decrypt the data. Again, the Hacker does not have the Server Symmetric Key and thus it is not possible for the hacker to decrypt the process result.
4.	The Client receives the encrypted process result and decrypts it with the previously received Server Symmetric Key.
5.	The Client returns to regular processing.	Server goes to sleep.	The Dumb Hacker cries and but the intelligent hacker might be thinking, “Hey there is something to hack”.

In this revised version an extra step just been introduced -the step 2, where both the parties exchanging Symmetric key (Lets call it session key from now on) instead of the actual data like in the initial concept. Now you might be thinking does not the RSA length limitation applies when encrypting the session key. The Answer is no, it does not, even we are exchange the most secure Rijndael key(256 bit) , the data length becomes 44 byte (256/8) which we can easily encrypted/decrypted with RSA. The only weakness of the above solution is, if the intelligent hacker can somehow discover the session key of the both parties, but which I assume is very difficult to discover based upon the encrypted data.

Where is the Digital Signature?

You might be thinking this article starts with Digital Signature, but still no flavor of it. Let me tell you, although the digital signature is not required but you can implement it to make the solution more secure and ensure that even the intelligent hacker can revive both the parties session key but there will be no way the hacker can tamper the data. Here is the final solution.

The Final Solution

	The Client	The Server	The Hacker
1.	Client creates random Public/Private Key Pair and sends the Server the Public Part.	The Server in turns creates a random Public/Private Key Pair and returns Public Part to client.	The Hacker can only view both parties Public Key.
2.	Client Creates a random Session key, encrypts it with the previously received Server Public Key, Digitally Signs it with the Client Private Key and sends it to Server.	The Server receives the client session key, Checks it integrity with the Client Public Key, if the Key has not been tampered, it decrypts the Client Session Key with the previously generated Server Private Key. Now the Server Creates a Random Session Key encrypts it with the Client Public Key, Digitally Signs it with the Server Private Key and returns it back to the Client.	Same thing happens for the Hacker as in the previous two solutions. The Hacker cannot decrypt the data, as it requires the Server Private key, moreover the Hacker founds the data is now Digitally Signed. Again, the Hacker only has the Client Public Key and thus it is not possible for the Hacker to decrypt the Server Session.
3.	Client Receives the encrypted Session Key of Server. It verifies it with the Server Public Key, if not tampered, it decrypts it with previously generated Client Private Key. Client Encrypts the target data with the previously generated Client Session Key, Digitally Signs it and sends to Server.	Server Receives the Encrypted data, Server verifies it with Client Public Key, if not tampered it decrypts the data with the previously received Client Session Key. Server Process the Data. Server encrypts the process result with the previously generated Server Session Key, Digitally Signs it with the Server Private Key and returns it back to Client	The Hacker does not have the Client Session Key and thus it is not possible to decrypt the data, also founds the data is Digitally Signed. Again, the Hacker does not have the Server Session Key and so it is not possible for the hacker to decrypt the process result.
4.	The Client receives the encrypted process result, verifies it with Server Public Key and decrypts it with the previously received Server Session Key.
5.	The Client returns to regular processing.	Server goes to sleep.	Both the Hacker cries now.

In the final solution, the digital signature has been introduced to ensure the data has not been tampered between the communications.

Creating Audit functionality using ASP.NET 2.0

2007-01-29T02:24:00.000-08:00

Introduction

This article describes how to create an audit tool that tracks and records the changes of any fields with their old and new values using an ASP.NET web page. To see the project in action, you can download the code, but you must change your connection properties for the database interaction.

Background

When posed with this issue, I found various solutions that range from SQL Server Triggers to more custom solutions that require additional knowledge of web controls. However, my solution has the following benefits:

Records and tracks only the changes made to the field or fields that are changed. One solution, for example, used a trigger that would populate the audit changes, but this included ALL fields that were in the UPDATE SQL statement. What this means it that it would populate the Audit table with fields that were also not changed, which filled my audit table exponentially.
Plays on events already part of the .NET framework. In this example, I used a GridView that has two key events that allow us to record and compare data changes. You can also use this same logic on the DetailsView as well.
Uses existing properties that allow audit tracking to work. One event property that is part of the GridView control allows us to see the new values via a returned collection object.
Builds a custom and portable data object using generics that you can use for any situation. The only modifications you will need to do is change the connection properties for ADO.NET.

Using the code

The first thing to do is review the CurrentFields.cs code, which you can import into the App_Code folder. This is the class that sets up the column name and column value of the current field values as they are in the database.

public string CurrColName
{
   get { return _colName; }
   set { _colName = value; }
}
public string CurrColValue
{
   get { return _colValue; }
   set { _colValue = value; }
}

When you instantiate the object at runtime, we can then invoke the GetCurrentFields(string empid) method that returns for us a generic list full of CurrentFields objects.

public List GetCurrentFields(string empid) { ...}

       When using the GridView control, the two events of interest are:
OnRowUpdated="GridView1_RowUpdated"
OnRowEditing="GridView1_RowEditing"
              
              Remember, you will have to wire up these GridView events.  This is were the         audit tracking starts.  First, the RowEditing event is fired when the user         clicks the "Edit" command button from the GridView.  This is where you create         the CurrentFields object to copy the column names and values as they currently reside         in the database.  In my example, I pass the empid to get the employee id, which         is unique for each person.
     List currFieldList;  //define globally in the webpage code

protected void GridView1_RowEditing(Object sender,
                                   GridViewEditEventArgs e)
{
 //get empid field to query database
 string strId = GridView1.Rows[e.NewEditIndex].Cells[4].Text;
 //instantiate object
 CurrentFields currFlds = new CurrentFields();          
 //get collection of objects of col names and values
 currFieldList = currFlds.GetCurrentFields(strId); 
 //store in Session for RowUpdated event          
 Session["CurrData"] = currFieldList;                        
}
          
              Next, the user will make a change to the data from the web form.  Now, we look         at the RowUpdated event of the GridView and use the e.NewValues Dictionary collection         returned in the GridViewUpdatedEventArgs event.  This is where we will compare         the current values with the new values.  NOTE:  We have to do two things         here, first, keep track of what fields we are comparing and also, account for null         values IF your database field allows for it.  The downloaded         code includes the class that inserts the audit table changes.  This         is shown here:
     
protected void GridView1_RowUpdated(object sender,
                                   GridViewUpdatedEventArgs e)
{
 //case Session obj
 currFieldList = (List)Session["CurrData"];  
 foreach (DictionaryEntry myDE in e.NewValues)
 {
   int i = 0;
   string key = myDE.Key.ToString();
      
   foreach (CurrentFields currFld in currFieldList)
   {
     if (currFld.CurrColName == key)
     {
       break;   // got a colname match, break out
     }
     i++;
   }

   if (myDE.Value != null)
   {
     string newVal = myDE.Value.ToString();
     if (currFieldList[i].CurrColValue != newVal)
     {
       InsertAuditData.insertAuditChanges("TestNames", "UPDATE", "Harry",
           key, currFieldList[i].CurrColValue, newVal);
       }
        }
   else
   {
     if (currFieldList[i].CurrColValue != "")
   {
        InsertAuditData.insertAuditChanges("TestNames", "UPDATE", "Harry",
           currFieldList[i].CurrColName, currFieldList[i].CurrColValue,
           "");
   }
        }
   }
}



The "using" key word in C#
2007-01-29T02:08:00.000-08:00
Introduction
 So, I was at work today, and someone asked me what happens when you use the  
using
 keyword in C#. Most people will tell you that you that it is something you use that will clean up any unmanaged resources for the specified object, which is not incorrect. But what actually happens at the IL level? How does it "clean up" unmanaged resources? I had my assumptions, but I really didn't know. So, I set out to find out for myself.  Tests
 So, I decided I was going to run through a couple of tests:
 Test #1:
  I wanted to see what the generated MSIL code looks like when I use the  
using
 keyword. So, I wrote some very simple sample code in C#, compiled it, then I decompiled it using ildasm to see the MSIL code. 
 Test #2:
  I wanted to see if I could write code without using the  
using
 keyword that would generate the exact same MSIL code. This process was a little more trial and error, but was fairly easy.  
 Test #1
 Here is my sample code:
[STAThread]
private static void Main(string[] args)
{
     using (Bitmap bitmap1 = new Bitmap(100, 100))
     {
           Console.WriteLine("Width: {0}, Height: {1}", bitmap1.Width, bitmap1.Height);
     }
     Console.ReadLine();
}

    
  As you can see... nothing special in the code. Create a new Bitmap inside a using statement, write some output to the console, wait for user input, then exit.
 So, what does this look like when we build the app, then decompile it into MSIL? Check it out:

.method private hidebysig static void Main(string[] args) cil managed
{
     .custom instance void [mscorlib]System.STAThreadAttribute::.ctor()
     .entrypoint
     .maxstack 4
     .locals init (
           [0] [System.Drawing]System.Drawing.Bitmap bitmap1)
     L_0000: ldc.i4.s 100
     L_0002: ldc.i4.s 100
     L_0004: newobj instance void [System.Drawing]System.Drawing.Bitmap::.ctor(int32, int32)
     L_0009: stloc.0
     L_000a: ldstr "Width: {0}, Height: {1}"
     L_000f: ldloc.0
     L_0010: callvirt instance int32 [System.Drawing]System.Drawing.Image::get_Width()
     L_0015: box int32
     L_001a: ldloc.0
     L_001b: callvirt instance int32 [System.Drawing]System.Drawing.Image::get_Height()
     L_0020: box int32
     L_0025: call void [mscorlib]System.Console::WriteLine(string, object, object)
     L_002a: leave.s L_0036
     L_002c: ldloc.0
     L_002d: brfalse.s L_0035
     L_002f: ldloc.0
     L_0030: callvirt instance void [mscorlib]System.IDisposable::Dispose()
     L_0035: endfinally
     L_0036: call string [mscorlib]System.Console::ReadLine()
     L_003b: pop
     L_003c: ret
     .try L_000a to L_002c finally handler L_002c to L_0036
}
       
  Test #1 Results:
 So the results from test #1 are interesting. the  
using
 keyword is basically a   try
 -  finally
 block, without a  catch
, where IDisposable.Dispose() is called in the  finally
. One interesting thing to note is that the Bitmap constructor is called before the  try
 block begins. This tells me that if, in the Bitmap constructor, an unmanaged resource is allocated, but not freed, then the constructor throws an exception, the unmanaged resource will not get freed by a call to the IDisposable.Dispose(). This also assumes that the IDisposable is implemented properly.  Therefore the  
using
 keyword is useless if the IDisposable is not implemented properly. The constructor should clean up resources if it fails, and the Dispose() method should clean up all unmanaged resources. Chances are good that Microsoft has implemented IDisposable correctly in their classes, so whatch out for this if you are implementing your own IDisposable.   Test #2:
 Based on the MSIL code that resulted from Test #1, I decided to write the same code using  
try
 -  finally
 blocks. This is what I came up with:  [STAThread]
private static void Main(string[] args)
{
     Bitmap bitmap1 = new Bitmap(100, 100);
     try
     {
           Console.WriteLine("Width: {0}, Height: {1}", bitmap1.Width, bitmap1.Height);
     }
     finally
     {
           if (bitmap1 != null)
           {
                 bitmap1.Dispose();
           }
     }
     Console.ReadLine();
}
 And this is what it looked like in MSIL:

.method private hidebysig static void Main(string[] args) cil managed
{
     .custom instance void [mscorlib]System.STAThreadAttribute::.ctor()
     .entrypoint
     .maxstack 4
     .locals init (
           [0] [System.Drawing]System.Drawing.Bitmap bitmap1)
     L_0000: ldc.i4.s 100
     L_0002: ldc.i4.s 100
     L_0004: newobj instance void [System.Drawing]System.Drawing.Bitmap::.ctor(int32, int32)
     L_0009: stloc.0
     L_000a: ldstr "Width: {0}, Height: {1}"
     L_000f: ldloc.0
     L_0010: callvirt instance int32 [System.Drawing]System.Drawing.Image::get_Width()
     L_0015: box int32
     L_001a: ldloc.0
     L_001b: callvirt instance int32 [System.Drawing]System.Drawing.Image::get_Height()
     L_0020: box int32
     L_0025: call void [mscorlib]System.Console::WriteLine(string, object, object)
     L_002a: leave.s L_0036
     L_002c: ldloc.0
     L_002d: brfalse.s L_0035
     L_002f: ldloc.0
     L_0030: callvirt instance void [System.Drawing]System.Drawing.Image::Dispose()
     L_0035: endfinally
     L_0036: call string [mscorlib]System.Console::ReadLine()
     L_003b: pop
     L_003c: ret
     .try L_000a to L_002c finally handler L_002c to L_0036
}
 Test #2 Results:
 It is almost exactly like the MSIL generated from the  
using
 keyword, except this calls Image.Dispose() rather than IDisposable.Dispose() , but the affect is the same.   Just for fun, I regenerated C# code using Lutz Roeder's .NET Reflector, which is an invaluable tool, and here is the result:
[STAThread]
private static void Main(string[] args)
{
     using (Bitmap bitmap1 = new Bitmap(100, 100))
     {
           Console.WriteLine("Width: {0}, Height: {1}", bitmap1.Width, bitmap1.Height);
     }
     Console.ReadLine();
}
 This looks exactly like our origional code!!!


Creating a ubuntu package repository in a local machine
2006-12-16T09:28:00.000-08:00
Install dpkg-dev package

- Go to the directory (folder) with deb packages

- Run dpkg-scanpackages, send the output to a file Packages
and compress it

dpkg-scanpackages . /dev/null > Packages
gzip Packages

- You can also do the previous step in one command

dpkg-scanpackages . /dev/null | gzip - > Packages.gz

- Then add the directory (say /foo/bar/) into /etc/apt/sources.list

deb file:///foo/bar ./

- Finally run `apt-get update' to update apt's list of packages.

chamildezoysa

Good Bye MFC!! (Throwing MFC Out of Windows)

Moving to Windows Vista x64 - .NET 3.0

Contents

A Vector Type for C#

Introduction

Using the code

Extended functionality

Usability functions

Summary

Artificial intelligence network load balancing using Ant Colony Optimisation - C# Algorithms

Introduction

Background

Routing algorithms

Introducing ANTNet

The algorithm

The program

Program features

Classes

Simulation results

Linux on the Nokia 770 Internet Tablet

The Machine

Data Entry

The Hacks

File Manager

Change the Logo and Sounds

Data Entry

Use Wikipedia

Hacks of the Near Future

Conclusion

Using a vista side bar gadget to consume an image feed

Introduction

What is a Sidebar Gadget?

Gadget Development Tools & Resources

Globalization

Step 1: The manifest File

Step 2: The Settings Page

Step 3: The Gadget UI

gadget.js

ajax.js

Layout

Step 4: The Flyout

ajax.js

Event Sequence

gadget.js

External Links

Step 5: Packaging Your Gadget

The Finished Product

Exchange Data Securely Without HTTPS/SSL-C# Web Services

Introduction

Advantages

Requirements

Background

Symmetric VS Asymmetric

Digital Signature

Alternate Solutions

The Solution

Initial Proof of Concept

.NET Platform

Symmetric Algorithms

Asymmetric Algorithms

Implementation

Revised Solution

Where is the Digital Signature?

The Final Solution

Creating Audit functionality using ASP.NET 2.0

Introduction

Background

Using the code

The "using" key word in C#

Introduction

Tests

Test #1

Test #2:

Creating a ubuntu package repository in a local machine