DCSIMG
June 2008 - Posts - Shai Raiten's Blog

Shai Raiten's Blog

It's all about code...

June 2008 - Posts

Add an uninstall shortcut to setup project

Add an uninstall shortcut to setup project

1. Create simple project UninstallApp.
2.

Main()
{
    string[] arguments = Environment.GetCommandLineArgs();
    foreach(string argument in arguments)
    {
        string[] parameters = argument.Split('=');
        if (parameters[0].ToLower() == "/u")
        {
            string productCode = parameters[1];
            string path = Environment.GetFolderPath(Environment.SpecialFolder.System);
            Process proc = new Process();
            proc.StartInfo.FileName = string.Concat(path,"\\msiexec.exe");
            proc.StartInfo.Arguments = string.Concat(" /i ", productCode);
            proc.Start();
        }
    }
.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

3. Create new setup project
4. Add UninstallApp.exe to "Application Folder" in 'File System' part
5. In "User's Program menu" create shortcut to UninstallApp.exe and in properties of this shortcut in parameter 'arguments' insert value "/u=[ProductCode]".
6.Rebuild deployment project.

I'm Leaving Ness

I'm Leaving Ness image

After a long time of thinking and thoughts I decide to quit Ness.

Last week I announced Ness that I’m leaving them after 1.5 years and starting my way to the next big thing.

Ness was a wonderful home, I’ve learned a lot and thanks for Ness I found Team System.

No matter where I go the Blog is still my second house.

Wish me good luck.

Sigist Conference 2008 is Over

Sigist Conference 2008 is Over

I had the pleasure working with two wonderful people :
"Hila Lahav Rice" & "Guy Burstein".

Me & "Hila Lahav Rice" took one of the keynotes in this conference in order to show a case study of Team System implementation by v-ness in Easy Forex.
In this session we also shared the vision of Microsoft for ALM solutions,
future plans and key areas of focus (mostly around testing).

During the preparation Guy gave me one very important advice:

  • Don't come to your presentation with - sandals!

Working with Guy was a real pleasure and I have learned a lot from him,
business & personal, Hope I will work with you again.

I have to say it: "Guy Burstein" you are my Hero!

Thanks

IMG_2961 IMG_2962 IMG_2963 IMG_2964 IMG_2965 IMG_2966 IMG_2967 IMG_2968 IMG_2969 IMG_2971 IMG_2972 IMG_2978 IMG_2980 IMG_2981 IMG_2983 IMG_2984 IMG_2985 IMG_2991 IMG_2992

TeamBuild Properties

TeamBuild 2005 Properties

We have comprehensive inline documentation for the properties defined in TfsBuild.proj file. These are very commonly used properties. There are several additional properties and overridable targets defined in TeamBuild targets file (Microsoft.TeamFoundationBuild.targets) that can be helpful while customize the build process.
Here is the complete laundry list of these properties and targets.

Properties the can be overridden

SkipClean set this property to skip clean action
SkipGet set this property to skip get action
SkipLabel set this property to skip labeling action
SkipInitializeWorkspace set this property to skip workspace creation action

SkipPostBuild

set this property to skip gathering associated changesets and workitems
SkipDropBuild set this property to skip drop build action.
SkipWorkItemCreation set this property to skip workitem creation on build failure
AdditionalVCOverrides additional VC property sheets passed to the VC++ solutions in the build process
SolutionRoot points to the root of the sources folder on build machine

Other Properties

WorkspaceName

identifies the name of the workspace created for this build

IsDesktopBuild identifies if the build process is a desktop build
BinariesRoot points to the binaries folder on build machine
TestResultsRoot points to test results folder on build machine
BuildType identifies the name of the build type
BuildNumber identifies the name of the build, i.e., the build number
TeamFoundationServerUrl

identifies the URL of the team foundation server

Overridabe Targets

BeforeEndToEndIteration called immediately after the full stack build is started on the build machine
AfterEndToEndIteration called at the end of full stack build
BuildNumberOverrideTarget target to override to customize the build number
BeforeClean called before clean is attempted
AfterClean

called after clean is completed

BeforeGet

called before sources are get

AfterGet called after sources are fetched
BeforeLabel

called before sources are labeled

AfterLabel

called after labeling is completed

BeforeCompile called before compilation is started
AfterCompile called after compilation is completed
BeforeTest called before testing is attempted
AfterTest called after testing is completed
BeforeDropBuild called before dropping the built binaries, build log files and test results to release server
AfterDropBuild called after dropping the built binaries, build log files and test results to release server
BeforeOnBuildBreak called before creating workitem on the build break
AfterOnBuildBreak called after workitem is created on the build break

Load Testing Features of Visual Studio Team System

Load Testing Features of Visual Studio Team System image

Visual Studio Team Test Edition provides two capabilities that can allow users to create load tests of web sites quickly and easily: the Web Test recorder and the Load Test wizard.    In fact, these tools make this process so easy that it is tempting to use the web tests and the load tests that result from using these tools without much modification.   However, to use the web testing and load testing capability most effectively it is beneficial to understand how to use other web test and load test properties that are not set by these tools.    This note describes some of the most important considerations for load testing with Visual Studio Team System that are not addressed by the web test recorder and the load test wizard.

Verify web tests and unit tests

Before running a load test, it is a good idea to make sure that all of the tests contained in the load test will pass when run by themselves by running them from either the Test Explorer or Test View windows, or from the Web Test editor in the case of a web test.    For data bound web tests, run through all of the data values.

See Josh Christie’s excellent paper Web Test Authoring and Debugging Techniques at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnvs05/html/WTAuthDebug.asp for more details on creating and verify web tests.   Some additional web test properties that are useful when running load test are described later in this document.

Choose an appropriate load profile 1

 

Choose a load profile for each Scenario in your load test that is appropriate for your test goals.    Here is a summary of the three choices for load profile.

 

Using a Constant Profile

A Constant Load Profile can be used to run the same user load throughout a load test.   Be careful about using a Constant Load Profile with a high user count; doing so may place an unreasonable and unrealistic demand on your server(s) at the beginning of the load test.   For example, if your load test contains a web test that starts with a request to a home page, and you set up the load test with a constant load of 1,000 users, the load test will submit the first 1,000 requests to the home page as fast as possible.   This may not be a realistic simulation of real-world access to your web site.    To mitigate this, consider using a Step Load Profile that ramps up gradually to 1,000 users, or specify a warm-up period in the Load Test Run Settings.   If a warm-up period is specified, the load test will automatically increase the load gradually during the warm-up period.

Using a Step Load Profile

A Step Load Profile can be used to increase the load on the server(s) as the load test runs so that you can see how performance varies as the user load increases.     For example, to see how your server(s) perform as the user load increasing to 2,000 users, you might run a 10-hour load test using a Step Load Profile with the following properties:
Initial User Count: 100
Maximum User Count: 2000
Step Duration (seconds): 1800
Step Ramp Time (seconds): 20
Step User Count: 100
These settings have the load test running for 30 minutes (1800 seconds) at user loads of 100, 200, 300, up to 2,000 users.   The Step Ramp Time property is worth special mention here because it is the only one of these properties that is not available to choose in the Load Test Wizard.   This property allows the increase from one step to the next (for example from 100 to 200 users) to be gradual rather than immediate.   In this example, the user load would be increased from 100 to 200 users over a 20 second period (an increase of 5 users every second).

Using a Goal-Based Load Profile

A Goal Based Load Profile is useful when you want to determine the number of users that your system can support before reaching some level of resource utilization.    This option works best when you have already identified the limiting resource (i.e. the bottleneck) in your system.   For example, if you know that the limiting resource in your system is the CPU on your database server, and you want to see how many users can be supported when the CPU on the database server is approximately 75% busy, you could use a Goal Based Load Profile with the goal of keeping the value of the performance counter “%Processor Time” between 70% and 80%.    One thing to watch out for is that if some other resource is limiting the throughput of the system, the goal specified by the Goal Based Load Profile may never be reached, and the user load will continue to rise until the value specified for the Maximum User Count is reached.  This is usually not the desired load, so be careful about the choice of the performance counter in the Goal Based Load Profile, and also make a conscious decision about the value for the Maximum User Count to place an upper bound on the user load.

Choosing the location of the Load Test Results Store

2

When the Visual Studio Team Test Controller is installed, the Load Test Results Store is set up to use an instance of SQL Express that is installed on the controller computer.    SQL Express is limited to using a maximum of 4 GB of disk space.   If you are going to run many load tests and want to keep the results for a while, you should consider configuring the Load Test Results Store to use an instance of the full SQL Server product if available.    See the Visual Studio Team Test documentation for instructions on setting up the database to be used as the Load Test Results Store.
Increase the performance counter sampling interval for longer tests
Choose an appropriate value for the “Sample Rate” property in the Load Test Run Settings based on the length of your load test.  A smaller sample rate, such as the default value of five seconds, requires more space in the load test results database.  For longer load tests, increasing the sample rate reduces the amount of data collected.
Here are some guidelines for sample rates:

Load Test Duration Recommended Sample Rate
      < 1 Hour                   5 seconds
    1 - 8 Hours                 15 seconds
   8 - 24 Hours                 30 seconds
     > 24 Hours                 60 seconds

Consider including Timing Details to collect percentile data

3

There is a property on the Run Settings in the Load Test Editor named "Timing Details Storage".  If Timing Details Storage is enabled, then the time to execute each individual test, transaction, and page during the load test will be stored in the load test results repository.   This allows 90th and 95th percentile data to be shown in the load test analyzer in the Tests, Transactions, and Pages tables.   The amount of space required in the load test results repository to store the Timing Details data may be very large, especially for longer running load tests.  Also, the time to store this data in the load test results repository at the end of the load test is longer because this data is stored on the load test agents until the load test has finished executing at which time the data is stored into the repository.   For these reasons, Timing Details is disabled by default.  However if sufficient disk space is available in the load test results repository, you may wish to enable Timing Details to get the percentile data.  Note that there are two choices for enabling Timing Details in the Run Settings properties named "StatisticsOnly" and "AllIndividualDetails".  With either option, all of the individual tests, pages, and transactions are timed, and percentile data is calculated from the individual timing data.   The difference is that with the StatisticsOnly option, once the percentile data has been calculated, the individual timing data is deleted from the repository.   This reduces the amount of space required in the repository when using Timing Details.   However, advanced users may want to process the timing detail data in other way using SQL tools, in which case the AllIndividualDetails option should be used so that the timing detail data is available for that processing.

Consider enabling SQL Tracing

4

There is a set of properties on the Run Settings in the Load Test Editor that allow the SQL tracing feature of Microsoft SQL Server to be enabled for the duration of the load test.   If enabled, this allows SQL trace data to be displayed in the load test analyzer on the "SQL Trace" table available in the Tables dropdown.   This is a fairly easy-to-use alternative to starting a separate SQL Profiler session while the load test is running to diagnose SQL performance problems.  To enable this feature, the user running the load test (or the controller user in the case of a load test run on a rig) must have the SQL privileges needed to perform SQL tracing, and a directory (usually a share) where the trace file will be written must be specified.   At the completion of the load test, the trace file data is imported into the load test repository and associated with the load test that was run so that it can be viewed at any later time using the load test analyzer.

Don’t Overload the Agent(s)

If an agent machine has more than 75% CPU utilization or has less than 10% of physical memory available, add more agents to your test rig to ensure that the agent machine does not become the bottleneck in your load test.

Add an Analysis Comment

After the load test is complete and you have spent some time analyzing the results, you can a short one line description and an arbitrarily long analysis comment to be stored permanently with the load test result.    To do this, in the load test result viewer, right click and choose the “Analysis” option.   This brings up a dialog that allows you to enter your analysis text which is stored in the load test results database you click OK to close the dialog.

Consideration for Load Tests that contain Web Tests

5

The following considerations apply to load tests that contain web tests, but are not applicable to load tests that contain only unit tests.

Choose the Appropriate Connection Pool Model

The property WebTest Connection Model found on the RunSettings property sheet in the Load Test editor allows you to choose between two models for managing the connections that the load test runtime engine uses for connecting to the target web site(s):

The ConnectionPerUser model most closely simulates the behavior of a real browser: each virtual user running a web test creates one or two connections to the web server that are dedicated to that virtual user. The first connection is established when the first request in the web test is issued. A second connection may be used when a page contains more than one dependent request; these requests may be issued in parallel using the two connections. These connections are re-used for subsequent requests within the web test, and are closed when the web test completes. The only drawback to the ConnectionPerUser model is that the number of connections held open on the agent machine may be high (as high as 2 times the user load), and the resources required to support this high connection count may limit the user load that can be driven from a single load test agent.  The extra resources used are the memory associated with the connection and extra processing time to close and reopen the connections as web tests complete and new web tests are started.

The ConnectionPool model conserves the resources on the load test agent by sharing connections to the web server among multiple virtual web test users. When the Connection Model is ConnectionPool, the Connection Pool Size specifies the maximum number of connections to make between the load test agent and the web server. If the user load is larger than the connection pool size, then web tests executing on behalf of different virtual users will share a connection. This means that one web test will need to wait before issuing a request when another web test is using the connection.  The average time that a web test waits before submitting a request is tracked by the load test performance counter “Avg. Connection Wait Time”.  The value of this performance counter should be very low; if it is not, you need to increase the connection pool size to prevent it from limiting the throughput of your load test.

Consider setting response time goals for web test requests

6

One of the properties of a web test request is a response time goal.  If you define response time goals for your web test requests, when the web test is run in a load test, the load test analyzer will report the percentage of the web tests executed within the load test for which the page response time was less than the goal.  Note that the response time goal is for the entire page, meaning the time to receive the response for the request specified in the web test including the time to receive the response for any dependent requests.  By default there are no response time goals defined.

Consider setting timeouts for web test requests

Another property of a web test request is a timeout value.  By default there is no timeout value specified on web test requests, so if the web server never responds to the request, the web test request (and the web test) will never complete and no error will be reported.     If a timeout value (even a high one such as 300 seconds) is set, then the web test request will eventually timeout, an error will be reported, and the web test will continue.

Consider setting the “ParseDependentRequests” property of your web test requests to false

By default, when a web test is executed, HTML responses received from the web server are parsed and any dependent requests such as images or style sheets are automatically submitted.   This is usually desirable as it places the most realistic load on the server.    However, if the goal of your load test is to place the maximum load on your application server as opposed to your web server, you may wish to disable this behavior by setting the ParseDependentRequests on all of your web test requests to false.

Choose a value for the “Percentage of New Users” property

7

There is a property on each scenario in a Load Test named "Percentage of New Users”.  This property affects the way in which the load test runtime engine simulates the caching that would be performed by a web browser.   The default value for the “Percentage of New Users” property is 100%.   This means that each web test run in a load test is treated like a first time user to the web site who doesn't have any content from the web site in their browser cache from previous visits.   Thus all requests in the web test (including all dependent requests such as images) are downloaded (except in the case where the same cacheable resource is requested more than once in a web test).  If you are load testing a web site that has a significant number of return users who are likely to have images and other cacheable content cached locally, then using the default setting of 100% for “Percentage of New Users” will generate more download requests than would occur in real-world usage.   In this case, you should estimate the percentage of visits to your web site that are from first time users of the web site, and set “Percentage of New Users” accordingly.

For any questions or help, feel free to contact me.

Sigist 2008

image 2008             

SIGIST Israel purpose and goals are to create a place in Israel were testing professionals can meet, speak on common issues, suggest solutions to common problems, and be a place were they can come up with a request for an answer, and get it! It aims to be the driving force and the trigger for recognition for the testing profession in Israel and is hoping to increase the awareness toward certification for the different levels of testing (using ISTQB certification scheme).

A month ago I received a call from Guy Burstein asking me if I want to talk in Sigist about one of my Team System projects.

I had a real pleasure and honor working with Guy Burstein on this presentation.

Me and "Hila Lahav Rice" going to talk in Sigist about:
"Microsoft’s ALM solution – Implementing Testing  & QA Measurements"

image


Here is the Presentation schedule:
image

 

 

 

 

 

Hope to see you there!

Posted: Jun 16 2008, 12:05 AM by Shai Raiten | with 4 comment(s) |
תגים:

Managing builds through Command Line

Managing builds through Command Line

Start

Starting a build requires START_BUILD permissions. Scenarios where you might want to use the command line to start a build are:-

  1. You want to integrate kicking off a build into your workflow; where build might be just one of the steps.
  2. You want builds to be kicked off at regular intervals of time. You could use the Windows scheduler and the command line interface to accomplish this.
  3. You might want to extend the functionality of the TFSBuild system provided. An example is writing a system for continuous integration.
  4. You just love working on the console:). A simple telnet to the office from home would enable you to do the needful.

D:\Program Files\Microsoft Visual Studio 8\Common7\IDE>TFSBuild.exe start http://MyTfs:8080/MyTestTeamProject MyTestTeamBuildType
Microsoft (R) TfsBuild Version 8.0.0.0
for Microsoft (R) Visual Studio 2005 Team System
Copyright (C) Microsoft Corporation 2004. All rights reserved.
Build number: MyTestTeamBuildType_20050926.2
Initializing build
    Getting sources
    Compiling sources
    Compiling ConsoleApplication1.sln for Any CPU/Release
    Compiling ConsoleApplication1.csproj
    Compiling TestProject1.sln for Any CPU/Release
    Compiling TestProject1.csproj
    Compiling WindowsApplication1.sln for Any CPU/Release
    Compiling WindowsApplication1.csproj
    Getting changesets and updating work items
    Running tests
    Running tests for Any CPU/Release
    Copying binaries to drop location
    Copying log files to drop location
    Successfully Completed


Dump 1 Command line start console output

Compare this with the build report shown through the UI:-

  1. In the console output for the build; only the build steps are shown and the "Completed On" time for these steps is not shown. The idea is that when you are using the command line build; your main intent is to fire the build and nothing else.
  2. If you want to change the quality of the build or view the time taken for each build step you would have to go to the UI. For viewing other stuff such as change sets and work items; you could look up the Buildlog.txt in the drop share. 

Stop

A user starting the build or having ADMIN_BUILD permission can only stop the build.

  1. To stop a currently running build use this functionality.
  2. "Stopping" the build would kill the build process on the build machine and mark the status of the build as "Stopped".
  3. The event log on the AT has an entry per build stopped containing data about the user who stopped the build and the time when the build was stopped.
  4. There is no way to resume or unstop a stopped build.

D:\Program Files\Microsoft Visual Studio 8\Common7\IDE>tfsBuild stop http://MyTfs:8080 MyTestTeamProject MyTestTeamBuildType_20050926.3 /noprompt
Microsoft (R) TfsBuild Version 8.0.0.0
for Microsoft (R) Visual Studio 2005 Team System
Copyright (C) Microsoft Corporation 2004. All rights reserved. 
Stopping MyTestTeamBuildType_20050926.3... Done


Dump 2 Command line stop console output

Delete

ADMIN_BUILD permissions are required to delete a build. Deleting a build having tests enabled also requires PUBLISH_BUILD permisssions.

  1. Used for removing unneeded builds from the system
  2. Can delete multiple builds from the same Team Project on a single invocation.
  3. A delete on an ongoing build is not supported.
  4. Deleting a build entails deleting the entries for that build from the database and removing the binaries for that build from the drop location.
  5. Deleting the most recent build from a team project would cause the next build to have the same build number as the deleted build.
  6. Again an entry per deleted build is logged in the event log at the AT.

D:\Program Files\Microsoft Visual Studio 8\Common7\IDE>tfsBuild delete http://MyTfs:8080 MyTestTeamProject MyTestTeamBuildType_20050926.1 MyTestTeamBuildType_20050926.3 /noprompt
Microsoft (R) TfsBuild Version 8.0.0.0
for Microsoft (R) Visual Studio 2005 Team System
Copyright (C) Microsoft Corporation 2004. All rights reserved. 
Deleting MyTestTeamBuildType_20050926.1... Done
Deleting MyTestTeamBuildType_20050926.3... Failed
The path is not of a legal form.


Dump 3 Command line delete console output

The failure to delete the last build is a known issue in Beta3 bits. This happens when a build that was stopped in the "Initializing Build" step is attempted to be deleted. The build has been deleted from the database; but not from the drop location. The workaround is to manually delete the folder from the drop share.

Validation error when saving Work Items after a user is removed from Active Directory

Validation error when saving Work Items after a user is removed from Active Directory

Last week I was called to give a consulting about a Team System problem.
There is some Work Items that no one can change because validation error.

The problems begin when the IT manager start to Delete old employees from the Active Directory.

If you remove a user from Active Directory or the local machine (depending where you store your user accounts for TFS), all work items that were related to that user cannot modified anymore.

The following validation error appears that requires the value in the “Closed By” field to be a valid user:image

Changed By, Active By, Resolved By, Assign To etc etc........

Workaround 1 (NOT recommended): You can add the users back to the Active Directory and disable the account, this way the user still exists in the AD but cannot perform login into the system.(Security Reasons).

Workaround 2 (recommended): You can modify the Work Item Type definitions so those field does not re-validate values that have been successfully validated in the past. In this case there would be not error, however if you put in a new name into the field it would be validated.

1. Export the WIT (Work Item Type) of which you want to change the behavior as described in my post - Customizing Work Items in Visual Studio.
2. Just add the <ALLOWEXISTINGVALUE /> rule to the field definition.

"Assign To" Example:

<FIELD reportable="dimension" type="String" name="Assigned To" refname="System.AssignedTo">        
<ALLOWEXISTINGVALUE />
  <VALIDUSER />    
  <REQUIRED />
</FIELD>