August 2010 - Posts
Why I Became a .Net Developer?
I’m being asked from time to time why I’ve chosen to work with .Net framework and in Microsoft world. Today I got that question again so I thought it will be nice to share the answer with my Blog readers.
In my early days when I was a computer science student I started my first steps in development world with the Java language. At the end of my first year I took a summer course in .Net in order to learn what it was all about. To tell the truth, I wasn’t impressed by the language (.Net framework 1.0 was released at that time). I kept on taking advanced Java courses like J2EE for example and deepened my Java skills and thought I won’t see C# anymore.
At the fourth semester I had a mandatory course in analyzing and building databases. As part of this course I bumped into C# and .Net for the second time. This time we had to build a simple Outlook like project in C# with a SQL Server backend. Still, I preferred Java since I was more familiar with the language and felt more comfortable programming with it. During that time I started to search for a development job and couldn’t find any (these were the years after the Dot-com bubble exploded).
At the starting of my fifth semester I found a job (at last) as a junior .Net developer in a small company that was called then Addwise (now it is called Netwise). Even though I preferred to find a job in the Java world I took the rational decision and started to work at the end as a .Net developer. I guess that if I’ve waited more time and didn’t take that job I would have been at another place in life right now. But I don’t regret the decision I made at all and I’m glad I made it. So the real answer to why I became a .Net person can be expressed only in one word – accidently :-).
The Customer is Always Right
One part of my job is consulting in a lot of .Net development areas.
One of the annoying things that you have when you are technology consultant is that customers think that they can demand from you almost everything (even things that aren’t part of your job role). The problem with these thoughts is that if you don’t give them the things they want they can go to your boss and complain. That means that you’ll have to answer to whoever is paying your salary about things that aren’t exactly your everyday work. Moreover, since “the customer is always right” then you’ll have to make a really good case in order to get out of troubles.
I had such an event lately when I gave to a customer details about how to create some feature by some specifications that you can find in the internet. Since this customer is using another programming language I couldn’t take part in their development cycle so I was involved only in delivering the specifications. After some time they finished the development of the feature and launched it. Of course there were problems and the feature didn’t work properly.
So what do you do when something that you developed isn’t working properly? I for example debug and test it and try to find out what is wrong with it. What this customer did? they addressed me and wanted me to debug their feature… I tried to help them by sending them information about the problem they experienced. I also found out by using a web debugger tool that even though I sent them the specifications for how to create the feature they didn’t followed them (which is a big mistake to do since the tool they built the feature for demands the following of those specifications).
A week later I was contact by the person who is in charge of the consulting session. He wanted to understand why I’m not helping the customer… I explained him what I've done and delivered him my mail correspondence with the customer. He backed me up and understood that the problem exists with the customer implementation. Now the ball is in the customer’s court and they work to repair the feature according to the specifications I provided in the first place. The customer is always right (is he/she?).
Select N+1 Problem – How to Decrease Your ORM Performance
Today one of the developers at my main customer showed me a code
snippet he wrote against Entity Framework and made me very pale. The code included the horrible select N+1 problem. This post will introduce the select N+1 pitfall and will explain how avoid it in Entity Framework.
What is Select N+1 Problem?
ORMs can help you to address the impedance mismatch between relational databases and object oriented models and by that make your life simpler. But not knowing about some of their pitfalls can decrease your performance dramatically. One of those pitfalls is the select N+1 problem. This problem is being caused mainly because most of the ORMs out there are enabling lazy loading behavior by default. When we have a parent-children relation the problem can raise its ugly head. The problem is happening when we are executing a single query and then N following queries (N is the number of parent entities) in order to query for something. As you can expect doing N+1 queries instead of a single one will flood your database with queries that we can and should avoid. This is very unacceptable.
Select N+1 Example
To explain the problem more properly lets look at an example. Lets say that we have the following model:
A department can hold 0 or more courses (a typical parent-children relation). Since EF4 enables the lazy loading behavior by default then the following code will raise the select N+1 problem:
using (var context = new SchoolEntities())
{
foreach (var department in context.Departments)
{
foreach (var course in department.Courses)
{
Console.WriteLine("{0}: {1}", department.Name, course.Title);
}
}
}
And the result is:
All I wanted to do is to write to the output the titles of the courses and attach to them their parent department name. In the database I got one query to retrieve all the departments and then N queries to retrieve each and every one of the courses for that department. Since in my database there are only 4 departments then I got 5 queries (1 for departments and 4 for all the courses for each department). Now in real world scenario when there are many parents… you can figure the amount of queries you’ll be generating without even knowing you did that.
How to Avoid the Problem in Entity Framework?
One of the main solutions to the select N+1 problem in Entity Framework is to use the Include method.
The Include method is making an eager load for the children that you indicate to it. You give the method a path of all the children you like to load in the query (as long as you have a relation between the entities) and one query will be generated to bring back all the relevant entities. This isn’t a bullet proof solution! There are serious implications that you should understand when you use the Include method. The main implication is that it is doing a join between all the tables that you want to return and the data is retrieved in a flatten manner in order to materialize all the entities from it. Also the materialization process when having a lot of included entities can cause a downgrade of performance. So you will have to weight the balance between using Include or lazy loading. The following code will generate the same results as in the above figure but with only one query:
using (var context = new SchoolEntities())
{
foreach (var department in context.Departments.Include("Courses"))
{
foreach (var course in department.Courses)
{
Console.WriteLine("{0}: {1}", department.Name, course.Title);
}
}
}
and take a look at the generated query:
SELECT [Project1].[DepartmentID] AS [DepartmentID],
[Project1].[Name] AS [Name],
[Project1].[Budget] AS [Budget],
[Project1].[StartDate] AS [StartDate],
[Project1].[Administrator] AS [Administrator],
[Project1].[C1] AS [C1],
[Project1].[CourseID] AS [CourseID],
[Project1].[Title] AS [Title],
[Project1].[Days] AS [Days],
[Project1].[Time] AS [Time],
[Project1].[Location] AS [Location],
[Project1].[Credits] AS [Credits],
[Project1].[DepartmentID1] AS [DepartmentID1]
FROM (SELECT [Extent1].[DepartmentID] AS [DepartmentID],
[Extent1].[Name] AS [Name],
[Extent1].[Budget] AS [Budget],
[Extent1].[StartDate] AS [StartDate],
[Extent1].[Administrator] AS [Administrator],
[Extent2].[CourseID] AS [CourseID],
[Extent2].[Title] AS [Title],
[Extent2].[Days] AS [Days],
[Extent2].[Time] AS [Time],
[Extent2].[Location] AS [Location],
[Extent2].[Credits] AS [Credits],
[Extent2].[DepartmentID] AS [DepartmentID1],
CASE
WHEN ([Extent2].[CourseID] IS NULL) THEN CAST(NULL AS int)
ELSE 1
END AS [C1]
FROM [dbo].[Department] AS [Extent1]
LEFT OUTER JOIN [dbo].[Course] AS [Extent2]
ON [Extent1].[DepartmentID] = [Extent2].[DepartmentID]) AS [Project1]
ORDER BY [Project1].[DepartmentID] ASC,
[Project1].[C1] ASC
Summary
There are pitfalls when we are using ORMs and one of them is the select N+1 problem. This isn’t a problem of Entity Framework only. This problem exists in other ORMs like NHibernate, LINQ to SQL and more. You should be aware of those problems when you develop with ORMs and avoid them whenever it is possible. One way to do that is the Include method in Entity Framework but this solution also can generate problems.
My Article on This Week On Channel 9 Show
I got an email yesterday from Guy Burstein that pointed me to the “This week On 
Channel 9” show which includes a reference to one of the articles I published in
The Code Project site. The article “How To Use Unity Container In ASP.NET MVC Framework” explains how to use Unity IoC container inside an ASP.NET MVC application. You can see the show here. They mention the article about 09:28 minutes after the show starts.
Very flattering.
Using .Net WCF Syndication API to Read a Comments Feed
In the last few days I have started a migration process to my own domain
– www.gilfink.net. Currently this domain is redirecting to this blog but in
the near future I’ll move my blog to there. I’ll keep publishing in this blog along with the new one so don’t worry. One of the biggest problems I got stuck with was that this blog community server doesn’t include an export feature like in other blog platforms like BlogEngine.Net. So I’m currently building a migration tool to help me move my blog content. In this post I’ll show an example of how to read all the blog’s comments using the .Net WCF syndication API.
The Problem
I’m using the XmlRPC.Net library to read all the blog posts but it doesn’t read the post comments.
The Solution
Since the blogs platform has an RSS for all the comments then I can use the .Net WCF syndication API to read them and add them to my blog post structures.
What is The WCF Syndication Library?
Since .Net 3.5, WCF has a syndication library to help reading feeds of Atom and RSS. This library holds a simple API and supports Atom 1.0 and RSS 2.0 formats. In order to read or write a feed we can use the SyndicationFeed class that represent an instance of a feed or use the SyndicationItem class that represent an item in the feed. There are more classes to help parse feeds but the previous written classes are the major ones. The library can be found in the System.ServiceModel.Syndication namespace which is part of the System.ServiceModel namespace.
How to Use the Syndication API to Read a Comments RSS?
The first thing to do is to reference the System.ServiceModel library.
Then I have created a Comment struct to hold the comment data:
public struct Comment
{
#region Properties
public string title { get; set; }
public string link { get; set; }
public string author { get; set; }
public DateTime pubDate { get; set; }
public string description { get; set; }
#endregion
}
Now I can create a method that will return all the comments for a specific post id:
public static List<Comment> GetAllPostComments(int postid, string commentsRssPrefix)
{
List<Comment> result = new List<Comment>();
var commentsRSSURL = string.Concat(commentsRssPrefix, postid);
SyndicationFeed feed = SyndicationFeed.Load(new XmlTextReader(commentsRSSURL));
foreach (var item in feed.Items)
{
result.Add(new Comment
{
author = item.Authors.Count > 0 ? item.Authors[0].Name : string.Empty,
description = item.Content != null ? item.Content.ToString() : string.Empty,
link = item.Links.Count > 0 ? item.Links[0].Uri.ToString() : string.Empty,
pubDate = item.PublishDate.UtcDateTime,
title = item.Title.Text
});
}
return result;
}
In the GetAllPostComments method I create the Url for the comments RSS. Then I create a SyndicationFeed object by using the Load static method in the SyndicationFeed class. This will enable me to read the feed data. Then I read all the syndication items and create the comments from the item data.
Using this method can look like:
var comments = Migrator.GetAllPostComments(689817, "http://blogs.microsoft.co.il/blogs/gilf/rsscomments.aspx?PostID=");
foreach (var comment in comments)
{
Console.WriteLine(comment.title);
}
Now I can add my comments to my posts.
Easy as that.
Summary
The .Net WCF syndication API makes it very easy to read RSS/Atom feeds. All we need to do is to load the syndication into a SyndicationFeed object and then use the syndication API to read the data we need. The .Net syndication library also supports the creation of a syndication feed using the same API.
EFProf – Profiler Tool for Entity Framework
One of the important tools in your tool arsenal when you develop with an ORM tool 
is a profiler. Like SQL profiler to a SQL Server DBA an Entity Framework profiler is a must have tool to Entity Framework developer.
The main reasons to acquire such a tool are to understand what is going on underneath the hood in the query engine and for performance tuning. If you think that you can use Entity Framework without knowing what it is committing to database then you should think again. Things like select N+1 or avoiding too many joins will get back at you during runtime and kick you in the face.
EFProf is an Entity Framework profiler from Hibernating Rhinos.
It includes a very intuitive views/reports and supply a lot of valuable information about what is going on when you use Entity Framework. It also provides a lot of performance alerts which can lead you to find bottle necks or problems in writing the data access queries. Behind the tool you can find Oren Eini (Ayende) which is very known for his support for NHibernate and he brings tones of knowledge in the ORMs world to the tool.
How to Get Started with EFProf
When you want to use EFProf you first need to do the following things:
- Add reference to the HibernatingRhinos.Profiler.Appender dll in your application.
The dll can be found in the tool folder. - In the application start you will call the following line of code:
HibernatingRhinos.Profiler.Appender.EntityFramework.EntityFrameworkProfiler.Initialize();
- Now start the profiler application and then run your application and get the profiler input.
Easy as that.
Using the Tool
When you start the profiler you will get the following screen:
After I configured my test application and ran it I got the following screen with all the information I needed:
You can see the queries that I run at the bottom, application statistic at the left menu and more important stuff which gives you very crucial details about your running application. You can also get analysis of your code in the left menu such as:
One feature that I really liked and made me feel like a DBA was the ability to see the query plan of the query in a visual way:
And of course the ability to run your queries from the profiler:
The tool also supports alerts about common mistakes that you can do when you program against an ORM.
Summary
Profiling your application is a very important aspect for gaining better performance and learning what you are doing in code when you use a tool that you didn’t write. Blind development without understanding what is going inside a development tool is something that will get back at you after deployment. EFProf supply the insight on what is going on and is a very helpful tool to have.
As a last word, since EFProf is a commercial tool this is my opinion that is expressed in this post and I wasn’t contact by Hibernating Rhinos.
The Need for Source Control
Last week I’ve been at a customer.
The problem – non responsive UI since all the application run on the UI thread.
So I took a look at the code and…
I don’t like to tell about the horrible code I saw but to share a very painful problem I have seen.
Even though source control systems are very common, there are customers that can make you “shout” at them when you come to a consulting session. In this customer controlling your sources in a repository tool like TFS, SVN or VSS was a very unnatural thing to do.
They prefer to use the file system for versioning…
They have directories for every version they make…
The solution for the UI problem was very easy (just run the long processing in a background thread) but the remedy to the organization wasn’t so simple. Try to convince managers that the way they work is very unacceptable.
A lot was discussed about source control system and their crucial part in the life cycle of a project. When you start to develop, the code you are writing will refactor and change during the project life time. The ability to be able to change it without concerning about versions can make you more calm and happy. If you succeeded to wrack something you can always go back and take the previous version to continue from it. This safety net is very crucial when you start to search for some version which includes some code that was deleted by mistake or changed by another developer.
Also, you are not working alone. There is a team of developers that work on the same project with you. If every one of those developers will keep his/her source not in a shared repository but in his/her file system the project is doomed. Try to make a merge of source code between two developers that keep their source code of the same project and think about the headache it can cause. Now multiply the developers numbers…
you get get the point.
Once and for a while customers I’m consulting for can make me wonder how the hell did they make it with their development process or even succeed with their products. At the end, I succeeded to make the customer understand the problem of not using source control and hopefully they will integrate a source control in their project development process.
Web Platform Installer – Web Installation on Steroids
One of the most annoying things that I’m sometimes
get stuck with is to install my machine with the latest
components. Trying to grab all the installation packages
for every application or library that I need can be a
very painful thing and also consumes a lot of time.
This is the reason that a tool like Web Platform
Installer was very welcome to my arsenal of tools.
In the post I’ll describe what is the Web Platform Installer and show a scenario of installation.
Web Platform Installer
As I wrote at the beginning of the post, installation can be a very painful thing. The Web Platform Installer is a tool provided by Microsoft to ease this process when we install Microsoft Web Platform. You can download it from here.
So what is this tool?
Taken from Microsoft’s site - “The Microsoft Web Platform Installer 2.0 (Web PI) is a free tool that makes getting the latest components of the Microsoft Web Platform, including Internet Information Services (IIS), SQL Server Express, .NET Framework and Visual Web Developer easy. The Web PI also makes it easy to install and run the most popular free web applications for blogging, content management and more with the built-in Windows Web Application Gallery.”. The current version of the tool is 2.0.
Using the Web Platform Installer
After you install the Web Platform Installer you can run it and install your desired components. You should have a connection to the internet in order to use the installer. When first started the Web Platform Installer will go to the web and check for the latest Web Platform products. Then the main screen will be opened:
In the screen you can choose the relevant components that you like. You can use the following tabs to choose components by categories:
- What’s New? – shows all the new components available.
- Web Platform – enables the installation of web servers, frameworks and runtime, databases and tools.
- Web Applications – enables the installation of web applications such as DotNetNuke, WordPress, BlogEngine.Net and many more web applications.
You only need to select the components and then press the Install button in order to install them. You also have the ability to customize your installation. For example go to the Web Platform tab:
In the Web Server press the Customize link:
Now you can customize the server that will be installed.
After Choosing the components press the Install button and that is it the products will be installed.
In some components you will need to configure things. For example I installed the BlogEngine.Net and after the installation inside the Web Platform Installer I had to configure the web site for the BlogEngine.Net. So you get the help from the platform even in these parts.
Summary
If you haven’t installed the Web Platform Installer go and grab it from here. This tool is very useful and helpful and it can help you to save a lot of time and headaches.
OData Session Slide Deck and Demos
Yesterday’s evening I delivered an OData session at the Israeli Web Developer
Community (WDC) user group at Microsoft Ra’anana. The session included the
following agenda:
- The OData need.
- What is OData?
- Creating and consuming OData feeds.
I want to thank all the attendees who came to the session. I really enjoyed to deliver the content about OData and the audience was very cooperative with a lot of good questions that helped to highlight many crucial OData subjects. At the end of the session I gave a WebsiteSpark gift card that I got in MIX2010 conference (hopefully that the code in the card still work four months after the event).
As I promised, I uploaded the session slide deck and demos to my Skydrive account and you can download it from here. Pay attention that in the demos main directory there is a backup file for the demo database.
Enjoy!
OData Session is Occurring Today
Today at 17:30 I’m delivering an OData session at Microsoft 
office in Ra’anana. Come and hear about the Open Data
Protocol, a new protocol for exposing and consuming data on the
web. The OData ecosystem is a growing community of data producers and
consumers using the Open Data Protocol to exchange data.
The Open Data Protocol breaks down data silos and increases the shared
value of data and its associated business logic through the Web by enabling the exposure of any data source as a Web-friendly data feed. The easiest way to become an OData producer in the .Net world is by using WCF Data Services. In the session I’ll talk about what is OData, how to use WCF Data Services to expose OData feeds and how to consume OData feeds in lots of ways and examples. You still can register to the event from here.
See you there.
Changing The OutputCache Provider During Runtime
I got a question in the post I wrote about
ASP.NET Output Cache Provider which asks
whether we can change the cache provider
during runtime.
The post will try to answer the question.
Overriding The OutputCache Provider
One of the changes in ASP.NET 4 is the ability to override the OutputCache provider name in the Global.asax file. This enables us to create logic that stores the OutputCache in more then one type of cache according to some state or behavior. When you do that you need to supply the name of the OutputCache provider by overriding the GetOutputCacheProviderName method. For example, I register the OutputCache provider from the previous post in the web.config but I don’t indicate that it is my default provider:
<caching>
<outputCache>
<providers>
<add name="InMemory" type="InMemoryOutputCacheProvider"/>
</providers>
</outputCache>
</caching>
If I’ll run the application now the default OutputCache will be the ASP.NET supplied OutputCache. If I would like to override it then the following code which will be written in the Global.asax file will help me achieve it:
public class Global : System.Web.HttpApplication
{
public override string GetOutputCacheProviderName(HttpContext context)
{
return "InMemory";
}
protected void Application_Start(object sender, EventArgs e)
{
}
protected void Session_Start(object sender, EventArgs e)
{
}
protected void Application_BeginRequest(object sender, EventArgs e)
{
}
protected void Application_AuthenticateRequest(object sender, EventArgs e)
{
}
protected void Application_Error(object sender, EventArgs e)
{
}
protected void Session_End(object sender, EventArgs e)
{
}
protected void Application_End(object sender, EventArgs e)
{
}
}
Pay attention that I return the provider name that was registered earlier in the configuration file. Also, I could have provided whatever behavior that I want in order to return my needed provider per some situation.
The scope of the GetOutputCacheProviderName method is per request which means that every request can get it’s own OutputCache provider.
Summary
Lets sum up, we can register more then one OutputCache provider to our application and decide according to some behavior when to use them. The way to achieve that is by overriding the GetOutputCacheProviderName method and returning the desired provider.