Sunday, July 08, 2007

Entity Framework and Orcas Team Suite June 2007 CTPs

Microsoft just announced availability of the Microsoft Visual Studio Code Name "Orcas" Team Suite June 2007 Community Technology Preview (CTP) virtual hard disk (VHD) for Virtual PC 2007 and Virtual Server 2005 R2, which I'm currently downloading.

The Visual Studio Code Name "Orcas" June 2007 Community Technology Preview (CTP) Readme has a link to an Express Editions Readme, so it's probably safe to assume that the Visual Web Developer Express June 2007 CTP download for the Entity Framework June 2007 CTP is a member of this update wave. The Readmes provide no information on new features.

There was only one Microsoft blog post about the SDK drop as of Sunday morning. It was a Friday morning post from the Microsoft UK Developer Tools Team, which was subsequently deleted (but it's cached). Reports of the CTP's availability have come from the UK and Europe primarily.

Update 7/10/2007: The Microsoft UK post reappeared this morning with the following caveat:

The Product Group have released an newer version of Visual Studio, but they aren't releasing it as a CTP. It is aimed at fixing a couple of problems for some targeted customers. The reality of this isn't tested or up to the quality of a proper CTP. You can still use this download, but it has some problems and it isn't supported for TFS. I would recommend waiting until Beta 2. Sorry for any confusion I have caused. [Spelling corrected.]

If the download page says it's a CTP and the four Readmes say it's a CTP, they released it as a CTP.

Tamir Khason of Microsoft Israel posted Orcas June CTP for limited beta is available on July 4, 2007 with this announcement:

After releasing June CTP of .NET framework 3.5, June CTP bits of Visual Studio 2008 (Team Suite and Visual Web Developer) are also ready for limited beta as Layout or VPC image.

and invited readers to contact him for downloading details.

I intend to test the EF June 2007 CTP with WinForms as soon as I get the VHD installed. I've also posted a message in the ADO.NET vNext forum concerning compatibility of the EF drop with the Team Suite and VB/C# Express versions. I'll update this post as I learn more about compatibility.

I've also posted requests for lists of new and improved features, bug fixes, and (for Visual Basic) promised features dropped from Orcas in the LINQ Project General and Visual Basic Orcas forums.

There also is a Microsoft Visual Studio Code Name "Orcas" SDK - June 2007 Community Technology Preview download (127.5 MB) available but it's date is 6/1/2007 and appears to be dedicated to Orcas Beta 1.

Saturday, July 07, 2007

Entity Framework June 2007 CTP Visual Basic Samples

As usual, the sample code included with the Entity Framework June 2007 CTP was written in C#. So I ported the basic elements of the C# EFJuneCTPSamples Web Site project to the VB EFJuneCTPwithVB ASP.NET Web Site (file system) project to determine if there were unexpected issues with VB 9.0. Updated 7/6/2007: Added detail text and a GridView page populated by an ObjectQuery from an Entity SQL statement. Note: John Papa's ADO.NET Entity Framework Overview in MSDN Magazine's July 2007 Data Points column provides a summary of EF features of the Orcas Beta 1 version. LINQ to Entities ObjectQueries The LINQGrid.aspx page executes a relatively simple LINQ to Entities ObjectQuery query against the C# sample's default mapping of an extended version of the Northwind Sample database. ObjectQuery objects implement IOrderedQueryable, IQueryable, IEnumerable(Of T), IQueryProvider, IOrderedQueryable, IQueryable, and IEnumerable, so you can bind them to GridView and other databound Web controls:

Click images to display full-size versions

Notice that the GridView maps only String data to columns automatically but the CustomerID foreign-key value is missing. If you want to display foreign key values, you must obtain them from the primary key of the related entities. (There is a request pending to provide read-only access to foreign-key values so associated entities don't need to be loaded to obtain key values.) You can bind GridView columns to OrderDate, RequiredDate, ShippedDate and Freight properties manually. The Country value isn't passed as a parameter to keep the query expression simple.

ObjectQuery also supports the ToBindingList method, which (theoretically) enables two-way (read/write) data binding with DataGridView controls.

The LINQText.aspx page has a more complex query with a specific projection and uses the Take(10) partition operator to display the last 10 orders from Mexico with a VB clone of the ObjectDumper utility.

The preceding page displays the same fields as those of the WinForms test harness for LINQ to SQL performance tests in my earlier posts here and here.

References and Namespaces for LINQ to Entities Queries

The sample Web Site project adds references to System.Data.Entity.dll and System.Data.DataSetExtensions.dll in the \Program Files\Reference Assemblies\Microsoft\Framework\v3.5 folder. System.Data.Entity.dll contains all EF-specific namespaces.

Only the default Imports System.Linq directive from System.Core.dll is needed for ordinary queries.

Using Entity SQL Statements with ObjectQuery Objects

The following page demonstrates the current state of Entity SQL (eSQL) syntax for ObjectQuery. Use of the related Customer entity's Country property as the restriction is contrived because ShipCountry could have provided this information (although it might differ from the Customer.Country value.)

ObjectQuery objects returned by eSQL queries that specify projections of properties with SELECT (instead of SELECT VALUE) return MaterializedDataRecords, not a type that implements IEnumerable. Dealing with DbDataRecord types isn't fun by any stretch of the imagination. Stick with LINQ for Entities queries when you want to return specified property (field) values.

Note: For a short course on Entity SQL, read the ADO.NET Team's EntitySQL post of May 30, 2007 by Mark Shields and Fabio Valbuena. Mike Dodaro's Entity Data Model 101: Part 1 (January 30, 2007) and Entity Data Model 101 - Part 2 (February 12, 2007) explain the EDM.

The following page substitutes a simplified Entity SQL query for the LINQ to Entities expression to verify that the GridView control responds to the ObjectQuery identically.

The GridView is missing the same fields as the first example.

Namespaces for Entity SQL ObjectQuery Operations

In addition to the default Imports System.Linq directive used with LINQ to Entities queries, add Imports System.Data.EntityClient and Imports System.Data.Objects directives.

Monday, July 02, 2007

ADO.NET Entity Framework June 2007 CTP Available for Download

The ADO.NET team missed their June target by only one working day for the first CTP of the ADO.NET Entity Framework after it's "liberation" from Visual Studio 2008. The announcement appeared about an hour ago.

You can download the CTP bits from this link on the ADO.NET Team blog: ADO.NET Entity Framework June 2007 CTP now available.

The CTP works only with Visual Web Developer Codename "Orcas" Express Edition June 2007 CTP. (Sigh!)

Update 2/8/2007: The CTP should work with the recently-released Microsoft Visual Studio Code Name "Orcas" Team Suite June 2007 Community Technology Preview (CTP). See the Entity Framework and Orcas Team Suite June 2007 CTPs post for more details.

The Visual Web Developer Express Edition only section says:

This is a CTP and the only edition of Visual Studio that works for this release is Visual Web Developer Express. This means that you will only be able to author Web sites in Visual Studio. Nothing prevents you from building console applications or WinForms applications outside of Visual Studio and building them with MSBuild and a .NET Framework compiler, but you will not be able to work on those project types inside of Visual Web Developer Express.

Update 7/3/2007: Danny Simmons' "June" CTP of the Entity Framework is finally out item (posted at 3:13 a.m. today)  describes the new features of the June CTP in more detail than the ADO.NET Team's list.

Download the sample code from the CodePlex ADO.NET Samples home page.

Danny Simmons and Tommy Williams explain why the Visual Web Developer June 2007 CTP is required to run ADO.NET EF June 2007 CTP in this ADO.NET Orcas forum post. Tommy says:

We do plan to align our CTPs and Betas as closely as we can to releases of Orcas so you can expect something that lines up with Orcas Beta 2 about the time that it comes out.

Update 7/4/2007: The ADO.NET Team revised the installation instructions to add a step that appears to be required only for Windows Vista. The instructions omit this admonition. which affects all OSes:

You must uninstall Orcas Beta 1 in order to install the June CTP bits.

The above is from Tommy Williams in yesterday's 10:35 PM response to my VWD Express June 2007 CTP Side-by-Side with VS 2008 Beta 1 Crashes Opening Any .aspx File post in the forum. He also notes that:

We are also currently investigating a problem where the Visual Web Developer Express installer (or possibly the .NET Framework v3.5 bootstrapper that it includes) is downloading the Orcas Beta 1 version of .NET Fx 3.5 rather than the June CTP version on Windows Vista. The installation works correctly on Windows XP but we are seeing failures on Windows Vista.

Detailed Steps for Running the Sample Files Under Virtualized Windows Vista

For Windows Vista running under Virtual Server 2005 RS (and presumably Virtual PC 2007), here's the full drill:

  1. Create a new virtual machine from an existing image or from scratch. It's not necessary to enable IIS 7.0.
  2. Run Windows Update. If you create a Vista virtual machine from scratch, there are 19 critical and recommended updates (as of 7/4/2007) of which 18 install. A reboot is required.
  3. Download and install SQL Server 2005 Express Edition (SSX) SP2 and SQL Server Management Studio Express (SSMX). Alternatively, install SSX with Advanced Services SP2.
  4. Optionally, download and install the case-insensitive version of the AdventureWorks sample database (AdventureWorksDBCI.msi) and the latest (May 2007) SQL Server Books Online. (If you download the Northwind sample database, don't attach it. The sample files include NORTHWND_101.MDB and NORTHWND_101_LOG.LDB in the App_Data folder.)
  5. Download and install the Microsoft .NET Framework 3.5 June 2007 Community Technology Preview (CTP). This undoubtedly is the culprit that prevents side-by-side installation of the VWDX CTP.
  6. Download and install VWDX June 2007 CTP (1-GB), which consists of the following four elements:
    • Microsoft Web Designer Tools
    • Microsoft .NET Framework v3.5
    • Windows SDK Tools for Visual Studio Express (x86)
    • Microsoft Visual Web Developer Codename Orcas Express Edition
    and requires a reboot after installation of the Windows SDK Tools for Visual Studio Express (x86). Installing the Express Edition documentation is optional.
  7. Optionally, create, build and run a default C# or VB WebSite project. Doing this will confirm that you have the correct .NETFx v3.5 CTP installed. (If you don't, errors result from assemblies that won't compile.)
  8. Download and install the ADO.NET Entity Framework June 2007 CTP.
  9. Download the EFLinqJuneCTPSamples.zip file to a temporary folder and extract all files to the C:\EFJuneCTPSamples folder. This is the root location that the configuration string in the \App_Code\BasePage.cs folder uses to attach the Northwind sample database files. (The folder name is hard coded.)
  10. Run VWDX as an Administrator, choose Open Web Site, navigate to the C:\EFJuneCTPSamples folder and click open.
  11. When the .NET Framework 2.0 Web Site Found message box opens, make sure the Do the Same for All .NET Framework 2.0 Web Sites in the Solution checkbox is marked and click OK.
  12. Press F5 to build the project and display Default.aspx page with links to 10 categories of LINQ to Entities queries. 
  13. If you incur an "It is an error to use a section registered as
    allowDefinition='MachineToApplication' beyond application level. This
    error can be caused by a virtual directory not being configured as an application in IIS" error, open the Web.config file in the root folder and comment out the <authentication mode="Windows"/> element on line 37.

As far as I can determine, the current LINQ to Entities queries don't demonstrate any new features that Danny mentions. The project's pages and navigation techniques won't win any design awards, either.

Update 7/6/2007: Installing EF adds a Program Files | ADO.NET Entity Framework menu with Class Reference, Help Overview, Readme, Samples and Tools Help choices. The ADO.NET Team's July 5, 2007 Entity Framework June CTP Documentation post describes problems with cross-references between the three help files.

Many of the documentation's EntitySQL query examples won't compile because of unresolved type references to Customer, Product, and the like. An example of a VB Web Site project with an eSQL query that will compile and execute is at Entity Framework June 2007 CTP Visual Basic Samples.

Note: As I mentioned in my June 17, 2007 Entity Framework: Complex Types Redux in Beta 2 post, Danny Simmons said on June 16:

Fortunately, we did decide to implement complex types, and they will appear in an upcoming CTP.

However, they aren't in this CTP. Danny says:

P.S. One other small bit of bad news: Complex Types didn't make it into this CTP.  So if you are holding out for them, unfortunately you'll have to wait for the next round.

The LinqDataSource control is missing from the VWDX CTP's toolbox, so there's still no opportunity to test it.

Third-Party LINQ Implementations Update

LINQ to SharePoint

As promised, Bart De Smet has released the LINQ to SharePoint 0.2.0.0 Alpha for download from CodePlex with the following features:

  • SpMetal tool for SharePoint list entity creation
  • LINQ query provider for runtime LINQ query-to-CAML translation
  • A set of unit tests
  • Debugger visualizer for LINQ to SharePoint queries
  • Support for both C# 3.0 and VB 9.0

SharePoint Mapping Language

Bart's working on a major update to SpMetal.exe for Alpha version 0.3. He says:

However, there's more than just a new back-end to SpMetal. The cool thing about it is its potential for reuse elsewhere, including the VS 2008 IDE. Over time, the goal is to provide entity creation as easy as dragging and dropping lists from an add-in in Server Explorer to a designer surface. We're not there yet, but an important milestone is under development right now: SPML. Designers are just overlays on top of some source definition, for example a partial class with Windows Forms designer generated code or a resx file or ... In a similar way, SPML is the source-side of a SharePoint list mapping for LINQ to SharePoint. Currently it's very minimalistic, but over time it will get more and more expressiveness to drive the mapping process (e.g. you'll be able to decide which fields to include in the entity mapping and you'll be able to control a few aspects associated with entity updating, another 0.3 feature that's under development right now).

Note: For a recent OakLeaf post on LINQ to SharePoint see LINQ to SharePoint 0.2 Alpha to Meet Major WSS 3.0 Template/Apps Surge.

LINQ to Amazon Refactored

Rob Conery a developer from Kauai and the architect of the open-source Commerce Starter Kit for .NET 2.0 and the Subsonic "no-code" data access layer (DAL) has posted LINQ: How To Use LINQ To Query Just About Anything, in which he describes how he's refactoring Fabrice Marguerie's LINQ to Amazon project to .NET 3.5. Along the way, Rob provides one of the better explanations that I've see on expression trees. The IQueryable interface and expression trees enable LINQ to SQL and third-party LINQ implementations. He also includes a pointer to Paul Kimmel's June 28, 2007 Cigars, Lambda Expressions, and .NET article for DevLife that describes anonymous methods, lambda expressions. closures and currying in plain English.

Rob promises that the source code will be available the end of this week. Check his site or here for availability.

Update 8/8/2007: Rob published on August 2 a great LINQ to SQL screencast as SonicCast 6 - LINQ, LinqToSql, and SubSonic. His demo includes a simple technique to remove data source-related code from the page's code-behind file. He's considering adding support for LINQ to SubSonic or replacing its query generator with LINQ. No word so far on source code for his refactoring of Fabrice Marguerie's LINQ to Amazon to Beta 2.

i4o (Indexing for Objects) Article

Aaaron Erickson has written "Indexed LINQ" for .NET Developer's Journal. Subtitiled "Optimizing the performance of LINQ queries using in-memory indexes," the article covers the theory behind creating indexes on in-memory collections and applying an extension method to the Where() standard query operator to enable speeding queries by up to a factor of 100 or so with the indexes. You can download the source code, runtime binary or both for i4o at http://www.codeplex.com/i4o. (Update 8/7/2007)

Third-Party LINQ Providers List

A complete list of third-party LINQ implementations that I update periodically is: Third-Party LINQ Providers.

Rico Mariani's DLinq (LINQ to SQL) Performance Tips (Round 3)

While waiting for Rico to drop his other shoe and provide some Beta 2 LINQ to SQL performance numbers, it occurred to me that increasing the size of the object property (field) values would have an deleterious effect on LINQ to SQL performance ratios.

Update 7/6/2007: Rico dropped the other shoe yesterday at 5:30 PM: DLinq (Linq to SQL) Performance (Part 4). Matt Warren comments at LINQ to SQL: Rico drops the other shoe. There's not much more to say until we get Beta 2.

Update 7/16/2007: Prior to moving to his new position as Chief Architect of Visual Studio 2008, Rico summarized his LINQ to SQL performance tests with Beta 2 bits in his July 16, 2007 DLinq (Linq to SQL) Performance Part 5 post. Rico achieved 93% of the performance of the underlying SqlClient/SqlDataReader data provider with compiled LINQ to SQL query expressions that exaggerate LINQ overhead. I'll see if I can duplicate his results when Beta 2 becomes generally available in two to three weeks.

Per Row Costs

As noted in my updated response to Rico's part 2, his tests used a short-form version of the Northwind Orders table: Just the OrderID, CustomerID, EmployeeID, and ShippedDate fields, which total 26 bytes of SQL Server datatypes. The objects have an additional ShipCountry property that SELECT AVG(LEN(ShipCountry)) FROM Orders says is 5 Unicode characters or 10 bytes. So you don't need to look it up, here's a copy of the performance metrics for the lightweight objects with the 36-byte (average) load:


Rows

Base, s.
LINQ (Param), s. Param / Base LINQ (Compiled), s. Compiled / Base Param / Compiled
0 0.293 1.941 6.63 0.589 2.01 3.30
6 0.317 2.051 6.47 0.664 2.09 3.08
22 0.340 2.299 6.76 0.803 2.36 2.86
77 0.429 3.164 7.38 1.273 2.96 2.49
122 0.507 3.894 7.68 1.684 3.32 2.31

A reminder: Base is the time to create and populate 500 instances of the objects with an SqlDataReader from an SQL Server 2005 Express instance on the same machine as the test harness. The code adds one instance per row to a List<Order> to emulate populating LINQ to SQL's IEnumerable<Order>. (The C# test harness now has 330 lines of code.)The LINQ (Param) and LINQ (Compiled) are times to execute 500 conventional-parameterized and compiled-parameterized LINQ queries, respectively.

Following are execution times and performance ratios for objects created from the entire Orders table with rows that average 172 bytes. The LINQ (Param) and LINQ (Compiled) times had more scatter than those for the smaller objects, but the performance patterns are similar (and to be expected).


Rows

Base, s.
LINQ (Param), s. Param / Base LINQ (Compiled), s. Compiled / Base Param / Compiled
0 0.304 1.948 6.41 0.579 1.90 3.36
6 0.350 2.504 7.15 0.739 2.11 3.38
22 0.426 2.866 6.73 1.084 2.54 2.64
77 0.735 4.059 5.55 2.243 3.05 1.81
122 0.973 5.033 5.17 3.126 3.21 1.61

The moral of this story is that increasing the average width of the row (weight of the object) by a factor of 4.78 increases execution time of a compiled query by a factor of 1.13 for 6 rows to 1.86 for 122 rows. If you need high performance when querying heavyweight objects, you might consider adding a second object widget with only the fields you need to the designer. But don't fall victim to an "entity explosion" by creating a custom entity for every table with lengthy varchar or nvarchar fields.

Entity/Identity Management

Rico recommends providing a read-only DataContext type to improve performance. The way I read the tea leaves is that setting the DataContext.EnableObjectTracking property value to false makes the DataContext read-only. However, my tests show no measurable performance difference with object tracking on or off.

Data Binding

Rico didn't provide the source code for the "necessary helper for getting and storing the fields so that there is effectively straight line code calling the underlying provider," so I'm not sure what he had in mind. Perhaps SqlDataReader code similar to that for the base data?

Wednesday, June 27, 2007

Rico Mariani's DLinq Performance Tips (Part 2) and Compiled Queries

Rico Mariani continues his analysis of LINQ to SQL performance issues with a new DLinq (Linq to SQL) Performance (Part 2) post dated June 25, 2007 by recommending compiled LINQ query expressions.

LINQ to SQL query expressions take advantage of lazy evaluation. Your code doesn't hit the SQL Server instance until the the IEnumerable<T> sequence's iterator (foreach or For Each structure) requests the first row. However, before the iterator starts it's pass, the compiler builds an expression tree, which in turn generates the T-SQL statement, and sends it to the database. As Rico observes:

In many cases all that will be different from one invocation to another is a single integer filtering parameter.

The expression tree incorporates logic to send a parameterized query when the query expression contains one or more parameters in the Where clause. For example:

SELECT [t0].[OrderID], [t0].[CustomerID], [t0].[EmployeeID], [t0].[ShippedDate]
FROM [dbo].[Orders] AS [t0]
WHERE [t0].[ShipCountry] = @p0
ORDER BY [t0].[OrderID] DESC
-- @p0: Input NVarChar (Size = 3; Prec = 0; Scale = 0) NOT NULL [USA]

SQL Server caches the query plan and uses the cached version for different parameter values. Thus the only option remaining to the developer for improving performance is to compile the query, which eliminates building the expression tree and generating the T-SQL on each execution. 

Compiling Parameterized LINQ Query Expressions

Rico provides an example of a query delegate type that actually will compile. The only item missing is the delegate function:

private IEnumerable<Order> GetOrderById(int orderId)
    return q(Northwinds, orderId);

The LINQ to SQL: .NET Language-Integrated Query for Relational Data white paper by Dinesh Kulkarni, Luca Bolognese, Matt Warren, Anders Hejlsberg, and Kit George (March 2007) is one of the few (about 7) hits you'll get on a search for compiledquery.compile. The white paper has a very brief section that covers the the CompiledQuery class and its Compile method for compiling parameterized query expressions with a single example that will compile but won't work more than once during a session.

Note: The paucity of posts about compiled LINQ queries is surprising.

A bug—presumably in a System.Data.Linq class—causes a null reference exception if you create new DataContext objects for successive executions of the compiled query during a session. The sample code for the function that invokes the query delegate type does exactly that (and has a variable name error, to boot):

public IEnumerable<Customer> GetCustomersByCity(string city)
{
   Northwind db = new Northwind();
   return Queries.CustomersByCity(myDb, city);
}

Delete the Northwind db = new Northwind(); instruction, change the erroneous  myDB argument to your DataContext variable and the sample code should work as expected. However, this is another example of untested sample code. The authors might not have found the multiple invocations bug, but certainly wouldn't have been able to compile the function with the wrong argument name.

Note: Anders Borum confirmed the bug and pointed out the source of the problem in an answer to my Problem with Reuse of Compiled LINQ to SQL Query in Beta 1 post in the LINQ Project General forum. Matt Warren confirmed that the bug was detected and fixed in the Beta 1 timeframe but didn't make it into the Beta 1 bits. It's fixed for the forthcoming Beta 2 drop.

Where's the Meat?

Rico didn't offer any performance numbers in his Part 2 post, possibly because the missing function prevented him from comparing the execution time of his compiled query with baseline SqlDataReader data. VB 9.0 doesn't support lambda expressions in VS 2008 Beta 1, so I ported my original VB test harness to C# 3.0, added code to generate a combo box of Northwind country names with the number of orders per country, and wrote the code to add a parameterized query option for LINQ and SqlDataReader queries, and a compiled query option for LINQ. Here's a screen capture of the test harness's form:

The combo box contains the name and count of orders from 21 distinct ShipCountry values from the Northwind Orders table. The count varies from 6 (Norway) to 122 (USA and Germany.) The  DataContext's Orders object has only the fields required for the query expression (the three fields of the original test harness and Rico's first test object) plus the ShipCountry field for the Where clause constraint.

Following is the performance penalty data for executing 500 parameterized and compiled queries with differing numbers of rows:


Rows

Base, s.
LINQ (Param), s. Param / Base LINQ (Compiled), s. Compiled / Base Param / Compiled
0 0.293 1.941 6.63 0.589 2.01 3.30
6 0.317 2.051 6.47 0.664 2.09 3.08
22 0.340 2.299 6.76 0.803 2.36 2.86
77 0.429 3.164 7.38 1.273 2.96 2.49
122 0.507 3.894 7.68 1.684 3.32 2.31

Base is the parameterized SqlDataReader execution time in seconds. Param/Base is the ratio of parameterized LINQ query execution time to the Base time. Compiled/Base is the ratio of compiled, parameterized LINQ query execution time to the Base time. Param/Compiled is the performance ratio of compiled-parameterized to parameterized LINQ queries.

Data: Rows contain 36 bytes of date: 2 integers (8 bytes), 1 5-character CustomerID (10 bytes), one DateTime field (8 bytes), and an nvarchar ShipCountry field that averages 5 characters (10 bytes) for a total of 36 bytes. The rows of the Part 1 tests contained an average of 26 bytes (no ShipCountry field).

Hardware: 2005-era, single core 2.26-GHz Pentium 4 Dell 400SC server with 1 GB RAM and 80-GB 7,200 RPM UltraATA-100 drive (IBM IC35L090AVV207-0, 2MB buffer, 8.8 ms. seek time).

Conclusions: With Beta 1 bits, LINQ to SQL parameterized queries are about 7 times slower than the baseline SqlDataReader; LINQ to SQL compiled queries are about 2.5 times slower than the baseline SqlDataReader. The LINQ to SQL query performance penalty increases for both types of LINQ to SQL queries as the number of rows returned increases. The performance benefit of compiled over parameterized LINQ queries decreases as the number of rows increases.

Note: The the 330% performance factor of compiled queries in this scenario doesn't match the 500% performance factor reported by Anders Borum, probably because of the masking effect of database access.

What About Compiled Query Caching?

Frans Bouma, the developer of the LLBLGen Pro object/relational mapping (O/RM) tool for .NET, raises in a comment the issue of compiled query caching between DataContext refreshes. My assumption is that the compiled query is cached independently of the DataContext, just as the database connection is independent of the DataContext and is closed once the object(s) of interest have been hydrated.

Matt Warren says in a reply to the Problem with Reuse of Compiled LINQ to SQL Query in Beta 1 post:

We kind of knew we needed something like compiled queries before we even started seriously looking at perf.  We already had experience with ObjectSpaces to tell us that an ORM would have translation overhead.  Usually that overhead is not a big deal for a client-app, but we figured we could do better on server apps where the same query gets executed over and over again.

Due to the Beta 1 bug, I can't test inter-DataContext caching currently. I'll see what I can find out and update the post later.

Jomo Fisher, a member of the LINQ to SQL team, subsequently confirmed my assumption about DataContext-independent compiled queries in a June 28. 2007 comment:

Compiled queries are not bound to a particular [DataContext]. An intended usage is to compile a query once and use it across many DataContexts.

Where's Beta 2?

I'm seeing increasing numbers of "beta 2 fixes that," "beta 2 is faster," "wait for beta 2" and "it's better in beta 2" comments from the LINQ to ADO.NET folks. Rico's added his comment to the first post in his series:

Some things made it into Beta 1 but the bulk of what I'm going to post in the next few days didn't happen until after. You'll first see it in Beta 2. [Emphasis added.]

and Matt Warren seconds Rico's motion in this comment to his LINQ to SQL: Learning to Crawl post about Rico's item: 

Major improvements coming. :-)

 Let's hope bug fixes are coming, too.

Mini-Connectionless DataContext Is Gone from VS 2008

The "mini-connectionless DataContext" that Matt Warren described in his April 12, 2007 post to the Update existing disconnected objects (dlinq) thread in the LINQ Project General forum won't be in VS 2008's LINQ to SQL implementation. Matt says in June 18 and 27, 2007 posts to the LINQ: Next CTP thread:

No, this feature will not be in the first release of LINQ to SQL.

It is only missing because of lack of time.  When I mentioned it, I was talking about our forward thinking and a sample I'm trying to put together.

It will be interesting to see how Matt proposes to run LINQ to SQL v1.0 in the middle tier as a WCF service. I expected to see this feature in Beta 2, so I've updated my Changes Coming to LINQ for SQL post of May 15, 2007 with the bad news.

Update 6/28/2007: Fixed major formatting problems caused by <pre> elements, minor typos, and added sections on caching and the missing mini-connectionless DataContext.

Update 6/29/2007: Matt Warren said Rico suspects I have a faster disk drive, so I corrected and updated the drive specs on the earlier post and added a hardware section here. Clarified the row size (in bytes) and DataContext specs, also.

VB 9.0 Won't Support Collection Initializers (or Array Initializers!)

Paul Vick answered my Will Visual Basic 9.0 Have Collection Initializers? question in his June 27, 2007 What's in VB 2008? What's out? post. The answer, as I anticipated, is no. However, I was surprised to learn that VB 9.0 won't have array initializers, either. My limited tests indicated that array initializers were behaving as expected (i.e., like C# array initializers) in Beta 1. Paul says:

For VB 2008, we will only support initializing read-write fields of non-collection objects.

And explains the disappearance of array and collection initializers as follows:

Our original plans, going back to PDC05, included several more features for object initializers, such as being able to write to read-only properties, as well as collection and array initializers. In the end, the schedule for VS 2008 was not sufficient to implement these features with a high degree of confidence. Which unfortunately means that they will have to wait to a release beyond VS 2008.

Array and collection initializers deliver a major productivity boost, especially for unit testing where you don't want to hit the database because of greatly increased test times.

On the LINQ front, VB 9.0's lambda expressions won't support statement blocks, but that's been known for some time and probably isn't a major issue.

I was surprised to learn that the VB 9.0 compiler will deliver expression trees. Neither the September 2005 "Overview of Visual Basic 9.0" .doc file or the current HTML Overview of Visual Basic 9.0 version of February 2007 mentions expression trees, so I assumed

that support for writing them with VB was gone, too.

I doubt if you'll see many expression trees written in VB. Folks who write custom LINQ for Whatever implementations for specialized data domains are certain to do so with C# 3.0.

On the whole, I think most—if not all—developers who prefer writing VB code would rather have array and collection initializers than the capability to write expression trees.

Slightly off-topic: The more C# 3.0 LINQ code I write, the better I like C#'s "fat arrow" lambda (=>) operator (argument-list => expression-or-statement-block) rather than VB's forthcoming inline Function(argument-list) expression syntax.

Sunday, June 24, 2007

Visual Basic Team Starts LINQ Cookbook Series

Serial blogs about LINQ topics appear to be gaining popularity. The VB Team has embarked on writing the LINQ Cookbook and posted the first two recipes on 6/22/2007.

Update 7/12/2007: Added recipes 5, 6, and 7.

Note: O'Reilly Media appears to claim the word Cookbook™ as a trademark, which seems outrageous to me.

LINQ Cookbook, Recipe 1: Change the font for all labels on a windows form iterates a sample WinForm's controls collection and partially refactors your project by changing the fonts of Label controls to the Comic Sans MS family.

LINQ Cookbook, Recipe 2: Find all capitalized words in a phrase and sort by length (then alphabetically) uses the String.Split() method to divide a block of text into an array of words, then uses LINQ to iterate the array, discarding words that aren't initial-letter-capped, and populating a list box with a sequence sorted first by word length and then in alphabetical order.

LINQ Cookbook, Recipe 3: Find all the prime numbers in a given range accepts two integer arguments and returns all prime numbers between and including the argument values. This project requires Beta 2 because it uses Group By in the query expression; it won't compile in Beta 1 (and might not in Beta 2 because of what appears to be a conflict between Count(), an array, and Count, an integer).

LINQ Cookbook, Recipe 4: Find all complex types in a given assembly fills a list box with the names of "complex types" from Mscorlib.dll (the System.Core namespace). The project defines a complex type as "having more than 10 public methods, of which at least one has more than 3 arguments." The original code contained a typo: t.GetMethods() should be type.GetMethods(). After fixing the typo, the code compiles in Beta 1. Update: The typo is fixed.

LINQ Cookbook, Recipe 5: Concatenating the selected strings from a CheckedListBox is a contrived app that does what it says (comma-separated) with the LINQ Aggregate ... In construct. Several comments recommend code improvements.

Linq Cookbook, Recipe 6: Your first Linq Application using Northwind shows users new to LINQ to SQL how to install and connect to the Northwind database on an SQL Express database instance. As usual, the author recommends a connection to the NORTHWND.MDF file rather than using SQL Server Management Studio Express (SSMSX) to attach the .mdf file to the instance.

LINQ Cookbook, Recipe 7: Selecting Pages of Data from Northwind demonstrates use of the Skip()Skip and Take()methods for paging data items from LINQ to SQL queries.

Update 7/14/2007: Bill McCarthy's July 14, 2007 Inside Recipe 7 post observes that running Recipe 7's LINQ query within a function that has no return type requires setting Option Strict Off, which isn't a recommended practice. Bill also provides and example of the convoluted T-SQL syntax required to support the Skip and Take operators.

Recipe 42 is the most more interesting project so far, but it would have been more interesting if it returned LINQ-related types with a query such as this:

Dim q = From type In System.Reflection.Assembly.GetAssembly( _
        GetType(Funclet)).GetTypes(), _
        m In type.GetMethods() _
        Where (type.IsPublic) _
        AndAlso (type.ToString.Contains("Linq")) _
        Select type Distinct

After giving the projects a quick test drive, I have some recommendations for these and future recipes:

  • Use a conventional <pre> block for code to be copied rather than the inline style (MsoNormal with Courier), which double-spaces the code in VS 2008 and gives line continuations the fits. See Will Visual Basic 9.0 Have Collection Initializers? as an example of a post and VB 9.0 Code for the LINQ to SQL Performance Test Harness as a page with <pre> blocks.
  • Encourage folks to set Option Strict On by strongly typing variables and literals. For example, in Recipe 2 Dim Text As String and New Char() {CChar(","), CChar("."), CChar(";"), CChar(":"), CChar(""""), CChar("!"), CChar("?"), CChar(" ")} instead of New Char() {",", ".", "!", " "} (I added a few extra splitter chars to cover more bases.)  Some examples won't run with Option Strict On.
  • Break the habit of using & as the string concatenation character; + has been preferred since VB 7.0 (see the text variable in Recipe 2).
  • Don't establish recipe prerequisites that don't apply and aren't available. Most of us don't have VS 2008 Beta 2 and it isn't needed for the first two projects.

Making VS 2008 a prerequisite for these projects indicates to me that it's coming soon. Let's hope.

Update 6/25/2007: Bill McCarthy set me straight on use of the + and & operators for string concatenation. I got into the habit of using + in VB 7.0 as the result of using it in C# and, as I recall, some problem I encountered with & early on. As the "Concatenation Operators in Visual Basic" online help topic says:

The & operator is recommended for string concatenation because it is defined exclusively for strings and reduces your chances of generating an unintended conversion.

However, it's not going to be easy to break a seven-year (bad) habit. Also, &= looks a lot stranger to me than +=.

Surprise!: In doing a search to try to remember what problem I had encountered with the & operator, I found an example of Billy Hollis using + to concatenate strings (Gasp!) on page 1 of his "Straighten Out Your Strings" article for Visual Studio Magazine:

Dim sTest As String
For i As Integer = 0 To 15000
    sTest = sTest + "Strings in VB"
Next

To his credit, he uses & elsewhere.

Nick Randolph noted in a comment that there's a simpler VB syntax for specifying literal strings as Char: New Char() {","c, "."c, ";"c, ":"c, """"c, "!"c, "?"c, " "c}. (An even shorter syntax that I should have thought of is ",.;:""!? ".ToCharArray, which Billy Hollis discusses at length in his article.) I just used the syntax suggested by the Error Correction Options popup: "Replace "," with CChar(",")." Another case of typing when I should have been thinking. On the other hand, maybe the popup should have suggested ToCharArray.

Mea culpa. With this much egg on my face, I think I'll stop making VB syntax recommendations.

Update 7/4/2007: Added Kit George's Recipe 5.

Saturday, June 23, 2007

Rico Mariani Starts LINQ to SQL Performance Analysis Series

Rico Mariani is a "Performance Preacher" in Microsoft's Developer Division and a co-author of the "Improving .NET Application Performance and Scalability" patterns & practices white paper. His Channel9 video, "Rico Mariani: Writing better, faster code," was the featured "Behind the Code" show segment earlier this year. He also starred in "Rico Mariani - Making .NET Perform (Happy Birthday Video #2)" produced in February 2005.

DLinq (Linq to SQL) Performance (Part 1) is Rico's first post about LINQ to SQL performance on his Performance Tips blog. His tests pit a simple LINQ to SQL query against an SqlDataReader with the Northwind sample database's Orders table as the data source. Rico's results for 500 executions of a simple query that iterated the orders table and created an instance of an Order object with OrderID, CustomerID, EmployeeID, and ShippedDate properties were as follows:


Build/Test
Time for 500 Queries
Queries/Sec
May 2006 CTP LINQ to SQL 8.027s 62.29
Raw Cost (SQL Data Reader) 1.094s 457.04

To see if I could duplicate Rico's results with the VS 2008 Beta 1 bits running under Vista Ultimate, I built a simple test harness that emulated his test conditions (to the extent that I could determine them from the code provided in Rico's post.)

Note: You can copy and paste the code for my LINQtoSQLPerf.sln test harness from my VB.NET Code for the LINQ to SQL Performance Test Harness Web page to a VB 2008 WinForm project. The O/R Designer contains only the Order object with OrderID, CustomerID, EmployeeID, and ShippedDate properties, so loading of related entities isn't an issue.

Rico must have a considerably faster machine than my two-years-old, single core 2.26-GHz Pentium 4 Dell 400SC server with 1 GB RAM and 80-GB 7,200 RPM UltraATA-100 drive (IBM IC35L090AVV207-0, 2MB buffer, 8.8 ms. seek time). Here are my results:


Build/Test
Time for 500 Queries
Queries/Sec
VS 2008 Beta 1 LINQ to SQL 14.698s 34.01
Raw Cost (SQL Data Reader) 1.465s 341.30

Changes, such as running a networked instance of SQL Server Express or moving from Debug to Release configuration made less than a 0.5 second change to the LINQ to SQL query time or 0.2 second change to the SQL Data Reader time. Removing unneeded columns from the Order object in the O/R Designer didn't change the LINQ to SQL time, either; CPU usage was pinned at 100% for the entire ~14.5 seconds; about 500 MB physical memory was free during the tests.

Rico says:

In May 2006 DLinq is running at about 1/8 the speed of the underlying provider (13.62%).*  We can do better than that.  And we did... Stay tuned for the details and some modern era DLinq results.

*Remember no real application would ever see a result as poor as 13.62% because of course they would be doing "actual work" as well as the DLinq operations resulting in more comparable performance.

My test show LINQ to SQL to be an order of magnitude slower than the underlying SqlDataReader. I'll be interested to see how much better Rico did with "modern era" LINQ to SQL bits. Rico says in a comment:

Some things made it into Beta 1 but the bulk of what I'm going to post in the next few days didn't happen until after.  You'll first see it in Beta 2.

That's what they all say. Stay tuned.

Updated 6/29/2007: Corrected and added fixed disk specs.

Technorati tags: , ,

Friday, June 22, 2007

Will Visual Basic 9.0 Have Collection Initializers?

For reasons unknown, all mention of collection initializers for Visual Basic 9.0 has disappeared. The disappearance seems to have occurred without comment from the VB team or VB users.

The September 2005 "Overview of Visual Basic 9.0" .doc file covered the topic briefly, but the current HTML Overview of Visual Basic 9.0 version of February 2007 discusses only object initializers and array initializers. VS 2008 Beta 1 online help has a "connection initializers (C#)" topic but no corresponding VB topic.

The 2005 Overview's section 1.3 "Object and Collection Initializers" says this about collection initializers:

As we have seen, object initializers are also convenient for creating collections of complex objects. Any collection that supports an Add method can be initialized using a collection initializer expression. For instance, given the declaration for cities as the partial class,

Partial Class City
  Public Property Name As String
  Public Property Country As String
  Public Property Longitude As Float 
  Public Property Latitude As Float
End Class

we can create a List(Of City) of capital cities of our example countries as follows:

Dim Capitals = New List(Of City){ _
  { .Name = "Antanarivo", _
    .Country = "Madagascar", _
    .Longitude = 47.4, _
    .Lattitude = -18.6 }, _
  { .Name = "Belmopan", _
    .Country = "Belize", _
    .Longitude = -88.5, _
    .Latitude = 17.1 }, _
  { .Name = "Monaco", _
    .Country = "Monaco", _
    .Longtitude = 7.2, _
    .Latitude = 43.7 }, _
  { .Country = "Palau",
    .Name = "Koror", _
    .Longitude = 135, _
    .Latitude = 8 } _
}

This example also uses nested object initial[iz]ers, where the constructors of the nested initializers are inferred from the context. In this case, each nested initializer is precisely equivalent to the full form New City{…}.

Note: The preceding excerpt also appears in the session of the same name made to the XML 2005 Conference & Exhibition held in Atlanta November 14-18, 2005. I doubt if that code worked in any VB 9.0 preview.

Neither the class nor the sample collection initializer will compile. It's one thing to change the syntax for a new feature, but another to use a defective class declaration in sample code.

The authors of the original version, Erick Meijer, Amanda Silver, and Paul Vick, imported and modified the class code and modified the List(Of City) code to utilize the current New Type With syntax in "Object and Array Initializers" of the February 2007 version:

As we have seen, object initializers are also convenient for creating collections of complex objects. Arrays can be initialized and the element types inferred by using an array initializer expression. For instance, given the declaration for cities as the class,

Partial Class City
  Public Property Name As String
  Public Property Country As String
  Public Property Longitude As Long 
  Public Property Latitude As Long
End Class

we can create an array of capital cities for our example countries as follows:

Dim Capitals = { _
  New City With { _
    .Name = "Antanarivo", _
    .Country = "Madagascar", _
    .Longitude = 47.4, _
    .Lattitude = -18.6 }, _
  New City With { _
    .Name = "Belmopan", _
    .Country = "Belize", _
    .Longitude = -88.5, _
    .Latitude = 17.1 }, _
  New City With { _
    .Name = "Monaco", _
    .Country = "Monaco", _
    .Longtitude = 7.2, _
    .Latitude = 43.7 }, _
  New City With { _
    .Country = "Palau",
    .Name = "Koror", _
    .Longitude = 135, _
    .Latitude = 8 } _
}

After fixing the class declaration by removing Property and changing Long to Float, the array initializer expression won't compile because there's a missing New City() type declaration in the first line, which should read Dim Capitals = New City(){ _. It's clear that no one tested the code before publishing the update.

Note: It's a shame that VB didn't adopt the Dim corollary to C#'s var arrayName = New[]{ ... } syntax, as Ralf Ehlert mentions in his "Problems with New Features of VB9" post in the Visual Studio VB Express Orcas forum.

The authors made no mention of VB 9.0 collection initializers, which were also missing from Jean-Marie Pirelli's "C# 3.0 and VB 9.0" session at Microsoft TechDays 07 March 27-28, 2007 in Geneva. Uncharacteristically, Jean-Marie provided both C# and VB examples, but he dispensed with VB examples when he reached the Collection Initializers slide. (Anders Hejlsberg and Jay Schmelzer received credit for the slides.)

It's interesting that Jeff Bramwell's Collection Initializer post in the Visual Basic Orcas forum didn't receive a reply from a Microsoft representative. The suggestion he received from Klaus Even Enevoldsen wasn't apropos collection initializers.

Note: My March 26, 2007 Updated "Overview of Visual Basic 9.0" Stealth Post lists other problems with the Overview reported by Jim Wooley. These errors relate to unimplemented VB 9.0 features scheduled for Beta 2.

There also were errors in the LINQ to SQL documentation that I reported in my April 26, 2007 More Issues with VB 9.0 Samples in the Updated LINQ to SQL Documentation post.

Update 6/24/2007: Added comment about data type error in Cities class declaration and corrected minor typos.

Update 6/25/2007: Bill McCarthy takes issue with my "It's clear that no one tested the code before publishing the update" statement about the second sample that uses the New Type With syntax in his VB9 and collection initializers post.

Regardless of syntax issues with the array initializer code, the class declaration has both syntax (Property) and data type (Long) errors.

Regarding the failure of the initializer code to compile, Bill says:

[I]t is clear the document is forward looking, and talking about how things will/should be.

So, given that, if we stop and analyze the syntax for array initializers it should be basically as presented. Perhaps one could argue the syntax should be Dim Capitals() = {.    It should also support the syntax you can use today in VS 2005, e.g: Dim ints() As Int32 = {1,2,3} , or as per the code Roger suggests Dim ints() As Int32 = New Int32(){1,2,3}

Okay so given the minimum it needs to support (current syntax), in VB9 we add anonymous types and inferred typing.  Inferred typing means we remove the "As XXX" part.  So this means the syntax Dim Capitals As City() becomes Dim Capitals and the Dim Capitals() As City syntax becomes  Dim Capitals().  Anonymous types means we can't define the type name. So New City() { would become New() { _ , and given that the existing syntax doesn't even require the New Int32(), the New() becomes superfluous because the set brackets { } define the array data.

IOW: the syntax as shown is as it should be. (IMO)

Explicitly typing the array members by adding New City With { ... syntax precludes the array being of an anonymous type or use of inferred typing and the Dim Capitals = New City(){ ... statement is required. I didn't propose Dim Capitals() As City = {...} or Dim Capitals As City() = New City(){...}  as inferred by Bill's "Dim ints() As Int32 = {1,2,3} , or as per the code Roger suggests Dim ints() As Int32 = New Int32(){1,2,3}" sentence.

You can create anonymously-typed array elements by substituting New With for New City With, but Dim Capitals() = {...} returns an array of type Object with Option Strict Off and won't compile with Option Strict On. If VB 9.0 finally supports the New() construct for anonymously typed arrays, the the preceding code should compile with Dim Capitals = New(){ _, but I don't believe that New(){ _ would be superfluous. Here are two C# array initializer examples that use new[]:

/* Array initializer with object initializers (array is inferred type LineItem) */

var LineItems = new[]
{
    new LineItem {OrderID = 11000, ProductID = 11, Quantity = 10, QuantityPerUnit = "24  500-g bottles", UnitPrice = 15.55M, Discount = 0.0F},
    new LineItem {OrderID = 11000, ProductID = 21, Quantity = 20, QuantityPerUnit = "12 1-kg cartons", UnitPrice = 20.2M, Discount = 0.1F},
    new LineItem {OrderID = 11000, ProductID = 31, Quantity = 30, QuantityPerUnit = "24 1-kg bags", UnitPrice = 25.45M, Discount = 0.15F}
};

/* Array initializer with object initializers (array and elements are anonymous types) */

var LineItems2 = new[]
{
    new {OrderID = 11000, ProductID = 11, Quantity = 10, QuantityPerUnit = "24 500-g bottles", UnitPrice = 15.55M, Discount = 0.0F},
    new {OrderID = 11000, ProductID = 21, Quantity = 20, QuantityPerUnit = "12 1-kg cartons", UnitPrice = 20.2M, Discount = 0.1F},
    new {OrderID = 11000, ProductID = 31, Quantity = 30, QuantityPerUnit = "24 1-kg bags", UnitPrice = 25.45M, Discount = 0.15F}
};

However, I haven't heard anything about VB 9.0 implementing New(). On the other hand, I'm not sure why I would want to create an anonymously typed array or what its use would be.

Tuesday, June 19, 2007

LINQ to SharePoint 0.2 Alpha to Meet Major WSS 3.0 Template/Apps Surge

Bart De Smet's LINQ to SharePoint - Announcing the 0.2 alpha release post yesterday and Mary Jo Foley's SharePoint: Microsoft’s Web 2.0 hub column today are an interesting coincidence.

Bart's enhancing LINQ to SharePoint with these new features:

  • Enhanced support for SharePoint list field types, including Lookup and LookupMulti fields with lazy loading support and subquery support
  • Changes to the entity model used by LINQ to SharePoint, in preparation for update support down the road
  • Optimization of the CAML queries generated by the query parser
  • Support for First and FirstOrDefault query operators
  • Introduction of a CamlMethods static class with helper functions, including DateRangeOverlaps
  • Support for Now and Today elements
  • Multiple Where clauses per query are supported
  • SpMetal improvements and separation of front-end and back-end functionality, allowing for hosting of SpMetal functionality in other environments (such as an IDE)

Bart also announced that work is starting on the 0.3 Alpha version that will include update capability. Current code, a draft of the technical spec and the first unit tests for the query parser are available now on his CodePlex site. Final 0.2 Alpha code will be  ready in a few days.

Bart picked a great data domain for his third-party LINQ implementation. SharePoint is getting much more of Microsoft's attention and resources as the company prepares to repulse the attack on its Office hegemony by online competitors Google, et al.

Update 7/26/2007: Jeff Raikes reported during the Microsoft's annual financial analysts conference that SharePoint generated $800 million in revenue during fiscal 2007. (From Joe Wilcox of Microsoft Watch).

SharePoint "Social Computing" Enhancements

Almost simultaneously, Mary Jo Foley announced the availability of the Community Kit for SharePoint (CKS) 2.0 beta, which opened on the CKS CodePlex site Monday. CKS 2.0 adds these features to Windows SharePoint Services (WSS) 3.0 and its big brother, Microsoft Office SharePoint Services (MOSS) 2007:

The Microsoft SharePoint Products and Technologies Team Blog's June 18, 2007 Community Kit for SharePoint 2.0 Pre-Release announcement says:

The ultimate goal of the CKS is to enable community oriented features and solutions by leveraging, enhancing, and extending SharePoint as a social computing platform.

100 "Next-Generation" SharePoint Business Apps Coming

Mary Jo also reports that Derek Burney, general manager of Microsoft's SharePoint Platform and Tools group, will commit today at the Enterprise 2.0 Conference to delivering 100 "next-generation" business applications (not templates) over the next 12 months to SharePoint users for internal use by Microsoft employees.

Sandy Kemsley, who's covering Enterprise 2.0 in her EbizQ Column 2 blog, didn't mention anything about this topic in her Enterprise 2.0: Derek Burney item. The same is true for Michael Sampson (Michael's Thoughts, Notes on Derek Burney, "Amplify the Impact of Your People with Enterprise 2.0 Technologies"), John Eckman (openparenthesis, Liveblogging Enterprise 2.0 - Microsoft’s Derek Burney), and Mike Gotta (Collaborative Thinking, Amplify the Impact of Your People with Enterprise 2.0 Technologies). Michael, John and Mike mention "Next Generation Applications" but not that Microsoft is giving 100 of the them to users.

Update 6/20/2007: According to later reports in Network World (Lotus, Microsoft jostle to land social networking customers by John Fontana)and eWeek (IBM, Microsoft Show Web 2.0 Wares by Renee Boucher Ferguson and Darryl K. Taft) articles, the 100 apps will be for internal use only. Fontana writes:

In addition, Microsoft said it is committing to build 100 social networking business applications before June 2008 for use inside the company. One currently in development is SharePointPedia, which helps users find SharePoint technical and support information from both Microsoft and other sources.

If SharePointPedia is an example, at least some internal apps might make reach SharePoint customers in the form of templates. According to Lawrence Liu's post in the CodePlex site for CKS:SharePointPedia:

Microsoft is embarking on an ambitious project to create an application codenamed "SharePointPedia" that will be used to enable a "community driven and supplemented content lifecycle." ... [I]t's being designed (yes, the project kicked off just last week) to be used primarily by the community. ...

SPP is scheduled to go live ... by the end of October.

Community Kit for SharePoint Background

The Community Kit for SharePoint Vision and Scope Document describes CKS:

At the most basic level, the CKS is a site template that enables practically anyone to create very quickly a functional community website on Windows SharePoint Services 3.0 or Microsoft Office SharePoint Server 2007. The “Standard Edition” will require nothing more than the out-of-the-box Web Parts that come with WSS 3.0. In this way, the CKS:SE is just like the Application Templates for WSS, but that is where the similarity ends.

Instead of being solely developed by Microsoft, the CKS will be a collaborative development project hosted on CodePlex, an online software development environment for open and shared source developers to create, host, and manage projects throughout the entire software development lifecycle.

Here's the CKS vision statement from Project Management and Evangelism Lead Lawrence Liu:

  • A set of best practices, templates, Web Parts, tools, and source code that enables practically anyone to create a community website based on SharePoint technology for practically any group of people with a common interest.
  • A technology framework that sits on top of Windows SharePoint Services or Office SharePoint Server and can be further customized or extended to suit the community website implementer’s needs.
  • A shared source community development project that is provided at no cost and allows anyone to use for commercial or non-commercial purposes.

As mentioned in the earlier Vision and Scope Document quote, you don't need to run a pricey Microsoft Office SharePoint Server (MOSS) 2007 version:

Targeted Platform: Given that Windows SharePoint Services 3.0 was released on November 16, 2006 and is available for free to licensed customers of Windows Server 2003, the development efforts on the CKS should be targeted at this version of SharePoint. Opportunities for “feature light up” when Office SharePoint Server 2007 is present should also be considered.

Just Say No to Web and Enterprise Two-Point-Oh?

"Web 2.0" and "Enterprise 2.0" are two terms that I've come to distrust—if not despise—as overhyped and basically without meaning. However, Dion Hinchcliffe's May 2006 A round of Web 2.0 reductionism item and July 2006 Enable richer business outcomes: Free your intranet with Web 2.0 post shed some light on the two topics in the enterprise.