Saturday, March 11, 2006

Anomalies and Issues with VB 9 DLinq Code

The January 2006 VB DLinq Technical Preview finally lets VB coders create Visual Studio test projects that explore most LINQ capabilities. I find that I seldom discover bugs, coding pitfalls, performance hits, and other problems when executing simple test examples, such as those of the preview's SampleQueries.sln project. So my "Test Drive VB9 and DLinq" article for Visual Studio Magazine's April 2006 issue and the .NETInsight newsletter uses moderate- to large-size object collections in the accompanying VbLinqDemo.sln sample project. VbLinqDemo.sln illustrates how to take advantage of most new DLinq and XLinq features. The VB team didn't update the "DLinq Overview" documentation for the preview. The white paper states: "Although the samples in this document are shown in C#, DLinq can be used with the LINQ-enabled version of the VB.NET compiler as well." Current LINQ, DLinq, and XLinq documentation contains few—if any—VB code examples, so you're on your own when it comes to VB query syntax beyond simple Select ... From Alias In DataContext Where ... Order By ... operations.

Following are a few of the problems I encountered when writing the article and the accompanying VB 9 sample project, which has about 2,000 lines of original source code, plus an 850-line DataContext class generated by SqlMetal.exe. Clearly, the problems could be the result of cockpit error on my part. I'll post solutions to the problems, if and when they're solved. I'll also post links to the article and its VB 9 source code when they're available from FTPOnline.com.

DataContext.GetQueryText(objQuery) Doesn't Return the SQL String Amanda Silver, a VB Team program manager, provided this reply to a question posed during the February 2006 DLinq chat about the T-SQL generated for DLinq queries:

"A: The magic API is GetQueryText on the DataContext object. And you pass in an IQuery object."

Amanda didn't provide a code example for the "magic API." Windows Live Web Search returned only three distinct hits for DLinq GetQueryText and Google returned only two. One hit of both searches unearthed the following:

Keith Farmer, a Microsoft developer on the DLinq team (and an Expert for the DLinq chat), suggested the following code to return the generated T-SQL statement from a C# 3.0 DLinq query in response to question posted in Channel 9's "Anders Heljsberg - LINQ" video forum:

var customers = from c in db.Customers
select c; // base query
IQuery query; // final query
if (filterByCity)
{
   string cityName = "London";
   // modify the base query
   query = from c in customers
   where c.City == cityName select c;
else
   // use the base query directly
   query = customers;
// base query
Console.WriteLine(db.GetQueryText(customers));
// correctly modified query
Console.WriteLine(db.GetQueryText(query));
The preceding code executes as expected with the original PDC technical preview upgraded to the RTM version of VS 2005. Here's the corresponding—but simplified—VB 9 translation:
Dim qryCustomers As IQuery
Dim strSQL As String
Dim objCustomers = Select c _
   From c In dbNwind.Customers _
   Where c.City = "London"
qryCustomers = objCustomers
strSQL = dbNwind.GetQueryText(qryCustomers)
The two examples' db and dbNwind DataContexts bind the Northwind sample database running on SQL Server 2005 or SQL Express. The VB 9 code fails when attempting to execute the qryCustomers = objCustomers instruction. The error message is
Unable to cast object of type '<select>d__b`2[VbLinqDemo.Nwind.Customers, VbLinqDemo.Nwind.Customers]' to type 'System.Data.DLinq.IQuery'. There are even fewer search hits from linq IQuery. The obvious question is "Where or how do you get the IQuery object for GetQueryText's argument?" Logging T-SQL Sent to the Server A partial workaround for the GetQueryText problem is to capture the T-SQL statements sent to the server with a TextWriter object assigned to the DataContext.Log property. The most common log destination is the Console.Out object, but you can use add following code immediately after instantiating the DataContext object to assign a StringBuilder for logging:
dbNwind = _
  New Nwind.Northwind(My.Settings.NwindConn)
Dim sbLog As New StringBuilder
Dim twLog As TextWriter
twLog = New StringWriter(sbLog)
dbNwind.Log = twLog
sbLog contains T-SQL for parameterized SELECT queries and INSERT operations. But UPDATE and DELETE statements are missing (or I'm not privy to the legerdemain needed to capture them.) Alternatively, use SQL Server Profiler to view T-SQL statements and their parameter values. Issues with Bound Object Inserts in DataGridViews Databinding is critical to the success of LINQ and DLinq in mainstream Windows and Web client development. Smart client developers must be able to bind DLinq DataContext objects to design-time components—such as BindingSources—and DataGridViews, TextBoxes, and DateTimePickers. Web page designers need similar capabilities for ASP.NET server controls. To achieve ubiquity, DataContext objects must offer binding flexibility and performance equal to—if not better than—strongly typed DataSets.

One of the two Windows forms of the VbLinqDemo.sln sample project contains a three-level, hierarchical set of DataGridView controls bound to the Customers, Orders, and OrderDetails Table members of a DataContext object generated by SqlMetal.exe from the Northwind sample database.

I used the Data Source Configuration Wizard to generate an Object Data Source from the Nwind.Northwind DataContext class, and dragged the Customers node from the Data Sources window to the form to generate CustomersBindingSource, CustomersDataNavigator, and CustomersDataGridView controls. Dragging the Orders node (under the Customers node), which represents the Orders EntitySet of the Customers member, to the form added the OrdersBindingSource and OrdersDataGridView. Finally, dragging the OrderDetails node (under the preceding Orders node), which represents the OrderDetails EntitySet of the Orders member, to the form added the OrderDetailsBindingSource and OrderDetilsDataGridView.

An empty AddNew row appeared in the Orders and Order Details DataGridViews, but not in the CustomersDataGridView. Setting CustomersBindindingSource.AllowNew = True generates an AddNew row with the CustomerID column set read-only. You can type values in the remaining columns, but you can't persist a Customers member instance for the new row. Populating the cells with a DefaultValuesNeeded event handler doesn't solve the persistance problem. Clicking anywhere in the empty AddNew row of the Orders or OrderDetails DataGridView throws an untrappable "Value does not fall within the acceptable range" exception. Populating the rows with a DefaultValuesNeeded event handler doesn't work and thus doesn't solve the problem.

The incomplete workaround is to write code to populate the new objects with code similar to the following:

Private Sub AddNewCustomer()
   'Create some new OrderDetails instances
   Dim objLineItem1 As New Nwind.OrderDetails
   With objLineItem1
      .ProductID = 1
      .Quantity = 12
      .UnitPrice = 22.25
      .Discount = 0.05
   End With
   ...

   'Create a new Order instance
   'This requires changing Orders.EmployeeID and
   'Orders.ShipVia field data types from
   'Nullable(Of Integer) to Integer
   'Otherwise these field values remain Null,
   'regardless of the property values set
   Dim objOrder As New Nwind.Orders
   With objOrder
      .CustomerID = "ABOGC"
      .EmployeeID = 1
      .OrderDate = Now
      .RequiredDate = Now.AddDays(14)
      .ShippedDate = Nothing
      .ShipVia = 2
      .Freight = 15.15
      .ShipName = "A Bogus Customer, Inc."
      .ShipAddress = "1000 Broadway"
      .ShipCity = "Oakland"
      .ShipRegion = "CA"
      .ShipPostalCode = "94608"
      .ShipCountry = "USA"
      .OrderDetails.Add(objLineItem1)
      ...
   End With

   'Add the new Customer instance
   Dim objCustomer As New Nwind.Customers
   With objCustomer
      .CustomerID = "ABOGC"
      .CompanyName = "A Bogus Customer, Inc."
      .ContactName = "Joe Bogus"
      .ContactTitle = "President and CEO"
      .Address = "1000 Broadway"
      .City = "Oakland"
      .Region = "CA"
      .PostalCode = "94608"
      .Country = "USA"
      .Phone = "(510) 555-1212"
      .Fax = "(510) 555-1213"
      .Orders.Add(objOrder)
   End With
   dbNwind.Customers.Add(objCustomer)
   dbNwind.SubmitChanges()
End Sub

The most important issue with creating new Orders object instances is the need to modify the data type of the class's EmployeeID and ShipVia fields from Nullable(Of Integer) to Integer. Foreign key constraints with the Employees and Shippers tables preclude NULL values.

The need to add a new Orders object for an existing customer to the Customers.Orders EntitySet, rather than to the Table(Of Orders) object directly is an anomaly. Doing the latter operation adds an orphan Orders record with a NULL CustomerID value. The same is likely to be true for adding a new Order Details record, but hasn't been tested.

Adding new Customers or Orders records in a grid isn't a common practice in the real world. Thus, the sample project's literal values would be replaced by values supplied from TextBoxes, ComboBoxes, DateAndTimePickers, or the like on another form. The preceding code assumes a business rule that requires a new customer to process an initial order with at least one line item.

Deletion of Rows in Bound DataGridViews

You can delete Orders and OrderDetails instances and their corresponding table rows by selecting the row in the DataGridView and pressing Delete. The DataGridView.UserDeletingRow event handler displays a message box that lets the user cancel the pending deletion. The DataGridView.UserDeletedRow event handler contains or calls the code to delete the entity and related entities, if any.

Deleting a parent instance deletes it's child (and grandchild) entities, if present. However, the deletion process differs depending on the position of the deleted instance in the DataContext hierarchy. As an example, deleting a Customers entity and its related Orders and OrderDetails entities requires code similar to the following:

Dim objNewCust = (Select c _
   From c In dbNwind.Customers _
   Where c.CustomerID = "ABOGC").First()
dbNwind.Customers.Remove(objNewCust)
dbNwind.SubmitChanges()

Deleting one of two or more OrderDetails entities from an order requires the following code:

Dim objDelDetail = (Select d _
   From d In dbNwind.OrderDetails _
   Where d.OrderID = delOrderID AndAlso _
      d.ProductID = delProductID).First()
dbNwind.OrderDetails.Remove(objDelDetail)
dbNwind.SubmitChanges()

where the OrderDetailsDataGridView.UserDeletingRow event handler returns the delOrderID and delProductID values.

However, deleting one of two or more Orders entities requires invoking the dbNwind.SubmitChanges() method only. These apparent inconsistencies seem to be anomalies.

Persisting edits of existing entities appears to require invoking the dbNwind.SubmitChanges() method only.

Design-Time Errors Occur with an OrderBy Clause

The sample project's other form (VBLinqDemo.vb) demonstrates creating singleton and multi-row joins between a DLinq entity (Orders) and XLinq in-memory XML documents (Customers, LineItems, Products, Suppliers, Employees, and Shippers). The joined entities use VB 9 XML literal code to generate a complex, hierarchical XML SalesOrders document.

Adding the Order By o.OrderID Descending _ clause to the following code

Dim docSalesOrders As XDocument = _
<?xml version="1.0" standalone="yes"?>
<SalesOrders>
  <%= Select _
    <SalesOrder _
      EmployeeID=<%= o.EmployeeID %>
      ShipVia=<%= o.ShipVia %>>
      ...
    </SalesOrder> _
    From o In dbNwind.Orders _
    Where o.ShipCountry = strCountry _
    Order By o.OrderID Descending _
  %>
</SalesOrders>

displays an "Order By clause in query expression is invalid" error and the project won't compile.

Large Amounts of XLinq Code Confound the Debugger

Setting a breakpoint on a valid instruction in the code for the CreateXmlFromJoins procedure behind the VBLinqDemo.vb form and pressing F5 throws an exception with the following (typical) message: "The following breakpoint cannot be set: At VbLinqDemo.vb, line 917 character 13, ('Create from Joins' line 16). The Common Language Runtime was unable to set the breakpoint." (Windows Live Search returns a link to the Visual Studio Debugger Breakpoint Helper.)

Clicking OK to dismiss the message opens another message box that states: "Execution has stopped at the beginning of the program because one or more breakpoints could not be set." (No search returns.)

The lack of search hits indicates that this problem occurs infrequently.

Conclusion

LINQ and its DLinq and XLinq expressions are in the pre-beta stage, so it's not surprising to find that these VB 9 extensions have anomalies and problems. However, it would be useful if documentation covering the technical previews was more forthcoming about limitations and issues. Updated overview whitepapers with non-trivial VB 9 sample code also would be useful.

Leave a comment or e-mail me at Roger_Jennings@compuserve.com if you have questions, suggestions, or solutions.

Technorati:

Sunday, March 05, 2006

Changes on the Way for VB 9.0 and XLinq

The January 2006 technical preview of Visual Basic 9 and Language Integrated Query (LINQ) is about to reach its second month of existence, but the XLinq team has already announced a syntax change for VB 9 XML literals. According to Avner Aharoni, a program manager for WebData XML team, there will be "a new syntax for these late-bound [XML literal] properties. Angle brackets now wrap the Xml member name so it is clear that the name refers to the elements with that name. For example Person.<Name> returns the elements called Name in the above document. ... In addition to this change in the axis members’ syntax, we are planning to change the binding of the Xml attribute member @ to XLinq’s Attributes method as opposed to the Attribute method as the compiler did in the January CTP." From Avner's response to a reader's comment [emphasis added]:

Our design goal is to make the Xml axis members experience as close as possible to regular CLR members, so users have a very easy learning curve. Our preference was to avoid the special syntax for Xml members, however because of the ambiguity and double colon problems we had to choose special syntax. We want users to continue using the most intuitive way to access members, which is the "Dot". Our plan is to add the known Xml members to the intellisense drop down so that the Xml members and the new syntax are easily discoverable. We hope that the new syntax will contribute the intellisense experience since the Xml members will show up in the same area due to the sorting of the intellisense members.
You can read more about the forthcoming changes to XLinq and the VB 9.0 compiler in Avner's "VB 9.0 Xml Features - latest update" post to the XML Team's blog.
Mike Champion took over Dave Remy's duties as XLinq program manager in mid-January 2006. (Dave is now the Group Program Manager for the XML Team.) According to his "New LINQ / XLinq Links" post, Mike continues to own the XML "DOM and the business case for XQuery in the mid-tier." Only time will tell if there is a "business case for XQuery" in the .NET Framework 3.0 or later; XLinq has the potential to handle the most common use cases for XQuery 1.0. In the meantime, Mike will be updating the "XLinq Overview" whitepaper that was included with the original (PDC) LINQ technical preview.
Visual Basic 9.0 might lose the expected lambda functions feature if the VB syntax problems for single- and multi-statement lambdas that are outlined in Paul Vick's pessmistic February 13, 2006 post can't be worked out. As Paul observes, the "Just because it comes in your size, doesn't mean you should wear it" admonition applies to programming languages, as well as to clothing. From Paul's response to a reader's comment [emphasis added]:
We're definitely started with nested functions and working our way forward, since nested functions give us at least the variable capture semantics that we need to properly do stuff like LINQ. So I would expect (but won't promise) that nested functions will make an appearance in VB. The question really is about the next step -- moving the nested functions inline. True lambda functions might not be so bad, and we're looking at some options. What looks really difficult is multi-statement lambdas. It's not so much an issue with line continuations (although we're looking at that issue too, given the length of LINQ query statements), but more with block delimiters. The fact that VB uses words for block delimiters makes inline multi-statement lambdas look pretty wordy. I'm not saying the issues are insurmountable -- we're still designing it -- just that it's not an entirely obvious piece of work...
Technorati:

Erik Meijer to Deliver Two LINQ Tutorials at ECOOP 2006

Erik Meijer, who describes himself as a "architect/language pimper in the SQL Server division," will conduct two LINQ tutorials at the 20th annual European Conference on Object-Oriented Programming (ECOOP) gathering in Nantes (France) on July 3 - 7, 2006. Following are excerpts from the two tutorials' abstracts:

• An Introduction to .NET using Visual Basic 9 and C# 3.0: "[T]he next versions of C# and Visual Basic will bring many features such as local type inference, anonymous types, nested functions and lambda expressions, extension members, meta-programming, and monad comprehensions, that we know and love from programming language research into the mainstream. Visual Basic adds further functionality such as deep XML support, relaxed delegates, strong duck typing, and dynamic identifiers that leverage its unique trait of having static typing where possible and allowing dynamic typing where necessary. • A Language Geek Perspective of LINQ, XLINQ, DLINQ: "This tutorial explains the programming language theory roots (monads and monad comprehensions, lazy/co-inductive functional programming, meta-programming) behind language integrated queries (LINQ) and briefly discusses the language enhancements to support them. We will give an in-depth treatment of the three domain specific APIs that constitute the LINQ framework namely the standard query operators for objects, the new XLinq API for manipulating XML, and the new DLinq and ADO.Net object-persistence infrastructures."
Earlier this year, Erik Meijer and Brian Beckman presented an XLinq demonstration titled "XML Support in Visual Basic 9.0" at the Programming Language Technologies for XML (PLAN-X) Workshop on January 14, 2006. PLAN-X was colocated with the 33rd Annual ACM SIGPLAN - SIGACT Symposium on Principles of Programming Languages (POPL 2006) held in Charleston, SC on January 11 - 13, 2006. Erik and Brian recently posted to the Microsoft Research site what appears to be a preview of a future, full-length technical paper, "XML Support in Visual Basic 9," that's based on—or forms the basis for—the PLAN-X demo. The paper's sample XLinq code is based on the January 2006 LINQ technical preview. My "Compose XML Content with XLinq Expressions and VB 9.0 XML Literals" post uses similar syntax from the earlier LINQ technical preview for the release version of Visual Studio 2005. Click here for a Google Blog search to display links to recent posts about—or references to—Erik Meijer from the OakLeaf blog. Technorati:

Saturday, March 04, 2006

Anders Hejlsberg and LINQ: A Belated Interview with the C# Guru

Anders Hejlsberg said in this pre-PDC 2005 "Behind The Code" interview (02:20) that Bill Gates called the Clarity project "The most exciting thing he saw during his demo days." Clarity was the earlier code name for Language Integrated Query (LINQ) and C# 3.0. The "Life and Times of Anders Hejlsberg" traces the career of C#'s chief architect from his early days in Denmark to Scotts Valley (Calif.) as Borland International's Principal Engineer, and finally to Redmond and Microsoft, where he's now a Distinguished Engineer and Technical Fellow. The interview is laced with mentions of LINQ and related C# 3.0 technologies, despite interviewer Barbara Fox's attempts to minimize the episode's technical content. Anders' remarks on C# 1.0 and 2.0 start at 31:15. A brief discussion of "What's Next for C#" begins at 43:55. Type "LINQ" in the Search This Blog input box and click the link or click here to perform a Google Blog Search for the close-to-30 posts on the OakLeaf blog that relate to LINQ, DLinq, XLinq, or all three. Somewhat off-topic: The "Behind the Code" series appears to be an enormous resource consumer, at least compared with the traditional hand-held sessions conducted by Robert Scoble and his colleagues. This one-hour episode required(?) a 27-person staff: Producer, co-producer, director, line producer, floor manager, technical director, lighting director, gaffer, grip, audio mixer, two audio assistants, four cameramen, prop master, video (switcher?), engineer (for ?), tape operator, makeup person, script supervisor, media processor, editor, title designer, audio post (producer), and field unit camerman. The production overhead might explain why an interview taped "five weeks before PDC" wasn't posted to the Channel 9 lineup until February 1, 2006. I didn't discover the existence of the interview until I wrote the post for the "Behind the Code" Jim Gray interview. Technorati:

Jim Gray: An Interview with the "Database Legend"

MSDN's Channel 9 Forums released on March 3, 2006 a one-hour interview with Jim Gray, Technical Fellow of the Scaleable Servers Research Group and the manager of Microsoft Research's eSciences group at the Bay Area Research Center (BARC) in San Francisco. Jim probably is best known to the Web community for the TerraService and SkyServer databases, but his credentials as a database legend stem from his seminal work in transaction processing, concurrency, and scaleability theory and practice. Jim is editor of the Performance Handbook for Database and Transaction Processing Systems, co-author with Andreas Reuter of Transaction Processing Concepts and Techniques, and Editor of the Morgan Kaufmann series on Data Management. He won the Association for Computing Machinery's prestigious Turing Award in 1998 and the US Geological Survey's John Wesley Powell Award (with Tom Barclay) for TerraServer in 2000. (Jim and Tom discuss the TerraServer project starting at 46:50.) He also gained the title of Microsoft "Distinguished Engineer" in 2000. This episode is the second in Microsoft's fledgling "Behind the Code" interview series, which Channel9 describes as follows:

Occasionally the office of the CTO, David Vaskevitch, sits down with some of Microsoft’s most influential technical employees to capture their stories. Instead of examining specific technologies, BTC takes a closer look at the person, the career and what it takes to produce world-class software.
There's no explanation of how an "office ... sits down with .. technical employees," but the interviews differ greatly from Channel9's technically focused fare gathered with handheld camcorders. Barbara Fox, formerly Microsoft's senior security architect of cryptography and digital rights management, hosts the interviews, which are professionally produced and edited. (The credits for the Microsoft Studios crew were about as lengthy as that for the first episode—27 people.) The interviews eschew detailed technical coverage for insight into the subject's career, colleagues, and scientific or technical community before and during his or her employment by Microsoft. Other OakLeaf posts that refer to Jim Gray: Jim Gray Podcast About SQL Server 2005 (and Later) and Microsoft Web Services DevChannel Opens at FTPOnline (.NET Web services client for TerraServer.) From FTPOnline: Build Petabyte DBs With SQL Server (VSLive! 2002 keynote, slides and audio track), A Talk With Database Guru Jim Gray (February 2002 interview by Lee The), and SQL Server 2005 Amid DBMS Market Dynamics by Peter O'Kelly (November 2004).You can read more about the TerraService Web services client and download its Visual Basic .NET source code from my "Build Real-Time Web Images" article from the August 2004 issue of Visual Studio Magazine. Technorati:

Monday, February 06, 2006

Updated Source Code for "Visual Basic 2005 Database Programming"

I've uploaded to the WROX code download site a complete refresh of the Visual Basic 2005 source code for Chapters 1 through 9 of the source code files for Expert One-on-One Visual Basic 2005 Database Programming. The refresh includes source code for chapters 10 through 12, which is unchanged. The update contains fixes for code in chapters 1 through 5 that readers weren't able to open in Visual Studio 2005's Windows form designer, as well as other minor fixes. The individual chapter zip files (Chapter 01.zip through Chapter12.zip) have different names than the original versions, but the new contents will overwrite the original source code copied to your \VB2005DB\Chapter01 through \VB2005DB\Chapter12 folders. (Updated 2/15/2006.) Important: You must download Part 1 (57678X_code.part1.exe, 9.44 MB) and Part 2 (57678X_code.part2.rar) into the same temporary folder of your choice (e.g., C:\Temp). Then run 57678X_code.part1.exe to start the extraction process for the Chapter##.zip files into the same or another folder (e.g., C:\VB2005DB), as shown here.

Finally, extract each Chapter##.zip file to a \VB2005DB\Chapter##\... folder and subfolders by marking the Use Folder Names check box, as shown here:

Caution: If you want to save modifications you made to the original source code, please change the name of your current \VB2005DB folder before opening the Chapter##.zip files. Chapter 1's OleDbDataObjects project has a couple of problems: 1. SQL Server 2005 Express (SSX) doesn't install SQLXML 4.0, despite Adam Weiner's inference in an XMLTeam blog post that it does. Thus, I've modified the source code to CopyLocal Microsoft.Data.SqlXml.dll, which enables compiling the project when you're running SSX. The project won't run in this case, because the underlying SQLXML 4.0 COM object isn't installed. See the comments in the Main.vb file regarding changes for SSX users. There's no redistibutable for SQLXML 4.0, so you'll need to download SQLXML 3.0 SP3 from MSDN and install it, if you haven't installed the client tools for SQL Server 2005 Developer edition or higher. 2. I accidentally built the original updated version with a later version of the (8.0.50926) Visual Basic compiler than the RTM version. The later compiler is for the VB-LINQ January Technical Preview. Unless you've installed the VB-LINQ preview bits, you'll receive an error when you run the project (after making the preceding fixes, if necessary.) Here's more info on VB-LINQ. The updated version of the OleDbDataObjects.sln project from the OakLeaf FTP site includes fixes for the preceding problems. Due to the long lead time to update the WROX download site, these fixes haven't been applied to that ZIP file. If you want an updated version of the project, send me mail and I'll provide a file that contains this project only. The source code update also contains a new version of Chapter 2's ReportViewerDemo.sln WinForm project and adds a new ASP.NET 2.0 ReportViewerWebDemo WebForm project to Chapter 8. You can learn more about the new ReportViewer control for WinForms in my "Build Client-Side Reports Easily" article in the November 2005 edition of Visual Studio Magazine. Watch for the announcement of a similar article about the ReportViewer control's WebForms version. This OakLeaf Blog post has links to this and other recent VSM articles. --rj P.S.: This is a copy (with minor modifications) of the latest "Plog" post for the book to my Amazon Connect site.

Sunday, February 05, 2006

Squidoo Changes Lens Ranking Method

On Sunday, February 5, 2006, I received the following e-mail message from Seth Godin, the founder of Squidoo:

As one of our top 100 lensmasters, you ought to know about a new iteration of the Squidoo beta that will happen today or tomorrow. We're posting many new features (which Heath will mention on our Squidblog and we're also going to change the way we compute the top 100. Instead of using "all-time" stats, we're going to institute a two-week rolling stats window for computing rank. Simple version: if your lens is high-ranking just because you've been around for a while, it might not be ranked as high by Tuesday. I didn't want you to be shocked at the new stats, hence this note, and I hope you'll decide to juice up your lens and your traffic and use this as an opportunity to move on up now that the log jam is broken. Thanks for all your support so far! Seth Original Squid
I raised the issue of unexplained and unexpected ranking of my four Squidoo lenses in an earlier "Squidoo Beta Problems - December 2005" post. Most of the problems I reported in that post have been solved, as noted in the update text. For example, there's a new workaround I discovered that lets you claim your lens, which should (but doesn't) enable Technorati tagging; a new Text List module eliminates my problems with the 2,500-character limit on plain Text modules. Updated 2/15/2006: Starting Monday, 2/6/2006, my lenses were ranked as follows (commas separate data for Mon, Tue, Wed, Thu...): www.squidoo.com/surfboards: 15, 23, 23, 22, 30 www.squidoo.com/theblackscholar: 228, 245, 194, 152, 180 www.squidoo.com/microsoft_access: 246, 289, 197, 179, 235 www.squidoo.com/oakleafsystems: 337, 380, 164, 156. 199 Monday and Tuesday: All my blogs gained in rank (like in golf, lower scores are better), presumably as the result of the new ranking method. However, their ranking declined on Tuesday, probably because none of the lenses were modified that day. Wednesday: The Black Scholar lens was updated with a major new entry and the .../oakleafsystems (Databases) lens was updated by automatic addition of a new Atom link for this entry on Tuesday night (2/7/2006). The .../surfboards and .../microsoft_access lenses were last updated on Sunday, 2/5/2006. Thursday: Only .../surfboards was updated, yet all four lenses gained rank. This is probably due to lack of updates to lenses rated below mine as of Wednesday. Friday: All lenses except .../surfboards were updated with minor changes (primarily Technorati tags). All lenses lost ground in the ratings. Other Rankings:

Squidoo also added "Top 100" rankings within many categories. Here are some results for me and my four lenses as of 2/9/2006:

  • I was #21 on the "Top 100 Lensmasters (by LensRank)" list.
  • "No More U.S. Custom Surfboards?" was #58 on the "Top 100 Most Visited (most traffic)" and #3 on the "Top 100 by Topic (Computers and Technology)" lists.
  • "OakLeaf Database Lens" was #25 and "Microsoft Access Lens" was #28 on the "Top 100 by Topic (Computers and Technology)" list.
  • "The Black Scholar" was #7 on the "Top 100 by Topic (People and Society)" list.

Obviously, these rankings are likely to change as more authors create new lenses and existing authors update formerly high-rated lenses.

Mis-Ranking?

The low ranking of The Black Scholar lens, compared to the .../surfboards lens, continues to be questionable because of the former's extensive content, more than 200 external links, and very frequent modification. The Black Scholar lens ranks #2 in a Google search for "The Black Scholar". The journal's site is #1. I'll continue to update this post daily for a week or so and attempt to determine what effect minor text and link modifications have on lens rank. --rj

Technorati:

Wednesday, February 01, 2006

VSLive! 2006 San Francisco Keynotes

Fawcette Technical Publications' VSLive! 2006 conference opened Monday, January 30, 2006 at San Francisco's Moscone West venue. This post provides links to summaries and online videos of each morning's 9:00 a.m. keynote address from the VSLive! San Francisco ShowDaily site: Monday: S. "Soma" Somasegar, Corporate Vice President, Developer Division, Microsoft: "Visual Studio 2005: A World of New Opportunity." Somasegar announced that Microsoft Team Foundation Server will be available as a Release Candidate at the end of this week and will release to manufacturing in March. He and his keynote team also announced a Community Technical Preview (CTP) for the Visual Studio Tools for Applications (VSTA) SDK, as well as CTPs for Visual Studio "Orcas" technologies such as Atlas, LINQ, Cider, and WinFX. Read "Q&A: Microsoft's Somasegar touts value of ALM platform," InfoWorld Editor-at-Large Paul Krill's interview with Somasegar after his VSLive! presentation. Tuesday: David Chappell, Principal, Chappell & Associates: "Building Modern Software: Services, Workflow, Integration." David Chappell detailed the design patterns for a new breed of service-oriented applications (SOAs) to a standing-room-only crowd at his VSLive! keynote on Tuesday. This fourth-generation architecture integrates Microsoft Windows Communication Framework (WCF, formerly code-named "Indigo"), SOAP-based XML service components, Windows Workflow Foundation (WF), BizTalk Server 2006, and Visual Studio 2005. Wednesday: Larry Jordan, Development Manager, MSDN and TechNet Development Team, Microsoft: "MSDN Architectural Overview and Deep Dive." Jordan billed MSDN as “the source of truth in technical documentation” in his keynote presentation. Prior to the complete MSDN site makeover by Larry and his team that began two years ago, relying on MSDN Search to seek the “truth” frustrated developers no end. According to a show of hands from the keynote audience, most developers currently use Google to find the MSDN software documentation pages they need. MSDN Search is just one of the site’s technologies to be replaced. The MSDN team ported the original ASP site to ASP.NET 2.0, migrated document storage from the Windows file system to 32-bit SQL Sever 2005, and wrote XML Web services to serve as an API for metadata and document access. Visit the VSLive! San Francisco ShowDaily site for view the additional keynote presentations on Tuesday and Wednesday. P.S.: Microsoft's Rob Caron takes issue with the comments in my review of Soma's keynote on "dogfooding" Team Foundation Server in this blog post. Technorati:

Wednesday, January 18, 2006

Visual Basic Team Releases January LINQ CTP

Microsoft released the January CTP of the Language Integrated Query (LINQ) project's Visual Basic preview for Visual Studio 2005 on January 17, 2006. The big news is that the January CTP finally adds DLinq support to Visual Basic 9.0. You can download the January bits here. The new CTP replaces the September (PDC 2005) Visual Basic CTP that, unlike the C# version, didn't support DLinq for relational data sources. [Updated 2/7/2006]. The January CTP requires the release version of Visual Studio 2005 or Visual Basic Express, and works with SQL Server 2005 or SQL Server Express. (It probably also works with SQL Server 2000 or MSDE 2000, but I haven't tested that capability.) Microsoft hasn't tested the CTP with languages other than English. Although not stated in the Readme page, it's a good idea to use Control Panel's Add/Remove Programs tool to remove the September CTP version before executing Linq VB Preview (Jan 2006).msi. Running the CTP's installer adds a VB LINQ Preview item to your Programs menu with Readme, Visual Basic 9.0 Overview, Visual Basic 9.0 Samples, Visual Basic LINQ Project Overview, and XLINQ Overview choices.

SQLMetal.exe Updated for Visual Basic 9.0

The January CTP's version of the SQLMetal.exe utility creates the DatabaseName.vb class file that's necessary for mapping an SQL Server resultset to a LINQ object. (The September CTP's SqlMetal.exe didn't work with the /language:vb switch.) Later VB 9.0 versions will add a designer to simplify generating the mapping class file.

To create a Northwind.vb class file with nwind as its namespace by attaching the Northwind.mdf file in the C:\Program Files\VB LINQ Preview\Samples\Sample Queries\SampleQueries folder to your default SQL Server 2005 instance, execute the following from the "\Program Files\VB LINQ Preview\Samples\Sample Queries\SampleQueries folder:

"\program files\vb linq preview\bin\sqlmetal" /namespace:nwind /language:vb /code:Northwind.vb northwind.mdf

If you've already added a Northwind database to your default SQL Server instance, you receive a "Failed to attach an auto-named database for file northwind.mdf. A database with the same name exists ..." or "A connection was successfully established with the server, but then an error occurred during the loginprocess. (provider: Shared Memory Provider, error: 0 - No process is on the other end of the pipe.)" exception.

In this case, execute the following to create the class file from the server and database you specify in the following example, which assumes SQL Server Windows authentication:

"\program files\vb linq preview\bin\sqlmetal" /server:localhost /database:Northwind /namespace:nwind /language:vb /code:Northwind.vb

New January CTP Features

According to the January CTP's Readme page, the CTP adds the following new features to the September release:

LINQ Intellisense: Query statements now have statement completion, Intellisense, and Pretty Printing to improve overall discovery and readability of VB queries. DLinq Support: Previous tech previews of the LINQ CTP did not fully support leveraging the DLinq technology which is for querying against relational data stores, such as SQL Server. DLinq support in this release is limited in that only the working query syntax over objects will extend to DLinq. Editing support for XML Literals: The Visual Basic background compiler provides immediate feedback, with this release this support has extended to integrated XML. Specifically, XML literals are now automatically indented and XML literals plug into auto-correction. The XML literals are now completely colorized with customizability through the VS 2005 Options, Fonts and colors.
The following new features apply to XML late-binding:
Importing XML Namespaces: XML namespaces can be imported to the VB program using similar mechanism to importing CLR namespace. Accessing Elements with Namespaces: Once an XML namespace is imported, you can use qualified identifiers to access elements with namespaces. Indexer for IEnumerable(Of T): An indexer was added to IEnumerable of any type. This enables easy access to any element from a generic IEnumerable, specifically for accessing the first XElement from the XElements collection that is returned from XML late binding expressions.

Paul Vick Summarizes New January CTP Features

Paul's February 6, 2006 post on the availability of the January CTP was a bit late, but his informal summary of the new DLinq and XLinq features is quite useful:

  • We now support some Intellisense for Select expressions. This is a step forward in our investigation of the Select/From vs. From/Select question, so we’re definitely interested in feedback here.
  • We now support a lot more LINQ, specifically DLinq and variable capture (so you can now access local variables in queries). A huge chunk of my November/December went towards implementing lambda expressions and expression trees in Visual Basic and that, combined with some excellent work by another team member on variable capture, means a lot of stuff works now that didn’t before.
  • The editing/display experience for XML literals has been greatly enhanced — just having colorization makes a huge difference.
  • A bunch of extensions were made to the XLinq support to make working with namespaces and elements easier.
Here's a screen capture that demonstrates colorization of part of the code for generating XLinq XML literals with element values obtained with DLinq from the Northwind sample database: Online Demo and Newsgroup Paul Vick and Amanda Silver deliver "Visual Basic Futures: Latest Developments in the LINQ Project", a 10-minute video overview of the objectives of the LINQ project, plus a brief description of January CTP's new and improved DLinq and XLinq features. Amanda continues the presentation with a 20-minute demonstration of DLinq and XLinq in Visual Studio 2005 console applications that demonstrate basic DLinq and XLinq syntax in simple console applications, the LINQ VB Project Sample Query Explorer Windows form project, and a new (but apparently unfinished) visualizer that displays expression trees for LINQ queries, as shown here:

Unfortunately, the expression tree visualizer isn't included with the January CTP bits. Click here to read a trasncript of the overview and demo. You can post questions about the new VB 9.0 features and (hopefully) receive answers on the LINQ (General) newsgroup. The newsgroup's MVPs are mostly—or, more likely, entirely—C# coders. Sample Applications

The Programs, VB LINQ Preview, Visual Basic 9.0 Samples menu choice opens the [C:]\Program Files\VB LINQ Preview\Samples folder, which contains Hands On Labs, Sample Queries, Visual Basic 9.0 Survey, and XML DOM Survey subfolders. Following is a screen capture of the LINQ VB Project Sample Query Explorer form that opens when you double-click the [C:]\Program Files\VB LINQ Preview\Samples\Sample Queries\SampleQueries.sln solution file and run the 101 DLinq Samples JOIN SelectMany - 1 To Many - 1 sample (DLinq27()).

Note: The default connection string is: connString = "AttachDBFileName='C:\Program Files\VB LINQ Preview\Samples\Sample Queries\SampleQueries\Northwind.mdf'; Server='.\SQLEXPRESS';Integrated Security=SSPI;enlist=false". If you don't have SQL Server Express installed or you already have added the Northwind sample database, you must change the connection string in the DLinqSamples class's Declarations section to a value that suits your installation.

This project lets you execute the sample LINQ, DLinq, and XLinq queries and, for DLinq, display the T-SQL statement and, where applicable, the query resultset for the sample.

The other two sample projects are console applications that are similar to their corresponding September CTP counterparts.

Links to Earlier LINQ Articles and OakLeaf Blog Posts

"Comparing LINQ and Its Contemporaries" by Ted Neward from the .NET Framework Developer Center (December 2005)

"Late Binding in Visual Basic" by Paul Vick and Erik Meijer (December 1, 2005, 00:22:45 MSDN TV video)

"Streamline Mapping With Orcas and LINQ" article from the December 2005 issue of Visual Studio Magazine.

"Cure Data Type 'Impedance Mismatch' With LINQ," from Fawcette Technical Publishing's .NETInsight newsletters for PDC 2005 (September, 2005)

"The Language Integrated Query (LINQ) Project" (first LINQ post), September 14, 2005

"Erik Meijer on LINQ as a 6GL Language Feature," September 14, 2005

"LINQ Interview with Anders Hejlsberg and Paul Vick," September 19, 2005

"Jon Udell: Microsoft driving toward .Net unity," September 21, 2005

"XLinq Presentation at the XML 2005 Conference," September 23, 2005

"Comparing C# 3.0 and VB 9.0 LINQ Syntax," September 26, 2005

"More on Object-XML and Object-Relational Impedance Mismatch," September 29, 2005

"Jon Udell: LINQ 101," September 29, 2005

"Compose XML Content with XLinq Expressions and VB 9.0 XML Literals," October 1, 2005

"More on Visual Basic 9.0 and LINQ versus SQL," October 10, 2005

--rj Technorati:

Monday, January 16, 2006

InfoWorld Gives Thumbs Up to SQL Server 2005

InfoWorld magazine's "Special Report" for January 16, 2006, "SQL Server's extreme makeover," begins with this summary:

"Microsoft's flagship database pumps up to enterprise class with high availability, slick disaster recovery, rebuilt integration services, stronger analytics, and other heavy-duty enhancements."

Contributing Editor Sean McCown, who wrote "SQL Server bulks up," the five-page review of SQL Server 2005 and its management tools, says:
"More than five years in the re-making, SQL Server 2005 not only addresses shortcomings in high-availability and disaster recovery, but brings sweeping mprovements in almost every area. Management tasks have been streamlined, and monitoring and diagnostics have been beefed up. A default locked-down configuration, data encryption, and other new features strengthen security. Analysis Services, Reporting Services, and data Mining are now fully integrated and mature. DTS (Data Transformation Services) has been rewritten from the ground up, emerging as the more powerful and flexible SQL SSIS (Server Integration Services.)"

He gives SQL Server a score of 9.1 (Excellent) with 9 out of 10 points for managability, performance, availability and scalability, plus 10 points for value. I would have awarded 10 points for managability, which has been SQL Server's forte since version 7.0 introduced the "Starfighter" management tools. Starfighter was the code-name for the first version of SQL Server Enterprise Manager implemented as a Microsoft Management Console (MMC) application.

McCown reviews new SQL Server Integeration Services (SSIS), which replace Data Transformation Services (DTS) and feature .NET Framework 2.0 integration. He points out some annoyances with SQL Server Management Studio (SSMS), but gave the new Data Management Views (DMVs) high marks for providing DBAs with management assistance, server statistics, and reports. Increased security with DDL triggers to prevent undesired schema changes, better password management, control over schema changes, and column/cell-level encryption also received a nod. (See my "Encrypt and Decrypt Data in Yukon" article from Visual Studio Magazine for a developer's eye view of SQL Server 2005's encryption/decryption processing.)

High-availability (HA) features and disaster recovery capabilities get high marks for mirroring, online restores, and page-level restores. Although McCown noted that mirroring was unsupported in the November 2005 release version of SQL Server 2005, Microsoft promises a supported version in 2006. Online indexing also contributes to HA.

A "Development Snapshot" graphic's Pros section outlines new SQL Server 2005 features for developers, such as CLR integration, new T-SQL constructs, and native XML data storage. The Cons include lack of IntelliSense for T-SQL queries, which Microsoft promised and then reneged, problems with an "unresponsive interface" for SSMS (which I haven't encountered), and more complex queries for DVMs than their "deprecated counterparts." A "Cool Tools for SQL Server Coders" graphic lists the most important T-SQL enhancements. See "Take Advantage of New T-SQL Features" and "Exploit Yukon's XML Data Type" for more details and downloadable Visual Basic 2005 source code examples for these new features. The article doesn't review new SQL Server Service Broker (SSSB) features or native XML Web services capabilities. (See my "What’s New With VS and Data" interview, forthcoming Visual Studio Magazine article on SSSB and August 2005's "Create Interoperable Native Web Services.")

McCown gives SQL Server 2005’s business intelligence (BI) tools—SQL Server Analysis Services (SSAS), SQL Server Data Mining (SSDM), and SQL Server Reporting Services (SSRS)—a test drive and finds them "finally designed to work together to answer your intelligence needs." SSRS's Report Builder gets praise for its Report Model, which lets administrators define the data available to user-defined reports.

The review concludes with a rehash of what's missing in SQL Server 2005—Intellisense for T-SQL queries, a centralized code repository, and lack of a separate extract, transform, and load (ETL) server—and advice about if and when to upgrade to SQL Server 2005. McCown's closing remarks, however, say it all:

SQL Server 2005 is an excellent release that will gain Microsoft some real respect in the high-end database market. SQL Server’s tools now lead Oracle and DB2 in pretty much every area—management, monitoring, and especially development—and of the three vendors, Microsoft seems to be the most concerned about improving them.

About the Reviewer:

Sean McCown is a Contributing Editor for InfoWorld magazine and a frequent contributor to SQLServerCentral.com as well as the SQL Server Worldwide Users Group. He also writes the occasional product review for Computerworld and article for SQL Server Standard. His April 2004 "Databases flex their XML" review for InfoWorld drew this detailed response from Michael Rys, Microsoft's point man for SQL Server 2005's XML data type.

You can read Sean's book reviews on database programming, database and Windows security, and general database topics at his ITBookworm site, and read his advice for DBAs and DBA "war stories" on his DBA Rants blog. Sean is the Senior DBA for SourceCorp of Dallas, Tx, where he lives with his wife and two kids. He's practiced and taught Kenpo for the past 20 years. [Kenpo competency is very useful for DBAs, especially Senior (supervisory) DBAs.]

Sidebar: Taking the Database Beyond SQL

Tom Yeager, InfoWorld's Test Center chief technologist and new-technology blogger begins his analysis with this paean to integration of the .NET Framework 2.0 with SQL Server 2005:

Through integration with .Net, SQL Server 2005 takes its rightful place as an active peer, not a detached agent, in large-scale distributed applications. ... SQL Server 2005’s .Net integration creates a dream scenario in which SQL developers can incorporate the advanced functionality of .Net in their stored procedures, while C#, Visual Basic, and even JavaScript developers can work with databases as though they were language extensions.

But .NET integration isn't limited to creating stored procedures with native code; you can also write user-defined functions (including table-valued functions), user-defined aggregates, triggers, and—if you have the courage—user-defined data types with Visual Basic 2005 and C# 3.0. (JavaScript isn't a supported language for creating SQL Server objects.)

.NET might be the best thing since sliced bread (or Java integrated with competitors' RDBMSs), but native code running in the SQL Server 2005 process has the potential to be the most misused feature in this release. (See the "When to Use SQLCLR" sidebar to my November 2005 "Manage Data With VS 2005" Visual Studio Magazine article.

Ahead of the Curve: SQL Server could make 'better together' ring true

Tom Yeager says, "Windows [2003] Server [R2], Visual Studio [2005], and SQL Server [2005] may impress even hardened skeptics," and goes on to explain why "reality-based" is a better marketing slogan than "better together" for Microsoft's newly upgraded server systems and developer tools. Yeager contends that testing by Microsoft's IT department and tens or hundreds of thousands of independent beta testers creates a truly "reality-based" server suite.

Database Underground: An Answer to Open Source

Sean McCown's December 11, 2005 blog post argues that SQL Server and Oracle "freeware" (Express Edition) versions of their proprietary relational database management systems (RDBMSs) are more suited to mission-critical applications than "open-source" databases like MySQL and PostgresSQL. (See "Oracle Releases Free 10g Express Edition Beta," October 2005). McCown sees advantages to Express Editions' maturity, performance monitoring, and well-travelled upgrade path.

McCown didn't make an important argument in favor of Express Editions: There is little or no relief from vendor lock-in with "open-source" databases. Migrating databases from MySQL to PostgreSQL or—heaven forbid—HSQLDB is likely to be far more difficult than moving to and from Oracle, SQL Server, DB2, or Sybase. The commercial RDBMS vendors provide tools to accomplish the transation; with an "open-source" database, you're mostly on your own.

McCown wants Microsoft and Oracle to remove the 4-GB size limits from their Express Editions in the hope that doing this will induce Web site owners to migrate from "open-source" to "proprietary" databases. It seems to me that Microsoft and Oracle are delivering much a more useful and stable product than MySQL or PostfresSQL—the two major players in the open-source database market—and neither vendor charges for support. Microsoft provides an Express version of SSMS to handle simple database chores in a Windows UI. Bear in mind that the size limit is per database, not per server, and you can run many named instances of SQL Server Express on a single computer running Windows XP or 2003 Server. (See my May 2005 "Migrate to SQL Server 2005 Express Edition" Visual Studio Magazine article.)

Editor's Letter: SQL Server 2005 leaves open source databases in the dust InfoWorld editor Steve Fox supplies a pointer to Sean McCown's December 28, 2005 review of MySQL 5.0, "Long-awaited MySQL 5.0 makes its debut," and quotes McCown as referring to DBAs that use "open source" databases as “'tree huggers' who don’t want to pay for commercial software." The woods are full of folks who subscribe to the "software should (or must) be free" theory, so it's unlikely that this opinion piece will change many "open-source" proponents' minds. However, McCown's An Answer to Open Source might convince rational DBSs to at least consider an Express Editions of a commercial RDBMS. Note: eWeek's Lisa Vass wrote a similar column titled "MySQL vs. the Lite Databases: A Fair Comparison?" that appeared December 29, 2005. She published a related column, "Readers Respond to MySQL, Lite Databases Comparison," the following week. This second column described how readers took her to task for omitting PostgreSQL and Firebird from the initial comparisons. Technorati: P.S. In honor of Martin Luther King, Jr.'s birthday, celebrated Monday, January 16, 2006, I've finished a major update to my Squidoo lens (Web page) about The Black Scholar: Journal of Black Studies and Research, its history, editors and contributors.