Showing posts with label TerraServer. Show all posts
Showing posts with label TerraServer. Show all posts

Wednesday, January 09, 2008

LINQ and Entity Framework Posts for 1/7/2008+

Note: This post is updated daily or more often, depending on availability of articles. As of this issue, the publication date will be reset after adding significant new posts.

Update 1/9/2008: Date (year) errors fixed.

The VB Team: Writing UNION, TOP and Subqueries in LINQ

The Converting SQL to LINQ, Part 7: UNION, TOP, Subqueries (Bill Horst) post of January 8, 2008 covers use of LINQ Union, Intersect, Except, Take, TakeWhile, and Skip Standard Query Operators in VB 9.0 query expressions.

Added: January 9, 2008

Marcello Lopez Ruiz Clarifies ADO.NET Data Services' URI Format

Marcello's Updates to URL syntax for December CTP of ADO.NET Data Services post of January 8, 2008 advises that the current Astoria CTP uses only the compact URI syntax. Compact URI syntax uses the resource set name followed by one or more comma-separated, single-quoted identifiers enclosed in parenthesis as in: /Customers('ALFKI').

The long-form syntax, described in Mike Flasko and Pablo Castro's URI Format - Part 1 - Addressing resources using URI path segments post of September 21, 2007 in the Astoria Project Team blog, was not implemented, presumably because the team abandoned the Web3S wire format in favor of JSON, ATOM (and hopefully POX) formats only.

Added: January 9, 2008

Julie Lerman: SQL Server CE Provider Coming for Entity Framework December CPM

Julie's SQL Server CE compatibility with EF Beta 3 in a few weeks... post of January 8 quotes the EF team's Jaroslaw Kowalski with the timetable for an upgraded SSCE Entity Framework data provider.

Added: January 8, 2008

Dare Obasanjo Compares LINQ with Python for List Processing

Dare writes a very terse C# LINQ to Objects query to find and display anagrams within 1,219 of the most popular male names in the U.S. in 97 milliseconds or less. Doing the same for 4,275 female names takes about 557 ms.

Added: January 8, 2008

Scott Guthrie Starts Dynamic LINQ Query Series

Scott kicks off his new Dynamic LINQ series with Dynamic LINQ (Part 1: Using the LINQ Dynamic Query Library) post of January 7, 2008 (earlier than usual—11:03 PM) that makes use of the unpublicized LINQ Dynamic Query Library that's included in the C# and VB samples packages you can download from here (C#) or here (VB). Downloads require agreement to the new Microsoft Public License (Ms-PL).

Some bloggers refer to LINQ queries in which you substitute string or numerica variables for literal constants in the query expression as "dynamic." However, the LINQ Dynamic Query Library goes a big step further by letting you substitute composable strings for Standard Query Operator arguments in queries that use method call syntax. The library generates expression trees by parsing the strings. Of course, You exchange type safety and IntelliSense for flexibility when you employ this technique.

Scott offers VB and C# ASP.NET Web site samples, and points to the Albahari brothers' Dynamically Composing Expression Predicates post that describes their PredicateBuilder class. PredicateBuilder is part of their LINQKit productivivity kit for LINQ to SQL, which every LINQ partisan should have.

Update: In response to a comment, Scott adds the following advice:

LINQ to SQL uses type-safe data model classes, you are protected from SQL Injection attacks by default.  LINQ to SQL will automatically encode the values based on the underlying data type.

BTW - you can use the Where() extension method either like so:

   .Where(String.Format("CategoryID={0}" & Request.QueryString["id"])

Or:

   .Where(String.Format("CategoryID=@0", Request.QueryString["id"])

The second option allows you to specify any parameter markers you want in the query, and then in[j]ect the values as params that you pass separately.

Be sure to read the comments and Scott's replies.

Added: January 8, 2008

Stefan Cruysberghs Examines Dynamic LINQ Where and OrderBy Clauses

Apropos Scott's Dynamic LINQ queries post, Belgian developer Stefan Cruysberghs' .NET - LINQ to SQL - part 4 article of January 7, 2008 goes into the details of

  • Anonymous functions and IEnumerable
  • Lambda expression trees and IQueryable
  • PredicateBuilder class and combining expressions
  • Predicate library

for creating dynamic Where and OrderBy clauses.

The article also includes links to his three preceding LINQ-related Web pages. Click here for a complete list of Stefan's articles in English.

Update 1/9/2008: Stefan's recent articles cover a wide range of topics of interest to VS developers, including detailed data on the performance improvement you can expect from PLINQ. His English .NET RSS feed includes brief abstracts.

Added: January 8, 2008

LINQ to XML: Grouping and Aggregation Gotchas, Part I

LINQ to XML is a candidate to replace XQuery for grouping elements, transforming Infosets and aggregating numerical data. It also can supplant XPath for navigation and XSLT for grouping and transformation. Aggregating elements or element groups and generating sum, average, minimium and maximum values is one of the most common uses for XQuery, XPath and XSLT.

However, non-trivial LINQ to XML examples that demonstrate element grouping and numerical aggregation are few and far between. This LINQ to XML: Grouping and Aggregation Gotchas, Part I post of January 7, 2008 describes two Microsoft grouping samples that don't behave as expected.

C# Team Posts Custom LINQ to TerraServer Implemetation

Charlie Calvert's TerraServer Sample: A LINQ Provider post of January 6,2008 announces the arrival of yet another custom (not third-party in this case) LINQ implementation: LINQ to Terra Server.

Here's the project's description from the project's ReadMe.htm file:

This sample is a custom LINQ provider for the TerraServer-USA Web service. It also contains a sample client application that uses the custom LINQ provider to query the Web service for geographical data.

The TerraServer-USA Web service exposes a method that returns information about locations in the United States when the method is given part or all of a location name. This method, which is named GetPlaceList, is the method that the LINQ provider calls to obtain the data that the LINQ query is run against. The provider uses Windows Communication Foundation (WCF) to communicate with the Web service. For more information about the TerraServer-USA Web service, see Overview of the TerraServer-USA Web Services.

A LINQ provider that implements the System.Linq.IQueryable<T> interface, such as this one, enables LINQ queries to be written against the data source that the provider connects to. A provider may execute the query functionality on the data itself, or it may translate the LINQ query into a query language that is appropriate for the data source that it connects to. This provider obtains raw data from the Web service and then modifies the original query in such a way that LINQ to Objects handles the query execution.

And here's sample output from the console:

States that have a place named "Redmond":
Washington
Colorado
Oregon
Utah
West Virginia
Pennsylvania

Number of places whose names start with "Lond": 29

Places, ordered by state, whose name is either "Johannesburg", "Yachats", or "Seattle":
{ Name = Johannesburg, State = California }
{ Name = Johannesburg, State = Michigan }
{ Name = Yachats, State = Oregon }
{ Name = Seattle, State = Washington }
{ Name = Johannesburg, State = Wisconsin }

MSDN's new Walkthrough: Creating an IQueryable LINQ Provider article shows you how to create the provider and client from scratch. (The link in the ReadMe.htm is incorrect.)

The LINQ to TerraServer code is part of a set of C# samples that you can download from here (requires agreement to the new Microsoft Public License (Ms-PL). 

Note: A few years ago, I wrote a graphic Web service front end to TerraServer in VB 2003. "Build Real-Time Web Images" article in the August 2004 issue of Visual Studio Magazine described a Visual Basic .NET 2003 graphic consumer of TerraService.net Web Services, as well as Microsoft MapPoint services. You can download the source code for the project here.

David Hayden Continues His Dynamic Data Series with Metadata Providers Tutorial

David says in ASP.NET Dynamic Data Tutorial - BuddyMetadataProvider and Custom Metadata Providers of January 6, 2008:

This is my 5th tutorial on the new ASP.NET Dynamic Data that ships with the ASP.NET 3.5 Extensions CTP. ASP.NET Dynamic Data is an alternative to code generation in helping one build database-driven web applications in a more productive manner. ...

If you don't like adding metadata to partial classes via attributes, the MVCToolkit contains a couple of custom metadata providers that you can use as a vehicle for adding metadata to your Dynamic Data Applications in different ways:

  • BuddyMetadataProvider
  • XmlMetadataProvider

And goes on to explain how to use both.

Ruurd Boeke Demonstrates Entity Framework's Table Per Type Inheritance

Catch-up: Ruurd's How To: Table per type inheritance with Entity Framework post of January 2, 2008 provides an illustrated walkthrough for implementing TPT inheritance with the classic three tables and classes: Person and related/derived Employee, and Customer entities.

Update January 9, 2008: Following are links to some of Ruurd's earlier Entity Framework posts:

Update January 10, 2008: Fixed error in spelling of Ruurd. Sorry about that.

Scott Hanselman Endorses LINQ to Mock (Moq)

From the "How Did I Miss This?" department: Scott's Moq: Linq, Lambdas and Predicates applied to Mock Objects of December 19, 2007 starts with:

Kzu and friends have a new pet project called Moq, which may be the coolest derivative open source project name ever.

And ends with:

Slick. Moq is definitely a project to watch.

In between, Scott compares NMock, RhinoMock, TypeMock and Moq (a.k.a., LINQ to Mock for provider naming consistency.)

The "Daniel Cazzulino and Juan Wajnerman Create LINQ to Mock (Objects)" topic of LINQ and Entity Framework Posts for 12/17/2007+ has more details on the release of LINQ to Mock.

Saturday, February 17, 2007

"Friends of Jim" Halt Volunteer Search for Jim Gray

The Tenacious blog reports that the search for Jim Gray, the renown large-scale database and transaction-processing pioneer, by a team volunteers known as the Friends as Jim has ceased its activities as of February 16, 2006. The "Search Status" post by Mike Olson states:

In the last several days, the Friends of Jim group has reviewed all the data with Coast Guard officials. The fact is that we have no evidence as to what has happened to Tenacious or to Jim Gray. Neither we nor the Coast Guard can come up with a surface search plan that is likely to find either Tenacious or Jim, given everything that has been done already.

Accordingly, the Friends of Jim group is suspending its active effort to find Tenacious that has been centered here at the blog. For both the Coast Guard and the Friends of Jim, “suspension” means that the active search has been discontinued due to exhausting all present leads and the lack of new information. Of course, should we or the Coast Guard receive any new information, we will investigate it.

James Nicholas Gray 1944 - 2007? Rest In Peace

The "U.S. Coast Guard Searching for Jim Gray off San Francisco Coast" post on this blog includes a chronology of the search efforts with quotes from Coast Guard sources, video news segments, and print stories. There are links to additional articles about Jim and the search on the James N. Gray Wikipedia page. The Help Find Jim site has a photograph of Jim's sailboat, "Tenacious," and links to downloadable posters that volunteers hung at marinas and other Pacific Coast locations.

Microsoft Research's "Update on Search for Jim Gray" page expands on the Tenacious site's statement:

There is tremendous interest both inside and outside Microsoft in paying appropriate tribute to Jim. But his family believes it would be premature at this time to hold a memorial service, because it has been less than three weeks since he disappeared. We are respecting the family’s wishes.

This has been an extremely emotional time for all of us who consider Jim a colleague and a friend. The tremendous outpouring of support is a true testament to how many lives he has touched. We all miss him, and we continue to keep Jim and his family in our thoughts.

All who knew him or knew of his work will miss Jim greatly.

Update 5/2/2007: Mark Whitehorn has posted an eloquent "Tribute to Jim Gray" on the UK's The Register.

Monday, January 29, 2007

U.S. Coast Guard Searching for Jim Gray off San Francisco Coast

1/29/2007: According to a brief story from Oakland's KTVU, the U.S. Coast Guard is searching the waters outside the Golden Gate for Microsoft Research's ultra-large database and transaction-processing expert Jim Gray. (Last Update 2/3/2007, 12:09 PST).

I happened to be scanning the Bay Area TV channels at about 5:15 p.m. today (Monday, January 29, 2007) when I saw a photo of Jim Gray and heard the last few words of a sound byte about his apparent disappearance after leaving San Francisco Sunday morning for a trip to the Farallon Islands in his 40-foot sailboat "Tenacious."

Update 1/30/2007, 3:45 p.m.: The "Search Intensifies For Missing Microsoft Exec" story from from KNTV (NBC 11 San Jose) includes quotes from Jim's daughter, Heather, video news clips, and a 10 minute raw video interview with Coast Guard spokeswoman Lt. Amy Marrs. The Marrs interview explains the status of the Coast Guard's search efforts as of yesterday.

A KPIX (CBS 5, San Francisco) story, "Coast Guard Steps Up Search for Missing SF Boater," says the Coast Guard "is expanding their search for him based on new information" that Jim enjoyed sailing north of the Golden Gate. According to reporter Anna Duckworth, the Coast Guard is adding another plane to the search this afternoon (Tuesday).

KGO-TV (ABC 7, San Francisco) offered similar coverage with its "Coast Guard Resumes Search for Sailor" story. The search was terminated at 3:15 a.m. Tuesday morning and resumed at sunrise. A later "Coast Guard Expands Search For Missing Sailor" article mentions two possible sightings at Stinson Beach, but searchers found nothing at this exclusive Marin County resort village a few miles northwest of San Francisco.

Update 1/31/2007: The Coast Guard reports that the search has been extended to an area from Monterey Bay to the Oregon state line. The boat was equipped with an emergency beacon and it hasn't gone off, according to the Coast Guard.

"Search continues for Microsoft researcher missing at sea" from the San Jose Mercury-News is a full length story datelined today. The San Francisco Chronicle ran a similar "Vast search off coast for data wizard" story. Techmeme has a link to an InformationWeek story by Charles Babcock.

KTVU's Patti Lee reported in a 1/31/2007 video clip that Jim wasn't scheduled to work at BARC "today or tomorrow" presumably meaning Monday and Tuesday. KNTV also offered a brief updated report.

Update 2/1/2007: The Coast Guard planned to call of the search as of this morning, according to KNTV's "Coast Guard To Call Off Search For Missing Sailor" Wednesday night story that says searchers are "'Stumped' In Search For Gray" and "Search for Gray Could Be Over" news segment, and a similar Associated Press story and segment from KPIX.

However, the Coast Guard said this morning (Thursday) that the search would be extended to 300 miles west of the coastline and south to the Channel Islands, west of Santa Barbara. KTVU's "Coast Guard Extends Search For Missing SF Sailor" story, updated this morning, and Jade Hernandez' related video segment tell most, but not all, of the story.

KGO-TV reported in yesterday's "Time Is Running Out For Missing Sailor: Coast Guard To Give Up Search" story by Heather Ishimaru:

Lt. Commdr. Jonathon Copley, U.S. Coast Guard: "He had no hard obligation at work, or at home, with his wife who was out of town until today, don't know if he did that, but know he loved to sail, we do know that."

And they have considered he might not want to be found.

Lt. Commdr. Copley: "People who don't want to be found, it's hard to find them, because all our models are based on people who want to be found."

At 11:44 a.m., KNTV updated this "Coast Guard To Suspend Search For Jim Gray Tonight" story based on a Coast Guard press release that quoted Lt. Amy Marrs: "To suspend a search is never an easy decision. We've covered the area Mr. Gray is likely to have been with the resources we have, but closing a search only comes after it has been determined we have done everything we can do to find someone." KNTV's Gavin Thomas reports that "[f]riends of Gray said they are prepared to hire private aircraft to continue the search if the Coast Guard calls off their efforts.

Update 2/2/2007: The Coast Guard suspended the search for Jim Gray as of 5:30 p.m. Thursday (2/1/2007) according to this USCG press release. KGO-TV's "Coast Guard Calls Off Search For Missing Sailor" story has more details, including the fact that Jims cellphone was active on the Cingular network at 7:30 p.m. Sunday, which indicates his location was within 10 miles of the shore.

Microsoft Research has an "Update on Search for Jim Gray" page with a link to a recently started Tenacious Search blog. According to this 2/2/2007 post, the private searchers had three planes in the air Thursday and planned to have four on Friday.

Update 2/3/2007: Today's "Silicon Valley’s High-Tech Hunt for Colleague" article by the New York Times' Katie Hafner summarizes the unsuccessful Coast Guard search and detailed private efforts with satellite imagery and aircraft. (The story hit Techmeme.) Eric Newcomer posts his reminiscences of Jim at Digital, Microsoft's Don Dodge recalls working together at Digital and Microsoft, and a reader describes a dinner with Jim while moored at The Spinnaker restaurant's dock in Sausalito.

The Tenacious Search blog has updates on the private search activities. Folks in Cal Berkeley's Electrical Engineering/Computer Science departments have produced a "Missing" poster. Amazon.com CTO Werner Vogels has added a "Help Find Jim Gray" post and there's a "Jim Gray Missing: Help find him by searching satellite imagery" Mechanical Turk task that request visitors to scan images for foreign objects that could represent "Tenacious."

Jim Gray's Contributions to Database Technology

Jim was graduated in 1969 from my alma mater, UC Berkeley, with the first Ph.D. degree awarded by the University's fledgling Computer Science department. He won the Association for Computing Machinery (ACM) Turing Prize in 1998 for his work in online transaction processing and was promoted to a Microsoft Distinguished Engineer in 2000. Jim was the founder of Microsoft's Bay Area Research Center (BARC) in San Francisco and subsequently became the manager of BARC's eScience group.

If you're a longtime reader of this blog, you've seen many references to Jim Gray and his TerraServer, SkyServer and other gargantuan database projects, as well as details of his early use of XML Web services to access terrestrial (TerraService.net) and galactic data. My "Build Real-Time Web Images" article in the August 2004 issue of Visual Studio Magazine described a Visual Basic .NET 2003 graphic consumer of TerraService.net Web Services, as well as Microsoft MapPoint services (requires free registration.) Jim was kind enough to review part of the article for accuracy.

Jim was a staunch proponent of large database systems constructed from interconnected low-cost commodity computers and cheap SATA disk drives called CyberBricks. I've covered CyberBrick technology in several magazine articles. My "Build Data Service Centers With Bricks" article for Fawcette Technical Publication's Windows Server System Magazine (May 2003 Tech*Ed issue) summarized Microsoft's tenuous commercial interest in CyberBrick technology.

Still Waiting for More News About the Private Search

Until we learn more, our thoughts and prayers go out to Jim, his family and his colleagues in San Francisco and around the world.

Update 5/2/2007: Mark Whitehorn has posted an eloquent "Tribute to Jim Gray" on the UK's The Register.