MSDN Blogs
from ideas to solutions
by DynGpTeam
2 Sep 2010 at 12:54pm
Presented by the Dynamics GP User Group (GPUG), the GPUG Summit is the annual conference for Dynamics GP users, GPUG Partners and key Dynamics GP personnel from Microsoft. It's coming up in October and I think you should consider attending.
It is October 25-28 with additional trainings before and after. https://www.regonline.com/gpug_summit_2010
Partner Connections is the 23rd and 24th focused on consultants. http://www.gppartnerconnections.com/.
Pam
by Matt Thalman
2 Sep 2010 at 12:50pm
Securing your Visual Studio LightSwitch application begins with defining permissions. In the first version of LightSwitch, developers are responsible for defining and consuming their own permissions. Once permissions have been defined, you can write the business logic to consume them where appropriate. As part of the development process, only permissions and their associated logic need to be defined. Once the application has been published, it can be administered to add users and roles and assign permissions.
The following is a walkthrough showing what is involved in this process:
To start with, I?ve created a simple Customer table:
In this application, I want to secure the ability to delete customers. To do this, I need to create a new permission. Permissions can be defined in the Access Control tab of the application properties. To get to the application properties you can double-click the Properties node in Solution Explorer or right-click the Application node and select Properties. Before permissions can be added, you need to enable authentication by choosing whether to use Windows or Forms authentication (see my previous blog post for more info on this). For my purpose, I?ve chosen to use Forms authentication. Now that authentication is enabled, I can add permissions. I?ve defined a permission named CanDeleteCustomer that provides the ability for users to delete customers:
With the permission defined, I can now write some code to secure the customer table. To do this, I navigate back to the Customer table and select ?Customers_CanDelete? from the Write Code drop-down button:
This opens up the code file for the customer table and generates the method stub where I can write my logic.
partial void Customers_CanDelete(
ref bool result)
{
result =
this.Application.User.HasPermission(
Permissions.CanDeleteCustomer);
}
A few things to note about this code:
The result parameter is assigned a value that indicates whether the user can delete the customer.
The User object is accessible from the Application class. The User object provides all sorts of information about the current user, most notably whether they have a specific permission.
The Permissions class is a generated class that contains the permissions that were defined in the Access Control tab.
This code runs on the server not the client. This ensures that the customer table will always be protected from a client that may not enforce this permission.
Ok, let?s test this to ensure it?s behaving as expected. First, I create an editable grid screen for the customer table. Now I?ll run the app and see how it looks.
Here you?ll see that the Delete button is disabled. By default, the currently running user doesn?t have the CanDeleteCustomer and the UI for the grid is reflecting this. So now you may be wondering how we can run as a user that does have that permission. To do that, I can just go back to the Access Control tab and check the ?Granted for debug? checkbox for the CanDeleteCustomer permission.
Now when I run the app, the Delete button is enabled.
In a subsequent blog post, I?ll describe how you assign permissions in a published application.
by Omar Aviles
2 Sep 2010 at 12:49pm
Si quieres desarrollar aplicaciones para Windows Phone 7 y ser de los primeros en mostrarla al mundo una aplicación en el lanzamiento a finales de este año, no te puedes perder este entrenamiento gratuito
Te enseñaremos a desarrollar aplicaciones o juegos y al terminas el curso podrás probar tu aplicación en un Windows Phone 7 en vivo no solo en el emulador.
Regístrate en el evento del mes que mejor se ajuste a tu agenda
Fecha
Sep 28 y 29 Regístrate aquí Oct 19 y 20 Regístrate aquí (Próximamente) Nov 23 y 24 Regístrate aquí (Próximamente)
No olvides traer tu maquina con el ambiente instalado: http://bit.ly/WP7MexTools
Tu ambiente de trabajo debe incluir: Visual Studio 2010 Express for Windows Phone Beta o Visual Studio 2010 RTM Windows Phone Emulator Beta Silverlight for Windows Phone Beta Microsoft Expression Blend for Windows Phone Beta XNA Game Studio 4.0 Beta
Donde:
UVM Campus Guadalajara Sur
Periférico Sur No. 8100 Santa María Tequepexpan
Tlaquepaque, Jal. CP. 45601
Horario: Cada día de 9:00 AM a 5:00 PM
Agenda Workshop:
Día 1: 9:00 -17:00
Windows Phone 7 introduction & design philosophy Lap Around Windows Phone 7 platform Lap Around the Developer and Designer tools Introduction to Silverlight Building Silverlight Applications for Windows Phone 7 Hardware Access (camera, accelerometer, touch, etc.)
Día 2: 9:00 -17:00
Push notifications Marketplace Introduction to XNA XNA and Windows Phone 7 Performance tips
:
by joneil
2 Sep 2010 at 12:46pm
This post is part of a series diving into the implementation of the @home With Windows Azure project, which formed the basis of a webcast series by Developer Evangelists Brian Hitney and Jim O?Neil. Be sure to read the introductory post for the context of this and subsequent articles in the series.
So where were we before vacations and back-to-school preparations got in my way!? Part 5 of this continuing series talked about the underlying REST protocol that?s used by the Azure Storage API, and in that discussion, we touched on two query classes:
DataServiceQuery, which will return at most 1000 entities, along with continuation tokens you can manage yourself to retrieve the additional entities fulfilling a given query, and CloudTableQuery, which includes an Execute method you call once to return all of the data with no further client calls necessary. Essentially, it handles the continuation tokens for you and makes multiple HTTP requests (each returning 1000 entities or less) as you materialize the query. Recall, you can use the AsTableServiceQuery extension method on a DataServiceQuery to turn it into a CloudTableQuery and access this additional functionality.
In the status.aspx page for Azure@home there are two collection displays ? a Repeater control (InProgress) showing the work units in progress and a GridView (GridViewCompleted) displaying the completed work units. The original code we?ve been looking at has the following line to retrieve all of the work units:
var workUnitList = ctx.WorkUnits.ToList<WorkUnit>();
We know now that code will result in at most 1000 entities being returned. Presuming Azure@home has been chugging away for a while, there may be more than 1000 completed work units, and as implemented now, we?ll never get them all. Additionally, since the table data is sorted by PartitionKey, and the partition key for the workunit table is the Azure WorkerRole instance ID, the entities you do get may change over time ? it?s not as if you?re guaranteed to get the first 1000 work units completed or the last 1000.
It?s simple enough to replace the line above with
var workUnitList = ctx.WorkUnits.AsTableServiceQuery().ToList<WorkUnit>();
and all of the data will be returned, all 10 or all 10,000 entities ? whatever happens to be in the table at the time. Obviously we need a middle-ground here: control over the pagination without bringing down massive amounts of data that the user may never look at.
While it?s conceivable you could have thousands of in-progress work units (each running in a worker role), that?s costly and beyond what you?ll be able to deploy as part of a typical Windows Azure account (20 roles is the default limit). To save some time and complexity then, I?m not going to worry about paginating the InProgress Repeater control.
You could though certainly accumulate a lot of completed work units, especially if you are leveraging an offer such as the Windows Azure One Month Pass. So for purposes of this discussion, the focus will be on a pagination scheme for the GridView displaying those completed work units..
As you might have expected by the existence of two similar classes (DataServiceQuery and CloudTableQuery), there are actually two mechanisms you can use to implement the pagination, one synchronous (via DataServicesQuery) and the other asynchronous (via CloudTableQuery). This post will focus on the former, and in the next post, we?ll transform that into an asynchronous implementation.
Some Refactoring
It was my goal minimally disrupt the other code in Azure@home and confine modifications solely to status.aspx. To accomplish that I had to do a bit of refactoring and introduce a utility class or two. The completed implementation of the changes to status.aspx (and the code-behind files) are attached to this blog post, so you should be able to replace the original implementation with this code and give it a whirl.
PageData class
When you?re implementing a paging scheme with Azure table storage, you?ve got a couple of choices in terms of how to handle repeat requests for the same page.
Cache the data once it?s retrieved. This approach minimizes the number of requests back to the service providing the data, but it also can make things more complex. You have to maintain the cache, and if the data is volatile, refresh the cache periodically so the data is not stale. Re-request the data for each pagination request. This approach requires maintaining a list of continuation tokens that correspond to each page of results ? kind of like guide words in a dictionary (the printed kind). Scott Densmore uses this approach in his blog post.
Since Scott?s example covers one of the options, I thought I?d go for the other, and just use the session state as my cache. The completed data is never going to change, so why pay for an additional Azure storage transaction to get the same data? Granted, I?m carrying around some baggage on the Web server now, and in a perfect world, I might use something like Velocity, but let?s leave that for a different day.
To encapsulate the data I need to store in session state, I created a class called PageData:
protected class PageData
{
public List<WorkUnit> InProgressList =
new List<WorkUnit>();
public List<CompletedWorkUnit> CompletedList =
new List<CompletedWorkUnit>();
public String PartitionKey =
null;
public String RowKey =
null;
public Boolean QueryResultsComplete =
false;
}
maintaining the following information:
InProgressList ? a list of work units being processed now by WebRole instances. WorkUnit is defined as a TableServiceEntity in AzureAtHomeEntities.cs.
CompleteList ? a list of work units already processed, but including only those that the user has paged through so far. CompletedWorkUnit is a new class defined in status.aspx and defined below.
PartitionKey and RowKey ? continuation tokens used in pagination
QueryResultsComplete ? a boolean indicating all results have been downloaded, that is, that user has explicitly paged through all of the data.
CompletedWorkUnit class
Notice above that CompletedList is a collection of a new class, CompletedWorkUnit. Why a new class? The original implementation materialized the entire workunit table into a workUnitList enumerable, and then used LINQ to Objects expressions to project the data source for both the Repeater and the GridView, as below:
GridViewCompleted.DataSource =
(from w
in workUnitList
where w.CompleteTime !=
null
let duration = (w.CompleteTime.Value.ToUniversalTime() -
w.StartTime.ToUniversalTime())
orderby w.StartTime descending
select
new
{
w.Name,
w.Tag,
w.StartTime,
Duration = String.Format(
"{0:#0} h {1:00} m",
duration.TotalHours, duration.Minutes)
});
Then the magic of data binding matched each property of the anonymous class in the LINQ projection to a BoundField in the GridView by name. That worked because all of the data was pulled down into memory and then manipulated with LINQ to Objects.
Now that we?re trying to query on an as-needed basis, we?re shifting from LINQ to Objects to the ADO.NET Data Services Client Library, and there are a couple of catches when addressing Azure table storage:
You can?t include a null in the query expression (so the where clause in the LINQ expression above is invalid), and
You can?t do a projection; you can only return the complete entity.
Didn?t Microsoft rename ADO.NET Data Services to WCF Data Services, so shouldn?t that be the WCF Data Services Client Library, not the ADO.NET Data Services Client Library?
The name change officially applies to the .NET Framework 4 version of the protocol, and since Azure@home was developed under .NET 3.5, we?re using the older terminology here. The constraints mentioned here also apply to .NET Framework 4 and the Open Data Protocol.
It?s the second issue above that necessitates a new class, since the Duration property is computed and not actually present in the workunit table. What we?ll see later on is that after the query results are retrieved, the WorkUnit instances are reshaped into CompletedWorkUnits (below), which reintroduces a Duration property and enables the data binding to GridViewCompleted to succeed.
protected class CompletedWorkUnit
{
public String Name { get; set; }
public String Tag { get; set; }
public DateTime StartTime { get; set; }
public DateTime? CompleteTime { get; set; }
public CompletedWorkUnit(WorkUnit wu)
{
this.Name = wu.Name;
this.Tag = wu.Tag;
this.StartTime = wu.StartTime;
this.CompleteTime = wu.CompleteTime;
}
public String Duration
{
get
{
if (
this.CompleteTime ==
null)
return "";
else
{
TimeSpan duration =
this.CompleteTime.Value.ToUniversalTime()
-
this.StartTime.ToUniversalTime();
return String.Format(
"{0:#0} h {1:00} m",
duration.TotalHours, duration.Minutes);
}
}
set
{
}
}
}
Embracing the ASP.NET Page Lifecycle
The original attempt at status.aspx was pretty straightforward, and everything was in Page_Load, but now we need to be a bit more cognizant of when things happen in processing the page:
Pagination has been added to the GridView, so we?ve got a PageIndexChanging event to consider.
We?ve got session state to maintain.
The Refresh button now requires logic to clear the session state and reinitiate the retrieval from Azure table storage.
To accommodate the changes, the implementation of Page_Load has been broken up across a few methods and events in the ASP.NET page life cycle, as depicted below.
The stages of the lifecycle relevant to this discussion are highlighted in blue; the two green boxes represent data retrieval logic refactored from the previous version of Page_Load; and the red box represents the session state, namely an instance of the PageData class that is initialized when the page is loaded and saved back to the session when the page is unloaded.
Let?s next walk through the code implementing each stage of the lifecycle.
Page_Load
1: protected void Page_Load(
object sender, EventArgs e)
2: {
3: if (Session[
"PageData"] !=
null)
4: pageData = Session[
"PageData"]
as PageData;
5: else
6: pageData =
new PageData();
7:
8: var cloudStorageAccount =
9: CloudStorageAccount.FromConfigurationSetting(
"DataConnectionString");
10: var cloudClient =
new CloudTableClient(
11: cloudStorageAccount.TableEndpoint.ToString(),
12: cloudStorageAccount.Credentials);
13:
14: // get client info
15: ctx =
new ClientDataContext(
16: cloudStorageAccount.TableEndpoint.ToString(),
17: cloudStorageAccount.Credentials);
18:
19: if (!
this.IsPostBack)
20: {
21: // get name of the user (maintained in ViewState)
22: var clientInfo = ctx.Clients.FirstOrDefault();
23: litName.Text = ctx.Clients.FirstOrDefault() !=
null ?
24: clientInfo.UserName :
"Hello, unidentifiable user";
25:
26: // ensure workunit table exists and retrieve data from Table Storage
27: cloudClient.CreateTableIfNotExist(
"workunit");
28: if (cloudClient.DoesTableExist(
"workunit"))
29: {
30: RetrieveInProgessUnits();
31: RetrieveCompletedUnits();
32: }
33: else
34: {
35: System.Diagnostics.Trace.TraceError(
"Unable to create 'workunit' table in Azure storage");
36: }
37: }
38: }
The retrieval of the session state occurs in Lines 3-6, followed by setting up the cloud storage account and context (Lines 8-17) just as in the original implementation.
New is the check for a postback (Liine 19). Only on the first rendering of the page will you retrieve the client name (from the client table), check that the workunit table exists, and populate the Repeater and GridView with some initial data. Subsequent interaction occurs via the pagination of the GridView or pressing the Refresh button, both of which incur postbacks and are handled in other code on the page.
What about? clientInfo on Line 22? It?s part of the !this.IsPostback branch, so don?t we lose that data when paging through results?
Yes and no. clientInfo is lost on a postback, but we only needed the grab the UserName field once. It?s assigned to litName on the first page view, and since litName has its ViewStateEnabled property set to true, the client name is retained across pagination and page refreshes. litName, by the way, is the only control on this page with ViewState enabled.
Clearly the two methods RetrieveInProgressUnits (Line 30) and RetrieveCompletedUnits (Line 31) are the most significant ones, and they?re up next!
RetrieveInProgressUnits
The original implementation of this functionality in status.aspx is below:
InProgress.DataSource =
(from w
in workUnitList
where w.CompleteTime ==
null
let duration = (DateTime.UtcNow - w.StartTime.ToUniversalTime())
orderby w.InstanceId ascending
select
new
{
w.Name,
w.Tag,
w.InstanceId,
w.Progress,
Duration = String.Format(
"{0:#0} h {1:00} m",
duration.TotalHours, duration.Minutes)
});
and it?s been simplified to:
private void RetrieveInProgessUnits()
{
pageData.InProgressList =
(from w
in ctx.WorkUnits
where w.Progress < 100
select w).ToList<WorkUnit>();
}
which incorporates three significant changes
The projection has been removed, since projections are not supported in the ADO.NET Data Services Client against Azure table storage. In fact, the projection isn?t even needed for the original implementation: the Duration property isn?t part of the user interface (a fact I didn?t realize until writing this blog post!)
The w.CompleteTime == null test isn?t supported in Azure table storage either, so in this case I opted for a nearly synonymous test of the Progress field.
Instead of assigning the result to the Repeater?s DataSource property directly, the data is retained in a session variable, pageData, with binding to occur later (in PreRenderComplete). We won?t be implementing pagination on this control, but I am still caching the data so that a subsequent postback doesn?t automatically re-retrieve the data. That means that as a user pages though the GridView, the in-progress work unit data shown in the Repeater will not be refreshed. You may or may not agree with that implementation, but it?s certainly easy enough to modify to your tastes. My take is that if a user paging through, he?s focused on the completed data, not the in-progress data, and there is always the handy ?Refresh this Page? button for him to use to get the latest and greatest in-progress (and completed) data.
RetrieveCompletedUnits
Ok, this is the meat of this article. The original code looked something like the InProgress.DataSource assignment above, where workUnitList was completely materialized on the client before the LINQ (to Objects) expression was constructed ? I hereby officially deem that ?old school?! Now the plan is to query the Azure table service to show the completed workunits on demand, a page at a time, where the size of the page is determined by the PageSize of the GridView.
Here?s how it works. Say there are 1200 entities in the Azure table, and the GridView is configured with a PageSize of 100. The first time the page is displayed, you?ll see 100 entities, and the pager control will indicate that a second page is available, but no more. Behind the scenes a query is made for 101 entities. Why 101? 100 to fill the page, and then one more to force the Page 2 indicator to appear in the pager control.
Continuation tokens are saved from the response header of that first retrieval, so when the second page is requested via the pager control, another query can be made for just the next 100 entities, and so on until the user pages through all of the results. If she doesn?t page through all the results, there?s no wasted bandwidth, and if she backs up to previous pages, they?re served immediately from the cache in the session state. Every entity in the table is retrieved once - and only once.
Here?s the code:
1: protected void RetrieveCompletedUnits()
2: {
3: if (!pageData.QueryResultsComplete)
4: {
5: // select enough rows to fill a page of the GridView
6: // GridView.PageSize < 1000 or UI paradigm will fail
7: Int32 maxRows = GridViewCompleted.PageSize;
8:
9: // add one if first page, to force a page 2 indicator
10: if (pageData.PartitionKey ==
null)
11: maxRows++;
12:
13: // set up query
14: var qry = (from w
in ctx.WorkUnits
15: where w.Progress == 100
16: select w).Take(maxRows)
as DataServiceQuery<WorkUnit>;
17:
18: // add continuation token (if there is one) to query
19: if (pageData.PartitionKey !=
null)
20: {
21: qry = qry.AddQueryOption(
"NextPartitionKey",
pageData.PartitionKey);
22: if (pageData.RowKey !=
null)
23: {
24: qry = qry.AddQueryOption(
"NextRowKey", pageData.RowKey);
25: }
26: }
27:
28: // execute the query
29: var response = qry.Execute()
as QueryOperationResponse<WorkUnit>;
30:
31: // grab continuation token from response
32: if (response.Headers.ContainsKey(
"x-ms-continuation-NextPartitionKey"))
33: {
34: pageData.PartitionKey =
response.Headers[
"x-ms-continuation-NextPartitionKey"];
35: if (response.Headers.ContainsKey(
"x-ms-continuation-NextRowKey"))
36: {
37: pageData.RowKey =
response.Headers["x-ms-continuation-NextRowKey"];
38: }
39: }
40:
41: // if no continuation token, reached end of table
42: else
43: {
44: pageData.PartitionKey =
null;
45: pageData.RowKey =
null;
46: pageData.QueryResultsComplete =
true;
47: }
48:
49: // add newly retrieved data to the current collection
50: pageData.CompletedList.AddRange(
51: from wu
in response.AsEnumerable<WorkUnit>()
52: select
new CompletedWorkUnit(wu)
53: );
54: }
55: }
Lines 5-11 set up how many entities (rows) to return, and Lines 14-16 set up (but do not execute) the query. Prior to that initial retrieval, there are no continuation tokens set (PartitionKey == null), so the condition in Line 19 fails, and the query is actually executed in Line 29.
As you might recall from my last post, the Take extension method (Line 16) adds a $top query option to the resulting HTTP request, so the GET URL that goes out looks like:
http://snowball.table.core.windows.net/workunit()?
$filter=Progress%20eq%20100&$top=26
$filter comes from the where clause in Line 15, and the value of $top comes from the fact that the GridView?s PageSize is 25 (then add 1 for the initial retrieval to force a Page 2 link in the GridView pager).
26 entities are retrieved (presuming there are that many), and the HTTP response headers look something like:
HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 2e75d787-ca85-4044-b0cc-0d6e462d910c
x-ms-version: 2009-09-19
x-ms-continuation-NextPartitionKey: 1!16!SW5zdGFuY2VfMDI2
x-ms-continuation-NextRowKey: 1!44!Q29tcGxldGVkX1VuaXR8MDI2fGRvd25sb2FkdGltZQ--
Date: Thu, 02 Sep 2010 01:09:37 GMT
Content-Length: 34290
Note the inclusion of two x-ms-continuation headers specifying how to pick up the next page of results. Lines 32-39 record these values in the session state (via pageData variable). If there are no continuation headers, it signifies the query results are complete, so the pageData continuation tokens are reset, and a boolean flag is flipped.
Finally, Lines 50-53 take the result of the query, materializes it, and projects it into a new collection of CompletedWorkUnits (thus pulling in the calculated Duration property needed for data binding to the GridView).
Executed next in the page lifecycle are control events. There are two control events of interest, clicking the Refresh button and paginating through the GridView. These events fire only on a postback, which means that the Page_Load processing merely sets up the cloud account and context variables; no retrieval of data occurs in Page_Load since that code is enclosed by a if (!this.IsPostback) condition (Line 19).
btnRefresh_Click
Clicking the Refresh button is tantamount to retrieving the page the first time, so the code here essentially replicates what happens in Page_Load the first time through (with a couple of lines to reset the session state and GridView page).
protected void btnRefresh_Click(
object sender, EventArgs e)
{
// refresh all the data on the page when button is clicked
pageData =
new PageData();
RetrieveInProgessUnits();
RetrieveCompletedUnits();
GridViewCompleted.PageIndex = 0;
}
GridViewCompleted_PageIndexChanging
This is my favorite part of the code, it just seems amazingly simple and elegant!
1: protected void GridViewCompleted_PageIndexChanging(
object sender,
System.Web.UI.WebControls.GridViewPageEventArgs e)
2: {
3: // if on the last 'current' page, retrieve another chunk
4: if (e.NewPageIndex == GridViewCompleted.PageCount - 1)
5: RetrieveCompletedUnits();
6:
7: // set requested page
8: GridViewCompleted.PageIndex = e.NewPageIndex;
9: }
Because of the way the page size was setup, as long as there are more entities to be retrieved, there will always be a next page link in the pager for the GridView. The initial retrieve will imply there are two pages, and when the link for page 2 is selected, this event goes into action.
Line 4 checks to see if the user clicked the page link for the last page of the current GridView. That could truly be the last of the data in the table, or it could just be the last page the user had requested thus far. Which of these is the case is determined by RetrieveCompletedUnits (Line 5).
If you revisit that code earlier in this post, you?ll note the first test is to see if the query results have been completely retrieved; if so, the method quickly returns. If not, the pageData variable must be holding on to a continuation token value (PartitionKey and RowKey), and the next query built will include these values as query options (via AddQueryOption). What gets issued via HTTP is something like:
http://snowball.table.core.windows.net/workunit()?
$filter=Progress%20eq%20100&$top=25&
NextPartitionKey=1!16!SW5zdGFuY2VfMDI2&
NextRowKey=1!44!Q29tcGxldGVkX1VuaXR8MDI2fGRvd25sb2FkdGltZQ--
If the requested page index isn?t the last one (the condition in Line 4 evaluates to false), the results are already in the session variable (pageData.CompletedList), and the regular data binding semantics and GridView mechanisms will take care of displaying the right data.
Page_PreRenderComplete
Speaking of data binding, that?s where this event comes in. After all of the control events have fired and the appropriate data has been gathered in the pageData variable, binding the data to the Repeater and GridView controls (and updating a few literals for aesthetics) is a simple affair:
protected void Page_PreRenderComplete()
{
// bind session data to the Repeater control
InProgress.DataSource = pageData.InProgressList;
InProgress.DataBind();
// bind session data to the GridView control
GridViewCompleted.DataSource = pageData.CompletedList;
GridViewCompleted.DataBind();
// update the labels
litNoProgress.Visible = InProgress.Items.Count == 0;
litCompletedTitle.Visible = GridViewCompleted.Rows.Count > 0;
}
Why did? I choose to implement the data binding in the PreRenderComplete event instead of LoadComplete? Well, originally I actually had put it in LoadComplete, and that will work fine here, but I?m setting us up for the next post where we?ll talk about asynchronously handling the paging, and there PreRenderComplete figures prominently.
Page_Unload
protected void Page_Unload()
{
Session[
"PageData"] = pageData;
}
See, there is such a thing as self-documenting code!
Tidying Up Loose Ends
There are still a couple of gotchas to be aware of in the implementation above that robust production code would want to accommodate.
First, there?s little in the way of exception handling ? but that?s par for the course with blog posts like these!
The biggest loose end though is that my pagination logic in RetrieveCompletedUnits disregards a pernicious scenario. In the fine print of the MSDN topic on Query Timeout and Pagination it?s mentioned that
It is possible for a query to return no results but to still return a continuation header.
What the heck does that mean?! Steve Marx has dedicated a post to it, and while some of the references in the post are obsolete (e.g., ExecuteAll and ExecuteAllWithRetries are no longer part of the StorageClient API), the situation described can still occur, and my code above is far from accommodating! I have not run into such a scenario during my testing, but my application usage is light, and it?s quite likely my data partitions are collocated at the same storage node. (In Azure@home, the partitions are defined by the CurrentRoleInstance.Id of the Worker role recording the data).
How might I fix the code? In Line 29 of RetrieveCompletedUnits the fatal flaw is the assumption that I?m getting a complete set of data back from the query. That is, I?m getting maxRows or whatever?s left at the tail end of the result set. What could happen though, is that I get, say, six entities or even none, but also a continuation token. If I really want to make sure I?m getting enough entities to fill a GridView page, I need to beef up my implementation, something like the following, in which I?ve highlighted the modifications to the code discussed earlier in the post.
protected void RetrieveCompletedUnits()
{
if (!pageData.QueryResultsComplete)
{
// select enough rows to fill a page of the GridView
// GridView.PageSize < 1000 or UI paradigm will fail
Int32 maxRows = GridViewCompleted.PageSize;
// add one if first page, to force a page 2 indicator
if (pageData.PartitionKey ==
null)
maxRows++;
do
{
// set up query
var qry = (from w
in ctx.WorkUnits
where w.Progress == 100
select w).Take(maxRows)
as DataServiceQuery<WorkUnit>;
// add continuation token (if there is one) to query
if (pageData.PartitionKey !=
null)
{
qry = qry.AddQueryOption(
"NextPartitionKey", pageData.PartitionKey);
if (pageData.RowKey !=
null)
{
qry = qry.AddQueryOption(
"NextRowKey", pageData.RowKey);
}
}
// execute the query
var response = qry.Execute()
as QueryOperationResponse<WorkUnit>;
// grab continuation token from response
if (response.Headers.ContainsKey(
"x-ms-continuation-NextPartitionKey"))
{
pageData.PartitionKey = response.Headers[
"x-ms-continuation-NextPartitionKey"];
if (response.Headers.ContainsKey(
"x-ms-continuation-NextRowKey"))
{
pageData.RowKey = response.Headers[
"x-ms-continuation-NextRowKey"];
}
}
// if no continuation token, reached end of table
else
{
pageData.PartitionKey =
null;
pageData.RowKey =
null;
pageData.QueryResultsComplete =
true;
}
// add newly retrieved data to the current collection
var returnedRows = response.ToList<WorkUnit>();
pageData.CompletedList.AddRange(
from wu
in returnedRows
select
new CompletedWorkUnit(wu)
);
// still need rows to fill page size
maxRows -= returnedRows.Count;
}
while ((!pageData.QueryResultsComplete) && (maxRows > 0));
}
}
Presuming the code above is correct? it wasn?t too hard to address the scenario that Steve described. Instead of assuming I?ll get data back on each call to Execute, I built a loop around the retrieval logic so RetrieveCompletedUnits doesn?t return until it?s either got a full page of data or there?s no data left ? regardless of how many actual REST requests are issued in the process.
Granted this post was a tad lengthy (when are mine not!), but hopefully it was straightforward enough for you to understand the pagination mechanism and how to code for it in your own applications. The devil?s always in the details, but the primary takeaway here is
When using DataServiceQuery, always expect continuation tokens unless you?re specifying both PartitionKey and RowKey in the query (that is, you?re selecting a single entity.
In the next post, I?ll take this code and transform it using a later addition to the StorageClient API (CloudTableQuery) that will handle much of the continuation token rigmarole for you.
by Omar Aviles
2 Sep 2010 at 12:46pm
Si quieres desarrollar aplicaciones para Windows Phone 7 y ser de los primeros en mostrarla al mundo una aplicación en el lanzamiento a finales de este año, no te puedes perder este entrenamiento gratuito
Te enseñaremos a desarrollar aplicaciones o juegos y al terminas el curso podrás probar tu aplicación en un Windows Phone 7 en vivo no solo en el emulador.
Regístrate en el evento del mes que mejor se ajuste a tu agenda
Fecha
Sep 28 y 29 Regístrate aquí Oct 19 y 20 Regístrate aquí (Próximamente) Nov 23 y 24 Regístrate aquí (Próximamente)
No olvides traer tu maquina con el ambiente instalado: http://bit.ly/WP7MexTools
Tu ambiente de trabajo debe incluir: Visual Studio 2010 Express for Windows Phone Beta o Visual Studio 2010 RTM Windows Phone Emulator Beta Silverlight for Windows Phone Beta Microsoft Expression Blend for Windows Phone Beta XNA Game Studio 4.0 Beta
Donde:
Tecnológico de Monterrey Campus Monterrey
Av. Eugenio Garza Sada 2501 sur Col Tecnológico
Monterrey Nuevo León Mexico
Horario: Cada día de 9:00 AM a 5:00 PM
Agenda Workshop:
Día 1: 9:00 -17:00
Windows Phone 7 introduction & design philosophy Lap Around Windows Phone 7 platform Lap Around the Developer and Designer tools Introduction to Silverlight Building Silverlight Applications for Windows Phone 7 Hardware Access (camera, accelerometer, touch, etc.)
Día 2: 9:00 -17:00
Push notifications Marketplace Introduction to XNA XNA and Windows Phone 7 Performance tips
:
by Omar Aviles
2 Sep 2010 at 12:43pm
Si quieres desarrollar aplicaciones para Windows Phone 7 y ser de los primeros en mostrarla al mundo una aplicación en el lanzamiento a finales de este año, no te puedes perder este entrenamiento gratuito
Te enseñaremos a desarrollar aplicaciones o juegos y al terminas el curso podrás probar tu aplicación en un Windows Phone 7 en vivo no solo en el emulador.
Regístrate en el evento del mes que mejor se ajuste a tu agenda
Fecha
Sep 21 y 22 Regístrate aquí Oct 19 y 20 Regístrate aquí (Próximamente) Nov 23 y 24 Regístrate aquí (Próximamente)
No olvides traer tu maquina con el ambiente instalado: http://bit.ly/WP7MexTools
Tu ambiente de trabajo debe incluir: Visual Studio 2010 Express for Windows Phone Beta o Visual Studio 2010 RTM Windows Phone Emulator Beta Silverlight for Windows Phone Beta Microsoft Expression Blend for Windows Phone Beta XNA Game Studio 4.0 Beta
Donde:
Tecnológico de Monterrey Campus Monterrey
Av. Eugenio Garza Sada 2501 sur Col Tecnológico
Monterrey Nuevo León Mexico
Horario: Cada día de 9:00 AM a 5:00 PM
Agenda Workshop:
Día 1: 9:00 -17:00
Windows Phone 7 introduction & design philosophy Lap Around Windows Phone 7 platform Lap Around the Developer and Designer tools Introduction to Silverlight Building Silverlight Applications for Windows Phone 7 Hardware Access (camera, accelerometer, touch, etc.)
Día 2: 9:00 -17:00
Push notifications Marketplace Introduction to XNA XNA and Windows Phone 7 Performance tips
:
by anilkr
2 Sep 2010 at 12:39pm
Team Foundation Server 2010 build service can now be customized using .NET v4.0 workflow activities. I was recently working on a requirement to generate an email after the successful build which provides basic information about the contents of the build. Here are some basic requirements for the activity.
Send Email after the compilation and test runs
Include list of changesets in the email
Include list of associated work items with a changeset
Include list of associated files with a changeset
Here is a quick snapshot of the email that needs to be generated automatically.
Most of this information can be obtained from Team Foundation Server Object Model class Changeset. It gives you access to the associated work items and files (changes) which are part of the changeset. The default process template in VS includes a step to associate changesets and work items to the build, this step will give us access to the associated changesets. First step in the process is to create an activity class that inherits from System.Activities.CodeActivity. Second step is to add references to Microsoft.TeamFoundation.Build.Client.dll, Microsoft.TeamFoundation.Client.dll, Microsoft.TeamFoundation.VersionControl.Client.dll and Microsoft.TeamFoundation.WorkItemTracking.Client.dll. Most of these references will allow you to access, the build details, changeset information and work item information which can be found in C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.0\ folder. You also should add reference to Microsoft Anti-XSS library to protect from XSS vulnerabilities. So here is how the class will look like so far.
using System.ComponentModel;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Activities;
using System.Net.Mail;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.VersionControl.Client;
using Microsoft.TeamFoundation.WorkItemTracking.Client;
using Microsoft.Security.Application;
namespace ISTBuildActivityLibrary
{
[
BuildActivity(
HostEnvironmentOption.Agent)]
public sealed class SendEmailActivity :
CodeActivity
{
#region Activity Execution Logic
protected override void Execute(
CodeActivityContext context)
{
}
}
}
The next step is to define the properties that you need for the activity, these will help you retrieve the changeset details, smtp server and mail target information.
#region Message Properties
[
BrowsableAttribute(
true)]
[
CategoryAttribute(MessagePropertiesCategory)]
public InArgument<
IBuildDetail> BuildDetail {
get;
set; }
[
BrowsableAttribute(
true)]
[
CategoryAttribute(MessagePropertiesCategory)]
public InArgument<
IList<
Changeset>> BuildAssociatedChangesets {
get;
set; }
[
BrowsableAttribute(
true)]
[
CategoryAttribute(MessagePropertiesCategory)]
public InArgument<
string> SmtpServer {
get;
set; }
[
BrowsableAttribute(
true)]
[
CategoryAttribute(MessagePropertiesCategory)]
public InArgument<
string> Subject {
get;
set; }
[
BrowsableAttribute(
true)]
[
CategoryAttribute(MessagePropertiesCategory)]
public InArgument<
string> MailFrom {
get;
set; }
[
BrowsableAttribute(
true)]
[
CategoryAttribute(MessagePropertiesCategory)]
public InArgument<
string> Mailto {
get;
set; }
#endregion
Please note the BuildDetail property when set in the process workflow gives you access to the build information such as build number, drop location, start time, team project, requested user, label name, and more which can be part of the email to provide the summary of the build. BuildAssociatedChangesets provides access to the changesets associated with the build, the default workflow process includes variables which store this information, thus the activity does not have to get it again from the TFS server. The rest of properties provide information such as SMTP Server, Subject, Mail From and Mail To which are self explanatory. Enough with the properties lets look at the activity execution code to see how the mail is constructed.
protected override voidExecute(CodeActivityContext context)
{
try
{
IBuildDetail buildInformation = context.GetValue(this.BuildDetail);
StringBuilder sbMailHtml = newStringBuilder();
sbMailHtml.Append(Resources.EmailHtml);
IList<Changeset> associatedChangesets = context.GetValue(this.BuildAssociatedChangesets);
StringBuilder changesetHtml = newStringBuilder();
foreach (Changeset changeset inassociatedChangesets)
{
changesetHtml.AppendLine("<table align=\"center\" cellpadding=\"4\" cellspacing=\"0\" class=\"style1\" style=\'border:solid #7BA0CD 1.0pt\'>');
changesetHtml.AppendLine("<tr><td bgcolor=\"#4F81BD\" class=\"style2\" colspan=\"2\">");
changesetHtml.AppendLine("<a target=\"_new\" href=\"http://server:8080/tfs/web/UI/Pages/Scc/ViewChangeset.aspx?cs="+ changeset.ChangesetId.ToString() + "\"><font color=\"white\">Changeset #"+ changeset.ChangesetId.ToString() + "</font></a>");
changesetHtml.AppendLine("<tr><td colspan=\"2\">Changeset checked in by <b>"+ AntiXss.HtmlEncode(changeset.Committer) + "</b> with comments:<br /><i>"+ AntiXss.HtmlEncode(changeset.Comment) + "</i></td></tr>");
changesetHtml.AppendLine("<tr><td colspan=\"2\">");
IList<WorkItem> workitems = changeset.WorkItems;
changesetHtml.AppendLine("<table align=\"center\" cellpadding=\"4\" cellspacing=\"0\" class=\"style1\" style=\'border:solid #7BA0CD 1.0pt\'><tr><td bgcolor=\'#4F81BD\' class=\'style3\' colspan=\'5\'>Associated Work Items: '+ workitems.Count.ToString() + "</td></tr>");
changesetHtml.AppendLine("<tr><td>Type</td><td>Id</td><td>Title</td><td>State</td><td>Reason</td></tr>");
foreach (WorkItem workitem in workitems)
{
changesetHtml.AppendLine("<tr><td>" + workitem.Type.Name + "</td>");
changesetHtml.AppendLine("<td><a target=\"_new\" href=\"http://server:8080/tfs/web/UI/Pages/WorkItems/WorkItemEdit.aspx?id=" + workitem.Id + "\">" + workitem.Id + "</a></td>");
changesetHtml.AppendLine("<td>" + AntiXss.HtmlEncode(workitem.Title) + "</td>");
changesetHtml.AppendLine("<td>" + workitem.State + "</td>");
changesetHtml.AppendLine("<td>" + workitem.Reason + "</td></tr>");
}
changesetHtml.AppendLine("</table></td></tr>");
changesetHtml.AppendLine("<tr><td colspan=\"2\"> </td></tr>");
List<Change> files = GetFilesAssociatedWithBuild(changeset.VersionControlServer, changeset.ChangesetId);
changesetHtml.AppendLine("<tr><td colspan=\"2\"><table align=\"center\" cellpadding=\"4\" cellspacing=\"0\" class=\"style1\" style=\'border:solid #7BA0CD 1.0pt\'><tr><td bgcolor=\'#4F81BD\' class=\'style3\' colspan=\'2\'>Associated Files: ' + files.Count + "</td></tr>");
foreach (Change file in files)
{
changesetHtml.AppendLine("<tr><td>" + AntiXss.HtmlEncode(file.Item.ServerItem) + "</td><td>" + file.ChangeType.ToString() + "</td></tr>");
}
changesetHtml.AppendLine("</table></td></tr>");
changesetHtml.AppendLine("</table>");
changesetHtml.AppendLine("<br />");
}
sbMailHtml.Replace("<@ChangesetsHtml>", changesetHtml.ToString());
sbMailHtml.Replace("<@DateTime>", buildInformation.FinishTime.ToString());
sbMailHtml.Replace("<@BuildNumber>", buildInformation.BuildNumber);
sbMailHtml.Replace("<@DropPath>", buildInformation.DropLocation);
SmtpClient objSmtp = new SmtpClient(this.SmtpServer.Get(context));
objSmtp.UseDefaultCredentials = true;
MailMessage objMsg = new MailMessage();
objMsg.From = new MailAddress(this.MailFrom.Get(context));
objMsg.To.Add(this.Mailto.Get(context));
objMsg.Subject = this.Subject.Get(context);
objMsg.IsBodyHtml = true;
objMsg.Body = sbMailHtml.ToString();
objSmtp.Send(objMsg);
sbMailHtml.Clear();
sbMailHtml = null;
}
catch
{
throw;
}
}
private static List<Change> GetFilesAssociatedWithBuild(VersionControlServer versionControlServer, int changesetId)
{
List<Change> files = new List<Change>();
Changeset changeset = versionControlServer.GetChangeset(changesetId);
if (changeset.Changes != null)
{
foreach (Change changedItem in changeset.Changes)
{
files.Add(changedItem);
}
}
changeset = null;
versionControlServer = null;
return files;
}
GetFilesAssociatedWIthBuild returns the list of files that have changed for a specific changeset. Execute is the main method that constructs the HTML from the changeset data and it encodes the string data that is coming from various sources to mitigate any XSS issues. Attached to this post is the full source code of the activity. In the next post I will outline the way to test and integrate the activity in to your build definition.
Thanks
Anil RV
by PhilPenn
2 Sep 2010 at 12:32pm
The Microsoft Patterns & Practices team has announced completion of their latest publication subtitled Design Patterns for Decomposition and Coordination of Multicore Architectures. Draft chapters of this book have been available via CodePlex during the duration of the project. There are now several available channels for printed and eBook acquisition.
Where to get the book
The printed book is available for pre-order from O?Reilly at Parallel Programming with Microsoft® .NET
The eBook is also available for download (today) from O?Reilly and Safari Books Online. Expect to see it at your favorite book store real soon!
The content is also available on MSDN Library: Parallel Programming with Microsoft .NET.
Where to get the code samples
Accompanying the book are code samples for each chapter. This includes small code samples showing how to use each feature of the Task Parallel Library and a larger example for each chapter setting the pattern in a larger context.
You can download code samples from the Parallel Patterns CodePlex site. There are versions of the samples for C#, Visual Basic and now F#. You can also download answers to the questions at the end of each chapter from the CodePlex site.
by DanKasun
2 Sep 2010 at 12:27pm
Been a while since I've been able to post... thanks to a glut of internal planning and meetings... my head may be getting above water now, I hope. I had the chance to respond to a post up on GovLoop ( http://www.govloop.com/profiles/blogs/gov-20-is-open-source ) - and wanted to cross post here... I know there are a lot of strong feelings on this topic, and it's more than worthy of debate: Sharing government-developed solutions, particularly solutions that have repeatable value throughout many agencies...(read more)
by Heather O'Cull
2 Sep 2010 at 12:20pm
One of the more common questions I get around Visual Reports is “I selected a field (% complete, duration, some text custom field, etc.) to add to my report and it doesn’t show up in Excel – why isn’t it there?”. The field probably is there, it just is in a different spot.
Visual Reports are built off a data structure called a cube. Cubes have 3 kinds of data types – dimensions, measures, and properties.
Dimensions are anything you can pivot the data on –ex. tasks, resources, time dimensions, any custom field with a lookup table.
Measures are anything the rollup can be calculated on – ex. work, cost, actual work.
Properties are everything else, they are just associated with tasks or resources and provide supporting information – ex. % complete, duration, text fields. Percent complete is a good example of a property since it is a number so it seems like it could be rolled up but unfortunately two 50% complete tasks do not equal a 100% complete summary.
If you can’t find your field, it is probably a property. To add those to a pivotTable in Excel you can’t go through the PivotTable field list that you are used to. You have to first add the resources or task dimension to the report. Then right-click a resource/task, select “Show Properties in Report”, and select your field.
That will give you this:
So to recap:
Note that you have to be using Excel 2007 or later to display properties. For more information on Visual Reports, check out this help article.
by Faisal Jamil
2 Sep 2010 at 12:02pm
Hi,
In the latest release for MSDN Search, we have retired the Search Code funtionality. We currently have a number of solutions including Library Code Examples and Samples, Galleries and Codeplex which allows our users to search and discover code. We are continuing to invest on a richer search experience as proven by the enhancements we have made to search results with added metadata such as ratings, downloads, clicks, etc. Here are some examples:
http://social.msdn.microsoft.com/Search/en-US?query=openfile&refinement=117
http://social.msdn.microsoft.com/Search/en-US?query=xaml&refinement=456
http://social.msdn.microsoft.com/Search/en-US/?Refinement=203&Query=xaml
Stay tuned for further improvements in MSDN Search in the coming year.
Thank you.
Regards,
Faisal Jamil
MSDN/TechNet Search PM
by Wayne Walter Berry
2 Sep 2010 at 11:58am
SQL Azure doesn?t currently support page level or row level compression like the enterprise edition of SQL Server 2008. However, you can implement your own column level compression in your data access layer to take advantage of the performance and cost savings of compression. I will discuss how to do this and provide some example code in this blog post.
Column level compression is the concept of compressing your data before you write it to the database, and decompressing it when you it from the database. For example, instead of having varchar(max) column of text you have a varbinary(max) of compressed text that is holding on average 80% less data.
The Right Scenario
Only in certain scenarios does column level compression work well. When compressing columns consider that:
Large text columns are the best to compress; they gain you the most savings. The gain from compression must exceed the costs of creating the compression dictionary (a result of using deflate compression); this only happens when there is a large amount of data that repeats itself. This technique will benefit not only text but large xml and binary data as well depending on the content. For example, you don?t want to compress images blobs ? they are usually already compressed. Don?t compress columns that appear in the WHERE clause of your queries. With this technique you can?t query on the compressed text without decompressing it. You also won?t be able to query or access these fields through SSMS or load data directly using BCP.exe Compress columns that can be cached on the application side, to avoid multiple reads; this avoids the costs of decompressing the text.
For example scenario that would work well is a web based product catalogues where you compress the product description and where you don?t need to search within the description and it doesn?t change very often.
The Benefits
Compression can reduce the amount of data you are storing, creating potential cost savings. It can also potentially help you stay below the maximum 50 Gigabyte database size for SQL Azure and avoid the development costs of partitioning.
In certain scenarios compression can result in speed improvements for queries that are preforming full table scans on the clustered index. When dealing with large-value data types, if the data in the column is less than 8,000 bytes it will be stored in the page with the rest of the column data. If you can reduce the 8000 bytes, more rows can be paged at one time, giving you performance gains on full table scans of the table.
Table Modification
Compressed data should be stored in varbinary(max) columns. If you are compressing a nvarchar(max) column, you will need to create an additional column to compress you data into. You can do this with an ALTER TABLE command. Once you have the text compressed, you delete the nvarchar(max) column. Here is a little example Transact-SQL that adds a column to SQL Azure:
ALTER TABLE Images
ADD PageUriCompressed varbinary(
max)
NOT NULL DEFAULT(0x0)
The Data Layer
Fortunately, .NET CLR 2.0 has some great compression built into the System.IO.Compression namespace with the GZipStream class. The first thing I need to do is create a throw-away console application that I will use once to compress all the existing rows into the new column, here is what it looks like:
do
{
using (SqlConnection sqlConnection =
new SqlConnection(
ConfigurationManager.ConnectionStrings[
"SQLAzure"].ConnectionString))
{
String pageUri;
Int64 Id;
// Open the connection
sqlConnection.Open();
// Pull One Row At A Time To Prevent Long Running
// Transactions
SqlCommand sqlCommand =
new SqlCommand(
"SELECT TOP 1 ID, PageUri FROM [Images] WHERE PageUriCompressed = 0x0",
sqlConnection);
using (SqlDataReader sqlDataReader = sqlCommand.ExecuteReader())
{
// WWB: Exit Do Loop When There Is No More Rows
if (!sqlDataReader.Read())
break;
pageUri = (String)sqlDataReader[
"PageUri"];
Id = (Int64)sqlDataReader[
"ID"];
}
Console.Write(
".");
// Compress Into the Memory Stream
using (MemoryStream memoryStream =
new MemoryStream())
{
using (GZipStream gzipStream =
new GZipStream(memoryStream,
CompressionMode.Compress,
true))
{
// Unicode == nvarchar
Byte[] encodedPageUri = Encoding.Unicode.GetBytes(pageUri);
gzipStream.Write(encodedPageUri, 0, encodedPageUri.Length);
}
// Now Everything is compressed into the memoryStream
// Reset to Zero Because We Are Going To Read It
memoryStream.Position = 0;
// WWB: Stream for Writing
using (SqlStream sqlStream =
new SqlStream(sqlConnection,
"dbo",
"Images",
"PageUriCompressed",
"ID",
SqlDbType.BigInt, Id))
{
using (BinaryReader binaryReader =
new BinaryReader(memoryStream))
{
using (BinaryWriter binaryWriter =
new BinaryWriter(sqlStream))
{
Int32 read;
Byte[] buffer =
new Byte[1024];
do
{
read = binaryReader.Read(buffer, 0, 1024);
if (read > 0)
binaryWriter.Write(buffer, 0, read);
}
while (read > 0);
}
}
}
}
}
}
while (
true);
Console.WriteLine(
"");
This code uses the SqlStream class that was introduced in this blog post.It also tries to make good use of the local memory and not consume too much if the string being compressed is really big. However, this results in a very ?chatty? application that creates a lot of connections to SQL Azure and runs slower than I would like.
Evaluation
My next step is to evaluate if the compressed really helped me. I do this because it can be hard to evaluate what the benefits compression will have until you have compressed your real world data. To do this I use the DATALENGTH field in Transact-SQL to sum up the two columns, i.e. before and after compression. My query looks like this:
SELECT COUNT(1),
SUM(DATALENGTH(PageUri)),
SUM(DATALENGTH(PageUriCompressed))
FROM Images
The results look like this:
I can see that compressed is actually going to reduce the size of my database in this case. In some scenarios compression makes the data bigger, usually when the data dictionary exceeds the gains from compression. As a rule of thumb, to have effective compression on text you need to have multiple repeating phrases, which happen with longer text blocks
Code
Now that the column is compressed you need to be able to read the compressed data and write uncompressed data to the column compressed. Here is a little bit of example code to help you do that:
protected static String Read(Int64 id)
{
using (SqlConnection sqlConnection =
new SqlConnection(
ConfigurationManager.ConnectionStrings[
"SQLAzure"].ConnectionString))
{
sqlConnection.Open();
SqlCommand sqlCommand =
new SqlCommand(
"SELECT PageUriCompressed FROM [Images] WHERE ID = @Id",
sqlConnection);
sqlCommand.Parameters.AddWithValue(
"@Id", id);
using (SqlDataReader sqlDataReader = sqlCommand.ExecuteReader())
{
sqlDataReader.Read();
Byte[] compressedPageUri =
(Byte[])sqlDataReader[
"PageUriCompressed"];
using (MemoryStream memoryStream =
new MemoryStream(compressedPageUri))
{
using (GZipStream gzipStream =
new GZipStream(memoryStream,
CompressionMode.Decompress))
{
using (StreamReader streamReader =
new StreamReader(gzipStream, Encoding.Unicode))
{
return (streamReader.ReadToEnd());
}
}
}
}
}
}
For writing:
protected
static void
Write(Int64 id, String pageUri)
{
using (SqlConnection sqlConnection =
new SqlConnection(
ConfigurationManager.ConnectionStrings["SQLAzure"].ConnectionString))
{
//
Open the
connection
sqlConnection.
Open();
// Compress
Into the Memory Stream
using (MemoryStream memoryStream =
new MemoryStream())
{
using (GZipStream gzipStream =
new GZipStream(memoryStream,
CompressionMode.Compress,
true))
{
// Unicode == nvarchar
Byte[] encodedPageUri = Encoding.Unicode.GetBytes(pageUri);
gzipStream.
Write(encodedPageUri, 0, encodedPageUri.Length);
}
// Now Everything
is compressed
into the memoryStream
// Reset
to Zero Because We
Are Going
To Read It
memoryStream.
Position = 0;
// WWB: Stream
for Writing
using (SqlStream sqlStream =
new SqlStream(sqlConnection,
"dbo",
"Images",
"PageUriCompressed",
"ID",
SqlDbType.BigInt, id))
{
using (BinaryReader binaryReader =
new BinaryReader(memoryStream))
{
using (BinaryWriter binaryWriter =
new BinaryWriter(sqlStream))
{
Int32
read;
Byte[] buffer =
new Byte[1024];
do
{
read = binaryReader.
Read(buffer, 0, 1024);
if (
read > 0)
binaryWriter.
Write(buffer, 0,
read);
}
while (
read > 0);
}
}
}
}
}
}
SQL Compression
For a comparison, the compression built into an on-premise SQL Server is transparent to the application and benefits much wider range of types like decimal and bigint all the way to larger types because it scopes to the row to the page. In other words it can use the repetition dictionary in compression across all the columns in the page. You can read more about SQL Server 2008 compression here.
Summary
Do you have a better way to accomplish the same thing? Post it in the comments below. Do you have questions, concerns, comments? Post them below and we will try to address them.
by MSMaroc
2 Sep 2010 at 11:58am
L’environnement .Net et les outils
Voici un article fort intéressant (en français !!!) pour tous ceux qui souhaiteraient s’initier au développement .Net.
Cela commence par une brève description de l’environnement .Net - le framework – ainsi que des outils utiles pour développer avec ce framework. Ceux qui n’ont pas envie de lire peuvent aussi regarder cette vidéo : vous y trouverez des informations encore plus complètes sur le framework.
Suite à cette introduction, certains d’entre vous auront peut être envie de se frotter à Visual Studio… Rien de plus simple, l’article vous propose deux “cursus”.
Développement Web :
Les bases de la technologie ASP.NET (Introduction à ASP.NET, Concepts fondamentaux ASP.NET 2.0, Découverte des principaux langages de programmation)
Découverte des outils de développement Web (Outils indispensables et procédures d’installation, Visual Web Developer - Présentation des fonctionnalités, Présentation de Visual Web Developper 2008 Express Edition, Expression Web en 6 webcasts)
Vos premières lignes de code (Introduction au développement d’applications Web avec ASP.NET 2.0, Introduction à XHTML / CSS, Les ateliers du coach ASP .NET)
Développement Windows :
L’environnement de développement (Présentation de Windows en tant que plateforme, les atouts du framework .NET, Tour d’horizon de la plateforme .NET, .Net, un framework multi-langages, … et multiplateformes !)
Les outils de développement (Présentation de Visual Studio, Pourquoi utiliser Visual Studio ? Présentation de Visual Basic 2008 Express, Présentation de Visual C# 2008 Express, Présentation de Visual C++ 2008 Express)
Les bases de votre langage de prédilection (Introduction au langage de programmation Visual Basic, Introduction au langage C# et au .NET Framework, Introduction au langage de programmation Visual C++)
Vos premières lignes de code avec
Le coach Visual Basic 2008
Le coach Visual C#
Le coach Visual C++
by The StreamInsight Team
2 Sep 2010 at 11:57am
We have added a new sample solution to our samples package on CodePlex. This sample uses the StreamInsight extensibility framework to implement a pattern detection application. The solution can be found in Applications\PatternDetector contains - apart from the used adapters - the following projects:
AugmentedFiniteAutomaton (refers to a project in UserExtensions), which implements a pattern matching user-defined operator.
PatternDetector, which uses the user-defined operator to look for a V-pattern in a stock ticker stream
Our pattern matching user-defined operator is based on an augmented finite automaton (AFA) - which is basically a non-deterministic finite automaton (NFA) with additional information, called a register, that can be created and maintained as part of the automaton during runtime.
The PatternDetector example uses a sample input stream of stock prices as point events (read from a file with the SimpleTextFileReader input adapter), and defines an AFA that detects a sequence of downticks followed by an equal number of upticks. We detect the pattern over a tumbling window of one hour. The corresponding AFA is shown in the file AFAexample.pptx (included in the solution), and is implemented in AFA_EqualDownticksUpticks.cs. The output is simply dumped on the console.
For more information on pattern matching using AFA, see:
Badrish Chandramouli, Jonathan Goldstein, and David Maier. High-Performance Dynamic Pattern Matching over Disordered Streams. In VLDB 2010.
Regards,
Badrish
by Jason Howell
2 Sep 2010 at 11:43am
Today we got this error when using 32-bit Excel and the PowerPivot addin to connect to Oracle Express.
The full stack of the error appears when you expand the [Details >> ] button
Error Message:
============================
ORA-06413: Connection not open.
----------------------------
Failed to connect to the server. Reason: ORA-06413: Connection not open.
============================
Call Stack:
============================
at System.Data.OleDb.OleDbConnectionInternal..ctor(OleDbConnectionString constr, OleDbConnection connection)
at System.Data.OleDb.OleDbConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup)
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.OleDb.OleDbConnection.Open()
at Microsoft.AnalysisServices.Modeler.Storage.RelationalDataSourceConnection.InitializeConnectionObject(String connectionIdentifier)
----------------------------
at Microsoft.AnalysisServices.Modeler.Storage.RelationalDataSourceConnection.InitializeConnectionObject(String connectionIdentifier)
at Microsoft.AnalysisServices.Modeler.Storage.RelationalDataSourceConnection.Open(String& connectionIdentifier)
at Microsoft.AnalysisServices.Modeler.Storage.RelationalDataSourceConnection.Open()
at Microsoft.AnalysisServices.Modeler.DataImportWizard.DataSourceBasic.TestConnection()
at Microsoft.AnalysisServices.Modeler.DataImportWizard.DataSourceBasic.ClickTestConnection(Object progressControl)
There is also a potential secondary error
ORA-12154: TNS:could not resolve the connect identifier specified
We found out this is a very well known issue in the community with the Oracle OLEDB provider.
SSIS users launching 32-bit DTExec.exe or the DTSWizard.exe have probably known this for years, but PowerPivot for Excel 2010 users may be seeing this for the first time.
The cause is When the path of the application which launches the connection contains parenthesis, the provider fails to connect.
When using 32-bit Excel on a 64-bit machine, the 32-bit WOW program files folder has parenthesis in it
C:\Program Files (x86)\Microsoft Office\Office14\Excel.exe
1. Quick and Dirty Workaround:
To work around this is very easy? find the equivalent 8.3 short naming convention for Excel.exe, and use that to launch Excel. Build yourself a shortcut from the short-named path.
With this approach the Oracle provider won?t have any trouble from what we?ve seen. Each system?s folder names can be different so you may need to check your own folders to find the right ~1 ~2 ~3 suffix when there are duplicate short folder names.
For example, mine is: C:\PROGRA~2\MICROS~4\Office14\Excel.exe
Here is how I determined the short name? to explore your own short folder names, use this approach on the drive letter where Office is installed.
Start > Run > Cmd.exe (or Start > CMD.exe if you don?t have a Run box)
Dir C:\ /x
-- look for Program Files (x86) short name
Dir C:\PROGRA~2 /x
-- Look for Microsoft Office short name. Mine is MICROS~4 but yours may be a different number or abbreviation.
Dir C:\PROGRA~2\MICROS~4\ /x
-- Look for Office14 (short name is same as long name)
Dir C:\PROGRA~2\MICROS~4\Office14 /x
-- Look for Excel.exe (short name is same as long name)
Therefore, my short-name to launch Excel can be:
C:\PROGRA~2\MICROS~4\Office14\Excel.exe
To make this easy to remember next time, take the short path that you found, copy paste it, and right click on your desktop or start menu to make a new shortcut.
Of course this is just a workaround, but this is easy enough, and maybe even easier than the alternatives?.
When you install Office, pick a non-default folder that doesn?t contain parenthesis. That?s probably not a good option for you, since you already have Office 2010 installed!
2. Long Term ? The Real Solution:
Contact Oracle to get the patches for the client drivers which eliminates this problem.
Oracle Number 3807408 CANNOT EXTERNALLY AUTHENTICATE USER WITH QUOTE IN USERNAME
https://metalink.oracle.com/metalink/plsql/showdoc?db=Bug&id=3807408
This fix requires that both the client and database software be patched.
The patch is available only for the currently certified versions (9.2.0.7, 10.2.0.1) and not on lower versions such as (9.2.0.4). There is a secondary patch for the client machine (machine where PowerPivot connects from.
4928723 (Description: ORACLE 9I 9.2.0.7 PATCH 6 ON WINDOWS 32 BIT)
4928724 (Description: ORACLE 9I 9.2.0.7 PATCH 6 FOR WINDOWS (64 BIT) )
Technorati Tags: PowerPivot,Excel 2010,ErrorMessage,Oracle Connectivity
Newsfeed display by CaRP
