Getting started with Silverlight: Part 3 – Accessing Data
This is part 3 in a series on getting started with Silverlight. To view the index to the series click here. You can download the completed project files for this sample application in C# or Visual Basic.
Now that we have our initial layout outlined and some controls to work with, let’s start getting the data. Since we’re going to use Twitter search, we’ll be leveraging their web service API. In our application we won’t be hosting our own database or anything but I did want to point out the various ways you can access data via Silverlight before we go to work on ours.
Data access options
One of the bigger beginner misconceptions about accessing data in Silverlight is people looking for some ADO.NET class library in Silverlight. Stop looking, it isn’t there. Remember, Silverlight is a client technology that is deployed over the Internet. You wouldn’t want a browser plug-in to have direct access to your database…as you’d have to expose your database directly to the web. We all know that is generally a big no-no.
So the next logical step is to expose data via service layers. This is how Silverlight can communicate with data. Here are the primary means:
- Web services: SOAP, ASP.NET web services (ASMX), WCF services, POX, REST endpoints
- Sockets: network socket communication
- File: accessing static content via web requests.
Sockets
Sockets are probably the most advanced data access endpoints. These require a socket host to exist and, at the time of this writing, also require communication over a specific port range. If this is acceptable for you, this can be a very efficient and powerful means of communication for your application. I don’t think, however, that this is going to be the norm if your application is public facing on the web – I see sockets right now being more for business applications. Some resources for Sockets:
- SDK documentation
- Working with data in sockets (Dan Wahlin, MVP) – this is Silverlight 2 but still relevant to understanding the concepts
Working with Sockets requires you to really understand your deployment scenario first before you jump right in and think this will be the best method.
File Access
Silverlight can communicate with local data or data on the web. For local data access, the application does not have direct access to the file system, but rather can read/write data via user-initiated actions using the OpenFileDialog and SaveFileDialog APIs to request and save streams of data to the local user’s machine.
Additionally, you can use plain text files or XML files on the web and have your Silverlight application use standard HTTP commands to read/write that information. Here’s some helper information on using some of these methods:
You may find yourselves using these techniques to save settings-based data or use very simple data access.
Web Services
This is the heart of accessing data in Silverlight – through a service layer. Silverlight supports accessing standard ASP.NET web services (ASMX) or WCF-based services using the familiar Add Service Reference methods in Visual Studio that will generate strongly-typed proxy code for you.
Additionally you can use the standard HTTP verbs to access more POX (Plain old XML) or REST-based endpoints. Understanding how to consume these various service types is probably the best time spent a developer can educate themselves on understanding what will be right for their scenario. Here’s some resources:
- Consume WCF and ASP.NET Services
- Calling services over HTTPS
- .NET RIA Services
- Choosing a data access layer (Shawn Wildermuth, MVP)
The third point there, .NET RIA Services, is a new framework that aims to make accessing data simpler and more familiar. The link to the video will walk you through an introduction of that topic. RIA Services is best when you own the database and are hosting services in the same web application that will be serving up the Silverlight application.
Asynchronous access
All data access in Silverlight is asynchronous. This is perhaps another area that gets typical web developers tripped up initially. For example in the server world it would be reasonable to see something like this:
1: MyWebService svc = new MyWebService();
2: string foo = svc.GetSomeValue();
3: MyTextBox.Text = foo;
In Silverlight you wouldn’t be able to do that synchronous call. To those who haven’t done asynchronous programming before this can be confusing, but it is well worth the learn and will make you a better developer. Using the above pseudo code, in Silverlight you’d do something like this:
1: MyWebService svc = new MyWebService();
2: svc.Completed += new CompletedHandler(OnCompleted);
3: svc.GetSomeValue();
4:
5: void OnCompleted(object sender, EventArgs args)
6: {
7: MyTextBox.Text = args.Result;
8: }
Notice that you use the result of the service call in a completed event handler. This is the pattern that you will see over and over again with basic service access.
Cross-domain data access
Since Silverlight is a web client technology, it operates in the browser’s secure sandbox area and are limited to certain access policies. One of these restrictions is referred to as cross-domain access. That is that your application hosted in one domain cannot access services in another domain unless the service says you can. This “opt-in” approach is commonly known as cross-domain policy files. Silverlight, like other rich client plug-ins, conforms to these policies. This is an area that you as a Silverlight developer will likely hit at some point. Educate yourself sooner than later. Here are some pointers:
- Cross-domain policy files with Silverlight
- Cross-domain policy file helpers
- Troubleshooting cross-domain access with helpful tools
- My Silverlight application cannot access my service!
- Cross-domain communication overview
In our Twitter application, we actually will be accessing a service hosted elsewhere and will need to conform to these policies. Luckily, the search API for Twitter enables this access through their cross-domain policy file (http://search.twitter.com/crossdomain.xml). The other areas of Twitter do NOT, which is why, for now, you would not be able to access them directly through Silverlight. In this situation you would proxy those service calls through your own service that you could enable cross-domain access via a policy file for Silverlight. Confusing? It’s simpler than it sounds.
COMMON MYTH: You need the Silverlight and the Adobe cross-domain policy files in your service to enable access. This is NOT TRUE and I see it to often people saying I have crossdomain.xml and clientaccesspolicy.xml and it still doesn’t work. If you are building a service for Silverlight consumption via cross-domain, you only need the clientaccesspolicy.xml file format – that is what we look for first and is the most flexible and secure for Silverlight.
Now that we have a high-level overview, let’s start accessing our data!
Calling the Twitter API
The Twitter search API is a simple REST-based API that we’ll only really be calling GET requests on in our application. The format they provide is the Atom specification which makes our job a lot easier because it is a standard format and Silverlight has framework libraries that support direct consumption of that format.
We will initiate calling the API when the user clicks the SEARCH button in our application and there is content in the search input box. Let’s wire up an even to the click button like we did in step 1 in our Hello World application. In our Search.xaml I’m adding the click event to our SearchButton. Add the Click event handler to the SearchButton and use the name SearchForTweets as the function name:
1: <Button x:Name="SearchButton" Width="75" Content="SEARCH" Click="SearchForTweets" />
In Visual Studio, if you right-click now on the function name, you can Navigate to Event Handler and it will generate the stub code for you in the code page. In this function we’re going to search the Twitter API for postings matching our criteria. Since the API is a simple REST GET call, we’re going to use the simple WebClient Silverlight API. This is the simplest networking API to use and allows you to read/write data via GET/POST commands as long as you don’t need to alter headers. Before we do that I’m going to set up some member variables for some tracking that we’ll use to monitor our search terms:
1: const string SEARCH_URI = "http://search.twitter.com/search.atom?q={0}&since_id={1}";
2: private string _lastId = "0";
3: private bool _gotLatest = false;
Now we can wire up our SearchForTweets function. Remember that I mentioned that network activity is asynchronous in Silverlight? This is where we will start experiencing this. We’re going to use the OpenRead API on WebClient. Because the function will be asynchronous, we’ll need to wire up a Completed event handler to receive the response and do whatever we need with it. Here’s what we have so far:
1: private void SearchForTweets(object sender, RoutedEventArgs e)
2: {
3: WebClient proxy = new WebClient();
4: proxy.OpenReadCompleted += new OpenReadCompletedEventHandler(OnReadCompleted);
5: proxy.OpenReadAsync(new Uri(string.Format(SEARCH_URI, HttpUtility.UrlEncode(SearchTerm.Text), _lastId)));
6: }
7:
8: void OnReadCompleted(object sender, OpenReadCompletedEventArgs e)
9: {
10: throw new NotImplementedException();
11: }
Notice we first create a new WebClient instance. Then we set up the completed event handler, and finally call the OpenReadAsync funciton with a formatted URI. The result in our completed handler (e.Result) will be a stream. Because we are going to manipulate the response a bit and do some databinding, we’re going to create a local class to represent the structure of our search result. I called mine TwitterSearchResult.cs and is just a class file in my project under a folder called Model:
1: using System;
2: using System.Windows.Media;
3:
4: namespace TwitterSearchMonitor.Model
5: {
6: public class TwitterSearchResult
7: {
8: public string Author { get; set; }
9: public string Tweet { get; set; }
10: public DateTime PublishDate { get; set; }
11: public string ID { get; set; }
12: public ImageSource Avatar { get; set; }
13: }
14: }
With our model in place we can shape the result and do some data binding.
Other networking options: HttpWebRequest and ClientHttp
There are two other network APIs we could have used to access the Twitter API: HttpWebRequest and ClientHttp. HttpWebRequest is essentially what we *are* using with WebClient as it is a simple wrapper around that API. If you needed more granular control over the headers in the request, you’d want to use HttpWebRequest. Both WebClient and HttpWebRequest make use of the browser’s networking stack. This presents some limitations, namely not being able to receive all complete status codes or leverage some expanded verbs (PUT/DELETE). Silverlight has also introduced a ClientHttp option and uses a custom networking stack that enables you to use more verbs as well as receive status code results beyond 200/404.
More information:
As an example if we wanted to use ClientHttp, our call would look like this:
1: private void SearchForTweets(object sender, RoutedEventArgs e)
2: {
3: bool httpBinding = WebRequest.RegisterPrefix("http://search.twitter.com",
4: WebRequestCreator.ClientHttp);
5: WebClient proxy = new WebClient();
6: proxy.OpenReadCompleted += new OpenReadCompletedEventHandler(OnReadCompleted);
7: proxy.OpenReadAsync(new Uri(string.Format(SEARCH_URI, HttpUtility.UrlEncode(SearchTerm.Text))));
8: }
Note that we *are not* using this method, but just wanted to point these out for you. The call to RegisterPrefix denotes that we registered it to use the ClientHttp networking stack instead of the browser’s networking stack. In the above sample we registered only calls to the Twitter search domain, but we could have enabled it for all HTTP requests as well.
These are additional options for you to consider in your applications.
Start simple binding some data with smart objects
Because our application is going to ‘monitor’ search terms in Twitter, we want to really just set up binding to an object collection once, and then just manipulate that collection (in our case, add to it). To do this we are going to use two helpful objects in Silverlight: ObservableCollection<T> and PagedCollectionView. ObservableCollection is a collection type that automatically provides notifications when items are modified within the collection (added, removed, changed). PagedCollectionView will be used to help us automatically give some sorting to our objects.
We’ll create these as member variables in our project:
1: ObservableCollection<TwitterSearchResult> searchResults = new ObservableCollection<TwitterSearchResult>();
2: PagedCollectionView pcv;
Now that we have the member variables, let’s initialize the PagedCollectionView in the constructor of the control so it will be available quickly. We also want to bind our UI to our elements in the XAML. It is good practice not to do anything to your UI in the constructor of your UserControl (in our case Search.xaml). Because of this we’ll add a Loaded event handler in the constructor and set up the initial binding in that handler. Our combination of constructor and Loaded event handler now looks like this:
1: public Search()
2: {
3: InitializeComponent();
4:
5: pcv = new PagedCollectionView(searchResults);
6: pcv.SortDescriptions.Add(new System.ComponentModel.SortDescription("PublishDate", System.ComponentModel.ListSortDirection.Ascending));
7:
8: Loaded += new RoutedEventHandler(Search_Loaded);
9: }
10:
11: void Search_Loaded(object sender, RoutedEventArgs e)
12: {
13: SearchResults.ItemsSource = pcv;
14: }
Notice that in the loaded handler we set the ItemsSource property of our DataGrid (SearchResults) to be the PagedCollectionView that is sorted (see the SortDescription we added in line 6 above). Now our UI is bound to this PagedCollectionView…so we should probably populate it. Remember that it is actually a view of the data in our ObservableCollection<TwitterSearchResult> – so that is what we need to add items to in order for some data to be seen.
Populating our ObservableCollection
Now go back to the OnReadCompleted function we had before that will fire after our network request to the search API. We now are going to populate our ObservableCollection in this function. Here’s the code we’ll use to do that:
1: void OnReadCompleted(object sender, OpenReadCompletedEventArgs e)
2: {
3: if (e.Error == null)
4: {
5: _gotLatest = false;
6: XmlReader rdr = XmlReader.Create(e.Result);
7:
8: SyndicationFeed feed = SyndicationFeed.Load(rdr);
9:
10: foreach (var item in feed.Items)
11: {
12: searchResults.Add(new TwitterSearchResult() { Author = item.Authors[0].Name, ID = GetTweetId(item.Id), Tweet = item.Title.Text, PublishDate = item.PublishDate.DateTime.ToLocalTime(), Avatar = new BitmapImage(item.Links[1].Uri) });
13: _gotLatest = true;
14: }
15:
16: rdr.Close();
17: }
18: else
19: {
20: ChildWindow errorWindow = new ErrorWindow(e.Error);
21: errorWindow.Show();
22: }
23: }
24:
25: private string GetTweetId(string twitterId)
26: {
27: string[] parts = twitterId.Split(":".ToCharArray());
28: if (!_gotLatest)
29: {
30: _lastId = parts[2].ToString();
31: }
32: return parts[2].ToString();
33: }
A few things are happening here. First, the e.Result matches the Stream of response we’ll get from a successfull search. If an error occurs, we’ll use the ErrorWindow template which is provided for us in the navigation application template we chose. The _gotLatest member variable helps us track the whether or not we need to reset the max value (which is so future queries request only the latest since the previous query). After we get the stream we load it into an XmlReader for ease of parsing with our SyndicationFeed class. SyndicationFeed is a class in the System.ServiceModel.Syndication library that you’ll have to add a reference too in your project. It has built-in functions for parsing known syndication formats, like RSS and Atom.
NOTE (Here be dragons): System.ServiceMode.Syndication brings with it other dependency assemblies. It is not a small library, but convenient to have. Take caution in using it for your project and know when and why you need it. We are using it here so you can be aware of the features and productivity benefits. An alternative method (especially for just reading syndicated feeds) would be to actually just use LINQ to XML and query the resulting XDocument after loading it. Again, for demonstration purposes, I wanted to point out the productivity use of SyndicationFeed as a strongly-typed class available to you.
More information about reading Syndication data can be found here:
Once we have the SyndicationFeed data loaded, we simply iterate through it and add a new TwitterSearchResult to our ObservableCollection<TwitterSearchResult> object. You’ll notice we’re doing some conversion on the Image URI to an ImageSource for easier binding later. Additionally, we’re parsing out the ID of the tweet so we can set the first result (which is the latest) as the most recent ID for later querying (_lastId).
Giving feedback back to our users
In our final step here, we want to make sure we are giving some feedback to our users that we are doing something (searching). Luckily we have something easy for you in an ActivityControl. At the time of this writing, the ActivityControl was a part of the .NET RIA Services templates, but you can get it here on David Poll’s blog. You’ll have to build the control and then add a reference to it in your project (if you download the source to our projects the binary is already included in the Libraries folder for you).
UPDATE NOTE: While the contents of this tutorial remains unchanged, the ActivityControl is now called the BusyIndicator and is released as a part of the Silverlight Toolkit. Following the same techniques you can get the official control from the toolkit and have it deployed with your application. No more need to compile on your own.
Once you have the reference, then you’ll add the same xmlns notation in your Search.xaml like we did with the DataGrid in step 2. Then add the control as the root control and the Grid as it’s child. My resulting Search.xaml now looks like this:
1: <navigation:Page
2: xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
3: xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
4: xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
5: xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
6: mc:Ignorable="d"
7: xmlns:navigation="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Navigation"
8: xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" x:Class="TwitterSearchMonitor.Views.Search"
9: xmlns:activity="clr-namespace:System.Windows.Controls;assembly=ActivityControl"
10: d:DesignWidth="640" d:DesignHeight="480"
11: Title="Twitter Search Page">
12: <activity:Activity x:Name="ActivityIndicator">
13: <Grid x:Name="LayoutRoot">
14: <Grid.RowDefinitions>
15: <RowDefinition Height="32"/>
16: <RowDefinition/>
17: </Grid.RowDefinitions>
18:
19: <StackPanel HorizontalAlignment="Left" Margin="0,-32,0,0" VerticalAlignment="Top" Grid.Row="1" Orientation="Horizontal">
20: <TextBox x:Name="SearchTerm" FontSize="14.667" Margin="0,0,10,0" Width="275" TextWrapping="Wrap"/>
21: <Button x:Name="SearchButton" Width="75" Content="SEARCH" Click="SearchForTweets" />
22: </StackPanel>
23: <data:DataGrid x:Name="SearchResults" Margin="0,8,0,0" Grid.Row="1"/>
24: </Grid>
25: </activity:Activity>
26: </navigation:Page>
In our code for starting the search (SearchForTweets) simply add the line:
1: ActivityIndicator.IsActive = true;
and when the OnReadCompleted event is done add the line:
1: ActivityIndicator.IsActive = false;
And you’ll have visual progress to the end user. At this point we can run our application (F5) and enter a search term. You’ll see the activity control:
And then the resulting view of results in the DataGrid:
Add some monitoring via timers
Now since we call this a monitoring service, we want the search to automatically refresh for us. Silverlight provides a few different ways to trigger automatic activity. We’re going to use a DispatcherTimer for our application. This is nothing more than a timer that fires a Tick event handler at our defined interval. We’ll add another member variable:
1: DispatcherTimer _timer;
and then in the constructor initiate our timer with adding an event handler:
1: double interval = 30.0;
2:
3: _timer = new DispatcherTimer();
4: #if DEBUG
5: interval = 10.0;
6: #endif
7: _timer.Interval = TimeSpan.FromSeconds(interval);
8: _timer.Tick += new EventHandler(OnTimerTick);
Now we want to refactor some code, because we want the SearchForTweets to be fired on the timer tick event. We’re going to use Visual Studio refactoring tools to extract the method function code from SearchForTweets into a new method SearchForTweetsEx which we’ll call in our Tick event handler OnTimerTick. We’ll also modify our Loaded event to actually start the timer and start the initial search for us (note our Timer is going to have a DEBUG interval of 10 seconds, otherwise 30 seconds). Our refactored complete Search.xaml.cs now looks like this:
1: using System;
2: using System.Collections.ObjectModel;
3: using System.Net;
4: using System.Net.Browser;
5: using System.ServiceModel.Syndication;
6: using System.Windows;
7: using System.Windows.Browser;
8: using System.Windows.Controls;
9: using System.Windows.Data;
10: using System.Windows.Media.Imaging;
11: using System.Windows.Navigation;
12: using System.Windows.Threading;
13: using System.Xml;
14: using TwitterSearchMonitor.Model;
15:
16: namespace TwitterSearchMonitor.Views
17: {
18: public partial class Search : Page
19: {
20: const string SEARCH_URI = "http://search.twitter.com/search.atom?q={0}&since_id={1}";
21: private string _lastId = "0";
22: private bool _gotLatest = false;
23: ObservableCollection<TwitterSearchResult> searchResults = new ObservableCollection<TwitterSearchResult>();
24: PagedCollectionView pcv;
25: DispatcherTimer _timer;
26:
27: public Search()
28: {
29: InitializeComponent();
30:
31: // set interval value for Timer tick
32: double interval = 30.0;
33:
34: _timer = new DispatcherTimer();
35: #if DEBUG
36: interval = 10.0;
37: #endif
38: _timer.Interval = TimeSpan.FromSeconds(interval);
39: _timer.Tick += new EventHandler(OnTimerTick);
40:
41: // initialize our PagedCollectionView with the ObservableCollection
42: // and add default sort
43: pcv = new PagedCollectionView(searchResults);
44: pcv.SortDescriptions.Add(new System.ComponentModel.SortDescription("PublishDate", System.ComponentModel.ListSortDirection.Descending));
45:
46: Loaded += new RoutedEventHandler(Search_Loaded);
47: }
48:
49: void OnTimerTick(object sender, EventArgs e)
50: {
51: SearchForTweetsEx();
52: }
53:
54: void Search_Loaded(object sender, RoutedEventArgs e)
55: {
56: SearchResults.ItemsSource = pcv; // bind the DataGrid
57: _timer.Start(); // start the timer
58: SearchForTweetsEx(); // do the initial search
59: }
60:
61: // Executes when the user navigates to this page.
62: protected override void OnNavigatedTo(NavigationEventArgs e)
63: {
64: }
65:
66: private void SearchForTweets(object sender, RoutedEventArgs e)
67: {
68: SearchForTweetsEx();
69: }
70:
71: /// <summary>
72: /// Method that actually does the work to search Twitter
73: /// </summary>
74: private void SearchForTweetsEx()
75: {
76: if (!string.IsNullOrEmpty(SearchTerm.Text))
77: {
78: _timer.Stop(); // stop the timer in case the search takes longer than the interval
79: ActivityIndicator.IsActive = true; // set the visual indicator
80:
81: // do the work to search twitter and handle the completed event
82: WebClient proxy = new WebClient();
83: proxy.OpenReadCompleted += new OpenReadCompletedEventHandler(OnReadCompleted);
84: proxy.OpenReadAsync(new Uri(string.Format(SEARCH_URI, HttpUtility.UrlEncode(SearchTerm.Text), _lastId)));
85: }
86: }
87:
88: /// <summary>
89: /// Method that fires after our SearchForTweetsEx runs and gets a result
90: /// </summary>
91: /// <param name="sender"></param>
92: /// <param name="e"></param>
93: void OnReadCompleted(object sender, OpenReadCompletedEventArgs e)
94: {
95: if (e.Error == null)
96: {
97: _gotLatest = false; // reset the latest detector
98: XmlReader rdr = XmlReader.Create(e.Result); // load stream into a reader
99:
100: SyndicationFeed feed = SyndicationFeed.Load(rdr); // load syndicated feed (Atom)
101:
102: // parse each item adding it to our ObservableCollection
103: foreach (var item in feed.Items)
104: {
105: searchResults.Add(new TwitterSearchResult() { Author = item.Authors[0].Name, ID = GetTweetId(item.Id), Tweet = item.Title.Text, PublishDate = item.PublishDate.DateTime.ToLocalTime(), Avatar = new BitmapImage(item.Links[1].Uri) });
106: _gotLatest = true; // reset the fact that we already have the max id needed
107: }
108:
109: rdr.Close(); // close the reader
110: }
111: else
112: {
113: // initialize our ErrorWindow with exception details
114: ChildWindow errorWindow = new ErrorWindow(e.Error);
115: errorWindow.Show();
116: }
117: ActivityIndicator.IsActive = false; // reset the UI
118: _timer.Start(); // reset the timer
119: }
120:
121: /// <summary>
122: /// Parses out the Tweet ID from the tweet
123: /// </summary>
124: /// <param name="twitterId"></param>
125: /// <returns></returns>
126: private string GetTweetId(string twitterId)
127: {
128: string[] parts = twitterId.Split(":".ToCharArray());
129: if (!_gotLatest)
130: {
131: _lastId = parts[2].ToString();
132: }
133: return parts[2].ToString();
134: }
135: }
136: }
The flow will be that our Search page will load, start the timer and start the initial search. After the interval time fires, the search will be executed again, but remember it will use the last known ID to start from as not to load all the repeats again. This future search data will be added to our ObservableCollection and since the DataGrid is already bound to that, it will be automatically represented in the UI in the appropriate sort order.
We also added some checking to make sure the search term is there and we’re not searching for a blank value.
