Looking for my RSS feed? Here it is!

Squid - Data Feed Framework for .NET 3.5

Monday, March 17, 2008

A few years ago I designed a system that would greatly ease data syndication, data aggregation, and reporting.  The first two components of the system were repackaged and release early last year under the incredibly horrible name "Data Feed Framework".  The idea behind the system was two fold.  The first concept was that you write a SQL statement and you immediately get a fully functional RSS feed with absolutely no more work required.  Here's an example of a DFF SQL statement that creates an RSS feed of SQL Server jobs:

select Id=0,
Title=name,
Description=description
from msdb.dbo.sysjobs
where enabled = 1

The second part of DFF was it's ASP.NET control named InfoBlock that would accept an RSS or ATOM feed and display it in a mini-reader window.  The two parts of DFF combine to create the following:

Given the following SQL statement (or more likely a stored procedure)...

select top 10
Id=pc.ContactID, 
Title=pc.FirstName + ' ' + pc.LastName + ': $' + convert(varchar(20), convert(numeric(10,2), sum(LineTotal))), 
Description='', 
LinkTemplate = '/ShowContactInformation/{id}'
from Sales.SalesOrderDetail sod
inner join Sales.SalesOrderHeader soh on soh.SalesOrderID = sod.SalesOrderID
inner join Person.Contact pc on pc.ContactID = soh.SalesPersonID
group by pc.FirstName, pc.LastName, pc.ContactID
order by sum(LineTotal) desc

...we have an automatically updating RSS feed and when that RSS feed is given to an InfoBlock, you get the following:

image

InfoBlocks could be placed all over a web site or intranet to give quick and easy access to continually updating information.  The InfoBlock control would also register the feed with modern web browsers that had integrated RSS support.  Furthermore, since it was styled properly in CSS, there's no reason for it to be a block at all.  It could be a horizontal list, a DOM-based window, or even a ticker as CSS and modern AJAX techniques allow.

DFF relied on RSS.NET for syndication feed creation and both RSS.NET and Atom.NET for aggregation.  It also used LLBLGen Pro a bit to access the data from SQL Server.  As I've promised with all my projects, they will update as new technologies are publicly released.  Therefore, DFF has been completely updated for .NET 3.5 technologies including LINQ and WCF.

I've also decided to continue down my slippery slope of a change in product naming philosophy.  Whereas before I would follow the Microsoft marketing philosophy of "add more words to the title until it's so long to say that you require an acronym" to the more Linux or O'Reilly approaches of "choose a random weird sounding word and leave it be" and "pick a weird animal", respectively.  I've also been moving more towards the idea of picking a cool name and leaving it as is.  This is in contrast to Microsoft's idea of picking an awesome name and then changing it to an impossibly long name right before release (i.e. Sparkle, Acrylic, and Atlas)  Therefore, I decided to rename DFF to Squid.  I think this rivals my Dojr.NET and Prominax (to be released-- someday) projects as having the weirdest and most random name I've ever come up with.  I think it may have something to do with SQL and uhhhh.. something about a GUID.  Donno.

Squid follows the same everything as DFF, however the dependencies on RSS.NET and ATOM.NET were completely removed.  This was possible due to the awesome syndication support in WCF 3.5.  Also, all reliance on LLBLGen Pro was removed.  LLBLGen Pro (see my training video here) is an awesome system and is the only enterprise-class O/R mapping solution in existence.  NHibernate should not be considered enterprise-class and it's usability is almost through the floor.  Free in terms of up-front costs, does not mean free in terms of usability (something Linux geeks don't seem to get).  However, given that LINQ is built into .NET 3.5, I decided that all my shared and open-source projects should be using LINQ, not LLBLGen Pro.  The new LLBLGen Pro uses LINQ and when it's released, should absolutely be used as the primary solution for enterprise-class O/R mapping.

Let me explain a bit about the new syndication feature in WCF 3.5 and how it's used in Squid.  Creating a syndication feed in WCF is required a WCF endpoint just like everything else in WCF.  This endpoint will be part of a service and will have an address, binding, and contract.  Nothing fancy yet as the sweetness is in the details.  Here's part of the contract Squid uses for it's feed service (don't be jealous of the VS2008 theme -- see Scott Hanselman's post on VS2008 themes):

namespace Squid.Service
{
    [ServiceContract(Namespace = "http://www.netfxharmonics.com/services/squid/2008/03/")]
    public interface ISquidService
    {
        [OperationContract]
        [WebGet(UriTemplate = "GetFeedByTitle/{title}")]
        Rss20FeedFormatter GetFeedByTitle(String title);

        //+ More code here
    }
}

Notice the WebGet attribute.  This is applied to signify that this will be part of a HTTP GET request.  This relates to the fact that we are using a new WCF 3.5 binding called the WebHttpBinding.  This is the same binding used by JSON and POX services.  There are actually a few new attributes, each of which provides it's own treasure chest (see later in this post when I mention a free chapter on the topic).  The WebGet attribute has an awesome property on it called UriTemplate that allows you to match parameters in the request URI to parameters in the WCF operation contract.  That's beyond cool.

The service implementation is extremely straight forward.  All you have to do is create a SyndicationFeed object, populate it with SyndicationItem objects and return it in the constructor of the Rss20FeedFormatter.  Here's a non-Squid example:

SyndicationFeed feed = new SyndicationFeed();
feed.Title = new TextSyndicationContent("My Title");
feed.Description = new TextSyndicationContent("My Desc");
List<SyndicationItem> items = new List<SyndicationItem>();
items.Add(new SyndicationItem()
{
    Title = new TextSyndicationContent("My Entry"),
    Summary = new TextSyndicationContent("My Summary"),
    PublishDate = new DateTimeOffset(DateTime.Now)
});
feed.Items = items;
return new Rss20FeedFormatter(feed);

You may want to make note that you can create an RSS or ATOM feed directly from an SyndicationFeed instance using the SaveAsRss20 and SaveAsAtom10 methods.

As with any WCF service, you need a place to host it and you need to configure it.  To create a service, I simply throw down a FeedService.svc file with the following page directive (I'm really not trying to have the ugliest color scheme in the world-- it's just an added bonus):

<%@ ServiceHost Service="Squid.Service.SquidService" %>

The configuration is also fairly straight forward, all we have is our previously mentioned ending with an address(blank to use FeedService.svc directly), binding (WebHttpBinding), and contract(Squid.Service.ISquidService).  However, you also need to remember to add the WebHttp behavior or else nothing will work for you.

<system.serviceModel>
    <behaviors>
        <endpointBehaviors>
            <behavior name="FeedEndpointBehavior">
                <webHttp/>
            </behavior>
        </endpointBehaviors>
    </behaviors>
    <services>
        <service name="Squid.Service.SquidService">
            <endpoint address=""
                      binding="webHttpBinding"
                      contract="Squid.Service.ISquidService"
                      behaviorConfiguration="FeedEndpointBehavior"/>
        </service>
    </services>
</system.serviceModel>

That's seriously all there is to it: write your contract, write your implementation, create a host, and set configuration.  In other words, creating a syndication feed in WCF is no different than creating a WsHttpBinding or NetTcpBinding service.  However, what about reading an RSS or ATOM feed? This is even simpler.

To read a feed all you have to do is create an XML reader with the data source of the feed and pass that off to the static Load method of the SyndicationFeed class.  This will return an instance of SyndicationFeed which you may iterate or, as I'm doing in Squid, transform with LINQ.  I actually liked how my server-control used an internal repeater instance and therefore wanted to continue to use it.  So, I kept my ITemplate object (RssListTemplate) the same and used the following LINQ to transform a SyndicationFeed to what my ITemplate what already using:

Object bindingSource = from entry in feed.Items
                       select new SimpleFeedEntry
                       {
                           DateTime = entry.PublishDate.DateTime,
                           Link = entry.Links.First().Uri.AbsoluteUri,
                           Text = entry.Content != null ? entry.Content.ToString() : entry.Summary.Text,
                           Title = entry.Title.Text
                       };

Thus, with .NET 3.5 I was able to remove RSS.NET and ATOM.NET completely from the project.  LINQ also, of course helped me with my database access and therefore remove my dependency on my LLBLGen Pro generated DAL:

using (SquidLINQDataContext db = new SquidLINQDataContext(Configuration.DatabaseConnectionString))
{
    var collection = from p in db.FeedCreations
                     where p.FeedCreationTitle == title
                     select p;

    //+ More code here
}

Thus, you can use Squid in your existing .NET 3.5 system with little impact to anything.  Squid is what I use in my Minima blog engine to provide the boxes of information in the sidebar.  I'm able to modify the data in the Snippet table in the Squid database to modify the content and order in my sidebar.  Of course I can also easily bring in RSS/ATOM content from the web with this as well.

You can get more information on the new web support in WCF 3.5 by reading the chapter "Programmable Web" (free chapter) in the book Essential WCF for .NET 3.5 (click to buy).  This is an amazing book that I highly recommend to all WCF users.

Links

(0 comments)

March 2008 Web Technology Update

Wednesday, March 5, 2008

Recently a bunch of technologies have been released and/or updated and I would like to mention a few of them briefly.

First and foremost, Silverlight 2 Beta 1 has finally been released and you may download it immediately.  There is also an accompanying SDK.  You can find a nice development tutorial series on Scott Guthrie's blog.  If you are already familiar with WPF, you can just skim this entire series in less than 5 minutes.  Given that this technology isn't the same as the full WPF and given that it's designed for the web, there will obviously be differences.  It's important to remember that Silverlight 2 isn't simply WPF for the web.  I would call WPF 3.5's XBAP support for IE/Firefox "WPF for the web".  No, this is possibly the biggest web technology improvement since the release of Firefox 1.0, which in turn was the biggest technology release since the printing press.  Alight, alight... since .NET 1.1.  It's support for the dynamic language runtime is going to completely revolutionize our web development.

When reading through Scott's tutorial series (serious, at least skim it), it's interesting to note that Silverlight 2 allows cross-domain communication.  It does this by reusing the Flash communication policy files.  This is really awesome as it means that you can start accessing resources that Flash has been using for a while.  Being able to dynamically access resources from different domains is critical to the success of web architecture in the future.

Speaking of cross-domain communication, John Resig and I received a very depressing e-mail the other day telling us horrible news: cross-domain communication will probably be removed from Firefox 3 before it's official release.  Apparently a bunch of paranoid anti-architects were complaining about the dreaded evils of being able to access resources from different domains.  Um ok.  Fortunately, however, Firefox 3 has a feature called postMessage that allows you to get around this.  Malte Ubl has produced a library called xssinterface to demonstrate just this concept.  You could, of course, get around this completely with some iframe hacks or some other scripting magic.

Speaking of web browsers, I would like to bring people's attention to a technology that I've been following for some time now: Apple WebKit.  This is basically the brains inside Safari.  I absolutely love the Safari web browser.  It's by far and away the easiest web browser to use.  It also has the same keyboard short-cuts as Firefox, which is how I'm able to use it.  It's also incredibly fast, but I should mention that it uses even more memory than Firefox.  My last instance passed 500MB.  Given it's lack of an extension or configuration (i.e. about:config) system, it's obviously no where near the same caliber as Firefox though.  It is, however, my primary web browser as has been since October '07.

The reason I mention WebKit is because as very few people know, this is an open source project and has nightly binaries released on their webkit.org web site.  One of the most interesting thing about nightlies that you can actually watch the progress of development as time goes on.  About every month or so I like to get the latest Firefox nightly.  It's always interesting to see the major experiments that the developers try about 2 months after a major release of Firefox.  There's always some really awesome "teaser" feature in there that later grows into a fully grown technology.  The same can be said for WebKit.

None of that is, however, my primary reason for mentioning WebKit.  As, most web developers know, the Acid2 test has been the standard for checking a web browsers compatibility with the CSS standard.  I've been pushing this test for a long time, but I've never pushed it as the only test.  There are many things that a web browser must do and many features a web browser must have before it can be considered appropriate for use.  Merely focusing on CSS, while completely ignoring DOM support, JavaScript, and general user usability can lead a browser to be as impossible to use as Opera 9.

As I've said time and time again, I'm not a CSS specialist.  Part of the definition of being a professional web develop is that I have a solid understand of the inner workings of CSS including specificity, the various selectors, and how to merge absolute, floating, and relative position on the same elements, tasks "coders" see as nearly impossible to learn.  However, my focus is on AJAX interaction as seen from the JavaScript and DOM worlds.  Therefore, we need to have a test for browsers that goes beyond the simple Acid 2 test for CSS.  I'm not the only one thinking this way, because recently the Acid3 test was published and it tests CSS, JavaScript and DOM support.  This is the new standard for web browsers.

So far no web browser has even gotten close, with the lowest score from a web browser being 39% in Safari to the best score being 50% in Firefox 2.0.0.12.  However, in terms of non-released software, Firefox 3.0b3 has a score between 59% and 61%, depending on its mood (update: b4 is steady at 67%) and the latest WebKit nighty has a score is 90% (watch WebKit progress on Acid 3 at http://bugs.webkit.org/show_bug.cgi?id=17064).  That's phenomenal!  The newly released Internet Explorer 8 beta 1 has a score of 17%.  Those of you who have naively praising the IE team for being YEARS late on getting near the Acid 2 test need to wake up and realize this is 2008.  Time moves-- keep up.  Firefox has been close for the longest time and has always had the next-gen's next-gen JavaScript and DOM support, but has only recently completely passed the finish line of the Acid 2 test.  So, they are finally off my watch list there, but I will not stop bugging them until they pass the Acid 3 test.

For more information on the Acid 3 test, see John Resig's most entitled "Acid 3 tackes EMCAScript".  He's about as passionate as I am for web standards and Firefox and his blog is an invaluable resource for all things JavaScript.  His work is so good that I would like to take the time to plug his book he is currently writing: Secrets of the JavaScript Ninja.  I absolutely guarantee you that this book will redefine the entire world of JavaScript and will raise the bar incredibly out of the reach of "coders". To all of you coders who think you know JavaScript, do a view-source on the Acid 3 source code (you may want to bring a change of underwear with you).

Lastly, it's not necessarily a "new" technology, but it's so incredibly phenomenal that I need to mention it: Prototype 1.6.  It's amazing to me that people actually go out of their way to use ASP.NET AJAX 3.5 (I still find the ICallbackEventHandler interface more productive).  ASP.NET AJAX 3.5 is not nearly as bad as extremists think, but the design is still flawed.  Prototype on the other hand is absolutely incredible.  I've written about Prototype before, but this version 1.6 is even more powerful.  There a A LOT of changes from Prototype 1.5.  It's so good that I no longer call it "prototype/script.aculo.us".  Script.aculo.us is a great animation system, but, honestly, the main reason I used it was for the DOM abstraction in the Builder object.  Prototype now has an Element object to help create DOM objects, thus allowing me to remove Script.aculo.us from most of my projects (it's not as complete as the Builder object, but it allows object chaining-- which greatly increases code readability, conciseness and understanding!).  The Template object is also amazing as it gives you the ability to go far beyond simple String.Format formatting.  The new Class object for OOP is also great.  It's so much easier to use than Prototype 1.5.  Also, being able to hide all elements with a particular CSS pattern with one shot is very useful! (for example, $$('div span .cell-block').invoke('hide')).  It even allows you to use CSS 3 selectors on the most dead of web browsers.  It really makes developing for Internet Explorer 6 and 7 bearable!  Even if I have to use ASP.NET AJAX 3.5, I'll still including prototype.js.  If you do anything with JavaScript, you need Prototype!

 

Links

(0 comments)

Creative Commons License
This work is licensed under a Creative Commons Attribution 2.5 License.

Powered by the Minima Blog Engine, a .NET 3.5/LINQ based blog engine.

Mini-icons are part of the Silk Icons set of icons at famfamfam.com