XmlHttp Service Interop - Part 3 (Service XML Serialization)

David Betz, MVP (ASP.NET)
del.icio.us LinkedIn e-mail

This entry is the third is a series on XmlHttp Service Interop.

In part 1 of this series, I introduced a simple manually created service that accepts some form of XML via the HTTP POST method. The service then used XPath to obtain appropriate values from the XML, processed the information, and returned a "0" for success and an exception message for failure. This service would allow XmlHttp service interop for just about any type of service. Once you know the XML format that is going to be used, you can just modify your own XPath queries to access the information via the new paths It's a very simple approach to setting up a service and reading the message, but perhaps it's a bit too manual.

While this service would probably serve it's purpose well, we can clean it up using a nice little something built directly into .NET: XML serialization. Using XML Serialization we can take an object and magically turn it into XML as well as take XML and magically turn it into a usable object. This latter scenario would be really nice for us to have as it would completely replace the need for XPath or manual XML parsing in our manually created services. To explain serialization, it's probably best to see it in action, but before we see XML serialization we should discuss what we want to serialize.

In our original service, our service message was a simple peice of XML with various XML nodes. The XML format we used looked like this:

<Mail>
 <ToAddresses>
   <ToAddress>johndoe@tempuri.org</ToAddress>
 </ToAddresses>
 <CcAddresses>
   <CcAddress>cc@tempuri.org</CcAddress>

 </CcAddresses>
 <BccAddresses>
   <BccAddress>bcc@tempuri.org</BccAddress>
 </BccAddresses>
 <From>no-reply@tempuri.org</From>

 <Subject>My Subject</Subject>
 <DateTime>03/09/2007 12:30PM</DateTime>
 <Body>This is the message body.</Body>
 <Attachments>

   <Attachment>filepath1</Attachment>
   <Attachment>filepath2</Attachment>
   <Attachment>filepath3</Attachment>
 </Attachments>

</Mail>

Looking at this XML message we we can see that it has a number of properties like "Subject", "Datetime", and "Body". Furthermore, a few of its properties looks like they are collections. You may not have noticed it, but I changed our terminology from XML terminology to .NET terminology, probably without offending anyone. Actually we can change more than our terminology. Here is the entire message rewritten in C# (as EmailService.cs):

using System;
using System.Collections.Generic;

namespace EmailService
{
    public class Mail
    {
        public List<String> ToAddresses = new List<String>();

        public List<String> CcAddresses = new List<String>();

        public List<String> BccAddresses = new List<String>();

        public String FromAddress;

        public String Subject;

        public DateTime DateTime;

        public String Body;

        public List<String> Attachments = new List<String>();
    }
}

Anyone who knows me, knows that I would fail any one of my students without prejeduge for even joking about using the above class. Having public fields in a class not only shows a blatant disregard for the .NET rules, but also a lack of respect for programming ettiquite. However, as you will see in a bit, we are creating something that is actually an exception this cardinal .NET rule. Someone may also say that using the generic List publicly is also taboo, but, again, the scenario we are going to be working with is the exception to the rule.

As you can see in the above type, we have everything that was in the XML message. Not only that, we are strongly typing the DateTime and using a generic List instead of a simple string array. At this point we have everything we need to move on to creating something that magically turns this class into XML via XML serialization.

The easiest way to play with XML serialization is through a console application. So, to start off with lets create a new console application and add the above EmailService.cs to the VS2005 project. After this we can get into the real coding of it by creating a new instance of our Mail class and populating it with some data:

Mail mail = new Mail();
mail.Subject = "subject";
mail.Body = "body";
mail.DateTime = DateTime.Now;
mail.FromAddress = "no-reply@tempuri.org ";
mail.Attachments.Add("Attachment1");
mail.Attachments.Add("Attachment2");
mail.Attachments.Add("Attachment3");
mail.ToAddresses.Add(" johndoe@tempuri.org");

Next, we need to create an instance of the System.Xml.Serialization.XmlSerializer class. This type has many different constructors, but the only one we are going to use simply takes an instance of System.Type representing the type we are going to serialize. This can be seen below:

XmlSerializer xml = new XmlSerializer(typeof(EmailService.Mail));

Finally, we simply have to call the Serialize method of this instance giving our object a place to go. The reason using a console application is nice for XML serialization testing is because we have seamless access a TextWriter in the form of Console.Out, which allows us to serialize our object direct to the console for our immediate inspection. We can see this here:

xml.Serialize(Console.Out, mail);

Now, by running our application we get the following output:

<?xml version="1.0" encoding="IBM437"?>
<Mail xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance " xmlns:xsd="http://ww
w.w3.org/2001/XMLSchema">
  <ToAddresses>

    <string>dfb@davidbetz.net</string>
  </ToAddresses>
  <CcAddresses />
  <BccAddresses />
  <FromAddress>no-reply@tempuri.org</FromAddress>

  <Subject>subject</Subject>
  <DateTime>2007-03-12T14:24:31.8417291-06:00</DateTime>
  <Body>body</Body>
  <Attachments>

    <string>Attachment1</string>
    <string>Attachment2</string>
    <string>Attachment3</string>
  </Attachments>

</Mail>

This is very similar to what our original XML looked like. With an exception of an XML declaraion, two XML namespaces, a new DateTime format, and... what in the world is up with the <string></string> elements? We can handle a new XML declaration and some silly namespaces and while it will require us to either switch our type from DateTime to String or tell our client to send a new DateTime format, we can probably handle the new DateTime format, but there's no way that the <string></string> elements are going to fly. First, off it's nothing like our previous "Attachment" and "ToAddress", but now the ToAddresses and Attachments take the exact same children.

Fortunately, .NET gives us many different XML serialization attributes in the System.Xml.Serialization namespace to help us modify how XML serialization works in a declarative manner. These XML serialization attributes come in handle in a variety of ways, but for now we are only concerned with one that will allow us to rename array items, that is, the XmlArrayItem attribute. To use this attribute, we simply have to put it on the array we want to modify, giving it the new item name in the constructor. We can see this here:

using System;
using System.Collections.Generic;
using System.Xml.Serialization;

namespace EmailService
{
    public class Mail
    {
        [XmlArrayItem("ToAddress")]
        public List<String> ToAddresses = new List<String>();

        [XmlArrayItem("CcAddress")]
        public List<String> CcAddresses = new List<String>();

        [XmlArrayItem("BccAddress")]
        public List<String> BccAddresses = new List<String>();

        public String FromAddress;

        public String Subject;

        public DateTime DateTime;

        public String Body;

        [XmlArrayItem("Attachment")]
        public List<String> Attachments = new List<String>();
    }
}

Now, doing absolutely nothing else, we can run our console application again to get:

<?xml version="1.0" encoding="IBM437"?>
<Mail xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ToAddresses>
    <ToAddress>dfb@davidbetz.net</ToAddress>
  </ToAddresses>

  <CcAddresses />
  <BccAddresses />
  <FromAddress>no-reply@tempuri.org</FromAddress>
  <Subject>subject</Subject>
  <DateTime>2007-03-12T14:34: 24.6042947-06:00</DateTime>

  <Body>body</Body>
  <Attachments>
    <Attachment>Attachment1</Attachment>
    <Attachment>Attachment2</Attachment>

    <Attachment>Attachment3</Attachment>
  </Attachments>
</Mail>

Much better. Granted we still have that new date, but if you want to modify that, you can simply use a String type instead of a DateTime and then parse the date in code. I would actually recommend you try to get the client to agree on this DateTime format as it's the one seen all over web services and allows for strong typing. With regard to the namespaces, if you absolutely can't have namespaces in your XML message all you have to do is create a new instance of XmlSerializerNamespaces, add a blank namespace with a blank name to it, and add it as the third parameter of the XmlSerializer::Serialize method. For example, here is our new code:

XmlSerializerNamespaces ns = new XmlSerializerNamespaces( );
ns.Add("", "");
XmlSerializer xml = new XmlSerializer(typeof(EmailService.Mail));
xml.Serialize (Console.Out, mail, ns);

Now here is our new XML message:

<?xml version="1.0" encoding="IBM437"?>
<Mail>
  <ToAddresses>
    <ToAddress>dfb@davidbetz.net</ToAddress>

  </ToAddresses>
  <CcAddresses />
  <BccAddresses />
  <FromAddress>no- reply@tempuri.org</FromAddress>
  <Subject>subject</Subject>

  <DateTime>2007-03-12T14:46:08.7599218-06:00</DateTime>
  <Body>body</Body>
  <Attachments>
    <Attachment>Attachment1</Attachment>

    <Attachment>Attachment2</Attachment>
    <Attachment>Attachment3</Attachment>
  </Attachments>
</Mail>

Now we have a way to create our XML message, but how does that help our service? To implement it in our service, we simply have to add our EmailService.cs file to our service and use the same XmlSerializer class to deserialize the incoming XML steam (Request.InputStream) into an instance of our type using the XmlSerializer:Deserialize method. Deleting all our fancy XPath parsing and XmlDocument work from our VS2005 solution in part 1, we have the following in our ASP.NET based service:

XmlSerializer xml = new XmlSerializer(typeof(Mail));
Mail mail = (Mail)xml.Deserialize(Request.InputStream);

Lastly, it's a good idea to test our service. The test we want to use is the same as the one used in part 1, with the slight modification of using the new DateTime format. For more information regarding access of HTTP services via JavaScript, see part 1 of this series. Assuming you are familiar with our test client, here is our new JavaScript data:

var data = '';
data += '<Mail>';
data += '  <ToAddresses>';
data += '    <ToAddress>dfb@davidbetz.net</ToAddress>';
data += '  </ToAddresses>';
data += '  <CcAddresses></CcAddresses>';
data += '  <BccAddresses></BccAddresses>';
data += '  <FromAddress>no- reply@tempuri.org</FromAddress>';
data += '  <Subject>XmlHttp Service Interop - Part 3</Subject>';
data += '  <DateTime>0001-01-01T00:00:00</DateTime>';
data += '  <Body>This is the message body2.</Body>';
data += '</Mail>';

Granted, it's not testing every single attribute, but it does get the point accross.

At this point you have all the information you need to recreate the service in part 1 using XML serialization, but is there something else we could do to the service? In fact, there is. In part 2 we talked about XML SOAP messages as a means of communication. We saw that this is how web services communicate through an example of manually talking to WCF from JavaScript. However, I find that many of the services I have to consume look and feel like the SOAP services, but have absolutely nothing to do with proper SOAP. Though it kills my inner child, the new service I put up has to preserve the pseudo-SOAP it improperly accepts. It seems like people have gotten so used to Internet Explorer's inappropriate acceptance of sloppy, inappropriate and blatantly wrong HTML and XHTML that they allowed the desease of pragmatism take over their careers by throwing away the entire idea of specifications and standards in every possible form. Whatever the reason, I'm forced to workaround their mistakes.

For things like this, the service I put up needs to be manual to work around the flaws in the XML message, be it a missing namespace, an invalid namespace, or a missing SOAP body element. However, unlike the service we just built, SOAP services will have multiple XML namespaces. Fortunately, the same set of XML serialization attributes we used to rename the XML array item in our first solution here can be used to further modify our XML message.

For the sake of clarify and to reduce confusion we are not going to create an inproper service, but are going to create a service that is compatible with the SOAP protocol. To start off with, let's state the result we want: we want to have a service that accepts a SOAP message that looks something like this:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Body>

    <Send xmlns=" http://www.tempuri.org/services/email/2007/03/">
      <from>no-reply@tempuri.org</from>
      <to>dfb@ davidbetz.net</to>
      <cc></cc>
      <bcc></bcc>
      <subject>XmlHttp Service Interop - Part 3b</subject>
      <datetime>2007-03-08T15:07:23.927497-06:00</datetime>
      <body>This is the message body.</body>
      <attachments xmlns:a=" http://schemas.microsoft.com/2003/10/Serialization/Arrays" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <a:string>c:\test.txt</a:string>
      </attachments>
    </Send>
  </s:Body>

</s:Envelope>

Now there is one change in this message than in the previous messages and that's the flattening of the to, cc, and bcc arrays into comma separated strings. This changes is simply for demonstration purposes and has nothing to do with any XML serialization limitations. Since you can see see XML serialization of arrays by looking at the attachments element, having the to, cc, and bcc be arrays did nothing more than make the XML longer.

To create a message like this we need to dramatically modify our EmailService.cs file we used above. As you can see there are multiple levels in this XML message. At root there is an Envelope which contains a Body element, which then contains a Send element, which in turn contains all the elements we are used to seeing. Let's look at each of theme in turn starting with the Envelope.

Notice that the Envelope element declares a namespace and is using the same namespace. There's absolutely nothing magic about the "s" as it is merely saying that the Envelope is in the " http://schemas.xmlsoap.org/soap/envelope/" namespace, so we don't need to care about it being "s", it would be "elephant" for all we care. Just think of it as a variable name. Also notice that the Envelope element is the root and has only one child, namely Body. So, for envelope we need a class named Envelope with the XmlRoot attribute assigned to it with the Namespace of that attribute set to " http://schemas.xmlsoap.org/soap/envelope/". Here is an example:

[XmlRoot("Envelope", Namespace = " http://schemas.xmlsoap.org/soap/envelope/")]
public class Envelope
{
    [XmlElement("Body")]
    public Body Body;
}

As you can see there is no "s" in the namespace. The point is that the Envelope is in the namespace, not that it has a particular name. .NET will do all the namespace work upon serialization. Also notice that the XmlRoot element contains the string "Envelope" in its attribute constructor and it has an XmlElement attribute on the Body property with "Body" in its attribute constructor. The purpose of these two is to lower the coupling between the XML service and our .NET types. In practice, all I'm doing is telling the XML serializer what element names to use in the XML message. This allows me to rename the .NET field name all I want while leaving the XML message untouched. This is actually a very good practice to get in to.

Next let's look at the Body element. This element is really simple because it's almost exactly like that Envelope element in that is simply has one child. So, here is our Body type:

public class Body
{
    [XmlElement("Send", Namespace = "http://www.tempuri.org/services/email/2007/03/")]
    public Send Send;
}

Notice that there is no XmlRoot on this one. There are two reasons for this. First, we are renaming the element in the Envelope type and secondly the namespace didn't change for Body. Notice however that I am setting a namespace on the Send field of this type. If you notice in the XML message, Send has an different namespace than does Envelope and Body. In fact, the XML namespace syntax used in the message isn't the same as that which is used with envelope:

xmlns="http://www.tempuri.org/services/email/2007/03/"

Here, we can see that there is no ":s" or anything like that. Now the namespace is being directly flooded into the Send element, thus not requiring a prefix on itself. Notice also that we are setting the namespace on the Send field, not on the send type. If we were to put it on the below Send type via the XmlRoot attribute, our message would have that XML namespace on every element of Send, thus making out XML message almost unreadable. In any case, let's look at the Send type:

public class Send
{
    [XmlElement("to")]
    public String To;

    [XmlElement("cc")]
    public String Cc;

    [XmlElement("bcc")]
    public String Bcc;

    [XmlElement("from")]
    public String From;

    [XmlElement("subject")]
    public String Subject;

    [XmlElement("datetime")]
    public DateTime DateTime;

    [XmlElement("body")]
    public String MailBody;

    [XmlElement("attachments")]
    public StringSet Attachments = new StringSet( );
}

As you can see, this type is very straight-forward. We are simply declaring a bunch of fields and tacking the XmlElement on them decoupling the XML name from the .NET name. However, notice the Attachments fields. This one has a type of StringSet, which is another custom type we had to write in order to satisfy the needs of the other namespaces in the XML message. Assuming your XML client is very strict, you may have to deal with multiple namespaces on multiple levels as seen in the below StringSet class:

public class StringSet
{
    [XmlElement("string", Namespace = "http://schemas.microsoft.com/2003/10/Serialization/Arrays ")]
    public List<String> Data = new List<String>( );

    [XmlNamespaceDeclarations]
    public XmlSerializerNamespaces Namespaces;

    public void Add(String path) {
        this.Data.Add(path);
    }

    public StringSet( ) {
        this.Namespaces = new XmlSerializerNamespaces( );
        this.Namespaces.Add("a", " http://schemas.microsoft.com/2003/10/Serialization/Arrays");
        this.Namespaces.Add("i", "http://www.w3.org/2001/XMLSchema-instance");
    }
}

This is by far the most busy of any of the types we've been working with, but that doesn't mean it has to be difficult to understand. As a reminder, here is the attachments element from our XML message:

<attachments xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays" xmlns:i=" http://www.w3.org/2001/XMLSchema-instance">
<a:string>c:\test.txt</a:string>

</attachments>

By looking at the XML message, we see that the <string></string> element is in a namespace that is declared in it's parent element and you can see that in our .NET type, we tacked a XmlElement attribute on it, changed the XML element name and gave it that namespace. However, we don't want the namespace on the string type, we want it imported on it's parent. This is similar to saying that we don't want to write System.Text.StringBuilder, we just want to write StringBuilder, so we have to add System.Text as an import in our page.

To meet this requirement we simply add a property of any name with the type XmlSerializerNamespaces and tack the XmlNamespaceDeclarations attribute on it thus giving us a handle on what namespaces are imported on that XML element. This XmlSerializerNamespaces type is the same one we used earlier to remove a namespace from an XML message. In this case we are using it to add namespaces to the XML message. As you can see in our .NET type's constructor, we are instantiating the XmlSerializerNamespaces object and giving it two namespaces.

With that final touch we finally have the ability to easily serialze and deserialize a message of that specific XML format, thus giving us the ability to work with the XML messages in a strongly typed manner in our .NET service. In our case, however, we are probably not done. Since we are working with SOAP requests, there will probably be an expectation of an SOAP response. In this case all we need to do is recreate the following SOAP response message:

<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">

  <Body>
    <SendResponse xmlns=" http://www.tempuri.org/services/email/2007/03/">
      <SendResult>0</SendResult>
    </SendResponse>
  </Body>

</Envelope>

To create this we simply go through the same process we did when creating the request, that is, just look at the parent types and it's namespaces and then look at the children and their namespaces on down the line. Here is our .NET type set for our SOAP response:

[XmlRoot(Namespace = "http://schemas.xmlsoap.org/soap/envelope/")]
public class Envelope
{
    [XmlElement("Body", Namespace = " http://schemas.xmlsoap.org/soap/envelope/")]
    public Body Body;
}

public class Body
{
    [XmlElement("SendResponse", Namespace = " http://www.tempuri.org/services/email/2007/03/")]
    public SendResponse SendResponse;
}

public class SendResponse
{
    [XmlElement("SendResult")]
    public String SendResult;
}

Notice however that the .NET types Envelope and Body are resused here. Because of this, it's a good idea to put your SOAP request types into a SOAPRequest namespace and your SOAP response types into a SOAPResponse namespace.

Now putting everything together we can finally look at our complete service code:

Response.ContentType = "text/xml";

try {
    XmlReader reader = XmlReader.Create (Request.InputStream);
    XmlSerializer xml = new XmlSerializer(typeof(EmailService.SOAPRequest.Envelope));
    EmailService.SOAPRequest.Envelope envelope = (EmailService.SOAPRequest.Envelope)xml.Deserialize(reader);
    EmailService.SOAPRequest.Send send = envelope.Body.Send;

    Notification.Send(send.From, send.To, send.Cc, send.Bcc, send.Subject, send.DateTime, send.MailBody, send.Attachments.Data);
}
catch (Exception ex) {
    if (ex.Message.ToLower( ).Contains("user unknown")) {
        status = "0";
    }
    else {
        status = ex.Message;
    }
}

EmailService.SOAPResponse.Envelope soapResponseEnvelope = new EmailService.SOAPResponse.Envelope( );
EmailService.SOAPResponse.Body soapResponsebody = new EmailService.SOAPResponse.Body( );
EmailService.SOAPResponse.SendResponse sendResponse = new EmailService.SOAPResponse.SendResponse ( );
sendResponse.SendResult = status;
soapResponsebody.SendResponse = sendResponse;
soapResponseEnvelope.Body = soapResponsebody;

XmlSerializer responseSerializer = new XmlSerializer(typeof(EmailService.SOAPResponse.Envelope ));
responseSerializer.Serialize(Response.OutputStream, soapResponseEnvelope);

As you can see we are getting the stream information from ASP.NET's Request.InputStream , deserializing the information, processing the data, setting up the response message, serializing it, and then streaming it to ASP.NET's Response.OutputStream. This then finishes our custom SOAP service.

In this part of the XmlService Interop Service you saw how you can use XML serialization to turn .NET types into XML and visa verse as well as how to modify the creation of the XML messages. Furthermore, you saw how you can reverse engineer an XML message to create .NET types which map to the XML message.

There are other ways to do what we have done here inclusing using a completely different serializer. .NET 3.0, for example, comes with a new serializer called the DataContractSerializer which works with the DataContract and DataMember attributes, but they are not as flexible as the route we went here. There is also a type in the .NET 3.0 System.ServiceModel.Channels namespace called Message which allows you to create different types of SOAP messages and that is another route you can take to create SOAP messages. However, the most well documented method is by far the method I presented to you here.