Testing socket servers

When you're developing a TCP/IP server application it's easy to test it poorly. In this article we develop a test framework that does most of the hard work for you.

Overview

When you're developing a TCP/IP server application it's easy to test it poorly. It's easy to fire requests into a server, check the responses and assume that's enough. Even if you're testing using the actual production client application you may find that you are failing to fully test the server under heavy load or unusual network conditions. You may be using two or more machines, but your development network probably doesn't cause the kind of packet fragmentation and delays that you might encounter in the wild. Often when testing in the development environment your server only ever receives complete, distinct messages, and this can lead novice developers to assume that this is how it always is. As we pointed out in a previous article, the server developer is always responsible for breaking up the TCP byte stream into protocol specific chunks.

Luckilly it's quite easy to create a generic TCP/IP test framework that you can then plug protocol specific knowledge into. The framework can ensure that the byte stream being sent to the server suffers from fragmented messages, mutliple messages and delays, if appropriate. The protocol specific knowledge can then test the server at the application level whilst the testing framework tests the server at the byte stream processing level.

We developed this testing framework using C# and .Net because we were liked the way that the .Net framework made it easy to develop socket client code using XML for configuration and allowing easy dynamic loading of plug-ins for protocol specific code. First we'll present the design of the framework, then we'll cover how we implemented this design and finally we'll present plug-in test harnesses that allow you to test all of the servers that we've developed in the other articles in this series.

The conversation - an abstract protocol

Client/server systems use many different protocols to communicate. Our challenge, in writing a protocol independent test, tool is to determine the features common to all protocols and write our test tool in terms of those features. By thinking about how protocols are put together in an abstract sense we can provide all of the facilities that any protocol could require. To anchor these abstract concepts in our design we need to name them.

Client/server systems communicate by sending sequences of bytes to each other. Each distinct sequence of bytes is a message. Testing such systems involves sending sequences of messages from peer to peer and ensuring that the reply to the message and the action taken by the peer on receipt of the message is as expected. In the abstract, clients and servers have conversations consisting of one or more exchanges of messages and replies. A test consists of one or more conversaions. Messages are sent from the test tool to the program being tested and replies are sent from the program being tested to the test tool. Some conversations may begin with a reply rather than a message and some messages do not generate a reply. In an exchange of messages both the message and the reply are optional.

Interface based programming

Our requirements state that we should be protocol agnostic and support plug-in code to allow the user of the test tool to develop the protocol specific code. We have abstracted all protocols to the concept of a conversation and now need to move that abstract concept nearer to the code. Since the concrete detail will be protocol specific the best way to capture the abstract concepts is to define interfaces that the test tool can use to manipulate the protocol specific plug-in code in terms of the abstract definition of a protocol. The test tool can then work with any code that implements the appropriate interfaces, it's not concerned about the actual classes involved as long as they implement the correct interfaces in the correct way.

Often people find it difficult to move from the abstract design to the slightly more concrete world of interfaces. One of the best ways to do this is to look at the abstract design and note each of the nouns. These are usually a good choice for interfaces. So:

A test consists of one or more conversaions consisting of one or more exchanges of messages and replies.

So, we have the following potential interfaces:

• Test
• Conversation
• Exchange
• Message
• Reply

We could draw a diagram or two at this point if we felt it would help, something like this, perhaps:

Iterative design

Now that we know the interfaces involved we can start to flesh them out in code. Defining the interfaces will be an iterative thing, it needs to be because we don't know everything about the system we're designing yet. Expect the interfaces to change during the design, expect to add new interfaces and remove ones that don't actually end up adding any value. Also, expect to write code that uses these interfaces whilst they're still in a state of flux, you need to write some code to work out if the interfaces allow you to do what you need to do. Interfaces can and do change whilst you're developing the application, but they should stay fixed once you have released to ensure the compatability of future releases.

A first attempt at our interfaces could look something like this:

   public interface IMessage
   {
   }

   public interface IReply
   {
   }

   public interface IMessageExchange
   {
      IMessage GetMessage();
      IReply GetReply();
   }

   public interface IConversation
   {
      IMessageExchange GetMessageExchange();
   }

   public interface ISocketServerTest
   {
      IConversation GetConversation();
   }

We've made some assumptions in the interfaces shown above. We assume that you can call GetConversation() multiple times and that, eventually, it will return null to indicate that no further conversations should take place. Likewise we assume that we can call GetMessageExchange() multiple times until all messages exchanges for a conversation have been retrieved. Each MessageExchange object consists of an optional message and an optional reply. If the message exists then GetMessage() returns it, if it doesn't then GetMessage() returns null. Likewise with the reply.

Dealing with TCP and UDP servers

Obviously these interfaces are far from complete. To be able to send the message we need to access it as a sequence of bytes. Processing a reply is more complex. As all good TCP/IP programmers know, TCP presents a byte stream interface. It's up to the application to take data from that byte stream and break it into blocks that are meaningful to the application. This is protocol specific, so must be represented as an abstract concept in our interfaces. It would also be good to be able to support both TCP and UDP with one test tool. UDP works in terms of distinct datagrams rather than a stream. This means that TCP message exchanges consist of a message and a response stream to process and UDP message exchanges consist of a message and a response datagram. Revising our interfaces to cater for these changes gives us something like this:

   public interface IMessage
   {
      byte[] GetAsBytes();
   }

   public interface IMessageExchange
   {
      IMessage GetMessage();
   }

   public interface IResponseStreamHandler
   {
      void HandleResponse(IResponseStream responseStream);
   }   

   public interface ITcpMessageExchange : IMessageExchange
   {
      IResponseStreamHandler GetResponseStreamHandler();
   }

   public interface IResponseDatagramHandler
   {
      void HandleResponse(byte[] responseDatagram);
   }

   public interface IUdpMessageExchange : IMessageExchange
   {
      IResponseDatagramHandler GetResponseDatagramHandler();
   }

Notice how we've created an abstraction around the the TCP response stream. IResponseStream is an interface that the test tool can implement and which can provide controlled access to the read side of a .Net NetworkStream object. We could simply pass the NetworkStream object to the plug-in, but then the code that is supposed to only be able to process the reply data stream could write to the NetworkStream, or close it, etc. In this kind of situation it's better to provide a custom interface that allows the user only the access that we want to allow them. This tends to result in more robust software.

The IResponseStream interface might look something like this:

   public interface IResponseStream
   {
      int DefaultTimeoutMillis
      {
         get;
         set;
      }

      string ReadLine();
      string ReadLine(int timeoutMillis);

      int Read(byte[] buffer, int offset, int length);
      int Read(byte[] buffer, int offset, int length, int timeoutMillis);

      byte ReadByte();
      byte ReadByte(int timeoutMillis);

      void Close();
   }

To make it easy for the developer writing the protocol specific plug-ins we provide various methods of accessing the data in the response stream. We also provide timeout functionality on all reads, this ensures that the test can't hang if there's no data sent back. The protocol specific code can specify a timeout for each read operation, and simply set a default timeout for all read operations that don't explicitly supply a timeout. We allow the user to close the response stream, but note that this merely shuts down the receive side of the TCP connection.

Configuring the test

We will need to be able to pass configuration data to the protocol specific plug-in. This configuration can be in the form of an XML document, to isolate the plug-in from how we come by the XML we'll just pass it the root node of the XML that it is to process. This data is completely specific to the plug-in and the test tool doesn't need to understand it at all. Since protocol plug-ins can be for TCP or UDP (or perhaps both) we need to configure the plug-in to use the correct protocol. If the plug-in doesn't support the protocol then it can throw an exception.

Munging the byte stream

Our test tool will attempt to send messages as a non distinct byte stream. This tests that the server is correctly handling the incoming data as a byte stream and not assuming that it will receive distinct messages. There are three ways that a message could arrive:

• A single, complete, message. Just by chance, the message arrives intact and there are no other messages available at the time we call read.
• A fragment of a message. Only the first x bytes have arrived, the rest will arrive later.
• Multiple messages. The results of the read contains more than one message, or a few messages and a message fragment.

To aid us in debugging situations where our server doesn't behave as expected we need to be able to control how the messages are sent from the test tool. Since we may wish to do this at a protocol specific level we will allow the protocol specific plug-in to indicate if it supports fragmented messages and multiple messages. If the plug-in doesn't support these options then the test tool will not use them. When tranmitting multiple messages together as a single block, it may make sense to the protocol that some messages in a conversation could be transmitted together and some can not be. Imagine a protocol which requires a user to log in and then, once logged in, the user can upload a file. The login message will never form part of a multiple message block as further data will never be sent until the reply has been processed. The file upload however could consist of multiple messages that could quite easilly form part of a multiple message block. To allow the plug-in to control which messages can and cannot form part of a multiple message block we can adjust the conversation interface a little. Rather than simply returning messages until all messages in the conversation have been processed it could, instead, return an array of messages that could form part of a multiple message block. Thus the conversation now consists of one or more blocks of message exchanges. If the test tool is operating in multiple message mode then messages in a block may be sent as one transmission. As before, GetMessages() returns null when there are no more blocks in the conversation.

Note that for UDP testing the concept of sending fragmented messages is not relevant and sending multiple messages tests a different aspect of the server. In TCP sending multiple messages together tests how the server determines message boundaries. In UDP message boundaries are fixed and sending multiple messages merely tests how the server handles multiple messages concurrently.

Allowing for multi-threading

Our test tool will simulate multiple concurrent connections by operating in a multi-threaded manner. There seems little point in loading and configuring the plug-in seperately for each thread, but our existing design doesn't provide for requesting "the first" conversation multiple times. By adding the concept of a ConversationCreator we can obtain the ConversationCreator interface once in each thread and then request conversations from it until it returns null.

Some extra hooks

It's possible that some protocols may need exact timing information about when messages were transmitted. To cater for this possibility we can add a MessageTransmitted() method to the IMessage interface. This method will be called as soon after the message is transmitted as is possible, if it's critical that this method is called exactly after the message is transmitted then the plug-in should state that it doesn't support multiple messages, or present the message to the test tool as a single message block. Likewise, it may be useful to know when a Conversation is complete, or when all conversations provided by a ConversationCreation are complete.

The resulting interfaces look something like this, note that we've also added a method to enable the test tool to display some information about the plug-in that it's using.

   public interface IMessage
   {
      byte[] GetAsBytes();

      void MessageTransmitted();
   }

   public interface IMessageExchange
   {
      IMessage GetMessage();
   }

   public interface IResponseStreamHandler
   {
      void HandleResponse(IResponseStream responseStream);
   }   

   public interface ITcpMessageExchange : IMessageExchange
   {
      IResponseStreamHandler GetResponseStreamHandler();
   }

   public interface IResponseDatagramHandler
   {
      void HandleResponse(byte[] responseDatagram);
   }

   public interface IUdpMessageExchange : IMessageExchange
   {
      IResponseDatagramHandler GetResponseDatagramHandler();
   }

   public interface IConversation
   {
      IMessageExchange [] GetMessages();

      void ConversationComplete();
   }

   public interface IConversationCreator
   {
      IConversation GetConversation();

      void Completed();
   }

   public interface ISocketServerTest
   {
      void Initialise(XmlNode parameters, Protocol protocol);
      
      bool AllowFragments();

      bool AllowMultipleMessages();

      void DumpInformation(TextWriter outputStream);

      IConversationCreator GetConversationCreator();
   }

The test tool uses these interfaces as follows:

• Load test configuration. The configuration details which plug-in to use, and other configurable test parameters.
• Load the specified plug-in.
• Locate the entry point object. This object must implement the ISocketServerTest interface.
• Call Initialise() on the ISocketServerTest interface.
• Adjust the test configuration based on the results of calls to AllowFragments() and AllowMultiplMessages().
• Create worker threads and pass the ISocketServerTest interface to each.
• Wait for all worker threads to complete.

In each worker thread:

• Obtain the ConversationCreator.
• While GetConversation() doesn't return null.
• Create the correct type of connection (TCP or UDP) using the conversation, and converse.
• Call Completed()

Conversing via TCP consists of:

• While GetMessages() doesn't return null.
• If configured to send multiple messages, randomly decide how many messages to send, else send 1.
• Calculate the size of buffer required to send all of the message data.
• If configured to send fragmented messages, randomly decide how much of the message data to actually send.
• Create a buffer of the required size.
• For each message that we're going to send, copy as many of its bytes into the buffer as we can.
• Send the full buffer.
• For each message that we sent, call MessageTransmitted().
• For each message that we sent call GetResponseStreamHandler() and if non null, HandleResponse()
• If configured for delays, add a delay and then process more messages

Conversing via UDP consists of:

• While GetMessages() doesn't return null.
• If configured to send multiple messages, randomly decide how many messages to send, else send 1.
• For each message that we're going to send, send it and call MessageTransmitted().
• For each message that we sent call GetResponseStreamHandler() and if non null, HandleResponse()
• If configured for delays, add a delay and then process more messages

Implementation

The implementation of the test tool in C# is fairly straight forward. The .Net framework library provides us with easy to use networking classes in System.Net.Sockets. We use the TcpClient and the UdpClient as the basis of our client connection classes. The multi-threading is equally easy. We use System.Threading.Thread and simply have to provide the function that we wish our threads to execute. The main thread waits for the worker threads to complete using a ManualResetEvent. The threads decrement a counter using the Interlocked class. Use of the Interlocked class means that each thread is guaranteed to decrement the counter as a single atomic operation. Without it, two threads accessing the counter at the same time could lead to unexpected behaviour. The thread that moves the counter to 0 sets the event and the main thread shuts down. Configuration is a breeze using the classes from System.Xml to load an XML configuration document and walk the nodes of the tree.

Dynamically loading plug-ins

One of our major requirements of the tool is that the protocol specific work can be done by plug-ins that can be loaded and configured per test run. We took great care in the design of the interfaces to make it as easy as possible to create the plug-ins. Each plug-in is developed as a stand alone DLL assembly. The entry-point to a plug-in is an object that implements ISocketServerTest. Plug-ins can contain multiple entry-points and the entry-point used by a particular test run is dependent on the configuration file. Loading and configuring the plug-in is made easy using the classes in System.Reflection.

      Assembly assembly = Assembly.LoadFrom(Y); // path to assembly dll

      Object obj = assembly.CreateInstance(X);  // name of object that implements ISocketServerTest

      if (obj == null)
      {
         throw new Exception("Entry point not found");
      }

      ISocketServerTest serverTest = (ISocketServerTest)obj;

      serverTest.Initialise(config.TestParameters, config.Protocol);

      return serverTest;
   }

We were pleasantly surprised at how easy it was to translate our design into code using C# and .Net and the only gripe we have is the lack of true multiple inheritance. It would have been nice to be able to have the interfaces, or a class derived from the interfaces, provide default implementations for some methods, such as the MessageTransmitted() and ConversationComplete() methods and effectively give the user of the interface the option to implement if they require behaviour other than the default.

Configuring the test tool

The tool is configured using XML files. The name of the configuration file to use is passed to the tool on the command line. A configuration file might look something like this:

<xml version="1.0" encoding="utf-8" ?>
<SocketServerTest>
   <Threads>
      <Number>350</Number>
      <Batch>10</Batch>
      <DelayMillis>1000</DelayMillis>
   </Threads>
   <Protocol>TCP</Protocol>
   <Messages>
      <Fragments>true</Fragments>
      <Multiple>true</Multiple>
      <DelayMillis>1000</DelayMillis>
   <Messages>
   <RandomSeed>112</RandomSeed>
   <Test>
      <Host>localhost</Host>
      <Port>5001</Port>
      <Name>LargePacketEchoServerTest.dll</Name>
      <EntryPoint>ServerTest</EntryPoint>
      <Parameters>
         <Conversations>10</Conversations>
         <Blocks>10</Blocks>
         <Messages>10</Messages>
         <MessageSize>8000</MessageSize>
         <ShowDebug>false</ShowDebug>
      </Parameters>
   </Test>
</SocketServerTest>

As you can see, we can configure the number of worker threads to start, and start these threads in batches with a delay between each batch, if required. We can configure the protocol, TCP or UDP, and whether the test tool should send message fragments and multiple messages and if it should delay between message blocks. We can also specify the seed of the random number generator, this allows us to be able to reply test runs exactly even though they contain a 'random' element. This is especially useful in tracking down bugs which only show up under certain message fragmentation situations.

The elements within the specify the details of the test itself. The host and port to connect to define the server that will be tested. The node details the plug-in protocol test assembly to use and the node details the name of the class within the test assembly that is the entry point for the test. The contents of the node is treated as opaque data by the test tool and is simply passed to the plug-in during initialisation.

Structuring the code

We decided to place all of the interfaces in their own assembly. This allows both the plug-ins and the test tool to reference them without requiring the plug-ins to reference the tool itself. The code for the test tool lives in the JetByte.SocketServerTest namespace and the interfaces in the JetByte.SocketServerTest.Interfaces namespace.

Writing a plug-in

Now that we have a test tool that allows for plugging in protocol specific tests we need to write some plug-ins. The packet-based echo server that we developed in an earlier article provides quite a good test of the test tool and its interfaces. The packet echo server works with a very simple protocol. Upon connection, it sends a welcome message. From then on it expects to receive a single byte header which contains the length of the data packet (including the one byte header). The server reads the number of bytes specified by the header and then echoes the packet back to the client. We can use the test tool to ensure that the server obeys the protocol correctly by sending fragmented messages and multiple message blocks, we can check that the server operates correctly by comparing every byte of data sent with every byte received.

The first thing to do when writing a protocol test plug-in is to create a new dll assembly. Select File New, C# class library (although of course you could use any .Net language to create the plug-in). We need to add a reference to the JetByte.SocketServerTest.Interfaces assembly so that we can refer to the interfaces that are defined within it.

The entry point into the protocol specific plug-in is any class that derives from the ISocketServerTest interface. A plug-in can have multiple entry points and the one used by a particular test is determined by the configuration file. We could place the entry point in a namespace, but configuration is easier if we don't, so we'll simply define a class at global scope. Our entry point may look something like this:

   public class ServerTest : ISocketServerTest
   {
      public ServerTest()
      {
      }

      // Implement ISocketServerTest

      public void Initialise(XmlNode parameters, Protocol protocol)
      {
         if (protocol != Protocol.TCP)
         {
            throw new Exception("PacketEchoServerTest only supports TCP");
         }
         config = new PacketEchoServerTest.Configuration(parameters);
      }

      public bool AllowFragments()
      {
         return true;
      }

      public bool AllowMultipleMessages()
      {
         return config.MultipleMessages;
      }

      public void DumpInformation(TextWriter outputStream)
      {
         outputStream.WriteLine("Packet echo server test");
         outputStream.WriteLine("Fragmented Messages: {0}", AllowFragments());
         outputStream.WriteLine("Multiple Messages: {0}", AllowMultipleMessages());
         outputStream.WriteLine("Conversations {0}", config.Conversations);
         outputStream.WriteLine("Blocks per conversation {0}", config.Blocks);
         outputStream.WriteLine("Messages per block {0}", config.Messages);
         outputStream.WriteLine("Message Size {0} bytes", config.MessageSize);
      }

      public IConversationCreator GetConversationCreator()
      {
         return new ConversationCreator(config, Conversation.Type.Simple);
      }

      private PacketEchoServerTest.Configuration config;
   }

The rest of the classes involved will be defined in the PacketEchoServerTest namespace. Notice that we have a Configuration class which deals with the XML document for us and provides validation and a property-based access method for our configuration. The Configuration class uses a base class defined in the JetByte.SocketServerTest.Interfaces namespace that provides helper functions for accessing data from the XML nodes. Notice that we always allow fragmented messages, but are configurable as to whether we allow multiple message blocks. The reason for this is that we can then use this test harness to test both a packet echo server that ensures that packets are echoed in the sequence that they are received and one that doesn't.

Our ConversationCreator is pretty simple. Remember that this is merely an extra layer of indirection so that each test thread can create conversations concurrently. The ConversationCreator is the place to hold any state that needs to be passed between the Conversations that make up a test.

   internal class ConversationCreator : IConversationCreator
   {
      public ConversationCreator(Configuration config)
      {
         this.config = config;
      }
		
      // Implement IConversationCreator

      public IConversation GetConversation()
      {
         if (numConversations++ < config.Conversations)
         {
            return new Conversation(config);
         }

         return null;
      }

      public void Completed()
      {
         // Nothing to do here
      }
		
      private int numConversations = 0;

      private Configuration config;
   }

Our Conversation is fairly straight forward. When GetMessages() is called it generates the appropriate number of MessageExchange objects and returns them as an array. Notice how we deal with the fact that for this particular protocol the Conversation starts with one kind of message and then continues with another kind of message. The ServerSignOn message is a MessageExchange class that doesn't send a Message, it just waits for a "reply". This lets us deal with the fact that the server sends data to us first. When We connect and begin conversing, the first thing we do is wait for the server's sign on "reply". Once we have received the reply we then move to sending and receiving echo data packets. The packets themselves are of varying sizes.

   internal class Conversation : IConversation
   {
      public Conversation(Configuration config)
      {
         this.config = config;
      }

      // Implement IConversation

      public IMessageExchange [] GetMessages()
      {
         IMessageExchange [] messages = null;

         if (blocksSent == 0)
         {
            messages = new IMessageExchange[1];

            messages[0] = new ServerSignOn();
         }
         else if (blocksSent < config.Blocks + 1)
         {
            messages = new IMessageExchange[config.Messages];

            for (int i = 0; i < config.Messages; ++i)
            {
               int messageSize = config.MessageSize + (i % 55);

               messages[i] = new MessageExchange(messageSize);
            }
         }

         blocksSent++;

         return messages;
      }
		
      public void ConversationComplete()
      {
         // Nothing to do here	
      }

      private int blocksSent = 0;

      Configuration config;
   }

The ServerSignOn class is typical of how MessageExchange objects are structured. It implements both the ITcpMessageExchange interface and the IResponseStreamHandler interface. When the GetResponseStreamHandler() method of the ITcpMessageExchange interface is called it simply returns itself. As we'll see in the echo MessageExchange class, it's convenient for the MessageExchange object to implement both the Message and reply interface. Of course the interface definition doesn't mandate this, it just happens to be a convenient implementation strategy which places the response handler and message generator in the same class. As we'll see with the echo MessageEchange object, this is convenient as the response handler associated with a message has easy access to the original message data. The ServerSignOn object simply reads a line from the response stream and discards it. We could do some checking here to validate that the server response is as we expect it, but we don't bother.

   internal class ServerSignOn : ITcpMessageExchange, IResponseStreamHandler
   {
      // Implement IMessageExchange

      public IMessage GetMessage()
      {
         return null;
      }

      public IResponseStreamHandler GetResponseStreamHandler()
      {
         return this;
      }

      // Implement IResponseStreamHandler

      public void HandleResponse(IResponseStream responseStream)
      {
         responseStream.ReadLine();
      }
   }

The echo MessageExchange class is a little more complex. First we create a message of the size specified, the message consists of the single byte header and the data bytes. When the GetMessage() method of IMessageExchange is called we simply return ourselves as we are the Message. We can then return our message bytes when asked by a call to GetAsBytes(). To process the server's respose GetResponseStreamHandler() will be called on our ITcpMessageExchange interface. Again we simply return ourself as we also handle the response. When HandleResponse() is called we read a single byte from the response stream, check that the header specifies the correct number of bytes and then attempt to read the message body. We loop until we have read the correct number of bytes. If any read times out and returns 0 bytes then the Read() method of the IResponseStream interface will throw an exception for us. Finally we compare the contents of the reply with the contents of the original message.

   internal class MessageExchange : ITcpMessageExchange, IMessage, IResponseStreamHandler
   {
      public MessageExchange(int size)
      {
         if (size > 256)
         {
            throw new Exception("Size must be <= 256");
         }

         this.size = size;
       
         message = new byte[size];

         message[0] = (byte)size;

         for (int i = 1; i < size; ++i)
         {
            message[i] = (byte)(i + 1);
         }
      }

      // Implement IMessageExchange

      public IMessage GetMessage()
      {
         return this;
      }

      // Implement ITcpMessageExchange

      public IResponseStreamHandler GetResponseStreamHandler()
      {
         return this;
      }

      // Implement IMessage

      public byte[] GetAsBytes()
      {
         return message;
      }

      public void MessageTransmitted()
      {
         // Nothing to do
      }

      // Implement IResponseStreamHandler

      public void HandleResponse(IResponseStream responseStream)
      {
         // Now, read in size bytes from the stream and compare them to the message

         byte[] response = new byte[size];

         response[0] = responseStream.ReadByte();

         if ((int)response[0] != size)
         {
            throw new Exception("packetSize != size");
         }

         int bytesRead = 1;

         while (bytesRead != size)
         {
            bytesRead += responseStream.Read(response, bytesRead, size - bytesRead);
         }

         for (int i = 0; ok && i < size; ++i)
         {
            if (message[i] != response[i])
            {
               throw new Exception("response != message");
            }
         }
      }      

      private byte[] message;
      private int size;
   }

The way that we've structured the Packet Echo Server's test plug-in bears little relationship to the structure of the interfaces. We've managed to condense the objects required for message exchange into a single object which handles the message, message exchange and response interfaces. For more complex protocols the message exchange object may link back to the conversation object, thus allowing the results of one message exchange to affect future message exchanges - such as passing the number of available mail messages between message exchanges used to test a POP3 server.

Testing

The plug-in that we've just described can be used to test the Packet Echo Server that's available here. The server listens on two ports and has slightly different semantics on each port. On port 5001 the server uses write sequence numbers to ensure that all writes issued occur in the correct sequence (see the article on Read and Write Sequencing for more information). This server can be tested using the a configuration file that specifies that multiple messages should be sent before processing replies. Since write sequencing is being used and only a single Read() is being posted per connection we can guarentee that the server will echo the packets back in the order that they are received off of the wire. Use the PacketEchoServerTest1.xml file to test this server.

The server that listens on port 5002 does not use write sequencing, using PacketEchoServerTest1.xml will test this server with multiple messages turned on and some of the tests should fail. If they don't fail, you're lucky, try running the server on a multi-processor box... Because multiple messages are being received on a single connection in blocks, multiple writes can be outstanding on that connection. Since write sequencing is not being used the problem outlined in the Read and Write Sequencing article can occur and the test harness recognises this as the packets are echoed out of sequence. Use PacketEchoServerTest2.xml to test this behavior.

Running a test against the server on port 5002 with multiple message blocks turned off will work. This is because there can only ever be a single outstanding write request per connection, so there's no way for the writes to get out of sequence. Use PacketEchoServerTest3.xml to test this behavior.

An alternative strategy for dealing with the out of sequence writes is to make the client responsible for reordering the packets. We can test this strategy, and demonstrate the use of multiple entry points within a plug-in, by adding a second protocol test to our plug-in. This involves creating another class that implements ISocketServerTest and that uses a MessageExchange object which can determine which message a response is for. We do this matching by effectively extending the header to two bytes, though the server is oblivious to this. The second byte is a message number which we use when matching responses. When a response arrives we read the length and message number and the data packet, we then lookup the original message by message number before matching the data. This involves the MessageExchange objects being aware of each other and shows how state might be communicated between messages in an entire message block or conversation. Use PacketEchoServerTest4.xml to test this behavior.

Note that the difference in timings, or lack thereof, between running PacketEchoServerTest1.xml and PacketEchoServerTest4.xml should be taken with a pinch of salt. To realistically test the effect that adding write sequencing to a server has on performance you would need test clients which simply read and discarded the responses so that the client loading was comparable between the two server tests, we'll leave that as an exercise for the reader.

Notes about the source

The source archive contains a Visual Studio .Net solution which builds the test interface assembly, the test tool and the protolcol plug-ins to enable testing of all of the servers developed so far. The SocketServerTest project has references to the plug-ins but these references are not required to build the test tool, they are merely for convenience when running the tool in debug. By including a reference to the plug-in dlls the build of the test tool also builds the plug-ins and copies the plug-ins into the test tool's build directory, this makes it easier to run the test tool in the debugger. The archive also contains configuration files for all of the plug-ins.

Download

The following source was built using Visual Studio .Net. The pre-compiled binary release should run on any system which has the .Net framework installed.

 Download SocketServerTest.zip - A C# test tool for socket servers

 Download SocketServerTestSource.zip - Source code to the above

Revision history

• 15th July 2002 - Initial revision.
• 16th July 2002 - Added EchoServer and EchoServerEx tests. EchoServerEx can also test UDP servers.

	1 2 3 4 5 comment on this article read other visitors' comments