We left off at the end of my last article being able to grab and print a GET request from a web page. There were a few issues with the way we did things however. Firstly, we were reading in a buffer the size of the maximum possible buffer size, so when we were printing out our request it was followed by a ton of whitespace. In order to fix this I tried to trim the string for whitespace, this did not work.

It turns out that if you take a look at the ASCII characters of the string you get in the byte buffer you will notice that there are a ton of 0’s at the end of the string. This corresponds to the null character. When Receive() grabs data from the socket, it will fill up the remainder of the array with null characters, so to trim the string we simply use

    convertedData = convertedData.Trim('\0');

Our next issue is how to parse the request we get. Basically all we want to do is find out what page/resource they are looking for and then check to see if that page exists, if it does respond with it, otherwise respond with nothing. However, there is a special case we have to look for. When a person browses to a site like www.blah.com the first get request is always going to look like “GET / http/1.1″, followed by the rest of the GET request. What this means is that the browser is looking for the “/” file or resource. By convention this “/” is always index.html (unless otherwise defined) so we have to check for this in our server.

So I created a method to parse our get that just looks at the first line of the request and returns the proper resource.

private static byte[] ParseGet(String getRequest)
{
    // Set the directory to server from to desktop\sites.
    var path = Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory) + "\\sites";
    var words = getRequest.Split();
 
    if (!words[0].Equals("GET"))
    {
        return new byte[0];
    }
 
    if (!words[1].Equals("/"))
    {
        return !File.Exists(path + words[1]) ? new byte[0] : File.ReadAllBytes(path + words[1]);
    }
    else
    {
        var bytes = File.ReadAllBytes(path+"\\index.html");
        return bytes;
    }
}

Our final problem is that currently, we can only accept one request at a time. We listen on the port until a request is made, then we process that request until completion, then we listen again. We want to be able to handle multiple requests concurrently, so we must thread our request procession. To do this, we move our request processing out into its own method:

private static void ProcessRequest(Object o)
{
    var acceptedSocket = (Socket) o;
    // Read request.
    var data = new byte[acceptedSocket.ReceiveBufferSize];
    acceptedSocket.Receive(data);
    var convertedData = Encoding.ASCII.GetString(data);
 
    // Receive() fills the buffer with nulls after the data has been read.
    convertedData = convertedData.Trim('\0');
    char[] delimiters = { '\r', '\n' };
 
    var lines = convertedData.Split(delimiters, StringSplitOptions.RemoveEmptyEntries);
    Console.WriteLine("{0}", convertedData);
    var outData = new byte[0];
 
    // Write response.
 
    // Make sure there is data to parse.
    if (0 != lines.Length)
    {
        outData = ParseGet(lines[0]);
    }
 
    acceptedSocket.Send(outData);
    acceptedSocket.Close();
}

You will notice from the above code that ProcessRequest takes an Object instead of a socket. This is too bad, it would be nice if you could do that
ParameterizedThreadStart requires a method that takes an object and I could find no way around this.

Our final run method now looks like this:

public void Run()
{
    // Set up socket for listening.
    var hostIp = (Dns.GetHostAddresses("127.0.0.1")[0]);
    Console.WriteLine("Listening on {0}:{1}", hostIp, Port);
    var endPoint = new IPEndPoint(hostIp, Port);
    TcpSocket.Bind(endPoint);
 
    while (true)
    {
        TcpSocket.Listen(BackLog);
        var processThread = new Thread(new ParameterizedThreadStart(ProcessRequest));
        processThread.Start(TcpSocket.Accept());
    }
}

If you create a directory on your desktop, place an index.html in there and then navigate to http://127.0.0.1 it will now
serve your very own index.html. Here is mine:

If you would like to check out the code, it is available here: KevServer

It has been quite a while since I programmed something productive on my own time. I have been really busy lately at work as we are finishing up our major project from the last 6 months. Hopefully at some point I will write a blog post regarding that project as it is really interesting. I have also been reading up on some languages I have toyed around with (Javascript, Objective-C, C#, Clojure). I also haven’t updated this blog in months. Because of these facts I have decided that it is time to stop reading and start doing, so I have come up with a fun little project for myself.

Yesterday I was in my idea chamber (shower) and I came up with a fun little idea that shouldn’t be too hard and will teach me a bit about two subjects I would like to know more about. I have decided to write my own little web server from scratch using C#. This will teach me more about C# and its libraries as well a lot more about the HTTP/1.1 protocol.

Now I know that C# comes with some nice libraries that makes writing things like web servers pretty easy/trivial, so I have decided not to use any of them. I will not be using HTTPWebRequest or HTTPWebResponse. I will be doing this using straight sockets and a design from my own head, as well as a lot of help from MSDN and the HTTP spec.

The second part of this journey will be to blog about my exploits so you guys can critique, offer suggestions, bash my designs or praise my awesomeness. This way I can actually tackle three birds with one stone as in addition to learning C# and HTTP I will practice my writing by blogging about it. So without further ado I will take you through my initial steps. If anyone is interested all my code is available at Google Code and will be updated as I go.

The first step I set out to achieve is to create a socket that will listen to a single request on a specific port, pretty basic. However, being a C# beginner I had no idea what to do, so I did what any competent programmer would do, I Googled “C# sockets” as well as “Http” and some other related stuff. I quickly came upon the MSDN article on the Socket class located in System.Net.Sockets.

As an aside, I have to say, this is the first time I have ever used MSDN and it is pretty damn good. Not only does it give you the full API for whatever you are looking for, it also provides you with code snippets on how to use it! INVALUABLE! Much much better resource than the JavaDocs I am used to. I guess that is what you get when you have professional support for your API… I would probably still prefer the API to be open source but I digress. Anyway, I quickly found what I was looking for (Socket) and set to work. After about an hour of mucking about I finally achieved some code that actually works and displayed a single get request from my browser. Behold my work of art:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Net.Sockets;
 
namespace KevServer
{
    class Server
    {
        private int Port { get; set; }
        private Socket TcpSocket { get; set; }
        private int BackLog { get; set; }
 
        public Server()
        {
            TcpSocket = new Socket(AddressFamily.InterNetwork,
                SocketType.Stream, ProtocolType.Tcp);
            Port = 8080;
            BackLog = 5;
        }
 
        public void Run()
        {
            var hostIp = (Dns.GetHostAddresses("127.0.0.1")[0]);
            Console.WriteLine("Listening on {0}:{1}", hostIp, Port);
            var endPoint = new IPEndPoint(hostIp, Port);
            TcpSocket.Bind(endPoint);
            TcpSocket.Listen(BackLog);
            var acceptedSocket = TcpSocket.Accept();
 
            var data = new byte[acceptedSocket.ReceiveBufferSize];
            acceptedSocket.Receive(data, 0,
                acceptedSocket.ReceiveBufferSize, SocketFlags.None);
            var convertedData = System.Text.Encoding.ASCII.GetString(data);
            Console.WriteLine("{0}", convertedData);
            Console.Read();
        }
    }
}

When opening up my browser and pointing it to 127.0.0.1:8080 my console printed the following:

Great Success! First hurdle achieved, next step: parsing the request and returning a page.
More to come, stay tuned!

Last night I was hacking around with Ruby and decided that to teach myself how to program better I would do something interesting for Wezzle. I had the idea of taking the XML markup for our achievements and making some sort of generator using our achievement pseudo-language.

Our achievement language basically amounts to what is known in computing as a Domain Specific Language (DSL). Luckily for me Ruby is a pretty good language for writing DSL’s.

I decided to start with the achievements that I find to be the hardest to write, COLLISION. A COLLISION occurs between 1 or many items. There are two seperate types of collisions which can be defined as either an AND or an INTO relationship. These can be seen in the following two examples:

Example 1:

<achievement name="A Tale of Two Rockets" difficulty="SILVER">
  <description>Fire a rocket into another rocket.</description>
  <rule type="COLLISION" operation="BETWEEN">
    <item type="ROCKET" />
    <item type="ROCKET" />
  </rule>
</achievement>

Example 2:

<achievement name="A Tale of Two Rockets" difficulty="SILVER">
  <description>Fire a rocket into another rocket.</description>
  <rule type="COLLISION" operation="BETWEEN">
    <item type="ROCKET" >
      <item type="ROCKET" />
    </item>
  </rule>
</achievement>

While these two examples may look the same, they mean two different things. The first one can be written as COLLISION BETWEEN ROCKET AND ROCKET simply means that two rockets will be activated in a single turn. The second one however would be written like COLLISION BETWEEN ROCKET INTO ROCKET and means a rocket will be fired and activate another rocket by hitting it.

I have written a generator in ruby that takes an achievement (currently only collisions) in the following form:

BEGIN COLLISION
achievement:Rocketeer:BRONZE
description:Fire a rocket into a rocket and fire a rocket and a rocket.
rule:COLLISION BETWEEN ROCKET INTO ROCKET
rule:COLLISION BETWEEN ROCKET AND ROCKET
END

This is currently defined within the program and will output the properly formatted and tabbed XML which you can then paste into your achievements.xml file. The goal is to have it work for all achievements, and I will keep you updated as to the progress should I choose to continue. As it stands, you can find the current version here although it’s a little bit ugly. Please note, you must have ruby 1.9+ in order to run it.

Computers today have tons of processing power. Who really cares about algorithm design when the computer can perform millions of calculations per second? This was a common thought of mine until recently.

When I learned big O notation and performance estimation in university I often thought to myself that I would never need to remember it, and it was not really useful for programming today. I was way off. Big O is something I have recently started using again and encourage every programmer to use as a way of estimating algorithm performance.

When we started making Wezzle we initially began designing without real thought of efficiency. Our initial implementation was all about design. However, we were very inexperienced when it came to actually designing good quality code. We quickly ran into some severe performance issues, and the Wezzle design still has some major flaws that really aren’t worth fixing at this point. One thing I can say with certainty is that my understanding of good design has improved by a drastic amount from this experience.

Our initial game design included the use of Scalable Vector Graphics (SVG). We chose to use SVG because we liked the way they looked and the fact that window resizing was a non-issue. However, we failed to take into account how performance critical games truly are, and how incredibly slow SVG graphics are.

Our initial game was extremely choppy and virtually unplayable on old machines. We spent a lot of time trying to figure out why that was and how to boost performance. Eventually we decided to bitmap our graphics and noticed a SUBSTANTIAL increase in playability and performance.

I was pretty shocked by this entire scenario. First, I had always assumed that performance today was a non-issue, and second, I never thought that the type of graphics we used would make such a huge difference in terms of performance.

SVG wasn’t even close to all of our performance issues with early versions of Wezzle. We had a bunch of calls that were needlessly being made every game tick (we still find this problem with some of our code) as well as various other design flaws that affected overall performance. We had an inefficient algorithm to refactor the board and just in general a lot of areas that could be streamlined. We also had a bunch of performance issues with label generation (which we fixed with a cacheing scheme) and some troubles with phantom references (yuck!). We made extensive use of the built in profiler in netbeans during our first refactoring phase.

Since then Wezzle has gone through countless refactorings and countless optimizations. Another huge increase in performance came when we switched from Java2D to OpenGL (lwjgl). Perhaps in a later post I will talk about some of the various optimizations we have employed in Wezzle, but that is beyond the scope of this article.

Even with all the performance issues in Wezzle, I thought to myself that games are kind of a unique situation that is very performance critical and lots of calculations are happening, maybe performance is only relevant to gaming.

Again I was wrong. As anyone who does any web development can attest to, large database operations can be TERRIBLY inefficient. I am currently under contract developing some code that loads a large number of records into a grid structure. I am limited by the fact that this grid is only usable to us by passing it XML on the client side, as well as various restrictions about look and feel that were dictated by my client. Perhaps in a future post I talk about the multitude problems with the grid I am using, and there are many, but that would take way too much space to type in this post.

Anyway, my first implementation which contained some optimizations took around 15 minutes to update 500 records from the grid. The records were huge and data validation was being done on every individual cell and since the grid had to be submit as a whole (as an XML string) I was parsing and updating the entire grid.

This was not acceptable and also mildly surprising. Going through my code I have optimized even more. I implemented a dirty cell scheme, marking cells which have been edited as dirty so that on the submit I can only validate cells which have been marked as dirty. This assumes that all data already in the database is valid (not always the case!). I validate pasted in values at paste-time and that’s it (once they pass validation, they are considered good from the client side). Finally, I mark records which have been modified as dirty and even though I still have to submit the entire grid, I now only parse the records that have been marked and skip the rest. This has vastly improved update performance on the average case. It has improved updates in the worst case as well, but not by quite so much, a full submit still takes some time.

I’m going to end this by saying that while yes, performance is an issue, it is also sometimes a non-issue. First and foremost you should code something for clarity. A clear design that is slightly less efficient is always better than a streamlined but cryptic design. It is a mistake to optimize something that doesn’t actually need optimization and a big time waster. A popular saying is “design for clarity first and foremost and use a profiler to determine which areas need to be optimized.” This idiom does make sense and makes a valid point, but I would instead say to design for clarity and performance in conjunction as performance can often be a surprising bottleneck.