Network Byte Order

There are probably perfectly legitimate reasons for the world being this way but I don’t know what it is. In a pretty substantial chunk of the world, when we write numbers, we write them from left to right meaning the portion of the number with the largest value is on the left hand side and typically, we would write from left to right. What this means is that if I write the number 6785, what I mean is six thousand, seven hundred eighty-five. When we are talking about digital communications, however, everything is in the form of a byte. Rather than dealing with the all of the individual bits that a byte would normally be represented as, let’s shorthand it to hexadecimal. One hexadecimal digit pair is how we would represent a single byte. The reason for that is simple. Four bits gives me the values of 0-15 since 2^0 + 2^1 + 2^2 + 2^3 = 1 + 2 + 4 + 8 = 15 as the maximum value for a 4 bit number. Since a byte is 2 pairs of 4 bits and a single hexadecimal digit (values 0-F or 0-15) is 4 bits, 2 hexadecimal digits is a whole byte. Simple, right? 

Let’s move on to writing values, knowing that we are going to be talking about writing out bytes for now and we are going to represent them as hexadecimal. We are going to write out the word hello and it doesn’t much matter where we write this out because we can run into the same problem, no matter what we are doing. The title of this suggests we are talking about writing out to a network interface but we have the same problem on hard disks and in memory. No matter where we have to write bits and bytes, we have to decide how we are going to write it. When we write character values, we have to have a way of converting them to a number. As a result, we use a table lookup. The common table to lookup characters to get a numeric representation is the ASCII table. After doing the lookup, we get the following: 68 65 6C 6C 6F. Again, without getting into the bit level, we have to decide what order we are going to send these in. Do you send the h first or the o first and then follow with the rest of the characters?

Thinking about numbers where the result is more catastrophic if you get it wrong, let’s take a look at a 16-bit value. The value 1348 is 0x0544 in hexadecimal. This is two bytes. If I send the 44 followed by the 05, how does the receiving party interpret that. If I send the 05 before I send the 44, I am sending in big-endian form. The reason for that is that I am sending the most significant data first — the data that has the largest value or is the biggest. If I send the 44 first, I am sending in little-endian form. If the receiving end is used to doing things a different way, I could go from sending the value 1348 but on the receiving end getting 17413. This is a very big difference. The reason is that if I send 05 then 44, which is big-endian, but the other end assumes little-endian, it would view what I sent as 44 05. 

So, which is the right way? Neither, actually. But since little-endian systems need to talk to big-endian systems, there had to be some consensus. As a result, there are two ordering schemes. There is host-order, which is whatever order your particular system architecture uses (Intel uses little-endian, by the way) and then there is network order. Network byte order is a synonym for big-endian, since historically more hardware architectures used the big-endian form of storing data. Of course, these days, far more systems on the network use little-endian simply because of the ubiquity of systems with Intel processors. 

When you are storing data on your own system, it doesn’t much matter how it’s represented because the operating system has to take care of writing and reading so you get the real value at a programmatic layer. When you are trying to interface with values on disk at a raw level, as you might in the case of forensics, you have to be aware of multi-byte values and what architecture the data was written on. If you have a multi-byte value that was written from a little-endian system, you need to remember to reverse the order of the bytes. But only within that value. 

If you are talking to another system, something has to handle the translation from host to network form. Languages that are capable of talking to the network, generally have those functions available. As an example, we can see how the process works in Python, below. 

kilroy@opus:~$ python3

Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 16 2013, 23:39:35) 

[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import socket

>>> socket.ntohl(45)

754974720

>>> socket.htonl(45)

754974720

>>> 

 

The socket class has a number of conversion functions including the two above. In the first example, I am converting from network byte order to a host long. In this case, that means I am converting to a 32-bit value that is little-endian. In the second example, I am converting from a little-endian number to a network long. Again, a long data type is 32 bytes in this case. As a result, you take the value of 45 in bits and then just turn all the bits around and re-calculate back to decimal. You can see the result we get is significantly larger than the value we have put in.