I wanted to kick off my category on Unity (sometimes called Unity3D, though Unity is the proper name) by talking about Serialization, especially in conjunction with Json .NET (the Newtonsoft library) which I've ported to Unity 3.5+ and made to be compatible with iOS and the Web Player. I'll give examples of serialization and deserialization with both Json and Bson (Binary Json).
Serialization and Unity (The Game Engine)
I cannot stress how important serialization is in my day to day programming life (which isn't nearly as fun as game development, but it is just as gratifying). Over and over again I find myself needing to persist data to disk, to a database or to transport the data between an application and a service. Any of these scenarios are valid use cases for Unity developers. This may come in the form of saving game data, transporting multiplayer data over a network, or creating reusable data structures that are loaded when your game starts. There are many different methods of serialization, the most common being XML, JSON and Binary serialization. You could even write your own serialization engine and serialize to your own custom format, as long as you also provide the mechanism to read and deserialize that format as well. The most important thing is that you choose the serialization mechanism that's right for you and your product.
XML serialization was, for a long time, the most popular method of string based serialization. In many cases though, it was a bit overly descriptive and created some issues in deserialization. Different engines handled it differently. Often data types are explicitly declared within the XML. In the .NET world, while most simple primitive types can be serialized and deserialized without any special handling, more complex types such as objects, objects with object based properties, enumerations etc., have to provide method and property attributes that guide the serialization and deserialization engine as to how the type is handled. The omission of these attributes often means that your object likely will not serialize or deserialize properly.
Meet JSON
JSON serialization came along to address many problems. Primarily, it provided a serialization mechanism that was inherently Javascript compatible. It was also much more compact than XML, describing only what it needs to about the data structure. Additionally, the lack of specified typing makes JSON notation more platform agnostic, allowing it to be converted to objects in the technology platform of your choice. In C# for example, you may be converting your JSON to matching classes. This is most often achieved by using reflection to match properties of the target object type to properties found in the JSON and then attempting to parse the value in the JSON string to the target type of the property. In some cases the target type may be a class object in which case the deserializer drills down and repeats the process for that object. A well designed and developed serialization library also allows you flexibility in how you handle things such as circular dependencies.
One of the great things about JSON is that it allows you to serialize one type if you'd like, and deserialize as an entirely different type. Let's see some examples of serialization with the following enum and class definitions:
public enum PlayerType
{
Troll = 1,
Goblin = 2,
Human = 3,
StrangeNewSpecies = 4
}
public class Player
{
public string Name { get; set; }
public int Age { get; set; }
public int Health { get; set; }
public PlayerType PlayerType { get; set; }
}
Now let's say I wanted to create a new instance of that object and Serialize it. My serializer of choice is Json .NET. This is a library with a proven track record. It has been around for quite awhile and is actively maintained. Microsoft now uses it as the default serializer in MVC and for your regular .NET applications it's available via NuGet. The problem with using it with Unity has always been that differences in Unity's implementation cause it to break in the WebPlayer and the precompiled version is not compatible with Unity on iOS and causes AOT errors. For this reason I've created a Unity Port of JSON .NET which is available on the Unity Asset Store. You can find additional information on my own site dedicated to Unity. Since this is a direct port of Json .NET, the implementation between a .NET application and a Unity game is nearly identical. To create an instance of the above class and serialize it in either application you would use the following:
var p = new Player()
{
Name = "Garth The Impossible",
Age = 24,
Health = 100,
PlayerType = PlayerType.StrangeNewSpecies
};
var serializedObject = JsonConvert.SerializeObject(p);
The call to JsonConvert.SerializeObject will convert your object to its string representation. I wanted to use an Enum value because this is the one place where my Unity version of Json .Net differs from the Newtonsoft version. The Unity version will serialize Enum members by their name by default, while the Json .Net library itself defaults to the value representation. However, you can change the behavior of either library. To change the way my library handles Enums at a global level simply call:
JsonSerializerSettings.DefaultEnumSerializationHandling = EnumSerializationHandling.Value;
This tells the serializer to always use the Enum value when serializing (for example, when serializing my object above, the PlayerType property would be serialized to its integer value of 4 instead of the string representation of "StrangeNewSpecies"). Of course, with either serializer, you can also specify your converters as attributes, or pass then in when serializing as follows:
var serializedObject = JsonConvert.SerializeObject(p, new Newtonsoft.Json.Converters.StringEnumConverter());
This will tell the serializer to always use the string name of the enum member when serializing this object. The method you choose is up to you. Nothing special needs to be done to deserialize an object when you use the name instead of the value for enums. Json .Net (both the original and my port) will handle and properly deserialize both a string and a number to the target enum type. The benefit of using the name instead of the value is that it makes your serialized Json easier to read, while using the integer value makes it more compact. If you use the integer value and later you change the name of your Enum to say "StrangeOLDSpecies", the deserialization will still work properly and will deserialize the integer value of 4 to the new Enum member. However, if you change the integer value, it will not deserialize properly. In this case, had you used name, the deserialization would still follow. Use your best judgement.
Now that we've serialized our class instance, the serialized text will actually look like this:
{"Name":"Garth The Impossible","Age":24,"Health":100,"PlayerType":"StrangeNewSpecies"}
Now let's say I'm sending this player information to a website. The website stores the player information in a database. The Player class for the website will be almost identical to the Player class that I'm using in my Unity game except that I want to store "PlayerType" as a string, so my new class will look like this:
public class DBPlayer
{
public string Name { get; set; }
public string Name { get; set; }
public int Age { get; set; }
public int Health { get; set; }
//Here PlayerType is a string instead of an Enum
public string PlayerType { get; set; }
}
Since Json doesn't care as much about data types, I can simply use the following code in my Web Application:
var player = JsonConvert.DeserializeObject<DBPlayer>(serializedObject);
Since we serialized in Unity using the PlayerType's string name, this will deserialize properly and the value of PlayerType will be "StrangeNewSpecies". This opens up some nice scenarios for Unity developers. You can choose to treat your web services as APIs. In this instance, you would likely create your models (classes, such as Player in this instance) in a separate project and target .NET 3.5 (to be compatible with Unity 3.5+) and reference that library in your web project. With this approach you could drop the same DLL into your Unity project to ensure that your models are exactly the same between the web and the Unity application. However, there may be cases where you want your models to contain some logic specific to your game project or have different rules as to how data for your properties is managed. In this case, you'll likely create the same, or similar models on your service side without the extra logic. Or perhaps you want additional logic in your service models to ensure data is not spoofed or tampered with. With the platform agnostic JSON serialization, we can simply worry about supplying the values from one end to the other and allow the receiving application to apply the logic necessary to validate and process that data in the appropriate way.
Before moving on to the next section I also want to quickly note that it is possible to serialize one member type and deserialize to another. Primitive types do this easily, as do enumerations. As an example, let's say I define a different enum on my web service as follows:
public enum PlayerClass
{
A = 1,
B = 2,
C = 3,
D = 4
}
Now, in my DBPlayer model that I created on my service side, I can change the PlayerType member to type "PlayerClass" instead of String. In my Unity application, I would change the serialization setting for enumerations so that it serializes by value instead of name. So, for the PlayerType property, the serialized value will now be 4 instead of "StrangeNewSpecies". Upon deserialization, the DBPlayer class instance will have a value of "PlayerClass.D" for the PlayerType property because it contains the same integer value as the "PlayerType.StrangeNewSpecies" we used on our Unity side. This can be useful in multiple scenarios, such as presenting data to a more friendly way in your Unity application while handling it differently in your service application, or if you're storing a string value, allowing you to simply store "D" in your database rather than the entire "StrangeNewSpecies" string. When transporting back to Unity, so long as you serialize your DBPlayer class using the enumerations "value" rather than "name" (remember, value is the default for the standard JSON .NET implementation), you will be able to deserialize it appropriately on the client side.
Meet BSON (Binary JSON Serialization)
JSON .NET (both the original and my port) support BSON serialization in addition to JSON serialization. If you're not familiar with BSON, a quick Bing or Google search can get you up to speed. Essentially, BSON is a binary version of JSON, but with some additional features. BSON supports the same data types that JSON supports but also allows you to encode binary data directly. I will cover encoding additional binary data in a future article, but as a use case, let's say you wanted players of your game to capture a screenshot and push it to a profile they have created on your server. With BSON, you could create a class object that held all of the information you'd need about the player's account and you could also include the binary data of the screenshot and send it all to your server. Your server could deserialize it and store the screenshot locally.
BSON has some additional benefits as well. Performance seems to vary between different systems that implement BSON. For example, there are many articles referencing BSONs slower performance on MongoDB vs JSON. However, this has more to do with the actual implementation than it does the format itself. While I haven't run any real world benchmarks, JSON .NET claims faster serialization and deserialization. Additionally, BSON is more compact to store and more tamper resistant as it's stored as raw byte data rather than plain text. BSON does take a bit more code to perform the serialization as it works with streams. Here is an example of serializing our original Player class using BSON instead of JSON:
var p = new Player()
{
Name = "Garth The Impossible",
Age = 24,
Health = 100,
PlayerType = PlayerType.StrangeNewSpecies
};
using (var stream = new MemoryStream())
{
var serializer = new JsonSerializer();
var writer = new BsonWriter(stream);
serializer.Serialize(writer, p);
}
At this point, the object has been serialized to the MemoryStream (stream). You can write that stream to a file or pull it as a byte array (using stream.ToArray()) and push it to a web service. This is also handy for serializing game data that you want to use later, such as level data, map data, behavior or pathfinding. If you want to convert the serialized data to a string value that you can visualize you can do so by using:
var serializedText = BitConverter.ToString(stream.ToArray());
The result will look something like this:
49-00-00-00-02-4E-61-6D-65-00-15-00-00-00-47-61-72-74-68-20-54-68-65-20-49-6D-70-6F-73-73-69-62-6C-65-00-10-41-67-65-00-18-00-00-00-10-48-65-61-6C-74-68-00-64-00-00-00-10-50-6C-61-79-65-72-54-79-70-65-00-04-00-00-00-00
Deserializing your BSON encoded data is similar. You would simply read your binary data into a stream. We're going to use the example from the JSON .NET website that resets our existing stream and deserializes it. Here is an example that shows the object being serialized to BSON and then immediately deserialized back into another object to show you how it's done:
var p = new Player()
{
Name = "Garth The Impossible",
Age = 24,
Health = 100,
PlayerType = PlayerType.StrangeNewSpecies
};
Player newPlayer;
using (var stream = new MemoryStream())
{
var serializer = new JsonSerializer();
var writer = new BsonWriter(stream);
serializer.Serialize(writer, p);
//Reset the stream back to the beginning
stream.Seek(0, SeekOrigin.Begin);
//Create the BSON Reader and Deserialize
var reader = new BsonReader(stream);
newPlayer = serializer.Deserialize<Player>(reader);
}
Conclusion
As you move forward, you will find that proper and efficient serialization is essential to serious game development. This especially holds true as you start thinking about developing connected games that require interaction with the outside world via channels such as web services. In my next article I will give an introduction into hooking Unity up to an ASP .NET MVC website and communicating data back and forth.