canopas-logo

How to do Data Serialization and Deserialization in Golang

Covered popular formats such as JSON, XML, Gob, Protocol Buffers (protobuf), MessagePack, and YAML
May 30 2024 · 6 min read
dharti-r image
Dharti Ramoliya
Table of contents

Background

Serialization and deserialization are essential processes in modern software development, enabling data to be efficiently transported and stored. 

In Go (Golang), these processes are straightforward. Thanks to its robust standard library and the availability of third-party packages. 

In this blog post, we will delve into the basics of serialization and deserialization in Go, covering popular formats such as JSON, XML, Gob, Protocol Buffers (protobuf), MessagePack, and YAML.

So, Let’s get started! 🎯

Stop the habit of wishful thinking and start the habit of thoughtful wishes with Justly.

What are Serialization and Deserialization?

Serialization is the process of converting an in-memory data structure into a format that can be easily saved to a file or transmitted over a network.

Deserialization is the reverse process, where the serialized data is converted back into an in-memory data structure.

Common formats for serialization include JSON, XML, binary formats like Gob, Protocol Buffers, MessagePack, and YAML.

Each format has its own use cases and advantages:

  • JSON: Great for web APIs and interoperability.
  • XML: Useful when a strict schema is needed.
  • Gob: Ideal for Go-specific, high-performance serialization.
  • Protocol Buffers: Efficient and versatile, especially for large-scale systems.
  • MessagePack: Compact and efficient for bandwidth-sensitive applications.
  • YAML: Human-readable, often used for configuration files.

Choosing the right format depends on the use case, such as human readability, performance, and compatibility requirements.

JSON Serialization and Deserialization

JSON (JavaScript Object Notation) is a lightweight data-interchange format that’s easy for humans to read and write, and easy for machines to parse and generate. 

In Go, the encoding/json package provides methods to work with JSON.

Example

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name  string `json:"name"`
	Age   int    `json:"age"`
	Email string `json:"email"`
}

func main() {
	// Serialization
	p := Person{Name: "Alice", Age: 30, Email: "alice@example.com"}

	data, err := json.Marshal(p)
	if err != nil {
		fmt.Println("Error serializing:", err)
		return
	}

	fmt.Println("Serialized JSON:", string(data))

	// Deserialization
	var p2 Person

	err = json.Unmarshal(data, &p2)
	if err != nil {
		fmt.Println("Error deserializing:", err)
		return
	}

	fmt.Printf("Deserialized struct: %+v\n", p2)
}

Output

Serialized JSON: {"name":"Alice","age":30,"email":"alice@example.com"}
Deserialized struct: {Name:Alice Age:30 Email:alice@example.com}

In this example, json.Marshal is used for serialization and json.Unmarshal is used for deserialization.

Advanced Tips

  1. Custom JSON Field Names: Use struct tags to customize JSON field names.
  2. Omitting Empty Fields: Add omitempty to struct tags to omit empty fields from the serialized output.
  3. Error Handling: Always handle errors during serialization and deserialization to avoid data corruption or crashes.
  4. Streaming: Use json.Encoder and json.Decoder for working with streams, such as reading from or writing to files.

XML Serialization and Deserialization

XML (Extensible Markup Language) is another format used for serialization, particularly in applications that require strict data schemas. 

Go provides the encoding/xml package for working with XML.

Example

package main

import (
	"encoding/xml"
	"fmt"
)

type Person struct {
	XMLName xml.Name `xml:"person"`
	Name    string   `xml:"name"`
	Age     int      `xml:"age"`
	Email   string   `xml:"email"`
}

func main() {
	// Serialization
	p := Person{Name: "Bob", Age: 25, Email: "bob@example.com"}

	data, err := xml.MarshalIndent(p, "", "  ")
	if err != nil {
		fmt.Println("Error serializing:", err)
		return
	}

	fmt.Println("Serialized XML:\n", string(data))

	// Deserialization
	var p2 Person

	err = xml.Unmarshal(data, &p2)
	if err != nil {
		fmt.Println("Error deserializing:", err)
		return
	}

	fmt.Printf("Deserialized struct: %+v\n", p2)
}

Output

Serialized XML:
 <person>
  <name>Bob</name>
  <age>25</age>
  <email>bob@example.com</email>
</person>
Deserialized struct: {XMLName:{Space: Local:person} Name:Bob Age:25 Email:bob@example.com}

In this example, xml.MarshalIndent is used for serialization with indentation for readability and xml.Unmarshal is used for deserialization.

Advanced Tips

  1. Customizing Element Names: Use struct tags to customize XML element names.
  2. Attributes: Use struct tags to serialize struct fields as XML attributes.
  3. Namespaces: Manage XML namespaces using the XMLName field.
  4. Error Handling: Ensure proper error handling for robustness.

Gob Serialization and Deserialization

Gob is a binary serialization format specific to Go, which is efficient and compact.

The encoding/gob package facilitates working with Gob.

Example

package main

import (
	"bytes"
	"encoding/gob"
	"fmt"
)

type Person struct {
	Name  string
	Age   int
	Email string
}

func main() {
	// Serialization
	p := Person{Name: "Charlie", Age: 28, Email: "charlie@example.com"}

	var buffer bytes.Buffer
	encoder := gob.NewEncoder(&buffer)

	err := encoder.Encode(p)
	if err != nil {
		fmt.Println("Error serializing:", err)
		return
	}

	fmt.Println("Serialized Gob:", buffer.Bytes())

	// Deserialization
	var p2 Person

	decoder := gob.NewDecoder(&buffer)
	err = decoder.Decode(&p2)
	if err != nil {
		fmt.Println("Error deserializing:", err)
		return
	}

	fmt.Printf("Deserialized struct: %+v\n", p2)
}

Output

Serialized Gob: [17 255 129 3 1 1 6 80 101 114 115 111 110 1 255 130 0 1 1 4 78 97 109 101 1 12 0 1 3 65 103 101 1 4 0 1 5 69 109 97 105 108 1 12 0 0 0 26 255 130 1 6 67 104 97 114 108 105 101 1 28 1 15 99 104 97 114 108 105 101 64 101 120 97 109 112 108 101 46 99 111 109 0]
Deserialized struct: {Name:Charlie Age:28 Email:charlie@example.com}

In this example, gob.NewEncoder and Encode are used for serialization and gob.NewDecoder and Decode are used for deserialization.

Advanced Tips

  1. Efficiency: Gob is more efficient than JSON and XML in terms of size and speed.
  2. Type Safety: Gob ensures type safety during encoding and decoding.
  3. Streaming: Use encoders and decoders with network connections or files for large data sets.
  4. Registering Types: Register complex or interface types with gob.Register to avoid encoding errors.

Protocol Buffers (Protobuf)

Protocol Buffers, developed by Google, is a language-neutral and platform-neutral mechanism for serializing structured data. They are more efficient than JSON and XML, especially in terms of size and speed.

Example

First, you need to define your data structure in a .proto file:

syntax = "proto3";

message Person {
  string name = 1;
  int32 age = 2;
  string email = 3;
}

Next, compile the .proto file using the protoc compiler to generate Go code.

Here’s how you can serialize and deserialize using Protobuf:

package main

import (
	"fmt"
	"log"

	"github.com/golang/protobuf/proto"
	"your_project_path/personpb" // Import the generated protobuf package
)

func main() {
	// Create a new person
	p := &personpb.Person{
		Name:  "Dave",
		Age:   35,
		Email: "dave@example.com",
	}

	// Serialization
	data, err := proto.Marshal(p)
	if err != nil {
		log.Fatal("Marshaling error: ", err)
	}

	fmt.Println("Serialized Protobuf:", data)

	// Deserialization
	var p2 personpb.Person
	err = proto.Unmarshal(data, &p2)
	if err != nil {
		log.Fatal("Unmarshaling error: ", err)
	}

	fmt.Printf("Deserialized struct: %+v\n", p2)
}

Output

Serialized Protobuf: [10 4 68 97 118 101 16 35 26 14 100 97 118 101 64 101 120 97 109 112 108 101 46 99 111 109]
Deserialized struct: {Name:Dave Age:35 Email:dave@example.com}

Advanced Tips

  1. Backward Compatibility: Protobuf supports backward and forward compatibility.
  2. Schema Evolution: You can add new fields to your messages without breaking existing code.
  3. Performance: Protobuf is highly efficient in both size and speed compared to text formats.
  4. Third-party Libraries: Use libraries like protoc-gen-go for easy integration in Go projects.

MessagePack Serialization and Deserialization

MessagePack is an efficient binary serialization format that is more compact than JSON. It is useful for scenarios where bandwidth is a concern.

Example

First, install the MessagePack library:

go get -u github.com/vmihailenco/msgpack/v5

Here’s an example of how to use MessagePack:

package main

import (
	"fmt"
	"log"

	"github.com/vmihailenco/msgpack/v5"
)

type Person struct {
	Name  string
	Age   int
	Email string
}

func main() {
	// Create a new person
	p := Person{Name: "Eve", Age: 22, Email: "eve@example.com"}

	// Serialization
	data, err := msgpack.Marshal(p)
	if err != nil {
		log.Fatal("Marshaling error: ", err)
	}

	fmt.Println("Serialized MessagePack:", data)

	// Deserialization
	var p2 Person
	err = msgpack.Unmarshal(data, &p2)
	if err != nil {
		log.Fatal("Unmarshaling error: ", err)
	}

	fmt.Printf("Deserialized struct: %+v\n", p2)
}

Output

Serialized MessagePack: [129 164 78 97 109 101 163 69 118 101 162 65 103 101 22 165 69 109 97 105 108 171 101 118 101 64 101 120 97 109 112 108 101 46 99 111 109]
Deserialized struct: {Name:Eve Age:22 Email:eve@example.com}

Advanced Tips

  1. Efficiency: MessagePack is very efficient in both speed and size.
  2. Human Readability: Although binary, tools exist to convert MessagePack to JSON for readability.
  3. Compatibility: Ensure the client and server both support MessagePack.

YAML Serialization and Deserialization

YAML (YAML Ain’t Markup Language) is a human-readable data serialization format that is often used for configuration files.

Example

First, install the YAML package:

go get -u gopkg.in/yaml.v2

Here’s an example of how to use YAML:

package main

import (
	"fmt"
	"log"

	"gopkg.in/yaml.v2"
)

type Person struct {
	Name  string `yaml:"name"`
	Age   int    `yaml:"age"`
	Email string `yaml:"email"`
}

func main() {
	// Create a new person
	p := Person{Name: "Frank", Age: 40, Email: "frank@example.com"}

	// Serialization
	data, err := yaml.Marshal(p)
	if err != nil {
		log.Fatal("Marshaling error: ", err)
	}

	fmt.Println("Serialized YAML:\n", string(data))

	// Deserialization
	var p2 Person
	err = yaml.Unmarshal(data, &p2)
	if err != nil {
		log.Fatal("Unmarshaling error: ", err)
	}

	fmt.Printf("Deserialized struct: %+v\n", p2)
}

Output

Serialized YAML:
name: Frank
age: 40
email: frank@example.com

Deserialized struct: {Name:Frank Age:40 Email:frank@example.com}

Advanced Tips

  1. Configuration Files: YAML is commonly used for configuration files due to its readability.
  2. Indentation: Be careful with indentation as it is significant in YAML.
  3. Complex Data Structures: YAML supports complex data structures, including nested maps and lists.

That’s it for Today!!

Feel free to experiment with these examples and adjust them to fit your specific requirements.

Happy coding! 👋

Conclusion

In this blog post, we covered the basics of serialization and deserialization in Go using several formats: JSON, XML, Gob, Protocol Buffers, MessagePack, and YAML.

By leveraging Go’s standard libraries and third-party packages, you can efficiently serialize and deserialize data to suit your application’s needs.

Additionally, remember to handle errors properly and consider the specific requirements of your application when choosing a serialization format.

Similar Articles

Serialization
Deserialization
Golang
JSON serialization
Data serialization
dharti-r image
Dharti Ramoliya
Web developer at canopas | Eager to Elevate Your Web Experience
dharti-r image
Dharti Ramoliya
Web developer at canopas | Eager to Elevate Your Web Experience

Whether you need...

  • *
    High-performing mobile apps
  • *
    Bulletproof cloud solutions
  • *
    Custom solutions for your business.
Bring us your toughest challenge and we'll show you the path to a sleek solution.
Talk To Our Experts
footer
Subscribe Here!
Follow us on
2024 Canopas Software LLP. All rights reserved.