Unmarshalling JSON with Null Boolean Values in Go

Background

Whilst working on Monitoring Agent¹, which is written in Go, I added validation for the configuration file read by the Monitoring Agent process when it is started. The configuration file is formatted in JSON and unmarshalled to a struct type.

When a mandatory configuration item is added and an administrator upgrades Monitoring Agent, they should see a useful error message about the missing configuration item instead of having the process run with unpredictable or insecure results.

The program therefore needs to ensure certain values have been set. In Go, when storage is allocated for a variable without explicit initialization, the variable is given the zero value for it’s type. For a bool type, this is 0 which is also false.

That means when configuration data is read from a file and unmarshalled, the code needs to work out whether a bool value is either false or omitted.

Solution 1: Use Pointers

Instead of defining bool types for members of the struct that JSON is unmarshaled into, we could use pointers (*bool) instead.

When a value is not provided to json.Unmarshal() for a member of the struct, the pointer value will be nil. Thus mystruct.var == nil can be used to see the value was included in the JSON.

type JsonInput struct {
	IsWrittenInGo *bool
	IsGoodCode    *bool
	DoWeLikeGo    *bool
}

func main() {
	jsonData := []byte(`
{
    "IsWrittenInGo": true,
    "IsGoodCode": false
}`)

	var input = JsonInput{}

	err := json.Unmarshal(jsonData, &input)
	if err != nil {
		fmt.Print(err)
	}

	// Prints main.JsonInput{IsWrittenInGo:(*bool)(0xc000018150), IsGoodCode:(*bool)(0xc000018151), DoWeLikeGo:(*bool)(nil)}
	fmt.Printf("%#v\n", input)

	if *input.IsWrittenInGo {
		fmt.Println("Yes it is written in Go")
	}

	if input.IsGoodCode != nil {
		fmt.Print("The AI has determined whether this is good code. The verdict: ")

		if *input.IsGoodCode {
			fmt.Println("Good Code.")
		} else {
			fmt.Println("Bad Code.")
		}
	}

	if input.DoWeLikeGo == nil {
		fmt.Println("The AI has not yet determined if we like Go")
	}
}

Go Playground example.

The downside of this solution is that interrogating or changing the value means dereferencing or indirecting the variable, for example *input.IsGoodCode = false, as opposed to input.IsGoodCode = false.

Solution 2: Use a Custom Struct Type

I don’t like smearing code with unnecessary punctuation. Fortunately, there is another way. Instead of relying on pointers, a struct type can be used:

type NullBool struct {
	IsTrue   bool
	HasValue bool // true if bool is not null
}

This is a struct with two members. IsTrue is the actual value of the bool. HasValue indicates whether or not IsTrue has been explicitly set.

Instead of input.IsGoodCode != nil and *input.IsGoodCode = true we can write input.IsGoodCode.HasValue and input.IsGoodCode.IsTrue = yes.

Before the NullBool struct can be used, we have to create a struct to represent the JSON data:

type JsonInput struct {
	IsWrittenInGo NullBool
	IsGoodCode    NullBool
	DoWeLikeGo    NullBool
}

The final step is changing the way that the code unmarhals JSON into the NullBool struct.

Taking advantage of Go’s UnmarshalJSON() interface, a method on NullBool is implemented with the signature UnmarshalJSON(b []byte) error:

func (nullBool *NullBool) UnmarshalJSON(b []byte) error {
	var unmarshalledJson bool

	err := json.Unmarshal(b, &unmarshalledJson)
	if err != nil {
		return err
	}

	nullBool.IsTrue = unmarshalledJson
	nullBool.HasValue = true

	return nil
}

This method is run by any call to json.Unmarshal() when populating any instance of the NullBool struct, for which data is provided.

If a key defined in the struct representing the JSON data is omitted from the JSON string which is unmarshalled into it, then NullBool’s UnmarshalJSON() is not run.

With bool values initiatlised to 0 (false), both IsTrue and HasValue will be false for any instance of NullBool that has not been populated by the unmarshalling of JSON.

Conversely, we know that instances of NullBool with IsTrue:false and HasValue:true have a corresponding key in a JSON string with a value of false.

Click here for the full Go Playground example.

Conclusion

Go is well documented, widely supported and easy to learn. Like many languages that use zero type initialisation, extra care needs to be afforded in some situations, like unmarshalling JSON.

The UnmarshalJSON interface is very powerful because it gives developers control over how custom types are populated when unmarshalling JSON. Not only can it be used to change the way objects are hydrated, but it can be used to add additional validation.

References

1. Variables - golang.org
2. The Zero Value - golang.org
3. Interface Types - golang.org
4. Struct Types - golang.org