3 minutes
Go: Avoid an Infitine Loop With Custom JSON Unmarshallers
Background
The third article in this series1.
I wanted to share a mistake I made in writing a custom Go UnmarshalJSON() method that caused an infinite loop.
The Problem Unmarshaller
I wrote a JSON unmarshaller that looked like this:
func (jsonInput *JsonInput) UnmarshalJSON(b []byte) error {
err := json.Unmarshal(b, jsonInput)
if err != nil {
return err
}
err = validateStruct(jsonInput)
if err != nil {
return err
}
return nil
}
Running this unmarshaller causes a fatal stack overflow runtime exception:
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc0201614d0 stack=[0xc020160000, 0xc040160000]
fatal error: stack overflow
runtime stack:
runtime.throw({0x303582, 0x399060})
C:/Program Files/Go/src/runtime/panic.go:1198 +0x76
runtime.newstack()
C:/Program Files/Go/src/runtime/stack.go:1088 +0x5fd
runtime.morestack()
C:/Program Files/Go/src/runtime/asm_amd64.s:461 +0x93
This UnmarshalJSON() method is run as a result of calling json.Unmarshal() in func main(). When this method calls json.Unmarshal(), it passes in the same jsonInput object and thus json.Unmarshal() will invoke the object’s UnmarshalJSON method again.
We are therefore recursively calling the unmarshaller and causing an infinite loop.
The solution is to not unmarshal directly into the jsonInput object, but instead unmarshal into something else.
Solution 1: Unmarshal into [Key]Value map
Typically this problem can be solved by unmarshalling into a map defined with a key type of string and a value type of interface{}. We then write logic to iterate over the map, parsing the keys and storing the appropriate values in each member of the JsonInput struct:
func (jsonInput *JsonInput) UnmarshalJSON(b []byte) error {
var tmpJson map[string]interface{}
err := json.Unmarshal(b, &tmpJson)
if err != nil {
return err
}
for key, value := range tmpJson {
switch key {
case "Id":
jsonInput.Id = int(value.(float64))
case "Author":
jsonInput.Author = value.(string)
case "Draft":
jsonInput.Draft = value.(bool)
}
}
err = validateStruct(jsonInput)
if err != nil {
return err
}
return nil
}
This simple example is published here on Go Playground.
With larger structs, this technique can become unwieldy and although there are other published suggestions which solve the problem in a number of different and complex ways, there is another simple solution.
Solution 2: Using the Go Type Alias
func (jsonInput *JsonInput) UnmarshalJSON(b []byte) error {
type TmpJson JsonInput
var tmpJson TmpJson
err := json.Unmarshal(b, &tmpJson)
if err != nil {
return err
}
*jsonInput = JsonInput(tmpJson)
err = validateStruct(jsonInput)
if err != nil {
return err
}
return nil
}
In this example we create a type alias with type TmpJson JsonInput and then instantiate it on the following line.
In Go a type alias is in effect an identical copy of a type, save for its methods. When we use the variable tmpJson of type TmpJson which itself is aliased to JsonInput, we no longer need to be concerned about the infinite loop. TmpJson does not have the UnmarshalJSON() method.
The drawback with this approach, is that we are using type aliases in a way that they were not intended to be used. Type aliases were introduced in Go to help ease the refactoring of code, for example, when moving a type between packages.
Click here for the full Go Playground example.
Conclusion
When writing a custom UnmarshalJSON() method for a type, we cannot simply call json.Unmarshal() on the object itself without causing our program to crash with a stack overflow runtime exception.
Go type aliases provide a convenient mechanism for calling json.Unmarshal() without unintended consequence.
You may, however, be more comfortable investigating other methods of avoiding the infinite loop problem, given that it is an unintended use of Go’s type alias.