Recently I found myself wanting to write some code to load values into a Go object. The obvious approach is reflection. However, the Go documentation on reflection is somewhat impenetrable, and the accompanying article on The Laws of Reflection only has a single skimpy example involving structs right at the end.

After some trial and error I came up with some robust code to demonstrate examining a struct and altering its fields. I decided to write it up in detail here.

A simple example

Let’s start with the struct:

type Name string

type Person struct {
  FirstName Name
  LastName  Name
  Age       int
}

First of all we define a custom type, so that our fields aren’t all primitive types and our example is a bit more general. Then we assemble a simple struct.

Next, we instantiate an example struct of that type, and pass it to a function.

t := Person{"John", "Smith", 23}
reflectOn(t)

The signature for the function will be as follows:

func reflectOn(s interface{}) {
  // Code here
}

Why the function call for a simple example?

Well, in real code you aren’t going to be performing reflection in the same function where you create an object. By passing an interface{} argument, we manage to completely bypass all type safety, meaning our example will be forced to rely on reflection. In real code, of course, you’d ideally narrow the scope a bit with a more specific interface.

In addition, by putting the reflection code in a function we can call it twice:

reflectOn(t)
reflectOn(&t)

Now we can make sure our code deals with structs passed by reference as well as by value, and we can demonstrate the important difference that makes, concerning whether you can change the fields of the struct.

So, what is a function argument really?

Recall that unlike (say) Java, Go is all about interfaces, not classes. A given object (struct) can implement any number of interfaces, just by providing the right methods. Object oriented programming is done via composition, rather than a class hierarchy.

So when you define a function in Go which operates on objects, you specify the interface which the objects must implement in order to be acceptable to the function. Our function takes an argument of type interface{}. That’s the empty interface, one with no methods specified, so as per the spec absolutely anything implements it — even a primitive type such as an int.

So our function accepts objects with any interface type. What it receives is an interface value in an interface variable.

If you’ve read about the implementation of the Go type system or at least tried to digest The Laws of Reflection, you’ll know that an interface value in Go is a tuple consisting of a type descriptor (“this is a string”) and the type-dependent data representing the value.

So, the first step for our reflection function is to extract those two things:

ptyp := reflect.TypeOf(s) // a reflect.Type
pval := reflect.ValueOf(s) // a reflect.Value

Next we want to look at the type of thing we were passed, to make sure it’s what we expect. However, the Type we just extracted is the specific type of the value — a Person in this case. What we really want to know is whether it’s a struct, before we try to go looking at its fields. So we look at the Kind of the type, which we obtain from the Type by calling Kind().

You might want to try printing out the value of ptyp.Kind(). If you try it with the two function calls:

reflectOn(t)
reflectOn(&t)

…you will quickly discover that in the second case, the interface type’s Kind is Ptr. So although in Go you can often ignore the distinction between a struct and a pointer to a struct, when it comes to reflection the difference is exposed.

So our function needs to know how to deal with pointers and get at the thing pointed to. The reflect package provides a method Elem() which operates on a Value and dereferences to get the Value pointed at. A similar method does the same thing for the Type. So:

var typ reflect.Type
var val reflect.Value
if ptyp.Kind() == reflect.Ptr {
  fmt.Printf("Argument is a pointer, dereferencing.\n")
  typ = ptyp.Elem()
  val = pval.Elem()
} else {
  fmt.Printf("Argument is %s.%s, a %s.\n", ptyp.PkgPath(), ptyp.Name(),
    ptyp.Kind())
  typ = ptyp
  val = pval
}

At this point, our two new variables typ and val contain the Type and Value of the actual struct, whether we were given it as an actual value or via a pointer. Now we can make sure that it really is a struct:

if typ.Kind() != reflect.Struct {
  fmt.Printf("Not a struct.\n")
  return
}

If this seems like a lot of work, remember that in real code you would know whether you were going to call your function with a struct or a pointer to a struct and would just call Elem() or not (as appropriate) in the first line or two of code.

Next, let’s examine the key difference between passing a struct by value and passing it by reference:

if val.CanSet() {
  fmt.Printf("We can set values.\n")
} else {
  fmt.Printf("We cannot set values.\n")
}

If you try the code so far, you’ll discover that if your function call is reflectOn(t) then CanSet() will report that you can’t set values. If it’s reflectOn(&t), you can set values. If you learned programming by learning Java, this probably makes no sense to you at all, but it goes back to the invention of function calls in programming languages. A brief digression is in order. If you’re a C or C++ programmer, you can skip to the next section.

What is the stack?

Back in the 1950s, the programming language Algol 60 was being designed. One of its design goals was to support recursion. This meant allowing an unrestricted number of function calls — functions calling functions calling functions. To do this, Dijkstra invented the stack. (See: A Brief History Of The Stack.)

Each time a function was called:

  1. The arguments would be pushed onto the stack, in specified order.
  2. The code for the function would be called.
  3. The function would pop the arguments off of the stack.
  4. The function would perform its computations, and put its result on the stack.
  5. The function would end, returning to the calling code, which would pop the result from the stack.

In the 1960s, memory was scarce and computers were slow. A 1964 IBM 360 mainframe started out with 8KiB of memory and executed 34,500 instructions per second — so not even equivalent to a 1MHz clock. If you wanted to pass a string to a function, the idea of copying the entire string onto the stack and off again would have crippled performance. So instead, any argument whose data was larger than a few bytes would be replaced with a pointer to the argument.

The same methods were used for function calls in CPL, which was modeled on Algol. CPL gave way to the BCPL programming language, and its successor C.

Nowadays compilers use various tricks to speed up function calls. For example, if all the arguments will fit into processor registers, they get passed that way instead of via the stack. However, conceptually Go still uses the same stack-based argument passing as its programming language ancestors. One difference, however, is that Go will actually shove an entire string or other large data object onto the stack if you ask it to — conceptually, at least.

By value or by reference

When we call our function via reflectOn(t), Go pushes an entire copy of the struct t onto the stack. The function retrieves it as s. The function doesn’t have any way to know where the copy came from. Whatever it does with the copy, the original will remain unchanged.

When we call our function via reflectOn(&t), Go pushes a pointer to the struct onto the stack. The function retrieves the pointer. At that moment, it can access the original structure — so any changes it makes will be visible when the function returns and the original structure is examined.

So although our code makes sure that typ and val are the Type and Value of the struct, in one case they are the type and value of a copy of the struct, and any changes we try to make will be ignored — so Go warns us of this by returning false from val.CanSet(). Notice that whether the value is settable is a property of the value and how we obtained it, not a property of the type of the structure; the struct’s type is identical in both cases.

We’ll get back to this in a few more lines of code. First, let’s see how we look at the fields of the struct. Logically, the fields of the struct and their types are defined in the type definition of the struct, so we would expect to use the typ variable to access the individual fields. And so we do:

for i := 0; i < typ.NumField(); i++ {
  sfld := typ.Field(i)

At this point we have a value representing a field. If you’re used to how Java reflection works you might expect it to be some sort of field class of a particular type that you can use to access the data, but in Go there’s another step to go through.

In Go, the .Field(int) method, when called on a struct, always returns a special StructField object. To get the actual type of the field, we need to call Type() on the StructField. Just as we examined the underlying type or ‘kind’ of our function argument, so we can do the same for the field:

tfld := sfld.Type // The Type of the StructField of the struct
kind := tfld.Kind() // The Kind of the Type of the StructField

OK, now how about the value? Java gives you a single Field object which you can interrogate for both type and value information. Go has two separate sets of objects to handle that. So just as we called .Field() on the struct’s Type to get at the field’s type (via an intermediate StructField), so we need to call .Field() on the struct’s Value to get the field’s value. This time, however, there’s no intermediate StructValue:

vfld := val.Field(i)
fmt.Printf("struct field %d: name %s type %s kind %s value %v\n", 
	   i, sfld.Name, tfld, kind, vfld)

Running the code at this stage will produce output like this:

struct field 0: name FirstName type main.Name kind string value John
struct field 1: name LastName type main.Name kind string value Smith
struct field 2: name Age type int kind int value 23

So, we’ve decoded our struct completely, down to field level. We’ve extracted both the specific types (including custom types we defined), and the underlying primitive types. We’ve even read out the data.

Writing fields

Now that we can read from the struct, let’s work out how to change it.

You might wonder whether setting a value is an operation you perform on the type, or an operation you perform on the value. In a dynamic language like Ruby, you’d expect to call a type-dependent method to set the value. But Go is statically typed, so you can’t change the type of a field at runtime — only its value. So to change a value you use a Set method on the value of the individual field, as returned by the Field() method of the struct’s Value. And if you try to tell the Value to take on a value of a different incompatible type, Go will panic.

Also, you need to use the Set methods on the Value of the individual field you want to change — not the interim StructField. So let’s try it:

if kind == reflect.String && vfld.CanSet() {
  fmt.Printf("Overwriting field %s\n", sfld.Name)
  vfld.SetString("Anonymous")
}

Notice that the field’s Value has its own CanSet() method, just like the overall struct’s Value does.

So now I can restate the part that confused the heck out of me: You can’t modify the value of a struct in Go using a Type or StructField. To perform a reflection operation in Go you need to go through two separate interrogation processes: first you start with the struct and retrieve all the type information you want and check it, then you start again at the struct and work down the value chain to the field value and change it.

You can interleave the operations, as I’ve done in this example code, but fundamentally you’re dealing with two different trees of information.

You can get the complete code on GitHub with added comments. If you run it, you’ll see quite clearly the behavior difference between calling with a struct, versus calling with a pointer to a struct:

First, passing the actual structure:

Argument is main.Person, a struct.
We cannot set values.
struct field 0: name FirstName type main.Name kind string value John
struct field 1: name LastName type main.Name kind string value Smith
struct field 2: name Age type int kind int value 23
After reflection:
John Smith, 23 years old

Now, passing a pointer to the structure:

Argument is a pointer, dereferencing.
We can set values.
struct field 0: name FirstName type main.Name kind string value John
Overwriting field FirstName
struct field 1: name LastName type main.Name kind string value Smith
Overwriting field LastName
struct field 2: name Age type int kind int value 23
After reflection:
Anonymous Anonymous, 23 years old

Hopefully that covers everything you need to know about reflecting on structs.