With the release of Go 1.2, slices gained the ability to specify the capacity when performing a slicing operation. This doesn’t mean we can use this index to extend the capacity of the underlying array. It means we can create a new slice whose capacity is restricted. Restricting the capacity provides a level of protection to the underlying array and gives us more control over append operations.
Here are the release notes and design document for the feature request:
http://tip.golang.org/doc/go1.2#three_index
https://docs.google.com/document/d/1GKKdiGYAghXRxC2BFrSEbHBZZgAGKQ-yXK-hRKBo0Kk/pub
Let’s write some code to explore using the new capacity index. As with all my slice posts, I am going to use this InspectSlice function:
func InspectSlice(slice []string) {
// Capture the address to the slice structure
address := unsafe.Pointer(&slice)
// Capture the address where the length and cap size is stored
lenAddr := uintptr(address) + uintptr(8)
capAddr := uintptr(address) + uintptr(16)
// Create pointers to the length and cap size
lenPtr := (*int)(unsafe.Pointer(lenAddr))
capPtr := (*int)(unsafe.Pointer(capAddr))
// Create a pointer to the underlying array
addPtr := (*[8]string)(unsafe.Pointer(*(*uintptr)(address)))
fmt.Printf("Slice Addr[%p] Len Addr[0x%x] Cap Addr[0x%x]\n",
address,
lenAddr,
capAddr)
fmt.Printf("Slice Length[%d] Cap[%d]\n",
*lenPtr,
*capPtr)
for index := 0; index < *lenPtr; index++ {
fmt.Printf("[%d] %p %s\n",
index,
&(*addPtr)[index],
(*addPtr)[index])
}
fmt.Printf("\n\n")
}
To start, let's create a slice we will use as our source:
source := []string{"Apple", "Orange", "Plum", "Banana", "Grape"}
InspectSlice(source)
Output:
Slice Addr[0x210231000] Len Addr[0x210231008] Cap Addr[0x210231010]
Slice Length[5] Cap[5]
[0] 0x21020e140 Apple
[1] 0x21020e150 Orange
[2] 0x21020e160 Plum
[3] 0x21020e170 Banana
[4] 0x21020e180 Grape
We start with a slice of strings with a length and capacity of 5. This means the underlying array has 5 elements and we have access to the entire array.
Next, let's take a traditional slice of the
source and inspect the contents:
takeOne := source[2:3]
InspectSlice(takeOne)
Output:
Slice Addr[0x210231040] Len Addr[0x210231048] Cap Addr[0x210231050]
Slice Length[1] Cap[3]
[0] 0x21020e160 Plum
With this slice operation we only take the third element from the
source. You can see the first element of the
takeOne slice has the same address as the third element of the
source slice. The
takeOne slice has a length of one and a capacity of three. This is because there are three elements left in the underlying array that are available for use.
What if we didn't want the new slice to have access to the remaining capacity? Prior to version 1.2, this was not possible. Let's take the slice again, but this time restrict the capacity to one:
takeOneCapOne := source[2:3:3] // Use the third index position to
InspectSlice(takeOneCapOne) // set the capacity
Output:
Slice Addr[0x210231060] Len Addr[0x210231068] Cap Addr[0x210231070]
Slice Length[1] Cap[1]
[0] 0x21020e160 Plum
After creating the
takeOneCapOne slice, the length and capacity are now one. The
takeOneCapOne slice no longer has access to the remaining capacity in the underlying array.
Length and capacity is calculated using this formula:
For slice[ i : j : k ] the
Length: j - i
Capacity: k - i
If we attempt to set the capacity greater than the underlying array, the code will panic.
takeOneCapFour := source[2:3:6] // (6 - 2) attempts to set the capacity
// to 4. This is greater than what is
// available.
Runtime Error:
panic: runtime error: slice bounds out of range
goroutine 1 [running]:
runtime.panic(0x9ad20, 0x1649ea)
/Users/bill/go/src/pkg/runtime/panic.c:266 +0xb6
main.main()
/Users/bill/Spaces/Test/src/test/main.go:15 +0x24f
So what happens if we append an element to the
takeOneCapOne slice?
source := []string{"Apple", "Orange", "Plum", "Banana", "Grape"}
InspectSlice(source)
takeOneCapOne := source[2:3:3]
InspectSlice(takeOneCapOne)
takeOneCapOne = append(takeOneCapOne, "Kiwi")
InspectSlice(takeOneCapOne)
Here is the output:
Slice Addr[0x210231000] Len Addr[0x210231008] Cap Addr[0x210231010]
Slice Length[5] Cap[5]
[0] 0x21020e140 Apple
[1] 0x21020e150 Orange
[2] 0x21020e160 Plum
[3] 0x21020e170 Banana
[4] 0x21020e180 Grape
-- Before Append --
Slice Addr[0x210231040] Len Addr[0x210231048] Cap Addr[0x210231050]
Slice Length[1] Cap[1]
[0] 0x21020e160 Plum
-- After Append --
Slice Addr[0x210231080] Len Addr[0x210231088] Cap Addr[0x210231090]
Slice Length[2] Cap[2]
[0] 0x210231060 Plum
[1] 0x210231070 Kiwi
When we append an element to the
takeOneCapOne slice, a new underlying array is created for the slice. This new underlying array contains a copy of the elements being referenced from the
source and then is extended to add the new element. This is because the capacity of the
takeOneCapOne slice was reached and append needed to grow the capacity. Notice how the address changes in the
takeOneCapOne slice after the append.
How is this different from not setting the capacity?
source := []string{"Apple", "Orange", "Plum", "Banana", "Grape"}
InspectSlice(source)
takeOne := source[2:3] // Don't specify capacity
InspectSlice(takeOne)
takeOne = append(takeOne, "Kiwi")
InspectSlice(takeOne)
InspectSlice(source)
Here is the output:
Slice Addr[0x210231000] Len Addr[0x210231008] Cap Addr[0x210231010]
Slice Length[5] Cap[5]
[0] 0x21020e140 Apple
[1] 0x21020e150 Orange
[2] 0x21020e160 Plum
[3] 0x21020e170 Banana
[4] 0x21020e180 Grape
-- Before Append --
Slice Addr[0x210231040] Len Addr[0x210231048] Cap Addr[0x210231050]
Slice Length[1] Cap[3]
[0] 0x21020e160 Plum
-- After Append --
Slice Addr[0x210231060] Len Addr[0x210231068] Cap Addr[0x210231070]
Slice Length[2] Cap[3]
[0] 0x21020e160 Plum
[1] 0x21020e170 Kiwi
Slice Addr[0x210231080] Len Addr[0x210231088] Cap Addr[0x210231090]
Slice Length[5] Cap[5]
[0] 0x21020e140 Apple
[1] 0x21020e150 Orange
[2] 0x21020e160 Plum
[3] 0x21020e170 Kiwi
[4] 0x21020e180 Grape
This time the append uses the existing capacity and overwrites the value at element 4 in the underlying array. This could be a disaster if this was not our intent.
This new feature of setting the capacity can really help protect us and our data from unwanted overwrites. The more we can leverage the built-in functions and runtime to handle these types of operations the better. These types of bugs are very difficult to find so this is going to help immeasurably.
Here are other posts about slices:
Understanding Slices In Go ProgrammingCollections Of Unknown Length In GoSlices Of Slices Of Slices In GoIterating Over Slices In Go