Embed files at compile time with go:embed

@ Go

Introduction

Notes on using Go 1.16’s embed feature, which can include any file at compile time so you don’t have to worry about paths and environment issues.

When to use

When implementing an authorization feature, you may need to read a local JSON configuration; you could simply read the file:

if data, err := os.ReadFile("./rbac.json"); err != nil {
    panic(err)
}

But you may encounter:

  • Forgetting to COPY the file into Docker after deployment
  • Different working directory in CI / test environments
  • Binary moved to another machine and can’t find relative paths
  • Want to deploy as a single executable

Single file

I find dynamically loading files during testing troublesome. To avoid depending on external file settings, you can embed them into the binary at compile time using embed:

package constants


import _ "embed"


//go:embed rbac.json
var RBACJSON []byte
if err := json.Unmarshal(constants.RBACJSON, &parsed); err != nil {
  println("ERROR: Failed to unmarshal embedded rbac.json:", err.Error())
  return
}

This magic-like comment: //go:embed is a compiler directive; after compilation, the contents of rbac.json will already exist in the rbacData variable.

Variable types can be:

  • string
  • []byte
  • embed.FS

Multiple files

package main


import (
  _ "embed"
  "fmt"
)


//go:embed messages/*.txt
var messages embed.FS


func main() {
  files, _ := messages.ReadDir("messages")
  for _, file := range files {
    data, _ := messages.ReadFile("messages/" + file.Name())
    fmt.Printf("File: %snContent: %snn", file.Name(), data)
  }
}

Summary

os.ReadFileembed
File SourceRead at runtimeEmbed at compile time
Dependency PathYesNo
Distribution MethodRequires fileSingle binary
Suitable ScenariosDynamically changing filesStatic files

If data needs to be modified at runtime, embed is not suitable; but if it is part of the software, embed is the safer choice.

  • embed determines content at compile time; files must be recompiled after changes
  • Files increase binary size; not suitable for large assets (e.g., videos)
  • Cannot embed files outside the project

Further reading