How to Embed Files and Directories in Golang Binary

Golang is a compiled language. Go compiler take your code and compile it into a binary file.

Go compiler can also compile your code to a statically linked binary. Statically linked binary contain all the required libraries statically linked and packaged in the binary file. Therefore, statically linked binary run on an OS without depending on system libraries provided by the operating system.

Real-world projects contain additional files other than .go files. For example, a web project contains HTML, CSS, and JS files. These files are not included in the binary by default. But you can embed those files using the embed package.

In this blog, you will learn how to embed any file or assets in a go binary.

Benefits of Embedding Files on Go Binary

A Golang full-stack web application consists of HTML and CSS files for front-end and Go files for back-end. But during compilation, the compiler only compiles the go code. Therefore, you need to transfer those static fron-end files to your server in addition to the go binary. It is a bit of hassle for developers.

But if you embed the front-end files in the Go binary, your application become a single file application. As a result, it becomes very easy to move your application across hosts.

Distribution and hosting of an embeded binary file is easier. For example, take a look at Pocketbase. Poacketbase is a free and open source back-end as a service application. If you go to their GitHub release page, they distribute pocketbase as a single binary and embed all the front-end files in the binary. This helps distribution and hosting easier for the end user.

Embed a File and Read its Content

Follow these steps to embed a single file in the go binary and read the file content.

Blank import the embed package.

import _ "embed"

Use //go:embed directive and the fine name followed by a variable of type string or []byte.

//go:embed index.html
var index string

Now, Go read the content of the index.html file and populate the index variable. If you want the index variable to be a []byte, use the following code instead.

//go:embed index.html
var index []byte

Embed a Folder/Directory in Go Binary

Sometimes you need to embed a directory in your go binary.

Go embed provide embed.FS type for this use case. The embed.FS type implement the FS interface of io/fs package.

Note: The directory path must not contain any . or .. symbols, and should not begin and end with a slash /.

Once you provide the directory path, Go recursively embed all the files inside that directory.

import "embed"

//go:embed assets
var assets embed.FS

In this example, the assets directory is embedded in the variable assets.

Embed Nested Directories and How To Access Them

In the previous example, assets/ is a top level directory. But, What if you want to embed a nested directory.

Let’s assume you want to embed the view/static/js directory. This directory contains javascript files for your website.

Now, the embed tag for this directory look something like this.

//go:embed view/static/js
var jsdir embed.FS

The jsdir variable contain all the javascript files inside the view/static/js directory. But, instead of mounting the js directory as the top level directory in the jsdir variable, Go embed maintain the view/static/js/{some js files} structure.

For example, you have a file react.js inside the view/static/js directory. Therefore, its path will be view/static/js/react.js. When you embed the js directory in your go program, your rect.js file don’t present in the route js/react.js. It is deeply nested in the view/static/js/react.js route.

If you want to open the react.js file from the jsdir directory, it throws a PathError.

//go:embed view/static/js
var jsdir embed.FS

file, err := jsdir.Open("react.js")

// thows a *PathError
log.Print(err)

To solve this problem, you have to create a sub filesystem. The sub filesystem removes the parent view/static directory, and you can access the files directly from the js directory.

// creating a sub file system contain only the js directory and
// removing its parents
jssub, err := fs.Sub(jsdir, "view/static/js")

if err != nil {
	log.Fatal(err)
}

file, err := jssub.Open("react.js")
//success

Now you can also serve this directory on HTTP request.

http.Handle("GET /js/", http.StripPrefix("/js/", http.FileServer(http.FS(jssub))))

Embed Hidden Files

By default, go embed package ignore the files start with . or _. You can bypass this behaviour with the following 2 method.

If the file is not nested, just add a wildcard * to embed them. For example, assets/_hidden.png can be embedded by adding //go:embed assets/* tag.
If the file is nested deep inside the tree, and you want to embed all the hidden files, use the all: directive. For example, //go:embed all:assets embed all the hidden files nested in the assets directory recursively.

When to Use `string` or `[]byte` Embeds.

The string or []byte embeds are good for a single file embed. If you add a directory embed tag and provide a string or []byte, it throws you an error.

As I have previously said, use string or []byte to embed single files and embed.FS to embed a directory.

Parse Embedded HTML Templates Nested Inside a Directory

Embed the HTML directory by adding the embed tag.

//go:embed view/html
var h embed.FS

Create a subdirectory.

htmldir, err := fs.Sub(h, "view/html")
if err != nil {
	log.Fatal(err)
}

Parse HTML files present inside the view/html directory. The HTML files are divided into pages and partials. The partials directory contain common templates like header and footers.

func parseTemplates() map[string]*template.Template {

	// pages contain an slice containing all the html files
	// inside the pages directory
	pages, err := fs.Glob(htmldir, "pages/*.html")
	if err != nil {
		log.Print(err)
	}

	templates := map[string]*template.Template{}

	for _, p := range pages {
		//parsing the partials. These partials will be present in
		// each template
		t := template.Must(template.ParseFS(htmlSubFS, "partials/*"))
		t = template.Must(t.ParseFS(htmldir, p))
		templates[filepath.Base(p)] = t
	}

	return templates
}

Conclusion

This is an in-depth blog on how to embed a file or directory in a Go binary. If you learn something new, do consider subscribing to my newsletter.