How we define and load configuration from files.



Background

After we write purposes, we mainly use configuration recordsdata, from numerous shells to nginx, and so on., all have their very own configuration recordsdata. Though this isn’t too troublesome, the configuration objects are comparatively cumbersome, and parsing and verifying them might be troublesome. On this article, I’ll inform how we simplify the definition and parsing of configuration recordsdata.



Situation

If we need to write a Restful API service, the configuration objects will in all probability have the next contents.

  • Host, the listening IP, if not crammed, the default is 0.0.0.0
  • Port, the port to take heed to, required, can solely be a quantity, larger than or equal to 80, lower than 65535
  • LogMode, logging mode, solely file or console might be chosen
  • Verbose, see whether or not to output detailed logs, non-obligatory, default is false
  • MaxConns, the utmost variety of concurrent connections allowed, default 10000
  • Timeout, timeout setting, default 3s
  • CpuThreshold, units the brink for CPU utilization to set off load shedding, default 900, 1000m means 100%

Beforehand we used json for the configuration file, however there was an issue with json that we could not add feedback, so we switched to yaml format.

Now let’s have a look at how simple it’s to outline and parse such a configuration file with go-zero ~



Defining the configuration

First, we have to outline the above configuration necessities right into a Go construction, as follows.

RestfulConf struct {
    Host string `json:",default=0.0.0.0"`
    Port int `json:",vary=[80,65535)"`
    LogMode string `json:",options=[file,console]"`
    Verbose bool `json:",non-obligatory"`
    MaxConns int `json:",default=10000"`
    Timeout time.Period `json:",default=3s"`
    CpuThreshold int64 `json:",default=900,vary=[0:1000]"`
}
Enter fullscreen mode

Exit fullscreen mode

As you’ll be able to see, now we have sure definitions and restrictions for every configuration merchandise, a few of that are outlined as follows.

  • default, the default worth is used if the configuration will not be crammed, you’ll be able to see that 3s in it is going to be routinely resolved into time.Period sort
  • non-obligatory, you’ll be able to depart the merchandise clean, if not, use the zero worth of its sort
  • vary, which restricts the quantity sort to the given vary
  • choices, which restricts the configured worth to solely one of many given values

And, some attributes can be utilized collectively, corresponding to.

  • default and vary can be utilized collectively to each add a variety restriction and supply a default worth
  • default and choices can be utilized collectively so as to add choice restrictions and supply a default worth



Configuration recordsdata

As a result of we give plenty of default values when defining the configuration, in addition to utilizing non-obligatory to specify as non-obligatory, so our configuration file has comparatively few configuration objects that may use the default values with out writing explicitly, as follows.

# As a result of a lot of them have default values, you solely want to put in writing those
# that must be specified and those that do not have default values
Port: 8080
LogMode: console
# You may learn the values of surroundings variables
MaxBytes: ${MAX_BYTES}
Enter fullscreen mode

Exit fullscreen mode

Here’s a word, if the worth of the configuration chars are all numbers, and also you outline the configuration sort is string, for instance, some individuals usually use 123456 to check the password, however the password will usually be outlined as string, the configuration ought to be written as follows (simply an instance, passwords are usually not really helpful to be written within the configuration file)

Password: "123456"
Enter fullscreen mode

Exit fullscreen mode

Right here the double quotes can’t be missed, in any other case sort mismatch will likely be reported, as a result of yaml parser will parse 123456 as int.



Loading configuration recordsdata

We’ve got the configuration definition (config.go) and the configuration file (config.yaml), the subsequent step is to load the configuration file, which might be executed in 3 ways.

  • should be loaded efficiently, in any other case this system exits, we usually use so, if the configuration will not be right, this system will be unable to proceed
// If error, exit this system immediately
var config RestfulConf
conf.MustLoad("config.yaml", &config)
Enter fullscreen mode

Exit fullscreen mode

The default code generated by go-zero‘s goctl instrument additionally makes use of MustLoad to load the configuration file

  • Load the configuration and deal with the error by yourself
// deal with errors by yourself
var config RestfulConf
// For simplicity, LoadConfig will likely be modified to Load later,
// LoadConfig has been marked as Deprecated.
if err := conf.LoadConfig("config.yaml", &config); err ! = nil {
    log.Deadly(err)
}
Enter fullscreen mode

Exit fullscreen mode

  • Load the configuration and skim the surroundings variables
// Mechanically learn surroundings variables
var config RestfulConf
conf.MustLoad(configFile, &config, conf.UseEnv())
Enter fullscreen mode

Exit fullscreen mode

Right here why we have to explicitly specify conf.UseEnv(), as a result of if default to make use of, you could have to escape when writing particular characters within the configuration, so the default doesn’t learn the surroundings variables.



Implementation precept

We often use encoding/json or the corresponding yaml library when implementing yaml/json parsing, however for go-zero, we have to have extra exact management over unmarshal, which leads us to customise yaml/json parsing ourselves, the whole code is right here.

Configuration file code: https://github.com/zeromicro/go-zero/tree/master/core/conf

yaml/json parsing code: https://github.com/zeromicro/go-zero/tree/master/core/mapping

That is additionally an excellent instance of how mirror can be utilized and the way unit exams can be utilized to make sure correctness in complicated situations.



Abstract

I at all times advocate the concept of Fail Quick, we do the identical on loading configuration recordsdata, as soon as there’s an error, instantly exit, in order that ops will discover the issue in time when deploying companies, as a result of the method can’t rise up operating.

All of the configuration objects of go-zero companies are loaded and routinely verified on this means, together with the configuration of lots of the instruments I wrote are additionally based mostly on this, hope it helps!



Mission deal with

https://github.com/zeromicro/go-zero

Welcome to make use of go-zero and star to help us!

Add a Comment

Your email address will not be published. Required fields are marked *