Configuration Manager

Configuration

環境變量(配置)

Region、Zone、Cluster、Environment、Color、Discovery、AppID、Host，等之類的環境信息，都是通過在線運行時平台打入到容器或者物理機，供 kit 庫讀取使用。

靜態配置

資源需要初始化的配置信息，比如 http/gRPC server、redis、mysql 等，這類資源在線變更配置的風險非常大，我通常不鼓勵 on-the-fly 變更，很可能會導致業務出現不可預期的事故，變更靜態配置和發布 bianry app 沒有區別，應該走一次迭代發布的流程。

動態配置

應用程序可能需要一些在線的開關，來控制業務的一些簡單策略，會頻繁的調整和使用，我們把這類是基礎類型(int, bool)等配置，用於可以動態變更業務流的收歸一起，同時可以考慮結合類似 https://pkg.go.dev/expvar 來結合使用。

全局配置

通常，我們依賴的各類組件、中間件都有大量的默認配置或者指定配置，在各個項目裡大量拷貝複制，容易出現意外，所以我們使用全局配置模板來定制化常用的組件，然後再特化的應用裡進行局部替換。

Redis client example

Bad

// DialTimeout acts like Dial for establishing the
// connection to the server, writing a command and reading a reply.
func Dial(network, address string) (Conn, error) {
    // ...
}

"我要自定義超時時間！" "我要設定 Database！" "我要控制連接池的策略！" "我要安全使用 Redis，讓我填一下 Password！" "可以提供一下慢查詢請求記錄，並且可以設置 slowlog 時間？" 導致 code 補了一堆 Features 如下

// DialTimeout acts like Dial for establishing the
// connection to the server, writing a command and reading a reply.
func Dial(network, address string) (Conn, error)

// DialTimeout acts like Dial but takes timeouts for establishing the
// connection to the server, writing a command and reading a reply.
func DialTimeout(network, address string, connectTimeout, readTimeout, writeTimeout time.Duration) (Conn, error)

// DialDatabase acts like Dial but takes database for establishing the
// connection to the server, writing a command and reading a reply.
func DialDatabase(network, address string, database int) (Conn, error)

// DialPool
func DialPool
// ...

net/http

定義一個 structure

  1. 因為是用structur, 所以可統一定義
  2. 假如是 string 沒填, 就會是默認值
  3. 無法區分必選或可選
package main
import (
  "log"
  "net/http"
  "time"
)
func main() {
  s := &http.Server{
  Addr: ":8080",
  Handler: nil,
  ReadTimeout: 10 * time.Second,
  WriteTimeout: 10 * time.Second,
  MaxHeaderBytes: 1 << 20,
}
  log.Fatal(s.ListenAndServe())
}

Functional options

Self-referential functions and the design of options -- Rob Pike Functional options for friendly APIs -- Dave Cheney

// DialOption specifies an option for dialing a Redis server.
type DialOption struct {
  f func(*dialOptions)
}

type dialOptions struct {
    readTimeout  time.Duration
    writeTimeout time.Duration
    dial         func(network, addr string) (net.Conn, error)
    db           int
    password     string
}

// DialReadTimeout specifies the timeout for reading a single command reply.
func DialReadTimeout(d time.Duration) DialOption {
    return DialOption{func(do *dialOptions) {
        do.readTimeout = d
    }}
}

// DialDatabase specifies the database to select when dialing a connection.
func DialDatabase(db int) DialOption {
    return DialOption{func(do *dialOptions) {
        do.db = db
    }}
}

// DialPassword specifies the password to use when connecting to
// the Redis server.
func DialPassword(password string) DialOption {
    return DialOption{func(do *dialOptions) {
        do.password = password
    }}
}

// Dial connects to the Redis server at the given network and
// address using the specified options.
func Dial(network, address string, options ...DialOption) (Conn, error) {
  do := dialOptions{
    dial: net.Dial,
  }
  for _, option := range options {
    option.f(&do)
  } // ...
}

package main
import (
  "time"
  "github.com/go-kratos/kratos/pkg/cache/redis"
)
func main() {
  c, _ := redis.Dial("tcp", "127.0.0.1:3389",
  redis.DialDatabase(0),
  redis.DialPassword("hello"),
  redis.DialReadTimeout(10*time.Second))
}

但只解決初始化, 配置文件加載不夠好

nethttp server structure

另一方法 不用struct, 用指針

// DialOption specifies an option for dialing a Redis server.
type DialOption func(*dialOptions)


// Dial connects to the Redis server at the given network and
// address using the specified options.
func Dial(network, address string, options ...DialOption) (Conn, error) {
  do := dialOptions{
    dial: net.Dial,
  }
  for _, option := range options {
    option(&do)
  }
  // ...
}

另一方法不用struct, 用指針且返回option

type option func(f *Foo) option

// Verbosity sets Foo's verbosity level to v.
func Verbosity(v int) option {
  return func(f *Foo) option {
    prev := f.verbosity
    f.verbosity = v
    return Verbosity(prev)
  }
}
func DoSomethingVerbosely(foo *Foo, verbosity int) {
  // Could combine the next two lines,
  // with some loss of readability.
  prev := foo.Option(pkg.Verbosity(verbosity))
  defer foo.Option(prev)
  // ... do some stuff with foo under high verbosity.
}

grpc 作法

type GreeterClient interface {
  SayHello(ctx context.Context, in *HelloRequest, opts ...grpc.CallOption) (*HelloReply, error)
}

type CallOption interface {
  before(*callInfo) error
  after(*callInfo)
}
// EmptyCallOption does not alter the Call configuration.
type EmptyCallOption struct{}

// TimeoutCallOption timeout option.
type TimeoutCallOption struct {
  grpc.EmptyCallOption
  Timeout time.Duration
}

Hybrid APIs

// Dial connects to the Redis server at the given network and
// address using the specified options.
func Dial(network, address string, options ...DialOption) (Conn, error)

// NewConn new a redis conn.
func NewConn(c *Config) (cn Conn, err error)

"JSON/YAML 配置怎麼加載，無法映射 DialOption 啊！" "嗯，不依賴配置的走 options，配置加載走config"

Configuration & APIs

For example, both your infrastructure and interface might use plain JSON. However, avoid tight coupling between the data format you use as the interface and the data format you use internally. For example, you may use a data structure internally that contains the data structure consumed from configuration. The internal data structure might also contain completely implementation-specific data that never needs to be surfaced outside of the system. -- the-site-reliability-workbook 2

避免接口的數據初始化和內部數據初始化強耦合

// Dial connects to the Redis server at the given network and
// address using the specified options.
func Dial(network, address string, options ...DialOption) (Conn, error){ // ...}
  • 僅保留 options API;
  • config file 和 options struct 解耦;

配置工具的實踐：

  • 語義驗證
  • 高亮
  • Lint
  • 格式化

YAML + Protobuf

configuration APIs

func ApplyYAML(s *redis.Config, yml string) error {
  js, err := yaml.YAMLToJSON([]byte(yml))
  if err != nil {
    return err
  }
  return ApplyJSON(s, string(js))
}

// Options apply config to options.
func Options(c *redis.Config) []redis.Options {
  return []redis.Options{
    redis.DialDatabase(c.Database),
    redis.DialPassword(c.Password),
    redis.DialReadTimeout(c.ReadTimeout),
  }
}

可以使用 wrappers : https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/wrappers.proto 來決定參數要必填還是選填

syntax = "proto3";
import "google/protobuf/duration.proto";
package config.redis.v1;
// redis config.
message redis {
  string network = 1;
  string address = 2;
  int32 database = 3;
  string password = 4;
  google.protobuf.Duration read_timeout = 5;
}

func main() {
  // load config file from yaml.
  c := new(redis.Config)
  _ = ApplyYAML(c, loadConfig())
  r, _ := redis.Dial(c.Network, c.Address, Options(c)...)
}

Configuration Best Pratice

代碼更改系統功能是一個冗長且複雜的過程，往往還涉及Review、測試等流程，但更改單個配置選項可能會對功能產生重大影響，通常配置還未經測試。配置的目標：

  • 避免複雜
  • 多樣的配置
  • 簡單化努力
  • 以基礎設施 -> 面向用戶進行轉變
  • 配置的必選項和可選項
  • 配置的防禦編程
  • 權限和變更跟踪
  • 配置的版本和應用對齊
  • 安全的配置變更：逐步部署、回滾更改、自動回滾
© Kimi Tsai all right reserved.            Updated : 2023-07-12 09:04:53

results matching ""

    No results matching ""

    results matching ""

      No results matching ""