Support multiple namespaces in the Nomad Provider

This commit is contained in:
Romain 2023-03-21 15:50:06 +01:00 committed by GitHub
parent 358f47443e
commit f7be1e97df
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
22 changed files with 266 additions and 60 deletions

View file

@ -142,6 +142,7 @@ issues:
- 'SA1019: cfg.FeaturePolicy is deprecated'
- 'SA1019: c.Providers.ConsulCatalog.Namespace is deprecated'
- 'SA1019: c.Providers.Consul.Namespace is deprecated'
- 'SA1019: c.Providers.Nomad.Namespace is deprecated'
exclude-rules:
- path: '(.+)_test.go'
linters:

View file

@ -7,6 +7,7 @@ This page is maintained and updated periodically to reflect our roadmap and any
| [Pilot](#pilot) | 2.7 | 2.8 | 2.9 |
| [Consul Enterprise Namespace](#consul-enterprise-namespace) | 2.8 | N/A | 3.0 |
| [TLS 1.0 and 1.1 Support](#tls-10-and-11) | N/A | 2.8 | N/A |
| [Nomad Namespace](#nomad-namespace) | 2.10 | N/A | 3.0 |
| [Kubernetes CRDs API Group `traefik.containo.us`](#kubernetes-crds-api-group-traefikcontainous) | 2.10 | N/A | 3.0 |
| [Kubernetes CRDs API Version `traefik.io/v1alpha1`](#kubernetes-crds-api-version-traefikiov1alpha1) | N/A | N/A | 3.0 |
@ -29,6 +30,11 @@ please use the `namespaces` options instead.
Starting on 2.8 the default TLS options will use the minimum version of TLS 1.2. Of course, it can still be overridden with custom configuration.
### Nomad Namespace
Starting on 2.10 the `namespace` option of the Nomad provider is deprecated,
please use the `namespaces` options instead.
### Kubernetes CRDs API Group `traefik.containo.us`
In v2.10, the Kubernetes CRDs API Group `traefik.containo.us` is deprecated, and its support will end starting with Traefik v3. Please use the API Group `traefik.io` instead.

View file

@ -499,6 +499,12 @@ In `v2.9`, Traefik Pilot support has been removed.
## v2.10
### Nomad Namespace
In `v2.10`, the `namespace` option of the Nomad provider is deprecated, please use the `namespaces` options instead.
## v2.10
### Kubernetes CRDs
In `v2.10`, the Kubernetes CRDs API Group `traefik.containo.us` is deprecated, and its support will end starting with Traefik v3. Please use the API Group `traefik.io` instead.

View file

@ -442,24 +442,65 @@ For additional information, refer to [Restrict the Scope of Service Discovery](.
### `namespace`
??? warning "Deprecated in favor of the [`namespaces`](#namespaces) option."
_Optional, Default=""_
The `namespace` option defines the namespace in which the Nomad services will be discovered.
!!! warning
One should only define either the `namespaces` option or the `namespace` option.
```yaml tab="File (YAML)"
providers:
nomad:
namespace: "production"
# ...
```
```toml tab="File (TOML)"
[providers.nomad]
namespace = "production"
# ...
```
```bash tab="CLI"
--providers.nomad.namespace=production
# ...
```
### `namespaces`
_Optional, Default=""_
The `namespace` option defines the namespace in which the Nomad services will be discovered.
The `namespaces` option defines the namespaces in which the nomad services will be discovered.
When using the `namespaces` option, the discovered object names will be suffixed as shown below:
```text
<resource-name>@nomad-<namespace>
```
!!! warning
One should only define either the `namespaces` option or the `namespace` option.
```yaml tab="File (YAML)"
providers:
nomad:
namespace: "production"
namespaces:
- "ns1"
- "ns2"
# ...
```
```toml tab="File (TOML)"
[providers.nomad]
namespace = "production"
namespaces = ["ns1", "ns2"]
# ...
```
```bash tab="CLI"
--providers.nomad.namespace=production
--providers.nomad.namespaces=ns1,ns2
# ...
```

View file

@ -858,6 +858,9 @@ Expose Nomad services by default. (Default: ```true```)
`--providers.nomad.namespace`:
Sets the Nomad namespace used to discover services.
`--providers.nomad.namespaces`:
Sets the Nomad namespaces used to discover services.
`--providers.nomad.prefix`:
Prefix for nomad service tags. (Default: ```traefik```)

View file

@ -858,6 +858,9 @@ Expose Nomad services by default. (Default: ```true```)
`TRAEFIK_PROVIDERS_NOMAD_NAMESPACE`:
Sets the Nomad namespace used to discover services.
`TRAEFIK_PROVIDERS_NOMAD_NAMESPACES`:
Sets the Nomad namespaces used to discover services.
`TRAEFIK_PROVIDERS_NOMAD_PREFIX`:
Prefix for nomad service tags. (Default: ```traefik```)

View file

@ -181,6 +181,7 @@
prefix = "foobar"
stale = true
namespace = "foobar"
namespaces = ["foobar", "foobar"]
exposedByDefault = true
refreshInterval = "42s"
[providers.nomad.endpoint]

View file

@ -195,6 +195,9 @@ providers:
prefix: foobar
stale: true
namespace: foobar
namespaces:
- foobar
- foobar
exposedByDefault: true
refreshInterval: 42s
endpoint:

View file

@ -294,19 +294,19 @@ Kubernetes cluster before creating `TCPRoute` objects.
kind: TraefikService # [11]
```
| Ref | Attribute | Description |
|------|---------------|-----------------------------------------------------------------------------------------------------------------------------------|
| [1] | `parentRefs` | References the resources (usually Gateways) that a Route wants to be attached to. |
| [2] | `name` | Name of the referent. |
| [3] | `namespace` | Namespace of the referent. When unspecified (or empty string), this refers to the local namespace of the Route. |
| [4] | `sectionName` | Name of a section within the target resource (the Listener name). |
| [5] | `rules` | Rules are a list of TCP matchers and actions. |
| [6] | `backendRefs` | Defines the backend(s) where matching requests should be sent. |
| [7] | `name` | The name of the referent service. |
| [8] | `weight` | The proportion of traffic forwarded to a targetRef, computed as weight/(sum of all weights in targetRefs). |
| [9] | `port` | The port of the referent service. |
| [10] | `group` | Group is the group of the referent. Only traefik.io`, `traefik.containo.us` and `gateway.networking.k8s.io` values are supported. |
| [11] | `kind` | Kind is kind of the referent. Only `TraefikService` and `Service` values are supported. |
| Ref | Attribute | Description |
|------|---------------|------------------------------------------------------------------------------------------------------------------------------------|
| [1] | `parentRefs` | References the resources (usually Gateways) that a Route wants to be attached to. |
| [2] | `name` | Name of the referent. |
| [3] | `namespace` | Namespace of the referent. When unspecified (or empty string), this refers to the local namespace of the Route. |
| [4] | `sectionName` | Name of a section within the target resource (the Listener name). |
| [5] | `rules` | Rules are a list of TCP matchers and actions. |
| [6] | `backendRefs` | Defines the backend(s) where matching requests should be sent. |
| [7] | `name` | The name of the referent service. |
| [8] | `weight` | The proportion of traffic forwarded to a targetRef, computed as weight/(sum of all weights in targetRefs). |
| [9] | `port` | The port of the referent service. |
| [10] | `group` | Group is the group of the referent. Only `traefik.io`, `traefik.containo.us` and `gateway.networking.k8s.io` values are supported. |
| [11] | `kind` | Kind is kind of the referent. Only `TraefikService` and `Service` values are supported. |
### Kind: `TLSRoute`
@ -340,17 +340,17 @@ Kubernetes cluster before creating `TLSRoute` objects.
kind: TraefikService # [12]
```
| Ref | Attribute | Description |
|------|---------------|-----------------------------------------------------------------------------------------------------------------------------------|
| [1] | `parentRefs` | References the resources (usually Gateways) that a Route wants to be attached to. |
| [2] | `name` | Name of the referent. |
| [3] | `namespace` | Namespace of the referent. When unspecified (or empty string), this refers to the local namespace of the Route. |
| [4] | `sectionName` | Name of a section within the target resource (the Listener name). |
| [5] | `hostnames` | Defines a set of SNI names that should match against the SNI attribute of TLS ClientHello message in TLS handshake. |
| [6] | `rules` | Rules are a list of TCP matchers and actions. |
| [7] | `backendRefs` | Defines the backend(s) where matching requests should be sent. |
| [8] | `name` | The name of the referent service. |
| [9] | `weight` | The proportion of traffic forwarded to a targetRef, computed as weight/(sum of all weights in targetRefs). |
| [10] | `port` | The port of the referent service. |
| [11] | `group` | Group is the group of the referent. Only traefik.io`, `traefik.containo.us` and `gateway.networking.k8s.io` values are supported. |
| [12] | `kind` | Kind is kind of the referent. Only `TraefikService` and `Service` values are supported. |
| Ref | Attribute | Description |
|------|---------------|------------------------------------------------------------------------------------------------------------------------------------|
| [1] | `parentRefs` | References the resources (usually Gateways) that a Route wants to be attached to. |
| [2] | `name` | Name of the referent. |
| [3] | `namespace` | Namespace of the referent. When unspecified (or empty string), this refers to the local namespace of the Route. |
| [4] | `sectionName` | Name of a section within the target resource (the Listener name). |
| [5] | `hostnames` | Defines a set of SNI names that should match against the SNI attribute of TLS ClientHello message in TLS handshake. |
| [6] | `rules` | Rules are a list of TCP matchers and actions. |
| [7] | `backendRefs` | Defines the backend(s) where matching requests should be sent. |
| [8] | `name` | The name of the referent service. |
| [9] | `weight` | The proportion of traffic forwarded to a targetRef, computed as weight/(sum of all weights in targetRefs). |
| [10] | `port` | The port of the referent service. |
| [11] | `group` | Group is the group of the referent. Only `traefik.io`, `traefik.containo.us` and `gateway.networking.k8s.io` values are supported. |
| [12] | `kind` | Kind is kind of the referent. Only `TraefikService` and `Service` values are supported. |

View file

@ -171,7 +171,6 @@
samplingType = "foobar"
samplingParam = 42.0
localAgentHostPort = "foobar"
localAgentSocket = "foobar"
gen128Bit = true
propagation = "foobar"
traceContextHeaderName = "foobar"

View file

@ -186,7 +186,7 @@ type Providers struct {
Rest *rest.Provider `description:"Enable Rest backend with default settings." json:"rest,omitempty" toml:"rest,omitempty" yaml:"rest,omitempty" export:"true" label:"allowEmpty" file:"allowEmpty"`
Rancher *rancher.Provider `description:"Enable Rancher backend with default settings." json:"rancher,omitempty" toml:"rancher,omitempty" yaml:"rancher,omitempty" export:"true" label:"allowEmpty" file:"allowEmpty"`
ConsulCatalog *consulcatalog.ProviderBuilder `description:"Enable ConsulCatalog backend with default settings." json:"consulCatalog,omitempty" toml:"consulCatalog,omitempty" yaml:"consulCatalog,omitempty" label:"allowEmpty" file:"allowEmpty" export:"true"`
Nomad *nomad.Provider `description:"Enable Nomad backend with default settings." json:"nomad,omitempty" toml:"nomad,omitempty" yaml:"nomad,omitempty" label:"allowEmpty" file:"allowEmpty" export:"true"`
Nomad *nomad.ProviderBuilder `description:"Enable Nomad backend with default settings." json:"nomad,omitempty" toml:"nomad,omitempty" yaml:"nomad,omitempty" label:"allowEmpty" file:"allowEmpty" export:"true"`
Ecs *ecs.Provider `description:"Enable AWS ECS backend with default settings." json:"ecs,omitempty" toml:"ecs,omitempty" yaml:"ecs,omitempty" label:"allowEmpty" file:"allowEmpty" export:"true"`
Consul *consul.ProviderBuilder `description:"Enable Consul backend with default settings." json:"consul,omitempty" toml:"consul,omitempty" yaml:"consul,omitempty" label:"allowEmpty" file:"allowEmpty" export:"true"`
@ -326,11 +326,15 @@ func (c *Configuration) ValidateConfiguration() error {
}
if c.Providers.ConsulCatalog != nil && c.Providers.ConsulCatalog.Namespace != "" && len(c.Providers.ConsulCatalog.Namespaces) > 0 {
return fmt.Errorf("consul catalog provider cannot have both namespace and namespaces options configured")
return fmt.Errorf("Consul Catalog provider cannot have both namespace and namespaces options configured")
}
if c.Providers.Consul != nil && c.Providers.Consul.Namespace != "" && len(c.Providers.Consul.Namespaces) > 0 {
return fmt.Errorf("consul provider cannot have both namespace and namespaces options configured")
return fmt.Errorf("Consul provider cannot have both namespace and namespaces options configured")
}
if c.Providers.Nomad != nil && c.Providers.Nomad.Namespace != "" && len(c.Providers.Nomad.Namespaces) > 0 {
return fmt.Errorf("Nomad provider cannot have both namespace and namespaces options configured")
}
return nil

View file

@ -115,7 +115,9 @@ func NewProviderAggregator(conf static.Providers) ProviderAggregator {
}
if conf.Nomad != nil {
p.quietAddProvider(conf.Nomad)
for _, pvd := range conf.Nomad.BuildProviders() {
p.quietAddProvider(pvd)
}
}
if conf.Consul != nil {

View file

@ -4,6 +4,7 @@ import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/traefik/traefik/v2/pkg/config/dynamic"
)
@ -2509,5 +2510,57 @@ func Test_keepItem(t *testing.T) {
}
}
func TestNamespaces(t *testing.T) {
testCases := []struct {
desc string
namespace string
namespaces []string
expectedNamespaces []string
}{
{
desc: "no defined namespaces",
expectedNamespaces: []string{""},
},
{
desc: "deprecated: use of defined namespace",
namespace: "test-ns",
expectedNamespaces: []string{"test-ns"},
},
{
desc: "use of 1 defined namespaces",
namespaces: []string{"test-ns"},
expectedNamespaces: []string{"test-ns"},
},
{
desc: "use of multiple defined namespaces",
namespaces: []string{"test-ns1", "test-ns2", "test-ns3", "test-ns4"},
expectedNamespaces: []string{"test-ns1", "test-ns2", "test-ns3", "test-ns4"},
},
}
for _, test := range testCases {
test := test
t.Run(test.desc, func(t *testing.T) {
t.Parallel()
pb := &ProviderBuilder{
Namespace: test.namespace,
Namespaces: test.namespaces,
}
assert.Equal(t, test.expectedNamespaces, extractNamespacesFromProvider(pb.BuildProviders()))
})
}
}
func extractNamespacesFromProvider(providers []*Provider) []string {
res := make([]string, len(providers))
for i, p := range providers {
res[i] = p.namespace
}
return res
}
func Int(v int) *int { return &v }
func Bool(v bool) *bool { return &v }

View file

@ -2,6 +2,7 @@ package nomad
import (
"context"
"errors"
"fmt"
"strings"
"text/template"
@ -46,17 +47,79 @@ type item struct {
ExtraConf configuration // global options
}
// Provider holds configurations of the provider.
type Provider struct {
// ProviderBuilder is responsible for constructing namespaced instances of the Nomad provider.
type ProviderBuilder struct {
Configuration `yaml:",inline" export:"true"`
// Deprecated: Use Namespaces option instead
Namespace string `description:"Sets the Nomad namespace used to discover services." json:"namespace,omitempty" toml:"namespace,omitempty" yaml:"namespace,omitempty"`
Namespaces []string `description:"Sets the Nomad namespaces used to discover services." json:"namespaces,omitempty" toml:"namespaces,omitempty" yaml:"namespaces,omitempty"`
}
// BuildProviders builds Nomad provider instances for the given namespaces configuration.
func (p *ProviderBuilder) BuildProviders() []*Provider {
if p.Namespace != "" {
log.WithoutContext().Warnf("Namespace option is deprecated, please use the Namespaces option instead.")
}
if len(p.Namespaces) == 0 {
return []*Provider{{
Configuration: p.Configuration,
name: providerName,
// p.Namespace could be empty
namespace: p.Namespace,
}}
}
var providers []*Provider
for _, namespace := range p.Namespaces {
providers = append(providers, &Provider{
Configuration: p.Configuration,
name: providerName + "-" + namespace,
namespace: namespace,
})
}
return providers
}
// Configuration represents the Nomad provider configuration.
type Configuration struct {
DefaultRule string `description:"Default rule." json:"defaultRule,omitempty" toml:"defaultRule,omitempty" yaml:"defaultRule,omitempty"`
Constraints string `description:"Constraints is an expression that Traefik matches against the Nomad service's tags to determine whether to create route(s) for that service." json:"constraints,omitempty" toml:"constraints,omitempty" yaml:"constraints,omitempty" export:"true"`
Endpoint *EndpointConfig `description:"Nomad endpoint settings" json:"endpoint,omitempty" toml:"endpoint,omitempty" yaml:"endpoint,omitempty" export:"true"`
Prefix string `description:"Prefix for nomad service tags." json:"prefix,omitempty" toml:"prefix,omitempty" yaml:"prefix,omitempty" export:"true"`
Stale bool `description:"Use stale consistency for catalog reads." json:"stale,omitempty" toml:"stale,omitempty" yaml:"stale,omitempty" export:"true"`
Namespace string `description:"Sets the Nomad namespace used to discover services." json:"namespace,omitempty" toml:"namespace,omitempty" yaml:"namespace,omitempty" export:"true"`
ExposedByDefault bool `description:"Expose Nomad services by default." json:"exposedByDefault,omitempty" toml:"exposedByDefault,omitempty" yaml:"exposedByDefault,omitempty" export:"true"`
RefreshInterval ptypes.Duration `description:"Interval for polling Nomad API." json:"refreshInterval,omitempty" toml:"refreshInterval,omitempty" yaml:"refreshInterval,omitempty" export:"true"`
}
// SetDefaults sets the default values for the Nomad Traefik Provider Configuration.
func (c *Configuration) SetDefaults() {
defConfig := api.DefaultConfig()
c.Endpoint = &EndpointConfig{
Address: defConfig.Address,
Region: defConfig.Region,
Token: defConfig.SecretID,
TLS: &types.ClientTLS{
CA: defConfig.TLSConfig.CACert,
Cert: defConfig.TLSConfig.ClientCert,
Key: defConfig.TLSConfig.ClientKey,
InsecureSkipVerify: defConfig.TLSConfig.Insecure,
},
}
c.Prefix = defaultPrefix
c.ExposedByDefault = true
c.RefreshInterval = ptypes.Duration(15 * time.Second)
c.DefaultRule = defaultTemplateRule
}
// Provider holds configuration along with the namespace it will discover services in.
type Provider struct {
Configuration
name string
namespace string
client *api.Client // client for Nomad API
defaultRuleTpl *template.Template // default routing rule
}
@ -74,31 +137,26 @@ type EndpointConfig struct {
// SetDefaults sets the default values for the Nomad Traefik Provider.
func (p *Provider) SetDefaults() {
defConfig := api.DefaultConfig()
p.Endpoint = &EndpointConfig{
Address: defConfig.Address,
Region: defConfig.Region,
Token: defConfig.SecretID,
TLS: &types.ClientTLS{
CA: defConfig.TLSConfig.CACert,
Cert: defConfig.TLSConfig.ClientCert,
Key: defConfig.TLSConfig.ClientKey,
InsecureSkipVerify: defConfig.TLSConfig.Insecure,
},
}
p.Prefix = defaultPrefix
p.ExposedByDefault = true
p.RefreshInterval = ptypes.Duration(15 * time.Second)
p.DefaultRule = defaultTemplateRule
p.Configuration.SetDefaults()
}
// Init the Nomad Traefik Provider.
func (p *Provider) Init() error {
if p.namespace == api.AllNamespacesNamespace {
return errors.New("wildcard namespace not supported")
}
defaultRuleTpl, err := provider.MakeDefaultRuleTemplate(p.DefaultRule, nil)
if err != nil {
return fmt.Errorf("error while parsing default rule: %w", err)
}
p.defaultRuleTpl = defaultRuleTpl
// In case they didn't initialize Provider with BuildProviders
if p.name == "" {
p.name = providerName
}
return nil
}
@ -106,13 +164,13 @@ func (p *Provider) Init() error {
// using the given configuration channel.
func (p *Provider) Provide(configurationChan chan<- dynamic.Message, pool *safe.Pool) error {
var err error
p.client, err = createClient(p.Namespace, p.Endpoint)
p.client, err = createClient(p.namespace, p.Endpoint)
if err != nil {
return fmt.Errorf("failed to create nomad API client: %w", err)
}
pool.GoCtx(func(routineCtx context.Context) {
ctxLog := log.With(routineCtx, log.Str(log.ProviderName, providerName))
ctxLog := log.With(routineCtx, log.Str(log.ProviderName, p.name))
logger := log.FromContext(ctxLog)
operation := func() error {
@ -165,7 +223,7 @@ func (p *Provider) loadConfiguration(ctx context.Context, configurationC chan<-
return err
}
configurationC <- dynamic.Message{
ProviderName: providerName,
ProviderName: p.name,
Configuration: p.buildConfig(ctx, items),
}

View file

@ -66,7 +66,12 @@ func Test_globalConfig(t *testing.T) {
for _, test := range cases {
t.Run(test.Name, func(t *testing.T) {
p := Provider{ExposedByDefault: test.ExposedByDefault, Prefix: test.Prefix}
p := Provider{
Configuration: Configuration{
ExposedByDefault: test.ExposedByDefault,
Prefix: test.Prefix,
},
}
result := p.getExtraConf(test.Tags)
require.Equal(t, test.exp, result)
})
@ -148,7 +153,7 @@ func Test_getNomadServiceData(t *testing.T) {
require.NoError(t, err)
// fudge client, avoid starting up via Provide
p.client, err = createClient(p.Namespace, p.Endpoint)
p.client, err = createClient(p.namespace, p.Endpoint)
require.NoError(t, err)
// make the query for services

View file

@ -1157,6 +1157,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}

View file

@ -75,6 +75,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}

View file

@ -141,6 +141,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}

View file

@ -155,6 +155,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}

View file

@ -75,6 +75,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}

View file

@ -20,6 +20,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}

View file

@ -37,6 +37,9 @@ export default {
if (name.startsWith('consulcatalog-')) {
return `statics/providers/consulcatalog.svg`
}
if (name.startsWith('nomad-')) {
return `statics/providers/nomad.svg`
}
return `statics/providers/${name}.svg`
}