h2c server
This commit is contained in:
parent
bfdd1997f6
commit
e76836b948
7 changed files with 690 additions and 70 deletions
File diff suppressed because one or more lines are too long
Before Width: | Height: | Size: 146 KiB After Width: | Height: | Size: 186 KiB |
|
@ -1,30 +1,10 @@
|
||||||
# gRPC example
|
# gRPC examples
|
||||||
|
|
||||||
This section explains how to use Traefik as reverse proxy for gRPC application with self-signed certificates.
|
## With HTTP (h2c)
|
||||||
|
|
||||||
!!! warning
|
This section explains how to use Traefik as reverse proxy for gRPC application.
|
||||||
As gRPC needs HTTP2, we need HTTPS certificates on Træfik.
|
|
||||||
For exchanges with the backend, we will use h2c (HTTP2 on HTTP without TLS)
|
|
||||||
|
|
||||||
<p align="center">
|
### Træfik configuration
|
||||||
<img src="/img/grpc.svg" alt="gRPC architecture" title="gRPC architecture" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## gRPC Client certificate
|
|
||||||
|
|
||||||
Generate your self-signed certificate for frontend url:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./frontend.key -out ./frontend.cert
|
|
||||||
```
|
|
||||||
|
|
||||||
with
|
|
||||||
|
|
||||||
```
|
|
||||||
Common Name (e.g. server FQDN or YOUR name) []: frontend.local
|
|
||||||
```
|
|
||||||
|
|
||||||
## Træfik configuration
|
|
||||||
|
|
||||||
At last, we configure our Træfik instance to use both self-signed certificates.
|
At last, we configure our Træfik instance to use both self-signed certificates.
|
||||||
|
|
||||||
|
@ -32,14 +12,9 @@ At last, we configure our Træfik instance to use both self-signed certificates.
|
||||||
defaultEntryPoints = ["https"]
|
defaultEntryPoints = ["https"]
|
||||||
|
|
||||||
[entryPoints]
|
[entryPoints]
|
||||||
[entryPoints.https]
|
[entryPoints.http]
|
||||||
address = ":4443"
|
address = ":80"
|
||||||
[entryPoints.https.tls]
|
[entryPoints.http]
|
||||||
# For secure connection on frontend.local
|
|
||||||
[[entryPoints.https.tls.certificates]]
|
|
||||||
certFile = "./frontend.cert"
|
|
||||||
keyFile = "./frontend.key"
|
|
||||||
|
|
||||||
|
|
||||||
[api]
|
[api]
|
||||||
|
|
||||||
|
@ -62,23 +37,115 @@ defaultEntryPoints = ["https"]
|
||||||
!!! warning
|
!!! warning
|
||||||
For provider with label, you will have to specify the `traefik.protocol=h2c`
|
For provider with label, you will have to specify the `traefik.protocol=h2c`
|
||||||
|
|
||||||
## Conclusion
|
### Conclusion
|
||||||
|
|
||||||
We don't need specific configuration to use gRPC in Træfik, we just need to be careful that exchanges between client and Træfik are HTTPS communications.
|
We don't need specific configuration to use gRPC in Træfik, we just need to use `h2c` protocol, or use HTTPS communications to have HTTP2 with the backend.
|
||||||
For exchanges between Træfik and backend, you need to use `h2c` protocol, or use HTTPS communications to have HTTP2.
|
|
||||||
|
|
||||||
## A gRPC example in go
|
## With HTTPS
|
||||||
|
|
||||||
We will use the gRPC greeter example in [grpc-go](https://github.com/grpc/grpc-go/tree/master/examples/helloworld)
|
This section explains how to use Traefik as reverse proxy for gRPC application with self-signed certificates.
|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
<img src="/img/grpc.svg" alt="gRPC architecture" title="gRPC architecture" />
|
||||||
|
</p>
|
||||||
|
|
||||||
|
### gRPC Server certificate
|
||||||
|
|
||||||
|
In order to secure the gRPC server, we generate a self-signed certificate for backend url:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./backend.key -out ./backend.cert
|
||||||
|
```
|
||||||
|
|
||||||
|
That will prompt for information, the important answer is:
|
||||||
|
|
||||||
|
```
|
||||||
|
Common Name (e.g. server FQDN or YOUR name) []: backend.local
|
||||||
|
```
|
||||||
|
|
||||||
|
### gRPC Client certificate
|
||||||
|
|
||||||
|
Generate your self-signed certificate for frontend url:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./frontend.key -out ./frontend.cert
|
||||||
|
```
|
||||||
|
|
||||||
|
with
|
||||||
|
|
||||||
|
```
|
||||||
|
Common Name (e.g. server FQDN or YOUR name) []: frontend.local
|
||||||
|
```
|
||||||
|
|
||||||
|
### Træfik configuration
|
||||||
|
|
||||||
|
At last, we configure our Træfik instance to use both self-signed certificates.
|
||||||
|
|
||||||
|
```toml
|
||||||
|
defaultEntryPoints = ["https"]
|
||||||
|
|
||||||
|
# For secure connection on backend.local
|
||||||
|
rootCAs = [ "./backend.cert" ]
|
||||||
|
|
||||||
|
[entryPoints]
|
||||||
|
[entryPoints.https]
|
||||||
|
address = ":4443"
|
||||||
|
[entryPoints.https.tls]
|
||||||
|
# For secure connection on frontend.local
|
||||||
|
[[entryPoints.https.tls.certificates]]
|
||||||
|
certFile = "./frontend.cert"
|
||||||
|
keyFile = "./frontend.key"
|
||||||
|
|
||||||
|
|
||||||
|
[api]
|
||||||
|
|
||||||
|
[file]
|
||||||
|
|
||||||
|
[backends]
|
||||||
|
[backends.backend1]
|
||||||
|
[backends.backend1.servers.server1]
|
||||||
|
# Access on backend with HTTPS
|
||||||
|
url = "https://backend.local:8080"
|
||||||
|
|
||||||
|
|
||||||
|
[frontends]
|
||||||
|
[frontends.frontend1]
|
||||||
|
backend = "backend1"
|
||||||
|
[frontends.frontend1.routes.test_1]
|
||||||
|
rule = "Host:frontend.local"
|
||||||
|
```
|
||||||
|
|
||||||
|
!!! warning
|
||||||
|
With some backends, the server URLs use the IP, so you may need to configure `insecureSkipVerify` instead of the `rootCAS` to activate HTTPS without hostname verification.
|
||||||
|
|
||||||
|
### A gRPC example in go (modify for https)
|
||||||
|
|
||||||
|
We use the gRPC greeter example in [grpc-go](https://github.com/grpc/grpc-go/tree/master/examples/helloworld)
|
||||||
|
|
||||||
|
!!! warning
|
||||||
|
In order to use this gRPC example, we need to modify it to use HTTPS
|
||||||
|
|
||||||
|
So we modify the "gRPC server example" to use our own self-signed certificate:
|
||||||
|
|
||||||
We can keep the Server example as is with the h2c protocol
|
|
||||||
```go
|
```go
|
||||||
// ...
|
// ...
|
||||||
lis, err := net.Listen("tcp", port)
|
|
||||||
|
// Read cert and key file
|
||||||
|
BackendCert, _ := ioutil.ReadFile("./backend.cert")
|
||||||
|
BackendKey, _ := ioutil.ReadFile("./backend.key")
|
||||||
|
|
||||||
|
// Generate Certificate struct
|
||||||
|
cert, err := tls.X509KeyPair(BackendCert, BackendKey)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatalf("failed to listen: %v", err)
|
log.Fatalf("failed to parse certificate: %v", err)
|
||||||
}
|
}
|
||||||
var s *grpc.Server = grpc.NewServer()
|
|
||||||
|
// Create credentials
|
||||||
|
creds := credentials.NewServerTLSFromCert(&cert)
|
||||||
|
|
||||||
|
// Use Credentials in gRPC server options
|
||||||
|
serverOption := grpc.Creds(creds)
|
||||||
|
var s *grpc.Server = grpc.NewServer(serverOption)
|
||||||
defer s.Stop()
|
defer s.Stop()
|
||||||
|
|
||||||
pb.RegisterGreeterServer(s, &server{})
|
pb.RegisterGreeterServer(s, &server{})
|
||||||
|
@ -87,10 +154,6 @@ err := s.Serve(lis)
|
||||||
// ...
|
// ...
|
||||||
```
|
```
|
||||||
|
|
||||||
!!! warning
|
|
||||||
In order to use this gRPC example, we need to modify it to use HTTPS
|
|
||||||
|
|
||||||
|
|
||||||
Next we will modify gRPC Client to use our Træfik self-signed certificate:
|
Next we will modify gRPC Client to use our Træfik self-signed certificate:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
|
|
475
h2c/h2c.go
Normal file
475
h2c/h2c.go
Normal file
|
@ -0,0 +1,475 @@
|
||||||
|
// Copyright 2018 The Go Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a BSD-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
// Package h2c implements the h2c part of HTTP/2.
|
||||||
|
//
|
||||||
|
// The h2c protocol is the non-TLS secured version of HTTP/2 which is not
|
||||||
|
// available from net/http.
|
||||||
|
package h2c
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"bytes"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"net/textproto"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/containous/traefik/log"
|
||||||
|
|
||||||
|
"golang.org/x/net/http2"
|
||||||
|
"golang.org/x/net/http2/hpack"
|
||||||
|
"golang.org/x/net/lex/httplex"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
http2VerboseLogs bool
|
||||||
|
)
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
e := os.Getenv("GODEBUG")
|
||||||
|
if strings.Contains(e, "http2debug=1") || strings.Contains(e, "http2debug=2") {
|
||||||
|
http2VerboseLogs = true
|
||||||
|
}
|
||||||
|
http2VerboseLogs = true
|
||||||
|
}
|
||||||
|
|
||||||
|
// Server implements net.Handler and enables h2c. Users who want h2c just need
|
||||||
|
// to provide an http.Server.
|
||||||
|
type Server struct {
|
||||||
|
*http.Server
|
||||||
|
originalHandler http.Handler
|
||||||
|
}
|
||||||
|
|
||||||
|
// Serve Put a middleware around the original handler to handle h2c
|
||||||
|
func (s Server) Serve(l net.Listener) error {
|
||||||
|
originalHandler := s.Server.Handler
|
||||||
|
if originalHandler == nil {
|
||||||
|
originalHandler = http.DefaultServeMux
|
||||||
|
}
|
||||||
|
s.Server.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method == "PRI" && r.URL.Path == "*" && r.Proto == "HTTP/2.0" {
|
||||||
|
if http2VerboseLogs {
|
||||||
|
log.Debugf("Attempting h2c with prior knowledge.")
|
||||||
|
}
|
||||||
|
conn, err := initH2CWithPriorKnowledge(w)
|
||||||
|
if err != nil {
|
||||||
|
if http2VerboseLogs {
|
||||||
|
log.Debugf("Error h2c with prior knowledge: %v", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer conn.Close()
|
||||||
|
h2cSrv := &http2.Server{}
|
||||||
|
h2cSrv.ServeConn(conn, &http2.ServeConnOpts{Handler: originalHandler})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if conn, err := h2cUpgrade(w, r); err == nil {
|
||||||
|
defer conn.Close()
|
||||||
|
h2cSrv := &http2.Server{}
|
||||||
|
h2cSrv.ServeConn(conn, &http2.ServeConnOpts{Handler: originalHandler})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
originalHandler.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
return s.Server.Serve(l)
|
||||||
|
}
|
||||||
|
|
||||||
|
// initH2CWithPriorKnowledge implements creating a h2c connection with prior
|
||||||
|
// knowledge (Section 3.4) and creates a net.Conn suitable for http2.ServeConn.
|
||||||
|
// All we have to do is look for the client preface that is suppose to be part
|
||||||
|
// of the body, and reforward the client preface on the net.Conn this function
|
||||||
|
// creates.
|
||||||
|
func initH2CWithPriorKnowledge(w http.ResponseWriter) (net.Conn, error) {
|
||||||
|
hijacker, ok := w.(http.Hijacker)
|
||||||
|
if !ok {
|
||||||
|
return nil, errors.New("hijack not supported")
|
||||||
|
}
|
||||||
|
conn, rw, err := hijacker.Hijack()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("hijack failed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
expectedBody := "SM\r\n\r\n"
|
||||||
|
|
||||||
|
buf := make([]byte, len(expectedBody))
|
||||||
|
n, err := io.ReadFull(rw, buf)
|
||||||
|
|
||||||
|
if bytes.Equal(buf[0:n], []byte(expectedBody)) {
|
||||||
|
c := &rwConn{
|
||||||
|
Conn: conn,
|
||||||
|
Reader: io.MultiReader(bytes.NewBuffer([]byte(http2.ClientPreface)), rw),
|
||||||
|
BufWriter: rw.Writer,
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
conn.Close()
|
||||||
|
if http2VerboseLogs {
|
||||||
|
log.Printf(
|
||||||
|
"Missing the request body portion of the client preface. Wanted: %v Got: %v",
|
||||||
|
[]byte(expectedBody),
|
||||||
|
buf[0:n],
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return nil, errors.New("invalid client preface")
|
||||||
|
}
|
||||||
|
|
||||||
|
// drainClientPreface reads a single instance of the HTTP/2 client preface from
|
||||||
|
// the supplied reader.
|
||||||
|
func drainClientPreface(r io.Reader) error {
|
||||||
|
var buf bytes.Buffer
|
||||||
|
prefaceLen := int64(len([]byte(http2.ClientPreface)))
|
||||||
|
n, err := io.CopyN(&buf, r, prefaceLen)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if n != prefaceLen || buf.String() != http2.ClientPreface {
|
||||||
|
return fmt.Errorf("Client never sent: %s", http2.ClientPreface)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// h2cUpgrade establishes a h2c connection using the HTTP/1 upgrade (Section 3.2).
|
||||||
|
func h2cUpgrade(w http.ResponseWriter, r *http.Request) (net.Conn, error) {
|
||||||
|
if !isH2CUpgrade(r.Header) {
|
||||||
|
return nil, errors.New("non-conforming h2c headers")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Initial bytes we put into conn to fool http2 server
|
||||||
|
initBytes, _, err := convertH1ReqToH2(r)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
hijacker, ok := w.(http.Hijacker)
|
||||||
|
if !ok {
|
||||||
|
return nil, errors.New("hijack not supported")
|
||||||
|
}
|
||||||
|
conn, rw, err := hijacker.Hijack()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("hijack failed: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rw.Write([]byte("HTTP/1.1 101 Switching Protocols\r\n" +
|
||||||
|
"Connection: Upgrade\r\n" +
|
||||||
|
"Upgrade: h2c\r\n\r\n"))
|
||||||
|
rw.Flush()
|
||||||
|
|
||||||
|
// A conforming client will now send an H2 client preface which need to drain
|
||||||
|
// since we already sent this.
|
||||||
|
if err := drainClientPreface(rw); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
c := &rwConn{
|
||||||
|
Conn: conn,
|
||||||
|
Reader: io.MultiReader(initBytes, rw),
|
||||||
|
BufWriter: newSettingsAckSwallowWriter(rw.Writer),
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// convert the data contained in the HTTP/1 upgrade request into the HTTP/2
|
||||||
|
// version in byte form.
|
||||||
|
func convertH1ReqToH2(r *http.Request) (*bytes.Buffer, []http2.Setting, error) {
|
||||||
|
h2Bytes := bytes.NewBuffer([]byte((http2.ClientPreface)))
|
||||||
|
framer := http2.NewFramer(h2Bytes, nil)
|
||||||
|
settings, err := getH2Settings(r.Header)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := framer.WriteSettings(settings...); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
headerBytes, err := getH2HeaderBytes(r, getMaxHeaderTableSize(settings))
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
maxFrameSize := int(getMaxFrameSize(settings))
|
||||||
|
needOneHeader := len(headerBytes) < maxFrameSize
|
||||||
|
err = framer.WriteHeaders(http2.HeadersFrameParam{
|
||||||
|
StreamID: 1,
|
||||||
|
BlockFragment: headerBytes,
|
||||||
|
EndHeaders: needOneHeader,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for i := maxFrameSize; i < len(headerBytes); i += maxFrameSize {
|
||||||
|
if len(headerBytes)-i > maxFrameSize {
|
||||||
|
if err := framer.WriteContinuation(1,
|
||||||
|
false, // endHeaders
|
||||||
|
headerBytes[i:maxFrameSize]); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if err := framer.WriteContinuation(1,
|
||||||
|
true, // endHeaders
|
||||||
|
headerBytes[i:]); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return h2Bytes, settings, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getMaxFrameSize returns the SETTINGS_MAX_FRAME_SIZE. If not present default
|
||||||
|
// value is 16384 as specified by RFC 7540 Section 6.5.2.
|
||||||
|
func getMaxFrameSize(settings []http2.Setting) uint32 {
|
||||||
|
for _, setting := range settings {
|
||||||
|
if setting.ID == http2.SettingMaxFrameSize {
|
||||||
|
return setting.Val
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 16384
|
||||||
|
}
|
||||||
|
|
||||||
|
// getMaxHeaderTableSize returns the SETTINGS_HEADER_TABLE_SIZE. If not present
|
||||||
|
// default value is 4096 as specified by RFC 7540 Section 6.5.2.
|
||||||
|
func getMaxHeaderTableSize(settings []http2.Setting) uint32 {
|
||||||
|
for _, setting := range settings {
|
||||||
|
if setting.ID == http2.SettingHeaderTableSize {
|
||||||
|
return setting.Val
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 4096
|
||||||
|
}
|
||||||
|
|
||||||
|
// bufWriter is a Writer interface that also has a Flush method.
|
||||||
|
type bufWriter interface {
|
||||||
|
io.Writer
|
||||||
|
Flush() error
|
||||||
|
}
|
||||||
|
|
||||||
|
// rwConn implements net.Conn but overrides Read and Write so that reads and
|
||||||
|
// writes are forwarded to the provided io.Reader and bufWriter.
|
||||||
|
type rwConn struct {
|
||||||
|
net.Conn
|
||||||
|
io.Reader
|
||||||
|
BufWriter bufWriter
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read forwards reads to the underlying Reader.
|
||||||
|
func (c *rwConn) Read(p []byte) (int, error) {
|
||||||
|
return c.Reader.Read(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write forwards writes to the underlying bufWriter and immediately flushes.
|
||||||
|
func (c *rwConn) Write(p []byte) (int, error) {
|
||||||
|
n, err := c.BufWriter.Write(p)
|
||||||
|
if err := c.BufWriter.Flush(); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return n, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// settingsAckSwallowWriter is a writer that normally forwards bytes to it's
|
||||||
|
// underlying Writer, but swallows the first SettingsAck frame that it sees.
|
||||||
|
type settingsAckSwallowWriter struct {
|
||||||
|
Writer *bufio.Writer
|
||||||
|
buf []byte
|
||||||
|
didSwallow bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// newSettingsAckSwallowWriter returns a new settingsAckSwallowWriter.
|
||||||
|
func newSettingsAckSwallowWriter(w *bufio.Writer) *settingsAckSwallowWriter {
|
||||||
|
return &settingsAckSwallowWriter{
|
||||||
|
Writer: w,
|
||||||
|
buf: make([]byte, 0),
|
||||||
|
didSwallow: false,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Write implements io.Writer interface. Normally forwards bytes to w.Writer,
|
||||||
|
// except for the first Settings ACK frame that it sees.
|
||||||
|
func (w *settingsAckSwallowWriter) Write(p []byte) (int, error) {
|
||||||
|
if !w.didSwallow {
|
||||||
|
w.buf = append(w.buf, p...)
|
||||||
|
// Process all the frames we have collected into w.buf
|
||||||
|
for {
|
||||||
|
// Append until we get full frame header which is 9 bytes
|
||||||
|
if len(w.buf) < 9 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// Check if we have collected a whole frame.
|
||||||
|
fh, err := http2.ReadFrameHeader(bytes.NewBuffer(w.buf))
|
||||||
|
if err != nil {
|
||||||
|
// Corrupted frame, fail current Write
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
fSize := fh.Length + 9
|
||||||
|
if uint32(len(w.buf)) < fSize {
|
||||||
|
// Have not collected whole frame. Stop processing buf, and withhold on
|
||||||
|
// forward bytes to w.Writer until we get the full frame.
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
// We have now collected a whole frame.
|
||||||
|
if fh.Type == http2.FrameSettings && fh.Flags.Has(http2.FlagSettingsAck) {
|
||||||
|
// If Settings ACK frame, do not forward to underlying writer, remove
|
||||||
|
// bytes from w.buf, and record that we have swallowed Settings Ack
|
||||||
|
// frame.
|
||||||
|
w.didSwallow = true
|
||||||
|
w.buf = w.buf[fSize:]
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
// Not settings ack frame. Forward bytes to w.Writer.
|
||||||
|
if _, err := w.Writer.Write(w.buf[:fSize]); err != nil {
|
||||||
|
// Couldn't forward bytes. Fail current Write.
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
w.buf = w.buf[fSize:]
|
||||||
|
}
|
||||||
|
return len(p), nil
|
||||||
|
}
|
||||||
|
return w.Writer.Write(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Flush calls w.Writer.Flush.
|
||||||
|
func (w *settingsAckSwallowWriter) Flush() error {
|
||||||
|
return w.Writer.Flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
// isH2CUpgrade returns true if the header properly request an upgrade to h2c
|
||||||
|
// as specified by Section 3.2.
|
||||||
|
func isH2CUpgrade(h http.Header) bool {
|
||||||
|
return httplex.HeaderValuesContainsToken(h[textproto.CanonicalMIMEHeaderKey("Upgrade")], "h2c") &&
|
||||||
|
httplex.HeaderValuesContainsToken(h[textproto.CanonicalMIMEHeaderKey("Connection")], "HTTP2-Settings")
|
||||||
|
}
|
||||||
|
|
||||||
|
// getH2Settings returns the []http2.Setting that are encoded in the
|
||||||
|
// HTTP2-Settings header.
|
||||||
|
func getH2Settings(h http.Header) ([]http2.Setting, error) {
|
||||||
|
vals, ok := h[textproto.CanonicalMIMEHeaderKey("HTTP2-Settings")]
|
||||||
|
if !ok {
|
||||||
|
return nil, errors.New("missing HTTP2-Settings header")
|
||||||
|
}
|
||||||
|
if len(vals) != 1 {
|
||||||
|
return nil, fmt.Errorf("expected 1 HTTP2-Settings. Got: %v", vals)
|
||||||
|
}
|
||||||
|
settings, err := decodeSettings(vals[0])
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("Invalid HTTP2-Settings: %q", vals[0])
|
||||||
|
}
|
||||||
|
return settings, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// decodeSettings decodes the base64url header value of the HTTP2-Settings
|
||||||
|
// header. RFC 7540 Section 3.2.1.
|
||||||
|
func decodeSettings(headerVal string) ([]http2.Setting, error) {
|
||||||
|
b, err := base64.RawURLEncoding.DecodeString(headerVal)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(b)%6 != 0 {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
settings := make([]http2.Setting, 0)
|
||||||
|
for i := 0; i < len(b)/6; i++ {
|
||||||
|
settings = append(settings, http2.Setting{
|
||||||
|
ID: http2.SettingID(binary.BigEndian.Uint16(b[i*6 : i*6+2])),
|
||||||
|
Val: binary.BigEndian.Uint32(b[i*6+2 : i*6+6]),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
return settings, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// getH2HeaderBytes return the headers in r a []bytes encoded by HPACK.
|
||||||
|
func getH2HeaderBytes(r *http.Request, maxHeaderTableSize uint32) ([]byte, error) {
|
||||||
|
headerBytes := bytes.NewBuffer(nil)
|
||||||
|
hpackEnc := hpack.NewEncoder(headerBytes)
|
||||||
|
hpackEnc.SetMaxDynamicTableSize(maxHeaderTableSize)
|
||||||
|
|
||||||
|
// Section 8.1.2.3
|
||||||
|
err := hpackEnc.WriteField(hpack.HeaderField{
|
||||||
|
Name: ":method",
|
||||||
|
Value: r.Method,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = hpackEnc.WriteField(hpack.HeaderField{
|
||||||
|
Name: ":scheme",
|
||||||
|
Value: "http",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = hpackEnc.WriteField(hpack.HeaderField{
|
||||||
|
Name: ":authority",
|
||||||
|
Value: r.Host,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
path := r.URL.Path
|
||||||
|
if r.URL.RawQuery != "" {
|
||||||
|
path = strings.Join([]string{path, r.URL.RawQuery}, "?")
|
||||||
|
}
|
||||||
|
err = hpackEnc.WriteField(hpack.HeaderField{
|
||||||
|
Name: ":path",
|
||||||
|
Value: path,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// TODO Implement Section 8.3
|
||||||
|
|
||||||
|
for header, values := range r.Header {
|
||||||
|
// Skip non h2 headers
|
||||||
|
if isNonH2Header(header) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, v := range values {
|
||||||
|
err := hpackEnc.WriteField(hpack.HeaderField{
|
||||||
|
Name: strings.ToLower(header),
|
||||||
|
Value: v,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return headerBytes.Bytes(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Connection specific headers listed in RFC 7540 Section 8.1.2.2 that are not
|
||||||
|
// suppose to be transferred to HTTP/2. The Http2-Settings header is skipped
|
||||||
|
// since already use to create the HTTP/2 SETTINGS frame.
|
||||||
|
var nonH2Headers = []string{
|
||||||
|
"Connection",
|
||||||
|
"Keep-Alive",
|
||||||
|
"Proxy-Connection",
|
||||||
|
"Transfer-Encoding",
|
||||||
|
"Upgrade",
|
||||||
|
"Http2-Settings",
|
||||||
|
}
|
||||||
|
|
||||||
|
// isNonH2Header returns true if header should not be transferred to HTTP/2.
|
||||||
|
func isNonH2Header(header string) bool {
|
||||||
|
for _, nonH2h := range nonH2Headers {
|
||||||
|
if header == nonH2h {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
|
@ -1,15 +1,10 @@
|
||||||
defaultEntryPoints = ["https"]
|
defaultEntryPoints = ["http"]
|
||||||
|
|
||||||
rootCAs = [ """{{ .CertContent }}""" ]
|
debug=true
|
||||||
|
|
||||||
[entryPoints]
|
[entryPoints]
|
||||||
[entryPoints.https]
|
[entryPoints.http]
|
||||||
address = ":4443"
|
address = ":8081"
|
||||||
[entryPoints.https.tls]
|
|
||||||
[[entryPoints.https.tls.certificates]]
|
|
||||||
certFile = """{{ .CertContent }}"""
|
|
||||||
keyFile = """{{ .KeyContent }}"""
|
|
||||||
|
|
||||||
|
|
||||||
[api]
|
[api]
|
||||||
|
|
||||||
|
|
25
integration/fixtures/grpc/config_h2c_termination.toml
Normal file
25
integration/fixtures/grpc/config_h2c_termination.toml
Normal file
|
@ -0,0 +1,25 @@
|
||||||
|
defaultEntryPoints = ["https"]
|
||||||
|
|
||||||
|
[entryPoints]
|
||||||
|
[entryPoints.https]
|
||||||
|
address = ":4443"
|
||||||
|
[entryPoints.https.tls]
|
||||||
|
[[entryPoints.https.tls.certificates]]
|
||||||
|
certFile = """{{ .CertContent }}"""
|
||||||
|
keyFile = """{{ .KeyContent }}"""
|
||||||
|
[api]
|
||||||
|
|
||||||
|
[file]
|
||||||
|
|
||||||
|
[backends]
|
||||||
|
[backends.backend1]
|
||||||
|
[backends.backend1.servers.server1]
|
||||||
|
url = "h2c://127.0.0.1:{{ .GRPCServerPort }}"
|
||||||
|
weight = 1
|
||||||
|
|
||||||
|
|
||||||
|
[frontends]
|
||||||
|
[frontends.frontend1]
|
||||||
|
backend = "backend1"
|
||||||
|
[frontends.frontend1.routes.test_1]
|
||||||
|
rule = "Host:127.0.0.1"
|
|
@ -88,9 +88,27 @@ func getHelloClientGRPC() (helloworld.GreeterClient, func() error, error) {
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func callHelloClientGRPC(name string) (string, error) {
|
func getHelloClientGRPCh2c() (helloworld.GreeterClient, func() error, error) {
|
||||||
client, closer, err := getHelloClientGRPC()
|
conn, err := grpc.Dial("127.0.0.1:8081", grpc.WithInsecure())
|
||||||
|
if err != nil {
|
||||||
|
return nil, func() error { return nil }, err
|
||||||
|
}
|
||||||
|
return helloworld.NewGreeterClient(conn), conn.Close, nil
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
func callHelloClientGRPC(name string, secure bool) (string, error) {
|
||||||
|
var client helloworld.GreeterClient
|
||||||
|
var closer func() error
|
||||||
|
var err error
|
||||||
|
|
||||||
|
if secure {
|
||||||
|
client, closer, err = getHelloClientGRPC()
|
||||||
|
} else {
|
||||||
|
client, closer, err = getHelloClientGRPCh2c()
|
||||||
|
}
|
||||||
defer closer()
|
defer closer()
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", err
|
return "", err
|
||||||
}
|
}
|
||||||
|
@ -149,7 +167,7 @@ func (s *GRPCSuite) TestGRPC(c *check.C) {
|
||||||
|
|
||||||
var response string
|
var response string
|
||||||
err = try.Do(1*time.Second, func() error {
|
err = try.Do(1*time.Second, func() error {
|
||||||
response, err = callHelloClientGRPC("World")
|
response, err = callHelloClientGRPC("World", true)
|
||||||
return err
|
return err
|
||||||
})
|
})
|
||||||
c.Assert(err, check.IsNil)
|
c.Assert(err, check.IsNil)
|
||||||
|
@ -168,6 +186,44 @@ func (s *GRPCSuite) TestGRPCh2c(c *check.C) {
|
||||||
}()
|
}()
|
||||||
|
|
||||||
file := s.adaptFile(c, "fixtures/grpc/config_h2c.toml", struct {
|
file := s.adaptFile(c, "fixtures/grpc/config_h2c.toml", struct {
|
||||||
|
GRPCServerPort string
|
||||||
|
}{
|
||||||
|
GRPCServerPort: port,
|
||||||
|
})
|
||||||
|
|
||||||
|
defer os.Remove(file)
|
||||||
|
cmd, display := s.traefikCmd(withConfigFile(file))
|
||||||
|
defer display(c)
|
||||||
|
|
||||||
|
err = cmd.Start()
|
||||||
|
c.Assert(err, check.IsNil)
|
||||||
|
defer cmd.Process.Kill()
|
||||||
|
|
||||||
|
// wait for Traefik
|
||||||
|
err = try.GetRequest("http://127.0.0.1:8080/api/providers", 1*time.Second, try.BodyContains("Host:127.0.0.1"))
|
||||||
|
c.Assert(err, check.IsNil)
|
||||||
|
|
||||||
|
var response string
|
||||||
|
err = try.Do(1*time.Second, func() error {
|
||||||
|
response, err = callHelloClientGRPC("World", false)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
c.Assert(err, check.IsNil)
|
||||||
|
c.Assert(response, check.Equals, "Hello World")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *GRPCSuite) TestGRPCh2cTermination(c *check.C) {
|
||||||
|
lis, err := net.Listen("tcp", ":0")
|
||||||
|
_, port, err := net.SplitHostPort(lis.Addr().String())
|
||||||
|
c.Assert(err, check.IsNil)
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
err := starth2cGRPCServer(lis, &myserver{})
|
||||||
|
c.Log(err)
|
||||||
|
c.Assert(err, check.IsNil)
|
||||||
|
}()
|
||||||
|
|
||||||
|
file := s.adaptFile(c, "fixtures/grpc/config_h2c_termination.toml", struct {
|
||||||
CertContent string
|
CertContent string
|
||||||
KeyContent string
|
KeyContent string
|
||||||
GRPCServerPort string
|
GRPCServerPort string
|
||||||
|
@ -191,7 +247,7 @@ func (s *GRPCSuite) TestGRPCh2c(c *check.C) {
|
||||||
|
|
||||||
var response string
|
var response string
|
||||||
err = try.Do(1*time.Second, func() error {
|
err = try.Do(1*time.Second, func() error {
|
||||||
response, err = callHelloClientGRPC("World")
|
response, err = callHelloClientGRPC("World", true)
|
||||||
return err
|
return err
|
||||||
})
|
})
|
||||||
c.Assert(err, check.IsNil)
|
c.Assert(err, check.IsNil)
|
||||||
|
@ -233,7 +289,7 @@ func (s *GRPCSuite) TestGRPCInsecure(c *check.C) {
|
||||||
|
|
||||||
var response string
|
var response string
|
||||||
err = try.Do(1*time.Second, func() error {
|
err = try.Do(1*time.Second, func() error {
|
||||||
response, err = callHelloClientGRPC("World")
|
response, err = callHelloClientGRPC("World", true)
|
||||||
return err
|
return err
|
||||||
})
|
})
|
||||||
c.Assert(err, check.IsNil)
|
c.Assert(err, check.IsNil)
|
||||||
|
|
|
@ -25,6 +25,7 @@ import (
|
||||||
"github.com/containous/traefik/cluster"
|
"github.com/containous/traefik/cluster"
|
||||||
"github.com/containous/traefik/configuration"
|
"github.com/containous/traefik/configuration"
|
||||||
"github.com/containous/traefik/configuration/router"
|
"github.com/containous/traefik/configuration/router"
|
||||||
|
"github.com/containous/traefik/h2c"
|
||||||
"github.com/containous/traefik/healthcheck"
|
"github.com/containous/traefik/healthcheck"
|
||||||
"github.com/containous/traefik/log"
|
"github.com/containous/traefik/log"
|
||||||
"github.com/containous/traefik/metrics"
|
"github.com/containous/traefik/metrics"
|
||||||
|
@ -89,7 +90,7 @@ type EntryPoint struct {
|
||||||
type serverEntryPoints map[string]*serverEntryPoint
|
type serverEntryPoints map[string]*serverEntryPoint
|
||||||
|
|
||||||
type serverEntryPoint struct {
|
type serverEntryPoint struct {
|
||||||
httpServer *http.Server
|
httpServer *h2c.Server
|
||||||
listener net.Listener
|
listener net.Listener
|
||||||
httpRouter *middlewares.HandlerSwitcher
|
httpRouter *middlewares.HandlerSwitcher
|
||||||
certs *safe.Safe
|
certs *safe.Safe
|
||||||
|
@ -747,7 +748,7 @@ func (s *Server) startServer(serverEntryPoint *serverEntryPoint) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Server) prepareServer(entryPointName string, entryPoint *configuration.EntryPoint, router *middlewares.HandlerSwitcher, middlewares []negroni.Handler) (*http.Server, net.Listener, error) {
|
func (s *Server) prepareServer(entryPointName string, entryPoint *configuration.EntryPoint, router *middlewares.HandlerSwitcher, middlewares []negroni.Handler) (*h2c.Server, net.Listener, error) {
|
||||||
readTimeout, writeTimeout, idleTimeout := buildServerTimeouts(s.globalConfiguration)
|
readTimeout, writeTimeout, idleTimeout := buildServerTimeouts(s.globalConfiguration)
|
||||||
log.Infof("Preparing server %s %+v with readTimeout=%s writeTimeout=%s idleTimeout=%s", entryPointName, entryPoint, readTimeout, writeTimeout, idleTimeout)
|
log.Infof("Preparing server %s %+v with readTimeout=%s writeTimeout=%s idleTimeout=%s", entryPointName, entryPoint, readTimeout, writeTimeout, idleTimeout)
|
||||||
|
|
||||||
|
@ -792,14 +793,16 @@ func (s *Server) prepareServer(entryPointName string, entryPoint *configuration.
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return &http.Server{
|
return &h2c.Server{
|
||||||
Addr: entryPoint.Address,
|
Server: &http.Server{
|
||||||
Handler: internalMuxRouter,
|
Addr: entryPoint.Address,
|
||||||
TLSConfig: tlsConfig,
|
Handler: internalMuxRouter,
|
||||||
ReadTimeout: readTimeout,
|
TLSConfig: tlsConfig,
|
||||||
WriteTimeout: writeTimeout,
|
ReadTimeout: readTimeout,
|
||||||
IdleTimeout: idleTimeout,
|
WriteTimeout: writeTimeout,
|
||||||
ErrorLog: httpServerLogger,
|
IdleTimeout: idleTimeout,
|
||||||
|
ErrorLog: httpServerLogger,
|
||||||
|
},
|
||||||
},
|
},
|
||||||
listener,
|
listener,
|
||||||
nil
|
nil
|
||||||
|
|
Loading…
Add table
Reference in a new issue