原文: medium.com/@amsokol.co…
在Medium上。
在此我并不打算重复它的内容。我希望通过一步步的引导让大家学会如何去开发一个简单的CRUD “To Do List”的微服务通过使用gPRC以及HTTP/REST的后端接口。我会演示如何编写测试用例以及加入中间件(请求ID以及日志记录与追踪)在服务当中。最后甚至还会提供一些例子讲述如何构建以及发布我们的微服务到Kubernetes上。
整个教程会分为4个部分:
这句话有什么意义?
“To Do List”微服务允许去管理“To Do”列表,ToDo项包括以下字段/属性:
ToDo服务包含典型的增删改查以及获取全部项方法。
Part1完整代码 戳这里
开始之前我们先构建一个项目的骨架, 这里 又一个非常优秀的Go项目的脚手架模板
我是用的是Windows 10 x64运行环境,但是我想你将我接下来的cmd指令转换成MacOs/Linux BASH应该不是什么大问题
首先创建文件夹并初始化项目
mkdir go-grpc-http-rest-microservice-tutorial cd go-grpc-http-rest-microservice-tutorial go mod init github.com/<you>/go-grpc-http-rest-microservice-tutorial 复制代码
在你的项目里面创建文件目录如下
mkdir -p api/proto/v1 复制代码
这里的v1就是我们API的版本号
API版本化:通过将不同版本的API代码放到不同文件夹里面并以此命名是我的最佳实践
下一步就是在刚创建的 v1 文件夹里面创建 todo-service.proto 文件并加入ToDo服务的定义,我们先从Create方法写起:
syntax = "proto3";
package v1;
import "google/protobuf/timestamp.proto";
// Taks we have to do
message ToDo {
// Unique integer identifier of the todo task
int64 id = 1;
// Title of the task
string title = 2;
// Detail description of the todo task
string description = 3;
// Date and time to remind the todo task
google.protobuf.Timestamp reminder = 4;
}
// Request data to create new todo task
message CreateRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity to add
ToDo toDo = 2;
}
// Response that contains data for created todo task
message CreateResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// ID of created task
int64 id = 2;
}
// Service to manage list of todo tasks
service ToDoService {
// Create new todo task
rpc Create(CreateRequest) returns (CreateResponse);
}
复制代码
戳我 查看Proto的编写语法
如你所见,API定义是跟编程语言,通讯协议以及网络传输无关的,这也是protobuf的一个重要标志
为了能编译proto文件我们需要安装一些工具和依赖
go get -u github.com/golang/protobuf/protoc-gen-go 复制代码
# Windows: ./third_party/protoc-gen.cmd # MasOS/Linux: ./third_party/protoc-gen.sh 复制代码
运行以后会创建一个名为 todo-service.pb.go 的文件在 pkg/model/v1 下
接下来让我们将剩下的方法驾到ToDo服务当中并编译
syntax = "proto3";
package v1;
import "google/protobuf/timestamp.proto";
// Taks we have to do
message ToDo {
// Unique integer identifier of the todo task
int64 id = 1;
// Title of the task
string title = 2;
// Detail description of the todo task
string description = 3;
// Date and time to remind the todo task
google.protobuf.Timestamp reminder = 4;
}
// Request data to create new todo task
message CreateRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity to add
ToDo toDo = 2;
}
// Contains data of created todo task
message CreateResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// ID of created task
int64 id = 2;
}
// Request data to read todo task
message ReadRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Unique integer identifier of the todo task
int64 id = 2;
}
// Contains todo task data specified in by ID request
message ReadResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity read by ID
ToDo toDo = 2;
}
// Request data to update todo task
message UpdateRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity to update
ToDo toDo = 2;
}
// Contains status of update operation
message UpdateResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Contains number of entities have beed updated
// Equals 1 in case of succesfull update
int64 updated = 2;
}
// Request data to delete todo task
message DeleteRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Unique integer identifier of the todo task to delete
int64 id = 2;
}
// Contains status of delete operation
message DeleteResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Contains number of entities have beed deleted
// Equals 1 in case of succesfull delete
int64 deleted = 2;
}
// Request data to read all todo task
message ReadAllRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
}
// Contains list of all todo tasks
message ReadAllResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// List of all todo tasks
repeated ToDo toDos = 2;
}
// Service to manage list of todo tasks
service ToDoService {
// Create new todo task
rpc Create(CreateRequest) returns (CreateResponse);
// Read todo task
rpc Read(ReadRequest) returns (ReadResponse);
// Update todo task
rpc Update(UpdateRequest) returns (UpdateResponse);
// Delete todo task
rpc Delete(DeleteRequest) returns (DeleteResponse);
// Read all todo tasks
rpc ReadAll(ReadAllRequest) returns (ReadAllResponse);
}
复制代码
再一次运行以下命令来更新Go的代码
# Windows: ./third_party/protoc-gen.cmd # MasOS/Linux: ./third_party/protoc-gen.sh 复制代码
到此为止,API的定义就完成了
我是使用Google Cloud上的MySQL作为数据库来持久化做存储的。你可以使用其他你喜欢的SQL数据库。
创建ToDo table的MySQL脚本
CREATE TABLE `ToDo` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `Title` varchar(200) DEFAULT NULL, `Description` varchar(1024) DEFAULT NULL, `Reminder` timestamp NULL DEFAULT NULL, PRIMARY KEY (`ID`), UNIQUE KEY `ID_UNIQUE` (`ID`) ); 复制代码
我会跳过如何安装配置SQL数据库以及创建数据表的步骤
创建文件 pkg/service/v1/todo-service.go 以及下面的内容
package v1
import (
"context"
"database/sql"
"fmt"
"time"
"github.com/golang/protobuf/ptypes"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/api/v1"
)
const (
// apiVersion is version of API is provided by server
apiVersion = "v1"
)
// toDoServiceServer is implementation of v1.ToDoServiceServer proto interface
type toDoServiceServer struct {
db *sql.DB
}
// NewToDoServiceServer creates ToDo service
func NewToDoServiceServer(db *sql.DB) v1.ToDoServiceServer {
return &toDoServiceServer{db: db}
}
// checkAPI checks if the API version requested by client is supported by server
func (s *toDoServiceServer) checkAPI(api string) error {
// API version is "" means use current version of the service
if len(api) > 0 {
if apiVersion != api {
return status.Errorf(codes.Unimplemented,
"unsupported API version: service implements API version '%s', but asked for '%s'", apiVersion, api)
}
}
return nil
}
// connect returns SQL database connection from the pool
func (s *toDoServiceServer) connect(ctx context.Context) (*sql.Conn, error) {
c, err := s.db.Conn(ctx)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to connect to database-> "+err.Error())
}
return c, nil
}
// Create new todo task
func (s *toDoServiceServer) Create(ctx context.Context, req *v1.CreateRequest) (*v1.CreateResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
reminder, err := ptypes.Timestamp(req.ToDo.Reminder)
if err != nil {
return nil, status.Error(codes.InvalidArgument, "reminder field has invalid format-> "+err.Error())
}
// insert ToDo entity data
res, err := c.ExecContext(ctx, "INSERT INTO ToDo(`Title`, `Description`, `Reminder`) VALUES(?, ?, ?)",
req.ToDo.Title, req.ToDo.Description, reminder)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to insert into ToDo-> "+err.Error())
}
// get ID of creates ToDo
id, err := res.LastInsertId()
if err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve id for created ToDo-> "+err.Error())
}
return &v1.CreateResponse{
Api: apiVersion,
Id: id,
}, nil
}
// Read todo task
func (s *toDoServiceServer) Read(ctx context.Context, req *v1.ReadRequest) (*v1.ReadResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
// query ToDo by ID
rows, err := c.QueryContext(ctx, "SELECT `ID`, `Title`, `Description`, `Reminder` FROM ToDo WHERE `ID`=?",
req.Id)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to select from ToDo-> "+err.Error())
}
defer rows.Close()
if !rows.Next() {
if err := rows.Err(); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve data from ToDo-> "+err.Error())
}
return nil, status.Error(codes.NotFound, fmt.Sprintf("ToDo with ID='%d' is not found",
req.Id))
}
// get ToDo data
var td v1.ToDo
var reminder time.Time
if err := rows.Scan(&td.Id, &td.Title, &td.Description, &reminder); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve field values from ToDo row-> "+err.Error())
}
td.Reminder, err = ptypes.TimestampProto(reminder)
if err != nil {
return nil, status.Error(codes.Unknown, "reminder field has invalid format-> "+err.Error())
}
if rows.Next() {
return nil, status.Error(codes.Unknown, fmt.Sprintf("found multiple ToDo rows with ID='%d'",
req.Id))
}
return &v1.ReadResponse{
Api: apiVersion,
ToDo: &td,
}, nil
}
// Update todo task
func (s *toDoServiceServer) Update(ctx context.Context, req *v1.UpdateRequest) (*v1.UpdateResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
reminder, err := ptypes.Timestamp(req.ToDo.Reminder)
if err != nil {
return nil, status.Error(codes.InvalidArgument, "reminder field has invalid format-> "+err.Error())
}
// update ToDo
res, err := c.ExecContext(ctx, "UPDATE ToDo SET `Title`=?, `Description`=?, `Reminder`=? WHERE `ID`=?",
req.ToDo.Title, req.ToDo.Description, reminder, req.ToDo.Id)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to update ToDo-> "+err.Error())
}
rows, err := res.RowsAffected()
if err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve rows affected value-> "+err.Error())
}
if rows == 0 {
return nil, status.Error(codes.NotFound, fmt.Sprintf("ToDo with ID='%d' is not found",
req.ToDo.Id))
}
return &v1.UpdateResponse{
Api: apiVersion,
Updated: rows,
}, nil
}
// Delete todo task
func (s *toDoServiceServer) Delete(ctx context.Context, req *v1.DeleteRequest) (*v1.DeleteResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
// delete ToDo
res, err := c.ExecContext(ctx, "DELETE FROM ToDo WHERE `ID`=?", req.Id)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to delete ToDo-> "+err.Error())
}
rows, err := res.RowsAffected()
if err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve rows affected value-> "+err.Error())
}
if rows == 0 {
return nil, status.Error(codes.NotFound, fmt.Sprintf("ToDo with ID='%d' is not found",
req.Id))
}
return &v1.DeleteResponse{
Api: apiVersion,
Deleted: rows,
}, nil
}
// Read all todo tasks
func (s *toDoServiceServer) ReadAll(ctx context.Context, req *v1.ReadAllRequest) (*v1.ReadAllResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
// get ToDo list
rows, err := c.QueryContext(ctx, "SELECT `ID`, `Title`, `Description`, `Reminder` FROM ToDo")
if err != nil {
return nil, status.Error(codes.Unknown, "failed to select from ToDo-> "+err.Error())
}
defer rows.Close()
var reminder time.Time
list := []*v1.ToDo{}
for rows.Next() {
td := new(v1.ToDo)
if err := rows.Scan(&td.Id, &td.Title, &td.Description, &reminder); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve field values from ToDo row-> "+err.Error())
}
td.Reminder, err = ptypes.TimestampProto(reminder)
if err != nil {
return nil, status.Error(codes.Unknown, "reminder field has invalid format-> "+err.Error())
}
list = append(list, td)
}
if err := rows.Err(); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve data from ToDo-> "+err.Error())
}
return &v1.ReadAllResponse{
Api: apiVersion,
ToDos: list,
}, nil
}
复制代码
为API逻辑实现创建文件 pkg/service/v1/todo-service.go 最后文件目录如下
不管我们开发什么都应该编写测试用例。这是 强制遵守 的规定。
这里有一个很棒的模拟库,用于测试SQL数据库的交互 go-sqlmock .我将会使用它为我们的ToDo服务编写测试用例。
将 这个文件 放到 pkg/service/v1 目录下,当前项目文件结构如下
创建文件 pkg/protocol/grpc/server.go 并写入
package grpc
import (
"context"
"log"
"net"
"os"
"os/signal"
"google.golang.org/grpc"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/api/v1"
)
// RunServer runs gRPC service to publish ToDo service
func RunServer(ctx context.Context, v1API v1.ToDoServiceServer, port string) error {
listen, err := net.Listen("tcp", ":"+port)
if err != nil {
return err
}
// register service
server := grpc.NewServer()
v1.RegisterToDoServiceServer(server, v1API)
// graceful shutdown
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
go func() {
for range c {
// sig is a ^C, handle it
log.Println("shutting down gRPC server...")
server.GracefulStop()
<-ctx.Done()
}
}()
// start gRPC server
log.Println("starting gRPC server...")
return server.Serve(listen)
}
复制代码
RunServer函数负责注册ToDo服务以及启动gRPC服务
你需要给gPRC服务配置TLS,查看 示例 学习如何配置
接下来创建 pkg/cmd/server/server.go 以及对应内容
package cmd
import (
"context"
"database/sql"
"flag"
"fmt"
// mysql driver
_ "github.com/go-sql-driver/mysql"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/protocol/grpc"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/service/v1"
)
// Config is configuration for Server
type Config struct {
// gRPC server start parameters section
// gRPC is TCP port to listen by gRPC server
GRPCPort string
// DB Datastore parameters section
// DatastoreDBHost is host of database
DatastoreDBHost string
// DatastoreDBUser is username to connect to database
DatastoreDBUser string
// DatastoreDBPassword password to connect to database
DatastoreDBPassword string
// DatastoreDBSchema is schema of database
DatastoreDBSchema string
}
// RunServer runs gRPC server and HTTP gateway
func RunServer() error {
ctx := context.Background()
// get configuration
var cfg Config
flag.StringVar(&cfg.GRPCPort, "grpc-port", "", "gRPC port to bind")
flag.StringVar(&cfg.DatastoreDBHost, "db-host", "", "Database host")
flag.StringVar(&cfg.DatastoreDBUser, "db-user", "", "Database user")
flag.StringVar(&cfg.DatastoreDBPassword, "db-password", "", "Database password")
flag.StringVar(&cfg.DatastoreDBSchema, "db-schema", "", "Database schema")
flag.Parse()
if len(cfg.GRPCPort) == 0 {
return fmt.Errorf("invalid TCP port for gRPC server: '%s'", cfg.GRPCPort)
}
// add MySQL driver specific parameter to parse date/time
// Drop it for another database
param := "parseTime=true"
dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?%s",
cfg.DatastoreDBUser,
cfg.DatastoreDBPassword,
cfg.DatastoreDBHost,
cfg.DatastoreDBSchema,
param)
db, err := sql.Open("mysql", dsn)
if err != nil {
return fmt.Errorf("failed to open database: %v", err)
}
defer db.Close()
v1API := v1.NewToDoServiceServer(db)
return grpc.RunServer(ctx, v1API, cfg.GRPCPort)
}
复制代码
RunServer函数负责读取命令行输入的参数,创建数据库连接,创建ToDo服务实例以及调用之前gPRC服务中的 RunServer 函数
最后创建以下文件 cmd/server/main.go 以及内容
package main
import (
"fmt"
"os"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/cmd"
)
func main() {
if err := cmd.RunServer(); err != nil {
fmt.Fprintf(os.Stderr, "%v/n", err)
os.Exit(1)
}
}
复制代码
以上就是服务端所有的代码了,当前项目目录如下
创建文件 cmd/client-grpc/main.go 以及以下内容
package main
import (
"context"
"flag"
"log"
"time"
"github.com/golang/protobuf/ptypes"
"google.golang.org/grpc"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/api/v1"
)
const (
// apiVersion is version of API is provided by server
apiVersion = "v1"
)
func main() {
// get configuration
address := flag.String("server", "", "gRPC server in format host:port")
flag.Parse()
// Set up a connection to the server.
conn, err := grpc.Dial(*address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := v1.NewToDoServiceClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
t := time.Now().In(time.UTC)
reminder, _ := ptypes.TimestampProto(t)
pfx := t.Format(time.RFC3339Nano)
// Call Create
req1 := v1.CreateRequest{
Api: apiVersion,
ToDo: &v1.ToDo{
Title: "title (" + pfx + ")",
Description: "description (" + pfx + ")",
Reminder: reminder,
},
}
res1, err := c.Create(ctx, &req1)
if err != nil {
log.Fatalf("Create failed: %v", err)
}
log.Printf("Create result: <%+v>/n/n", res1)
id := res1.Id
// Read
req2 := v1.ReadRequest{
Api: apiVersion,
Id: id,
}
res2, err := c.Read(ctx, &req2)
if err != nil {
log.Fatalf("Read failed: %v", err)
}
log.Printf("Read result: <%+v>/n/n", res2)
// Update
req3 := v1.UpdateRequest{
Api: apiVersion,
ToDo: &v1.ToDo{
Id: res2.ToDo.Id,
Title: res2.ToDo.Title,
Description: res2.ToDo.Description + " + updated",
Reminder: res2.ToDo.Reminder,
},
}
res3, err := c.Update(ctx, &req3)
if err != nil {
log.Fatalf("Update failed: %v", err)
}
log.Printf("Update result: <%+v>/n/n", res3)
// Call ReadAll
req4 := v1.ReadAllRequest{
Api: apiVersion,
}
res4, err := c.ReadAll(ctx, &req4)
if err != nil {
log.Fatalf("ReadAll failed: %v", err)
}
log.Printf("ReadAll result: <%+v>/n/n", res4)
// Delete
req5 := v1.DeleteRequest{
Api: apiVersion,
Id: id,
}
res5, err := c.Delete(ctx, &req5)
if err != nil {
log.Fatalf("Delete failed: %v", err)
}
log.Printf("Delete result: <%+v>/n/n", res5)
}
复制代码
以上就是客户端所有代码,当前项目目录如下
最后一步骤是确保gPRC服务能跑起来
开启一个终端build以及run gRPC服务(将下面数据库连接的参数替代成你自己的数据库配置)
cd cmd/server go build . server.exe -grpc-port=9090 -db-host=<HOST>:3306 -db-user=<USER> -db-password=<PASSWORD> -db-schema=<SCHEMA> 复制代码
如果能看到
2018/09/09 08:02:16 starting gRPC server... 复制代码
证明我们的服务已经被启动起来了
打开另一个终端build和run gRPC客户端
cd cmd/client-grpc go build . client-grpc.exe -server=localhost:9090 复制代码
如果能看到以下信息:
2018/09/09 09:16:01 Create result: <api:"v1" id:13 > 2018/09/09 09:16:01 Read result: <api:"v1" toDo:<id:13 title:"title (2018-09-09T06:16:01.5755011Z)" description:"description (2018-09-09T06:16:01.5755011Z)" reminder:<seconds:1536473762 > > > 2018/09/09 09:16:01 Update result: <api:"v1" updated:1 > 2018/09/09 09:16:01 ReadAll result: <api:"v1" toDos:<id:9 title:"title (2018-09-09T04:45:16.3693282Z)" description:"description (2018-09-09T04:45:16.3693282Z)" reminder:<seconds:1536468316 > > toDos:<id:10 title:"title (2018-09-09T04:46:00.7490565Z)" description:"description (2018-09-09T04:46:00.7490565Z)" reminder:<seconds:1536468362 > > toDos:<id:13 title:"title (2018-09-09T06:16:01.5755011Z)" description:"description (2018-09-09T06:16:01.5755011Z) + updated" reminder:<seconds:1536473762 > > > 2018/09/09 09:16:01 Delete result: <api:"v1" deleted:1 > 复制代码
所有东西都正常运作了!
以上就是Part1的全部内容了,我们成功构建了gRPC的客户端以及服务端
Part1的源代码在 此处
接下来 Part2是讲述如何在我们本章建立的gRPC服务上增加HTTP/REST接口,敬请期待。
感谢收看!:pray::pray::pray::pray: