Front end to grpc framework

RPC What is it? ？

RPC The full English name is Remote Procedure Call Both remote procedure calls , The definition given in Wikipedia is that a computer calls a function , But this function is not on this computer , In this way, programmers do not need to pay attention to how to call remotely , It's as like as two peas in a local function .

Listen, it's very tall , We're going to implement an example of summation ：

function sum(a, b) {
	return a + b
}
 Copy code 

As a client , Actually, I don't know sum It's logical , It just needs to pass a and b Two parameters are given to the server , The server returns the result .

Here you will have a question , Why should we call a function remotely ？

The answer is that we don't have it here , The above is sum Pure logic , But if the client has an account and password , To get User details What about the data? , We don't have it here , So be sure to call... Remotely .

PRC and HTTP Relationship of agreement ？

After our explanation , I believe we all understand , But there will be a new question , How does this process relate to http The request response model is so similar , What is the relationship between the two ？ In fact, in a broad sense ,http Namely rpc An implementation of ,rpc More like an idea ,http Request and response is an implementation .

gPRC What is it? ？

Has just said rpc It's more of an idea , And what we're talking about now gPRC It is PRC An implementation of , It can also be called a framework , And there is more than one framework , The industry also has thrift, However, it is widely used in microservices at present , So what we have to learn is it .

gRPC Official website The introduction of A high performance, open source universal RPC framework. A high performance 、 Open source universal RPC frame . It has the following four characteristics ：

• Simple service definition
• Cross language and platform
• Fast expansion and reduction
• be based on HTTP/2 Two way authentication

Reading ：

• Simple service definition ： be based on Protocol Buffers Defining services （ It will be explained later ）
• Cross language and platform ： adopt proto Define code that can generate various languages

This paper also focuses on two points , Illustrate with examples .

Protocol Buffer What is it? ？

VS Code Provides vscode-proto3 This plug-in is used for proto Highlights of

protocal buffer You can understand it as a language , But don't be afraid of , Its syntax is very simple , Its role is also clear , Is used to define functions 、 The parameters of the function 、 Evaluation of response results , And it can be realized by converting the command line into functions in different languages . Its basic grammar is ：

// user.proto

syntax = "proto3";

package user; //  Package name 

//  Request parameters 
message LoginRequest {
	string username = 1;
  string password = 2;
}

//  In response to the results 
message LoginResponse {
	string access_token = 1;
  int32 expires = 2;
}

//  User related interfaces 
service User {
	//  Login function 
	rpc login(LoginRequest) returns (LoginResponse);
}
 Copy code 

In order to understand , I translate the above definition into typescript Definition ：

namespace user {
  interface LoginRequest {
    username: string;
    password: string;
  }

  interface LoginResponse {
    access_token: string;
    expires: number;
  }

  interface User {
    login: (LoginRequest) => LoginResponse // ts  In the definition of type , Function arguments can have no name .
  }
}
 Copy code 

By comparison, we know ：

• syntax = "proto3"： This sentence is equivalent to proto3 Version of the agreement , What is unified now is 3, Every proto The documents are written in this way
• package： similar namespace Scope
• message： amount to ts Medium interface
• service： It's also equivalent to js Medium interface
• string、int32： It's the type , because ts The division of numbers in is not so detailed , therefore int32 It was turned into number
• User： amount to ts Classes or objects in
• login： amount to ts The method in
• Numbers 1、2： The most confusing thing is the number after the variable , It's actually grpc The key to the communication process , Is the sequence used to encode and decode data , Be similar to json Object to string , Then convert the string to json The colons and commas and semicolons in the object work the same , That is, the rules of serialization and deserialization .

from proto Define the node Code

Dynamic loading version

The so-called dynamic loading version refers to nodejs Load and process at startup proto, And then according to proto Define the encoding and decoding of data .

• Create directories and files

gRPC It is a framework for exchanging information between client and server , Let's build two js Files are divided into client and server , The client sends a login request , Server response , Its directory structure is as follows ：

.
├── client.js #  client 
├── server.js #  Server side 
├── user.proto # proto  Definition 
└── user_proto.js #  Both client and server need to load  proto  The public code of 
 Copy code 

• Installation dependency

yarn add @grpc/grpc-js  # @grpc/grpc-js： yes  gRPC node  The implementation of the （ Different languages have different language implementations ）
yarn add @grpc/proto-loader # @grpc/proto-loader： Used for loading  proto 
 Copy code 

• To write user_proto.js

user_proto.js It is important for both server and client , The client can know the data type and parameters to be sent , The server can know the parameters it accepts 、 The result of the response and the name of the function to be implemented .

// user_proto.js
//  load  proto
const path = require('path')
const grpc = require('@grpc/grpc-js')
const protoLoader = require('@grpc/proto-loader')

const PROTO_PATH = path.join(__dirname, 'user.proto') // proto  route 
const packageDefinition = protoLoader.loadSync(PROTO_PATH, { keepCase: true, longs: String, enums: String, defaults: true, oneofs: true })
const protoDescriptor = grpc.loadPackageDefinition(packageDefinition)

const user_proto = protoDescriptor. user

module.exports = user_proto
 Copy code 

• To write server.js

// service.js
//  Server side 
const grpc = require("@grpc/grpc-js"); //  introduce  gprc  frame 
const user_proto = require("./user_proto.js"); //  Load parsed  proto

// User Service  Realization 
const userServiceImpl = {
  login: (call, callback) => {
    // call.request  Is to request relevant information 
    const { request } = call;
    const { username, password } = request;

    //  The first parameter is the error message , The second parameter is the response information 
    callback(null, {
      access_token: `username = ${username}; password = ${password}`,
      expires: "zhang",
    });
  },
};

//  and  http  equally , Need to listen to a port , Wait for others to link 
function main() {
  const server = new grpc.Server(); //  initialization  grpc  frame 
  server.addService(user_proto.User.service, userServiceImpl); //  add to  service
 	//  Start monitoring services （ Fixed writing ）
  server.bindAsync("0.0.0.0:8081", grpc.ServerCredentials.createInsecure(), () => {
      server.start();
      console.log("grpc server started");
    }
  );
}

main();

 Copy code 

because proto In, we only define , did not login The real realization of , So we need to server.js Chinese vs login To implement . We can console.log(user_proto) notice ：

{
  LoginRequest: { 
  	// ... 
  },
  LoginResponse: {
  	// ...
  },
	User: [class ServiceClientImpl extends Client] {
    service: { login: [Object] }
  }
}
 Copy code 

therefore server.addService We can fill in user_proto.User.service.

• To write client.js

// client.js
const user_proto = require("./user_proto");
const grpc = require("@grpc/grpc-js");

//  Use  `user_proto.User`  Create a  client, The target server address is  `localhost:8081`
//  That is, we just  service.js  Listening address 
const client = new user_proto.User(
  "localhost:8081",
  grpc.credentials.createInsecure()
);

//  Initiate a login request 
function login() {
  return new Promise((resolve, reject) => {
      //  Agreed parameters 
      client.login(
        { username: 123, password: "abc123" },
        function (err, response) {
          if (err) {
            reject(err);
          } else {
            resolve(response);
          }
        }
      );
  })
}

async function main() {
  const res = await login();
  console.log(res)
}

main();
 Copy code 

• Start the service

First node server.js Start server , Keep it listening , then node client.js Start client , Send a request .

We see that there is already a response result .

• A bad eye

We make a bad heart , If the sent data format is not proto What happens to the types defined in ？ The answer is to be Type of enforcement Convert to proto Type defined in , Let's say we have server.js Lieutenant general expires The return value of the field is changed to zhang Then he will be converted into numbers 0, And the client sends past 123 Also converted to string type .

Statically compiled version

Dynamic loading is runtime loading proto, Static compilation is to advance proto File compiled into JS file , We just need to load js Just a file , No compilation proto Time for , It is also a more common way at work .

• New projects

Let's create a new project , This time there are only four files in the folder , Respectively ：

.
├── gen #  Folder , Used to store the generated code 
├── client.js #  Client code 
├── server.js #  Server code 
└── user.proto # proto  file , Remember to copy the content 
 Copy code 

• Installation dependency

yarn global add grpc-tools #  For from  proto -> js  File tool 
yarn add google-protobuf @grpc/grpc-js #  Run time dependencies 
 Copy code 

• Generate js Code

grpc_tools_node_protoc \
--js_out=import_style=commonjs,binary:./gen/ \
--grpc_out=grpc_js:./gen/ user.proto
 Copy code 

We see that we have generated user_pb.js and user_grpc_pb.js Two documents ：

• grpc_tools_node_protoc： It's installation grpc-tools Command line tools generated after
• --js_out=import_style=commonjs,binary:./gen/： Is to generate user_pb.js The order of
• --grpc_out=grpc_js:./gen/： Is to generate user_grpc_pb.js The order of .

pb yes protobuf Abbreviation

If you look closely at the contents of both, you will find ：

user_pb.js： It's mainly about proto Medium message Define and extend various encoding and decoding methods , That is to say LoginRequest and LoginResponse Do the processing .

user_grpc_pb.js： That's right. proto Medium service Define various methods .

• To write server.js

const grpc = require("@grpc/grpc-js");

const services = require("./gen/user_grpc_pb");
const messages = require("./gen/user_pb");

const userServiceImpl = {
  login: (call, callback) => {
    const { request } = call;

    //  Use  request  The method in gets the parameters of the request 
    const username = request.getUsername();
    const password = request.getPassword();

    //  Use  message  Set the response result 
    const response = new messages.LoginResponse();
    response.setAccessToken(`username = ${username}; password = ${password}`);
    response.setExpires(7200);

    callback(null, response);
  },
};

function main() {
  const server = new grpc.Server();

  //  Use  services.UserService  Add service 
  server.addService(services.UserService, userServiceImpl);
  server.bindAsync(
    "0.0.0.0:8081",
    grpc.ServerCredentials.createInsecure(),
    () => {
      server.start();
      console.log("grpc server started");
    }
  );
}

main();
 Copy code 

We found that the difference between the dynamic version and the dynamic version is addService The exported UserService Definition , And then login when , We can use various encapsulated methods to process request and response parameters .

• To write client.js

// client.js

const grpc = require("@grpc/grpc-js");

const services = require("./gen/user_grpc_pb");
const messages = require("./gen/user_pb");

//  Use  services  initialization  Client
const client = new services.UserClient(
  "localhost:8081",
  grpc.credentials.createInsecure()
);

//  launch  login  request 
function login() {
  return new Promise((resolve, reject) => {
    //  Use  message  Initialize parameters 
    const request = new messages.LoginRequest();
    request.setUsername("zhang");
    request.setPassword("123456");

    client.login(request, function (err, response) {
      if (err) {
        reject(err);
      } else {
        resolve(response.toObject());
      }
    });
  });
}

async function main() {
  const res = await login()
  console.log(res)
}

main();
 Copy code 

As can be seen from the notes above , We directly from the generated JS Load content in file , And it provides a lot of encapsulation methods , Let's make the transmission more controllable .

from JS To TS

From the above we can see , Restrictions on parameter types , More is cast , In the writing stage, you can't find , This is very unscientific , however , We need to get through proto Generate ts Type definition to solve this problem .

Online about from proto To generate ts There are many solutions , We chose to use protoc + grpc_tools_node_protoc_ts + grpc-tools .

• New projects

mkdir grpc_demo_ts && cd grpc_demo_ts #  Create project directory 

yarn global add typescript ts-node @types/node #  install  ts  and  ts-node

tsc --init #  initialization  ts
 Copy code 

• install proto Tools

yarn global add grpc-tools grpc_tools_node_protoc_ts #  install  proto  Tools to the whole 
 Copy code 

• The installation runtime depends on

yarn add google-protobuf @grpc/grpc-js #  Runtime dependency 
 Copy code 

• create a file

mkdir gen #  Create a directory to store the output files 
touch client.ts server.ts user.proto #  create a file 
#  Remember to  user.proto  Copy your content 
 Copy code 

• install protoc

Then we need to install protoc This tool , First of all to enter protobuf Of github, Get into release, Download the file of your platform , Then install , After installation, remember to add it to the setting environment variable , Make sure you can use it globally .

mac Can pass brew install protobuf Installation , After installation, there will be protoc command

• Generate js Document and ts The type definition

#  Generate  user_pb.js  and  user_grpc_pb.js
grpc_tools_node_protoc \
--js_out=import_style=commonjs,binary:./gen \
--grpc_out=grpc_js:./gen \
--plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` \
./user.proto

#  Generate  d.ts  Definition 
protoc \
--plugin=protoc-gen-ts=`which protoc-gen-ts` \
--ts_out=grpc_js:./gen \
./user.proto
 Copy code 

• To write server.ts

// server.ts

import * as grpc from "@grpc/grpc-js";
import { IUserServer, UserService } from "./gen/user_grpc_pb";
import messages from "./gen/user_pb";

// User Service  The implementation of the 
const userServiceImpl: IUserServer = {
  //  Implement login interface 
  login(call, callback) {
    const { request } = call;
    const username = request.getUsername();
    const password = request.getPassword();

    const response = new messages.LoginResponse();
    response.setAccessToken(`username = ${username}; password = ${password}`);
    response.setExpires(7200);
    callback(null, response);
  }
}

function main() {
  const server = new grpc.Server();
  
  // UserService  Definition ,UserImpl  It's the realization of 
  server.addService(UserService, userServiceImpl);
  server.bindAsync(
    "0.0.0.0:8081",
    grpc.ServerCredentials.createInsecure(),
    () => {
      server.start();
      console.log("grpc server started");
    }
  );
}

main();
 Copy code 

The type hint is perfect

• To write client.ts

// client.ts

import * as grpc from "@grpc/grpc-js";
import { UserClient } from "./gen/user_grpc_pb";
import messages from "./gen/user_pb";

const client = new UserClient(
  "localhost:8081",
  grpc.credentials.createInsecure()
);

//  Initiate a login request 
const login = () => {
  return new Promise((resolve, reject) => {
    const request = new messages.LoginRequest();
    request.setUsername('zhang');
    request.setPassword("123456");

    client.login(request, function (err, response) {
      if (err) {
        reject(err);
      } else {
        resolve(response.toObject());
      }
    });
  })
}

async function main() {
  const data = await login()
  console.log(data)
}

main();
 Copy code 

When we enter the wrong type ,ts Will be subject to mandatory inspection .

• Start the service

We use ts-node Start both , Found that the effect is normal .

from Node To Go

In the introduction above ,client and server It's all used js/ts To write it , But in practice, more is node As a client to aggregate and call interfaces written in other languages , That is to say BFF layer , We use go Language as an example .

• Original transformation ts project

We will ts The project is transformed into client and server Two directories ,client yes ts Project as client ,server yes go project , As a server , At the same time, we put the original server.ts Delete , hold user.proto Put it on the outside , Both share .

.
├── client #  Client folder , The content is the same as  ts  chapter , Just deleted  server.ts  Related content 
│   ├── client.ts
│   ├── gen
│   │   ├── user_grpc_pb.d.ts
│   │   ├── user_grpc_pb.js
│   │   ├── user_pb.d.ts
│   │   └── user_pb.js
│   ├── package.json
│   ├── tsconfig.json
│   └── yarn.lock
├── server #  Server files 
└── user.proto # proto  file 
 Copy code 

• install Go

We enter Go Language website , Find the latest version, download and install ：golang.google.cn/dl/

• Set up go agent

and npm equally ,go Language pull bag , You also need to set the image to pull packets faster .

go env -w GOPROXY=https://goproxy.cn,direct
 Copy code 

• initialization go project

similar yarn init -y The role of .

cd server #  Get into  server  Catalog 
go mod init grpc_go_demo #  Initialization package 
mkdir -p gen/user #  Used to store the code generated later 
 Copy code 

• install protoc Of go Language plug-ins

Used to generate go Code of language , The functions and grpc-tools and grpc_tools_node_protoc_ts identical .

go install google.golang.org/protobuf/cmd/[email protected]
go install google.golang.org/grpc/cmd/[email protected] 
 Copy code 

• The installation runtime depends on

We also need to install runtime dependencies , The function is similar to the above node Of google-protobuf and @grpc/grpc-js.

go get -u github.com/golang/protobuf/proto
go get -u google.golang.org/grpc
 Copy code 

• modify user.proto

syntax = "proto3";

option go_package = "grpc_go_demo/gen/user"; //  Add this sentence 

package user;

message LoginRequest {
	string username = 1;
  string password = 2;
}

message LoginResponse {
	string access_token = 1;
  int32 expires = 2;
}

service User {
	rpc login(LoginRequest) returns (LoginResponse);
}
 Copy code 

• Generate go Code

//  To be in  server  Directory 

protoc --go_out=./gen/user -I=../ --go_opt=paths=source_relative \
    --go-grpc_out=./gen/user -I=../ --go-grpc_opt=paths=source_relative \
    ../user.proto
 Copy code 

• install VS Code Plug in and create a new open project

When you click to view the generated user.pb.go perhaps user_grpc.pb.go when , You'll find that vscode Let you install plug-ins , Just pretend it's over , Then you may find go The package reports an error that cannot be found , Don't panic , We use server Reopen the project for the project root path .

• establish main.go Write server code

// server/main.go

package main

import (
	"context"
	"fmt"
	pb "grpc_go_demo/gen/user"
	"log"
	"net"

	"google.golang.org/grpc"
)

//  Declare an object 
type userServerImpl struct {
	pb.UnimplementedUserServer
}

//  The object has a  Login  Method 
func (s *userServerImpl) Login(ctx context.Context, in *pb.LoginRequest) (*pb.LoginResponse, error) {
	//  Return the response result 
    return &pb.LoginResponse{
		AccessToken: fmt.Sprintf("go: username = %v, password = %v", in.GetUsername(), in.GetPassword()),
		Expires: 7200,
	}, nil
}


//  Listen to the service and  server  Object registered to  gRPC  Server 
func main() {
    //  establish  tcp  service 
	lis, _ := net.Listen("tcp", ":8081")
	
    //  establish  grpc  service 
	server := grpc.NewServer()
    
    //  take  UserServer  Sign up to  server
	pb.RegisterUserServer(server, &userServerImpl{})
    
	log.Printf("server listening at %v", lis.Addr())

	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}
 Copy code 

Why gRPC Instead of HTTP?

Nowadays, most microservice architectures use gRPC Conduct inter service communication , So why not use what we are familiar with at the front end http Well ？

Some people say efficiency ,gRPC yes tcp agreement 、 Binary transmission , Efficient , High efficiency, yes , But it's relative to http There will be no obvious gap , One side http in json The encoding and decoding efficiency and the number of occupied space are not much worse than that of encoding and decoding into binary , secondly ,tcp and http In an intranet environment , I personally don't think the performance will be much worse （PS：gRPC The official website does not emphasize that it is relative to HTTP The high efficiency ）.

In fact, the core of the official website is its Language independence , adopt protobuf This intermediate form , Code that can be converted into various languages , Ensure code consistency , Instead of http Look at me like that swagger Or other document platforms to interface .

Conclusion

This article is just an introduction , as for gRPC How to combine node Framework development or deeper knowledge still needs you to explore by yourself .

Another bald day .