Unifying Communication Between Services with gRPC

Introduction

Most of the time in AI-assisted development, as long as the goal is described clearly, the implementation details are usually not a problem. But one issue has bothered me for a long time: “the correctness of communication across services” and “providing reasonable context for AI collaboration.” I found that when working across services, I spent a lot of time manually planning and providing the code between two services, feeding the right context to the AI.

For example, when recently handling communication among multiple services, the most troublesome part was that the same piece of data had different descriptions:

Pain Point 1: Different Services Use Different Names for the Same Data

Both sides are talking about “the same customer,” but the field names are inconsistent (userId vs customer_id)

Pain Point 2: Missing Data Fields Cause Problems

The gRPC Solution

The common root cause behind the two pain points above is: the definition of the same data is scattered across the codebases of different services. Naming differences and missing fields can only be discovered by manually comparing documentation, and they usually failed only on deployment.

gRPC🔗 takes the “interface” out of the code and turns it into a language-agnostic contract file, .proto, then uses tools to generate code for each language to reinforce type safety (details will be covered in the sections below):

Because there is only one contract, services can no longer diverge on the “data definition.” And because the code is generated from the contract, renaming a field or adding a new one will directly cause the side that has not caught up to fail compilation or type checking, instead of failing only at runtime.

Defining the Contract

Protocol Buffers (Protobuf) is gRPC’s default interface definition language (IDL) and serialization format. The data definitions from the two services above can be written as a contract:

// Specify that Protobuf version 3 syntax is used.
syntax = "proto3";

// The logical namespace inside Proto, used to avoid name conflicts between different Proto files.
package customer.v1;

// Tell the Protobuf compiler where to place the generated Go code for this file and what package name to use.
option go_package = "github.com/riceball/example/gen/customer/v1;customerv1";

message Customer {
  string id = 1;
  string full_name = 2;
  string email = 3;
  string phone_number = 4;
}

message GetCustomerRequest {
  string id = 1;
}

message GetCustomerResponse {
  Customer customer = 1;
}

service CustomerService {
  rpc GetCustomer(GetCustomerRequest) returns (GetCustomerResponse);
}

• Field Number: = 1 and = 2 are not default values; they are the identifiers of these fields in the binary format. Once a number is released, it cannot be changed. The name, on the other hand, can be changed, because what is transmitted in binary is the number rather than the name, for example: sending “number + data content” (for example, 2: “John Doe”).
• Naming conventions are handled by the generator: proto standardizes on snake_case; when generating Go, it becomes FullName, and when generating TypeScript, it becomes fullName. In other words, the userId vs customer_id dispute from Pain Point 1 does not exist at the contract layer. Each language receives the style it is accustomed to.

Generating Code

The official native tool is protoc, but its parameters and include paths are difficult to maintain. In practice, I recommend buf🔗:

version: v2
modules:
  - path: proto
lint:
  use:
    - DEFAULT
breaking:
  use:
    - FILE

version: v2
plugins:
  - remote: buf.build/protocolbuffers/go
    out: gen
    opt: paths=source_relative
  - remote: buf.build/grpc/go
    out: gen
    opt: paths=source_relative

# Generate code
buf generate

# Check whether naming and style follow conventions
buf lint

# Check for breaking changes against the main branch
buf breaking --against '.git#branch=main'

buf breaking is the most convenient tool here. It directly blocks changes such as “deleting a field that is still in use” or “changing a field number” in CI, so contract compatibility is guaranteed by the contract itself instead of relying on the eyesight of code reviewers.

What Does Code Generation Produce?

buf generate does not generate any business logic; it only translates the contract into Go code. The YAML above configures two plugins, each responsible for half of the work:

• protoc-gen-go (protocolbuffers/go)
  • Responsibility: the data structure (Message) layer.
  • Output: customer.pb.go
  • Content: Converts message Customer into a Go type Customer struct, and includes Getter methods as well as Protocol Buffers binary serialization/deserialization (Marshal/Unmarshal) logic.
• protoc-gen-go-grpc (grpc/go)
  • Responsibility: the network transport (RPC) layer.
  • Output: customer_grpc.pb.go
  • Content: Converts service CustomerService into Go interfaces, including the client-side call wrapper (Client Stub) and the handler interface to be implemented on the server side.

gen/
└── customer/
    └── v1/
        ├── customer.pb.go       # protocolbuffers/go: message types and serialization
        └── customer_grpc.pb.go  # grpc/go: Client and Server skeletons for service

customer.pb.go is the “data” part. It turns each message into a Go struct, attaches field numbers and serialization information, and provides nil-safe getters:

type Customer struct {
  Id          string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
  FullName    string `protobuf:"bytes,2,opt,name=full_name,json=fullName,proto3" json:"full_name,omitempty"`
  Email       string `protobuf:"bytes,3,opt,name=email,proto3" json:"email,omitempty"`
  PhoneNumber string `protobuf:"bytes,4,opt,name=phone_number,json=phoneNumber,proto3" json:"phone_number,omitempty"`
  // ...omitted private fields used internally by protobuf
}

// The generated getter handles a nil receiver, so res.GetCustomer().GetFullName() will not panic
func (x *Customer) GetFullName() string {
  if x != nil {
    return x.FullName
  }
  return ""
}

full_name retains three names here at the same time: proto’s full_name (for transport and JSON mapping), Go’s FullName (for program use), and json=fullName (camelCase for JSON conversion). This is why naming conventions can be left to the generator, without requiring each service to write its own conversion functions.

customer_grpc.pb.go is the “interface” part. Both the Client and Server sides grow out of this file, but the extent of generation is completely different.

Client Side: Even the Calls Are Written for You

const CustomerService_GetCustomer_FullMethodName = "/customer.v1.CustomerService/GetCustomer"

type CustomerServiceClient interface {
  GetCustomer(ctx context.Context, in *GetCustomerRequest, opts ...grpc.CallOption) (*GetCustomerResponse, error)
}

type customerServiceClient struct {
  cc grpc.ClientConnInterface
}

func NewCustomerServiceClient(cc grpc.ClientConnInterface) CustomerServiceClient {
  return &customerServiceClient{cc}
}

func (c *customerServiceClient) GetCustomer(ctx context.Context, in *GetCustomerRequest, opts ...grpc.CallOption) (*GetCustomerResponse, error) {
  out := new(GetCustomerResponse)
  err := c.cc.Invoke(ctx, CustomerService_GetCustomer_FullMethodName, in, out, opts...)
  if err != nil {
    return nil, err
  }
  return out, nil
}

The Client side is a complete implementation: the interface, struct, and the Invoke for every method are all generated. The caller only needs NewCustomerServiceClient(conn) to get a usable object. The path string /customer.v1.CustomerService/GetCustomer is also written as a constant. This is the address composed from proto’s package + service + rpc, eliminating the part that is easiest to mistype when writing HTTP requests by hand.

Server Side: Only the Skeleton Is Generated; You Write the Logic

type CustomerServiceServer interface {
  GetCustomer(context.Context, *GetCustomerRequest) (*GetCustomerResponse, error)
  mustEmbedUnimplementedCustomerServiceServer()
}

type UnimplementedCustomerServiceServer struct{}

func (UnimplementedCustomerServiceServer) GetCustomer(context.Context, *GetCustomerRequest) (*GetCustomerResponse, error) {
  return nil, status.Errorf(codes.Unimplemented, "method GetCustomer not implemented")
}

func RegisterCustomerServiceServer(s grpc.ServiceRegistrar, srv CustomerServiceServer) {
  s.RegisterService(&CustomerService_ServiceDesc, srv)
}

var CustomerService_ServiceDesc = grpc.ServiceDesc{
  ServiceName: "customer.v1.CustomerService",
  HandlerType: (*CustomerServiceServer)(nil),
  Methods: []grpc.MethodDesc{
    {MethodName: "GetCustomer", Handler: _CustomerService_GetCustomer_Handler},
  },
  // Omitted Streams and Metadata
}

What the Server side generates is holes to fill in:

• The CustomerServiceServer interface defines which methods exist, what they receive, and what they return. If the method signature is wrong (for example, one parameter is missing or the return type is wrong), compilation fails; it will not wait until runtime.
• UnimplementedCustomerServiceServer is the default implementation; every method returns codes.Unimplemented. The lowercase mustEmbedUnimplementedCustomerServiceServer() method in the interface cannot be implemented from an external package, which effectively forces you to embed it into your own struct. This way, when proto adds a new rpc, old Servers can still compile.
• ServiceDesc and the various _Handlers are the routing table and decoders: they deserialize incoming binary data into *GetCustomerRequest, call your method, then serialize the returned value back out.

So the division of labor between the two sides is:

• Never edit .pb.go by hand: the next buf generate will overwrite the entire file. If you need to add behavior, wrap it in your own struct.
• Should generated files be committed to version control? I tend to commit them, so go build does not require installing the protoc toolchain first, and IDEs and AI can directly read the types. The cost is that every proto change brings a chunk of diff. Conversely, generating them in CI ensures you do not forget to rerun generation, but the local development experience is a bit worse.

Implementing the Server

The generated customerv1 package provides a CustomerServiceServer interface. All the Server side needs to do is implement it:

package main

import (
  "context"
  "log"
  "net"

  customerv1 "github.com/riceball/example/gen/customer/v1"
  "google.golang.org/grpc"
  "google.golang.org/grpc/codes"
  "google.golang.org/grpc/status"
)

type customerServer struct {
  customerv1.UnimplementedCustomerServiceServer
}

func (s *customerServer) GetCustomer(ctx context.Context, req *customerv1.GetCustomerRequest) (*customerv1.GetCustomerResponse, error) {
  if req.GetId() == "" {
    return nil, status.Error(codes.InvalidArgument, "id is required")
  }

  // In practice, replace this with a DB query
  if req.GetId() != "123" {
    return nil, status.Errorf(codes.NotFound, "customer %s not found", req.GetId())
  }

  return &customerv1.GetCustomerResponse{
    Customer: &customerv1.Customer{
      Id:          "123",
      FullName:    "Riceball",
      Email:       "riceball@example.com",
      PhoneNumber: "0912345678",
    },
  }, nil
}

func main() {
  lis, err := net.Listen("tcp", ":50051")
  if err != nil {
    log.Fatalf("failed to listen: %v", err)
  }

  s := grpc.NewServer()
  customerv1.RegisterCustomerServiceServer(s, &customerServer{})

  log.Println("gRPC server listening on :50051")
  if err := s.Serve(lis); err != nil {
    log.Fatalf("failed to serve: %v", err)
  }
}

In addition to embedding UnimplementedCustomerServiceServer as mentioned earlier, the only new concept here is expressing errors with status: gRPC has its own error code system (codes.NotFound, codes.InvalidArgument, codes.DeadlineExceeded…). There are no custom fields stuffed into a response body; clients can directly use status.Code(err) to distinguish errors.

Implementing the Client

The Client side does not need any handwritten HTTP requests or JSON parsing. What you get is a type-safe function:

package main

import (
  "context"
  "log"
  "time"

  customerv1 "github.com/riceball/example/gen/customer/v1"
  "google.golang.org/grpc"
  "google.golang.org/grpc/credentials/insecure"
)

func main() {
  conn, err := grpc.NewClient("localhost:50051", grpc.WithTransportCredentials(insecure.NewCredentials()))
  if err != nil {
    log.Fatalf("failed to connect: %v", err)
  }
  defer conn.Close()

  client := customerv1.NewCustomerServiceClient(conn)

  ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
  defer cancel()

  res, err := client.GetCustomer(ctx, &customerv1.GetCustomerRequest{Id: "123"})
  if err != nil {
    log.Fatalf("GetCustomer failed: %v", err)
  }

  // PhoneNumber is part of the contract; it will not disappear because Order Service forgot to define it
  log.Println(res.GetCustomer().GetFullName(), res.GetCustomer().GetPhoneNumber())
}

insecure.NewCredentials() is only suitable for local development. In production, replace it with credentials.NewTLS(...).

In gRPC, context.WithTimeout is not just a local timeout. The deadline is sent to the Server along with the request via the grpc-timeout header. As long as the Server passes the same ctx downstream, downstream services will also share the remaining time. This is very different from REST, where you need to agree on headers yourself.

Looking Back at the Two Pain Points

• Pain Point 1 (different naming): Names are determined by the contract, and the generator for each language is responsible for converting them to local conventions. No manual mapping table is needed anymore.
• Pain Point 2 (missing fields): phone_number exists in the same message, so Order Service receives the complete structure. If Customer Service wants to delete this field someday, buf breaking will stop it in CI.

This also solves another thing mentioned at the beginning: .proto itself is excellent AI context. Instead of dumping the code from two repositories into the model and asking it to guess the interface, give it a contract of a few dozen lines, and it will know which services, methods, fields, and types exist.

Four Calling Patterns

gRPC is built on HTTP/2. In addition to the common request-response pattern, it also supports streaming:

service CustomerService {
  // 1. Unary: one request, one response
  rpc GetCustomer(GetCustomerRequest) returns (GetCustomerResponse);

  // 2. Server streaming: one request, multiple responses (for example, export or event subscription)
  rpc ListCustomers(ListCustomersRequest) returns (stream ListCustomersResponse);

  // 3. Client streaming: multiple requests, one response (for example, batch upload)
  rpc ImportCustomers(stream ImportCustomersRequest) returns (ImportCustomersResponse);

  // 4. Bidirectional streaming: both directions proceed simultaneously (for example, chat or real-time synchronization)
  rpc SyncCustomers(stream SyncCustomersRequest) returns (stream SyncCustomersResponse);
}

Most internal service communication only uses unary request-response, but when push or large-volume data streaming is needed, you no longer need to introduce an additional layer such as WebSocket or SSE.

Trade-offs

gRPC is not suitable for every scenario. Before adopting it in practice, it is worth confirming the following limitations:

The usual approach is gRPC for internal services, while externally exposed APIs continue to use OpenAPI. In between, you can use grpc-gateway🔗 to generate RESTful endpoints and OpenAPI documentation from the same .proto, so even the external documentation does not need to be maintained by hand.

Losing curl for debugging feels a bit unfamiliar, but with Server Reflection, grpcurl can call services directly without a proto file, provided that the Server has registered the reflection service (reflection.Register(s) from google.golang.org/grpc/reflection, usually enabled only on internal networks or in development environments):

# List all services
grpcurl -plaintext localhost:50051 list

# Call a method
grpcurl -plaintext -d '{"id": "123"}' localhost:50051 customer.v1.CustomerService/GetCustomer

Summary

gRPC consolidates service communication into a single contract

1. There is only one definition of the data between services, so low-level mistakes such as naming conversions and missing fields disappear
2. Breaking changes are blocked by buf breaking in CI, so changing data definitions no longer depends on Code Review
3. .proto becomes an interface document shared by humans and AI, making collaboration both precise and inexpensive

The cost is an additional code-generation build step and the need to become familiar with new debugging tools. For an internal multi-service architecture, this is a reasonable trade-off.

Further Reading

• Official gRPC Documentation🔗
• Protocol Buffers Language Guide (proto 3)🔗
• Buf Documentation🔗
• gRPC-Go Basics Tutorial🔗
• Easily Understanding gRPC - Coding with Yalco🔗