24. January 2023
by Swaneet Sahoo | 1020 words | ~5 min read
Do you have difficulties debugging Protocol-Buffers-based HTTP REST endpoints? Since Protobuf uses binary payloads, we face the problem, that we cannot easily write or read our Protobuf payloads with
curl directly on the terminal. Ideally, we would like to to use curl with Protobuf just like we use curl with JSON or XML with classic text-based HTTP REST endpoints. 1
To this problem, we present protoCURL - cURL for Protobuf: The command line tool to quickly and easily write requests in human-readable text formats on the command line against Protocol Buffer over HTTP endpoints and view the output in a text-based format as well.
This can become handy when debugging Protobuf-based HTTP endpoints when they are used instead of JSON.2
With protoCURL a request can be as simple as this:3
The command line arguments have this meaning:
-I test/protopoints to proto files in the filesystem
-o ..HappyDayResponsetell which message types are used for the input and expected for the output
..instructs protocurl to automatically infer the (unique) full package path of the message types
-u <url>is the HTTP REST endpoint url to send the request to
-d 'includeReason: true'describes the input data in the Protobuf Text Format
The Protobuf endpoint uses the proto file happyday.proto - which consists of these (condensed) request and response messages:
The server will tell us whether the UTC day of a specific timestamp is a happy day or not. We simply say, that everyday, except for Wednesdays, is a happy day.4
Concretely, given a
UTCand sets the boolean
isHappyDayif and only if the weekday of the date is Wednesday.
includeReasonis set, then a reason for the
isHappyDayboolean is given as a string.
The above protoCURL command will produce this output:
We can now easily see the request and response in the Protobuf Text Format without needing to use
protoc or a full-blown programming language to manually (de-) serialize the content. Since we didn’t provide a
date, the Protobuf server implicitly assumes all of the timestamp.proto fields to be zero - which corresponds to epoch time
In the background, protocurl essentially does the following:
curl, if possible) and receive the binary response payload
As a second example, let’s see, what happens, when we set the
date to the first Wednesday in 2023:
Note, the syntax for the text format above.
This will produce:
Furthermore, we can also use JSON as the text format:
The JSON format is automatically detected and also used for the output:
Want to try this out? You can reproduce these examples by taking the following steps:
In your macOS / Linux or Windows MinGW terminal, run:
# Clone the repository and enter the directory git clone protocurl cd protocurl # Ensure these are installed and available to you: bash, jq, zip, unzip and curl # e.g. sudo apt install bash jq zip unzip curl # Download the latest protoc binaries for the tests ./release/10-get-protoc-binaries.sh # Start server (source test/suite/setup.sh && startServer)
Simply follow the command line installation instructions.
When using windows, run
protocurl.exe in Powershell or cmd instead of MinGW.
If you use docker, then use the command
docker run -v /path/to/proto:/proto qaware/protocurl [...ARGS]
protocurl -I /path/to/proto [...ARGS]
Now you can send requests like the examples above.
We can use protocurl as a quick and ergonomic command line tool to interact with Protobuf-based HTTP REST endpoints while working with human-readable text formats.
Head over to protocurl on GitHub and
In specific scenarios, Protobuf is used instead of JSON, because the built-in schema format as well as the ability to generate code make it a viable choice for keeping consumers and producers of REST APIs consistent with each other and documented. It can also be the serialization for RPC frameworks such as Twirp. ↩︎
For Powershell, we need to replace
\ in the path and use a backtick ` instead of
\ as line separators. ↩︎