Skip to content
  • Colin Patrick McCabe's avatar
    KAFKA-7609; Add Protocol Generator for Kafka (#5893) · 71e85f5e
    Colin Patrick McCabe authored
    This patch adds a framework to automatically generate the request/response classes for Kafka's protocol. The code will be updated to use the generated classes in follow-up patches. Below is a brief summary of the included components:
    
    **buildSrc/src**
    The message generator code is here.  This code is automatically re-run by gradle when one of the schema files changes.  The entire directory is processed at once to minimize the number of times we have to start a new JVM.  We use Jackson to translate the JSON files into Java objects.
    
    **clients/src/main/java/org/apache/kafka/common/protocol/Message.java**
    This is the interface implemented by all automatically generated messages.
    
    **clients/src/main/java/org/apache/kafka/common/protocol/MessageUtil.java**
    Some utility functions used by the generated message code.
    
    **clients/src/main/java/org/apache/kafka/common/protocol/Readable.java, Writable.java, ByteBufferAccessor.java**
    The generated message code uses these classes for writing to a buffer.
    
    **clients/src/main/message/README.md**
    This README file explains how the JSON schemas work.
    
    **clients/src/main/message/\*.json**
    The JSON files in this directory implement every supported version of every Kafka API.  The unit tests automatically validate that the generated schemas match the hand-written schemas in our code.  Additionally, there are some things like request and response headers that have schemas here.
    
    **clients/src/main/java/org/apache/kafka/common/utils/ImplicitLinkedHashSet.java**
    I added an optimization here for empty sets.  This is useful here because I want all messages to start with empty sets by default prior to being loaded with data.  This is similar to the "empty list" optimizations in the `java.util.ArrayList` class.
    
    Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Ismael Juma <ismael@juma.me.uk>, Bob Barrett <bob.barrett@outlook.com>, Jason Gustafson <jason@confluent.io>
    71e85f5e