Skip to content

RepeatedFieldFixedSize<T, 0> retains sizeof(T) of storage on libc++ #93

Description

A repeated field configured with maxLength = 0 should occupy no per-element storage, but on libc++ it costs a full sizeof(T) the same as maxLength = 1. The cause is std::array<T, 0>, whose size is implementation-defined: libstdc++ collapses it to 1 byte, libc++ keeps sizeof(T) bytes so that data() can return a validly-aligned pointer.

This matters for the common pattern of building one set of generated messages and selecting per-product profiles through the MAX_LENGTH template parameters. A profile that disables an optional repeated message by passing 0 still pays for one complete element, which defeats the compile-time sizing model that Embedded Proto is built around. On a large nested message the difference is hundreds of bytes per instance.

Tested on 3.6.1 (94b6ed5).

Reproduction

#include <RepeatedFieldFixedSize.h>
#include <MessageInterface.h>
#include <cstdio>

using namespace EmbeddedProto;

// Stand-in for a fat nested message.
class FatMsg : public MessageInterface {
  public:
    Error serialize(WriteBufferInterface&) const override { return Error::NO_ERRORS; }
    Error deserialize(ReadBufferInterface&) override { return Error::NO_ERRORS; }
    void clear() override {}
    static constexpr uint32_t max_serialized_size() { return 4; }
    static constexpr uint32_t max_serialized_size(uint32_t) { return 4; }
  private:
    char payload_[512] = {};
};

int main() {
  printf("sizeof(FatMsg)                = %zu\n", sizeof(FatMsg));
  printf("sizeof(std::array<FatMsg, 0>) = %zu\n", sizeof(std::array<FatMsg, 0>));
  printf("RepeatedFieldFixedSize<FatMsg, 0> = %zu\n",
         sizeof(RepeatedFieldFixedSize<FatMsg, 0>));
  printf("RepeatedFieldFixedSize<FatMsg, 1> = %zu\n",
         sizeof(RepeatedFieldFixedSize<FatMsg, 1>));
}

Results

sizeof(FatMsg) is 520 on the host and 516 on the embedded targets (vtable
pointer width differs).

Toolchain Standard library array<FatMsg, 0> RFFS<FatMsg, 0> RFFS<FatMsg, 1>
Apple clang 21.0.0 (arm64) libc++ 520 536 536
riscv32-esp-elf-g++ 14.2.0 libstdc++ 1 12 524
xtensa-esp-elf-g++ 13.2.0 libstdc++ 1 12 524

On libc++ the zero-capacity field is byte-for-byte as expensive as a single-element one. On libstdc++ it is nearly free already, so the behaviour is inconsistent across the toolchains a single project may target we hit this because our firmware builds with the GCC cross-compilers while the host unit tests build with libc++, and only the host build ballooned.

Suggested fix

An explicit specialization for MAX_LENGTH == 0 that holds no DATA_TYPE storage while preserving the RepeatedField<DATA_TYPE> interface and wire behaviour. Since the field can never hold an element, every accessor is a constant and nothing is ever serialized:

template<class DATA_TYPE>
class RepeatedFieldFixedSize<DATA_TYPE, 0> : public RepeatedField<DATA_TYPE>
{
  public:
    RepeatedFieldFixedSize() = default;
    RepeatedFieldFixedSize(const RepeatedFieldFixedSize&) = default;
    RepeatedFieldFixedSize& operator=(const RepeatedFieldFixedSize&) = default;
    ~RepeatedFieldFixedSize() override = default;

    uint32_t get_length() const override { return 0; }
    uint32_t get_max_length() const override { return 0; }
    uint32_t get_size() const override { return 0; }
    uint32_t get_max_size() const override { return 0; }

    DATA_TYPE& get(uint32_t) override { std::abort(); }
    const DATA_TYPE& get_const(uint32_t) const override { std::abort(); }

    Error get_const(const uint32_t, DATA_TYPE&) const override
    {
      return Error::INDEX_OUT_OF_BOUND;
    }

    void set(uint32_t, const DATA_TYPE&) override { std::abort(); }

    Error set_data(const DATA_TYPE*, const uint32_t length) override
    {
      return 0 == length ? Error::NO_ERRORS : Error::ARRAY_FULL;
    }

    Error add(const DATA_TYPE&) override { return Error::ARRAY_FULL; }
    void clear() override {}

    const std::array<DATA_TYPE, 0>& get_data_const() const
    {
      static const std::array<DATA_TYPE, 0> empty{};
      return empty;
    }

    //! A zero capacity field never serializes anything, packed or unpacked.
    static constexpr uint32_t max_serialized_size(const uint32_t) { return 0; }
    static constexpr uint32_t max_packed_serialized_size(const uint32_t) { return 0; }
    static constexpr uint32_t max_unpacked_serialized_size(const uint32_t) { return 0; }
};

This brings the zero-capacity field down to 8 bytes on libc++ and 4 on libstdc++, the vtable pointer alone, and makes the cost consistent across standard library implementations.

Happy to open a pull request if this approach looks reasonable

Environment

  • Embedded Proto 3.6.1 (94b6ed5)
  • C++17
  • Apple clang 21.0.0 / libc++, riscv32-esp-elf-g++ 14.2.0 and
    xtensa-esp-elf-g++ 13.2.0 / libstdc++ (ESP-IDF crosstool-NG builds)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions